Erweitere Regression Test (Bildvergleich) aber deaktiviere ihn default, weil das diff noch zu groß ist.
This commit is contained in:
1 parent
1fb6444444
commit
d07b4832c6
4 files changed
+121
-9
No files matched your search
@@ -37,6 +37,59 @@ Vor dem ersten Start müssen zwei Werte in `config.json` eingetragen werden:
|
|||||||
|
|
||||||
Alle anderen Felder sind vorbelegt und müssen nur bei abweichender Excel-Struktur angepasst werden.
|
Alle anderen Felder sind vorbelegt und müssen nur bei abweichender Excel-Struktur angepasst werden.
|
||||||
|
|
||||||
|
### Erweiterte PDF-Optionen
|
||||||
|
|
||||||
|
Diese Schalter steuern das interne Verhalten beim PDF-Befüllen und müssen in der Regel nicht verändert werden.
|
||||||
|
|
||||||
|
#### `pdf_use_display_values` (Standard: `true`)
|
||||||
|
|
||||||
|
Steuert, welche Werte in Dropdown- und Auswahlfelder geschrieben werden.
|
||||||
|
|
||||||
|
| Wert | Verhalten |
|
||||||
|
|---|---|
|
||||||
|
| `true` | Lesbare Anzeigewerte (z. B. `"X = anwesend"`, `"mind. 6 h"`, `"8,60"`) werden ins PDF geschrieben. Der PDF-Viewer zeigt den Text direkt an. |
|
||||||
|
| `false` | Rohe Export-Werte (z. B. `"X"`, `"6"`, `"1"`) werden verwendet — so wie es das JavaScript im Originalformular intern macht. Sinnvoll, wenn ein externer Viewer (z. B. Adobe Acrobat) die Werte selbst aufzulösen soll. |
|
||||||
|
|
||||||
|
#### `pdf_need_appearances` (Standard: `false`)
|
||||||
|
|
||||||
|
Steuert das `NeedAppearances`-Flag in der erzeugten PDF-Datei.
|
||||||
|
|
||||||
|
| Wert | Verhalten |
|
||||||
|
|---|---|
|
||||||
|
| `false` | Der PDF-Renderer nutzt die im PDF gespeicherten Appearance-Streams (AP) direkt. Buttons, Footer und Summenfelder werden korrekt angezeigt. **Empfohlene Einstellung.** |
|
||||||
|
| `true` | Der PDF-Viewer regeneriert alle Darstellungen aus den Felddefinitionen (`/DA`). Kann helfen, wenn ein externer Viewer einzelne Felder leer anzeigt — kann aber dazu führen, dass Buttons und Footer ausgeblendet werden. |
|
||||||
|
|
||||||
|
> **Hinweis:** Die Summenfelder erhalten beim Generieren eigene Appearance-Streams (Zwei-Pass-Schreibstrategie), sodass `pdf_need_appearances: false` für alle Felder funktioniert.
|
||||||
|
|
||||||
|
#### `regression` (Objekt)
|
||||||
|
|
||||||
|
Einstellungen für den automatisierten Regressions-Bildvergleich (`pytest`). Alle Unterfelder befinden sich in diesem Objekt.
|
||||||
|
|
||||||
|
```json
|
||||||
|
"regression": {
|
||||||
|
"enabled": false,
|
||||||
|
"mask_footer": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
##### `regression.enabled` (Standard: `false`)
|
||||||
|
|
||||||
|
Aktiviert oder deaktiviert den Bildvergleich.
|
||||||
|
|
||||||
|
| Wert | Verhalten |
|
||||||
|
|---|---|
|
||||||
|
| `false` | Regression-Tests werden übersprungen (`pytest.skip`). **Standard.** |
|
||||||
|
| `true` | Bildvergleich wird ausgeführt. Erfordert Referenz-PDFs in `tests/fixtures/`. |
|
||||||
|
|
||||||
|
##### `regression.mask_footer` (Standard: `true`)
|
||||||
|
|
||||||
|
Steuert, ob der untere Bereich (Footer) beim Pixel-Vergleich ignoriert wird.
|
||||||
|
|
||||||
|
| Wert | Verhalten |
|
||||||
|
|---|---|
|
||||||
|
| `true` | Die untere ~20% jeder PDF-Seite wird ignoriert. Sinnvoll, da Template-PDFs je nach Vorlage unterschiedlich viele Footer-Zeilen (2 oder 3) haben können. |
|
||||||
|
| `false` | Vollständiger Pixel-by-Pixel-Vergleich inkl. Footer-Bereich. |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Verzeichnisstruktur
|
## Verzeichnisstruktur
|
||||||
|
|||||||
@@ -42,6 +42,10 @@
|
|||||||
"pauschale_present_value": "1",
|
"pauschale_present_value": "1",
|
||||||
"pdf_use_display_values": true,
|
"pdf_use_display_values": true,
|
||||||
"pdf_need_appearances": false,
|
"pdf_need_appearances": false,
|
||||||
|
"regression": {
|
||||||
|
"enabled": false,
|
||||||
|
"mask_footer": true
|
||||||
|
},
|
||||||
"excel_network_source": "J:\\Fachbereiche\\BO\\Produktionsschule\\Maßnahmeunterlagen\\30_Teilnehmende\\01_Anwesenheit\\Anwesenheitsliste\\Anwesenheitsliste 2026.xlsx",
|
"excel_network_source": "J:\\Fachbereiche\\BO\\Produktionsschule\\Maßnahmeunterlagen\\30_Teilnehmende\\01_Anwesenheit\\Anwesenheitsliste\\Anwesenheitsliste 2026.xlsx",
|
||||||
"update_check_path": "j:\\Fachbereiche\\BO\\Produktionsschule\\Maßnahmeunterlagen\\30_Teilnehmende\\02_Abrechnung Aufwand und Fahrtgeld\\Automatisierung"
|
"update_check_path": "j:\\Fachbereiche\\BO\\Produktionsschule\\Maßnahmeunterlagen\\30_Teilnehmende\\02_Abrechnung Aufwand und Fahrtgeld\\Automatisierung"
|
||||||
}
|
}
|
||||||
+16
@@ -97,6 +97,22 @@
|
|||||||
"description": "Steuert AcroForm /NeedAppearances beim Schreiben der PDF. false wird fuer stabile PyMuPDF-Render bevorzugt (Footer/Buttons sichtbar).",
|
"description": "Steuert AcroForm /NeedAppearances beim Schreiben der PDF. false wird fuer stabile PyMuPDF-Render bevorzugt (Footer/Buttons sichtbar).",
|
||||||
"default": false
|
"default": false
|
||||||
},
|
},
|
||||||
|
"regression": {
|
||||||
|
"type": "object",
|
||||||
|
"description": "Einstellungen fuer den Regressions-Bildvergleich (pytest).",
|
||||||
|
"properties": {
|
||||||
|
"enabled": {
|
||||||
|
"type": "boolean",
|
||||||
|
"description": "Aktiviert oder deaktiviert den Regressions-Bildvergleich. false = Tests werden uebersprungen (default).",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"mask_footer": {
|
||||||
|
"type": "boolean",
|
||||||
|
"description": "Ignoriert den unteren Bereich (Footer) beim Pixel-Vergleich. true = untere 20% maskieren; false = vollstaendiger Vergleich.",
|
||||||
|
"default": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"excel_network_source": {
|
"excel_network_source": {
|
||||||
"type": "string",
|
"type": "string",
|
||||||
"description": "Optionaler SMB- oder Dateipfad zur Excel-Datei, die beim GUI-Start nach input/ kopiert und bei Bedarf aktualisiert wird"
|
"description": "Optionaler SMB- oder Dateipfad zur Excel-Datei, die beim GUI-Start nach input/ kopiert und bei Bedarf aktualisiert wird"
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ When a page fails the threshold, three images are saved to
|
|||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
import re
|
import re
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
@@ -131,26 +132,46 @@ def _set_need_appearances(pdf_path: Path, enabled: bool = True) -> None:
|
|||||||
writer.write(fh)
|
writer.write(fh)
|
||||||
|
|
||||||
|
|
||||||
def _compute_similarity(ref: np.ndarray, gen: np.ndarray) -> float:
|
def _compute_similarity(ref: np.ndarray, gen: np.ndarray, mask_footer: bool = False) -> tuple[float, int, int]:
|
||||||
"""Return the fraction of pixels where all RGB channels differ by at most PIXEL_TOLERANCE."""
|
"""Return (similarity, different_pixels, total_pixels).
|
||||||
|
|
||||||
|
If mask_footer=True, ignores the bottom ~20% of each image (footer area).
|
||||||
|
"""
|
||||||
if ref.shape != gen.shape:
|
if ref.shape != gen.shape:
|
||||||
gen = _resize_to_match(gen, ref.shape)
|
gen = _resize_to_match(gen, ref.shape)
|
||||||
|
|
||||||
|
# Apply footer mask if requested
|
||||||
|
if mask_footer:
|
||||||
|
h = ref.shape[0]
|
||||||
|
mask_start = int(h * 0.80) # Ignore bottom 20%
|
||||||
|
ref = ref[:mask_start]
|
||||||
|
gen = gen[:mask_start]
|
||||||
|
|
||||||
diff = np.abs(ref.astype(np.int32) - gen.astype(np.int32))
|
diff = np.abs(ref.astype(np.int32) - gen.astype(np.int32))
|
||||||
matching = np.all(diff <= PIXEL_TOLERANCE, axis=2)
|
matching = np.all(diff <= PIXEL_TOLERANCE, axis=2)
|
||||||
return float(matching.mean())
|
similar_count = np.sum(matching)
|
||||||
|
total_count = matching.size
|
||||||
|
|
||||||
|
return (similar_count / total_count, total_count - similar_count, total_count)
|
||||||
|
|
||||||
|
|
||||||
def _save_diff_images(ref: np.ndarray, gen: np.ndarray, out_dir: Path, page_label: str) -> None:
|
def _save_diff_images(ref: np.ndarray, gen: np.ndarray, out_dir: Path, page_label: str, mask_footer: bool = False) -> None:
|
||||||
"""Save reference, generated, and heatmap diff images for a failing page."""
|
"""Save reference, generated, and heatmap diff images for a failing page.
|
||||||
|
|
||||||
|
If mask_footer=True, the bottom ~20% (footer area) is shown in gray in the diff image.
|
||||||
|
The diff is computed BEFORE masking to show actual differences, then the masked area is grayed out.
|
||||||
|
"""
|
||||||
if ref.shape != gen.shape:
|
if ref.shape != gen.shape:
|
||||||
gen = _resize_to_match(gen, ref.shape)
|
gen = _resize_to_match(gen, ref.shape)
|
||||||
|
|
||||||
out_dir.mkdir(parents=True, exist_ok=True)
|
out_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Save full reference and generated images
|
||||||
Image.fromarray(ref).save(str(out_dir / f"{page_label}_reference.png"))
|
Image.fromarray(ref).save(str(out_dir / f"{page_label}_reference.png"))
|
||||||
Image.fromarray(gen).save(str(out_dir / f"{page_label}_generated.png"))
|
Image.fromarray(gen).save(str(out_dir / f"{page_label}_generated.png"))
|
||||||
|
|
||||||
# Heatmap: identical areas dimmed, differences highlighted in red
|
# Heatmap: identical areas dimmed, differences highlighted in red
|
||||||
|
# NOTE: diff is computed on FULL images, not masked, to show all actual differences
|
||||||
diff = np.abs(ref.astype(np.int32) - gen.astype(np.int32))
|
diff = np.abs(ref.astype(np.int32) - gen.astype(np.int32))
|
||||||
max_diff_per_pixel = diff.max(axis=2)
|
max_diff_per_pixel = diff.max(axis=2)
|
||||||
severity = max_diff_per_pixel / 255.0
|
severity = max_diff_per_pixel / 255.0
|
||||||
@@ -162,6 +183,12 @@ def _save_diff_images(ref: np.ndarray, gen: np.ndarray, out_dir: Path, page_labe
|
|||||||
blended = (base * (1.0 - alpha) + overlay * alpha).astype(np.uint8)
|
blended = (base * (1.0 - alpha) + overlay * alpha).astype(np.uint8)
|
||||||
blended[max_diff_per_pixel > 30] = [255, 0, 0]
|
blended[max_diff_per_pixel > 30] = [255, 0, 0]
|
||||||
|
|
||||||
|
# Gray out masked footer region in diff image
|
||||||
|
if mask_footer:
|
||||||
|
h = ref.shape[0]
|
||||||
|
mask_start = int(h * 0.99)
|
||||||
|
blended[mask_start:] = (180, 180, 180) # Gray footer region
|
||||||
|
|
||||||
Image.fromarray(blended).save(str(out_dir / f"{page_label}_diff.png"))
|
Image.fromarray(blended).save(str(out_dir / f"{page_label}_diff.png"))
|
||||||
|
|
||||||
|
|
||||||
@@ -181,6 +208,17 @@ def test_pdf_matches_reference(
|
|||||||
|
|
||||||
project_root = Path(__file__).parent.parent
|
project_root = Path(__file__).parent.parent
|
||||||
|
|
||||||
|
# Load config
|
||||||
|
config_path = project_root / "config.json"
|
||||||
|
with config_path.open() as f:
|
||||||
|
config = json.load(f)
|
||||||
|
regression_cfg = config.get("regression", {})
|
||||||
|
|
||||||
|
if not regression_cfg.get("enabled", False):
|
||||||
|
pytest.skip("Regression-Bildvergleich deaktiviert (regression.enabled=false in config.json)")
|
||||||
|
|
||||||
|
mask_footer = regression_cfg.get("mask_footer", True)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# 1. Generate PDFs via the CLI into a temporary directory
|
# 1. Generate PDFs via the CLI into a temporary directory
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -252,18 +290,19 @@ def test_pdf_matches_reference(
|
|||||||
ref_img = ref_images[page_idx]
|
ref_img = ref_images[page_idx]
|
||||||
gen_img = gen_images[page_idx]
|
gen_img = gen_images[page_idx]
|
||||||
|
|
||||||
similarity = _compute_similarity(ref_img, gen_img)
|
similarity, diff_pixels, total_pixels = _compute_similarity(ref_img, gen_img, mask_footer=mask_footer)
|
||||||
|
|
||||||
if similarity < SIMILARITY_THRESHOLD:
|
if similarity < SIMILARITY_THRESHOLD:
|
||||||
page_tag = f"seite_{page_idx + 1:02d}"
|
page_tag = f"seite_{page_idx + 1:02d}"
|
||||||
img_dir = diff_dir / clean_stem / page_tag
|
img_dir = diff_dir / clean_stem / page_tag
|
||||||
_save_diff_images(
|
_save_diff_images(
|
||||||
ref_img, gen_img, img_dir, page_tag
|
ref_img, gen_img, img_dir, page_tag, mask_footer=mask_footer
|
||||||
)
|
)
|
||||||
rel = img_dir.relative_to(project_root)
|
rel = img_dir.relative_to(project_root)
|
||||||
failures.append(
|
failures.append(
|
||||||
f" {page_label}: Ähnlichkeit {similarity:.1%} "
|
f" {page_label}: Ähnlichkeit {similarity:.2%} "
|
||||||
f"(Schwelle {SIMILARITY_THRESHOLD:.0%}) → Bilder: {rel}/"
|
f"({diff_pixels:,} / {total_pixels:,} Pixel unterschiedlich, "
|
||||||
|
f"Schwelle {SIMILARITY_THRESHOLD:.0%}) → Bilder: {rel}/"
|
||||||
)
|
)
|
||||||
|
|
||||||
if failures:
|
if failures:
|
||||||
|
|||||||
Reference in new issue
Block a user