mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-10-14 19:27:04 -04:00
Tanish MankuandClaude Opus 5
f988cf87ed
Gap 12 of docs/gaps.md: until rendering is measured, every claim about its quality is a judgement. Text extraction had an oracle and written output had qpdf; rendering had nothing. `cargo xtask render-oracle` renders page 1 of every corpus file with our CLI and with Poppler's pdftoppm and reports how far apart they are. The metric is the point. A raw pixel comparison is useless here -- two correct renderers disagree on every glyph edge, so it calls a perfect page badly wrong and says nothing about whether a shading is missing. `pixels::compare` counts a pixel as differing only when it differs from every pixel in the other image's 3x3 neighbourhood by more than a tolerance, in both directions, taking the worse. A one-pixel shift is forgiven; absent or miscoloured content is not. First reading, 250 files at 72 dpi, tolerance 8: 229 compared, median 0.89%, mean 2.16%, p90 5.35%, 25 within 0.1%. It reports rather than gates -- the corpus is not on CI, and a first reading of a borrowed renderer is a baseline to improve on, not a bar to clear. The worst results cluster on JBIG2 `bitmap-*` files and on ZapfDingbats, which aims the renderer work at something specific for the first time. 15 files could not be compared because the two renderers round the page box to different pixel sizes; they are reported as unmeasurable rather than scored. Verified against the knob's bounds end to end: tolerance 255 scores every file 0.00%, tolerance 0 takes the median to 0.95% and p90 to 26.36%. Also turns on `extract_type3_shared_xobjects_matches_poppler_pixels`, which had never run anywhere. Poppler is now installed on the Linux runner, and the test is version-robust by construction -- both sides of its comparison go through the same pdftoppm, so it is about our two documents rather than about Poppler's output. Verified in all three states: runs and passes with pdftoppm present, skips without it, fails without it when PDFCRAFT_ORACLES_REQUIRED is set. `run_capture` keeps its exact behaviour, including leaving the buffer empty on non-UTF-8 output; the byte-oriented capture an image needs is a second entry point on the same timeout logic. Co-Authored-By: Claude Opus 5 <[email protected]>