fix(oracle): say what the render-oracle score does and does not mean

Investigating the worst file of the first 250 turned the headline finding around. bug1552113.pdf
scores 84%, and the cause is Poppler: it paints the link's `/Border [0 0 112]` as a 112-unit band
across a 250x50 page, hiding the two lines of text. The fixture is named "this text should be
visible", and in our rendering it is. The oracle is not ground truth, and the top of its worst
list needs a person to look at both images before it is read as a bug of ours.

That led to a plausible change that the measurement rejected. Poppler synthesises appearances for
annotations that have none, where we draw only what the document carries -- `/AP`-less link borders
are already recorded as not covered under view.render-annotation-appearances -- so hiding
annotations looked like the way to measure page content alone. It is worse: it then scores Poppler
for every appearance we do draw. Over the same 250 files, shown gives mean 2.16%, p90 5.35% and 25
files within 0.1%; hidden gives 2.18%, 5.55% and 22, with annotation-tx3.pdf jumping to 45.64% and
annotation-tx2.pdf to 18.79%. The two biases nearly cancel.

So the default is unchanged and `--hide-annotations` is an opt-in for explaining a single file,
with the numbers written down so the experiment is not repeated. Verified that the default
reproduces the merged baseline exactly and that the flag moves it.

Co-Authored-By: Claude Opus 5 <[email protected]>

xtask/src/gates.rs

@@ -253,6 +253,19 @@ pub fn render_oracle(args: &[String]) -> anyhow::Result<()> {
     let limit: usize = flag_num(args, "--limit").unwrap_or(usize::MAX);
     let dpi: u32 = flag_num(args, "--dpi").unwrap_or(72);
     let tolerance: u8 = flag_num(args, "--tolerance").unwrap_or(8);
+    // Annotations are shown by default, which is not the obvious choice and was settled by
+    // measurement rather than argument. Poppler synthesises an appearance for an annotation that
+    // has none, where we draw only what the document carries (`view.render-annotation-appearances`
+    // records link borders without `/AP` as not covered), so showing them scores us for a known
+    // gap: bug1552113.pdf is 84% purely because Poppler paints a 112-unit link border over the
+    // text that fixture exists to show -- and our rendering, not Poppler's, is the readable one.
+    //
+    // Hiding them is worse, though, because it scores Poppler for every appearance we *do* draw.
+    // Over the same 250 files: shown gives mean 2.16%, p90 5.35%, 25 files within 0.1%; hidden
+    // gives 2.18%, 5.55% and 22, with annotation-tx3.pdf jumping to 45.64%. The two biases very
+    // nearly cancel, so the default stays with the fuller picture and this flag exists to tell
+    // the two apart when a single file's score needs explaining.
+    let hide_annotations = args.iter().any(|a| a == "--hide-annotations");
     let dir =
         args.iter().position(|a| a == "--dir").and_then(|i| args.get(i + 1)).map_or_else(|| root().join("corpus/pdfjs/test/pdfs"), PathBuf::from);
 
@@ -287,9 +300,13 @@ pub fn render_oracle(args: &[String]) -> anyhow::Result<()> {
             .filter(std::process::ExitStatus::success)
             .and_then(|_| std::fs::read(&out_pam).ok())
             .and_then(|b| pdfcraft_oracle::pixels::decode_pam_gray(&b));
-        let theirs =
-            run_capture_bytes(pdftoppm.command().args(["-singlefile", "-gray", "-f", "1", "-l", "1", "-r"]).arg(dpi.to_string()).arg(file), 30)
-                .and_then(|b| pdfcraft_oracle::pixels::decode_pgm(&b));
+        let mut theirs_cmd = pdftoppm.command();
+        theirs_cmd.args(["-singlefile", "-gray", "-f", "1", "-l", "1", "-r"]).arg(dpi.to_string());
+        if hide_annotations {
+            theirs_cmd.arg("-hide-annotations");
+        }
+        theirs_cmd.arg(file);
+        let theirs = run_capture_bytes(&mut theirs_cmd, 30).and_then(|b| pdfcraft_oracle::pixels::decode_pgm(&b));
         // An oracle that cannot read a file says nothing about our fidelity, and a file we cannot
         // read is `xtask check`'s business, not this one's. Count each side and move on.
         if ours.is_none() {

xtask/src/main.rs

@@ -35,7 +35,7 @@ const COMMANDS: &[(&str, &str, Command)] = &[
     ("text-oracle", "Compare text extraction with pdftotext over corpus/ (word F1; target median ≥ 0.97)", gates::text_oracle),
     (
         "render-oracle",
-        "Compare page rendering with Poppler over corpus/ (pixel disagreement rate; --dir, --dpi, --limit, --tolerance)",
+        "Compare page rendering with Poppler over corpus/ (pixel disagreement rate; --dir, --dpi, --limit, --tolerance, --hide-annotations)",
         gates::render_oracle,
     ),
     ("screenshots", "Regenerate the README screenshots in docs/images/ and their ATTRIBUTION entries", screenshots::run),