Draw non-native art in .ai editing data from its PDF

A /ForeignObject (non-native art: the content of a placed PDF, which
Illustrator shows but doesn't edit) was left out with a note. The
letterhead of #861 keeps its fan of swooshes that way: a radial gradient
in a 3-point PDF, scaled 271 times and clipped by the swoosh paths, so
the page lost its main graphic.

The reader now decodes the object's ASCII85 /Data, has the PDF importer
read the PDF, and fits its art to the object's box as a group named
"Non-native art": /Bounds measured from /Origin with y down, then
/RTransform, which lands exactly on the clipping paths around both
objects of that file; the PDF's content is kept in its own y-up
coordinates, so it is flipped into the box. Ids are the document's and
the PDF's swatches stay unlinked; its images join the document by
content key. A PDF that can't be read is left out with a note, as
before.

The EPS crate now uses the PDF crate (same layer; allowed in the layer
check like render and metafile).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_017ZwZjADUZP4VzvrFGpiZch

Cargo.lock

@@ -4499,6 +4499,7 @@ dependencies = [
  "vectorcraft-effects",
  "vectorcraft-geom",
  "vectorcraft-metafile",
+ "vectorcraft-pdf",
  "vectorcraft-render",
  "vectorcraft-testkit",
  "vectorcraft-text",

crates/eps/Cargo.toml

@@ -17,6 +17,8 @@ vectorcraft-brush = { workspace = true }
 vectorcraft-text = { workspace = true }
 # Windows metafile previews, opened when a file's PostScript can't be read.
 vectorcraft-metafile = { workspace = true }
+# Non-native art in Illustrator editing data (a placed PDF's content, kept as a PDF).
+vectorcraft-pdf = { workspace = true }
 # The TIFF writer (previews) and PackBits, shared with raster export.
 vectorcraft-render = { workspace = true }
 kurbo = { workspace = true }

crates/eps/src/import/ai/read.rs

@@ -37,6 +37,7 @@ use vectorcraft_geom::{FillRule, PathData};
 use super::lex::{Lexer, Tok};
 use super::obj::{self, Obj, V};
 use super::paint::{self, GradientDef, Instance, Named};
+use crate::ps;
 
 /// Deepest nesting of layers, groups and dictionaries.
 const MAX_DEPTH: usize = 256;
@@ -248,6 +249,8 @@ struct Reader<'a> {
     hidden_ids: HashSet<NodeId>,
     /// CMYK samples converted, by value.
     cmyk_cache: HashMap<[u8; 4], [u8; 3]>,
+    /// A `/ForeignObject`'s `/Data`: its PDF as ASCII85 in comment lines, until the object closes.
+    foreign_data: Option<Vec<u8>>,
     /// The page ended: what follows isn't art.
     done: bool,
     /// The dictionary closed last was opened on a `%_` line.
@@ -316,8 +319,11 @@ const SKIPPED: &[(&str, &str)] = &[
     ("Alternate_Content", "End_Versioned_Content"),
 ];
 
-/// The note for a `/ForeignObject`, art the editor shows but doesn't edit.
-pub const NON_NATIVE_ART: &str = "non-native art (a placed PDF's content) is left out: VectorCraft doesn't draw it from the editing data yet";
+/// The note for a `/ForeignObject` (non-native art, the content of a placed PDF, which the editor
+/// shows but doesn't edit) whose PDF can't be read.
+pub const NON_NATIVE_ART_UNREAD: &str = "non-native art (a placed PDF's content) that VectorCraft can't read is left out";
+/// The name of the group a `/ForeignObject`'s art comes in as.
+pub const NON_NATIVE_ART: &str = "Non-native art";
 
 /// A section comment's marker without its `AI<version>_` prefix and value (`AI14_BeginSymbol` →
 /// `BeginSymbol`); DSC comments (`%%BeginProlog`) keep their `%`.
@@ -350,6 +356,7 @@ impl<'a> Reader<'a> {
             pending_stroke: None,
             spots: Vec::new(),
             raster: None,
+            foreign_data: None,
             after_object: false,
             nodes: 0,
             unsupported: BTreeSet::new(),
@@ -659,8 +666,10 @@ impl<'a> Reader<'a> {
                     // of it may start with `%_` and read as hidden tokens: a `(` there would open
                     // a string that swallows the rest of the layer.
                     let key = o.entries.last().and_then(|(k, _)| k.as_deref());
-                    if (o.ty == "Binary" && key == Some("ASCII85Decode")) || (o.ty == "ForeignObject" && key == Some("Data")) {
+                    if o.ty == "Binary" && key == Some("ASCII85Decode") {
                         self.lex.skip_past(b"~>");
+                    } else if o.ty == "ForeignObject" && key == Some("Data") {
+                        self.foreign_data = Some(self.lex.skip_past(b"~>").to_vec());
                     }
                 }
                 return Ok(());
@@ -839,9 +848,7 @@ impl<'a> Reader<'a> {
             "Document" => self.document(&o),
             "AI11Text" => self.text_slot(&o)?,
             "SymbolInstance" => self.unreadable("symbols"),
-            // Art Illustrator shows but doesn't edit (a placed PDF's content): its PDF is kept in
-            // the dictionary, which nothing draws from yet, so the file opens without it.
-            "ForeignObject" => self.warn(NON_NATIVE_ART),
+            "ForeignObject" => self.foreign_art(&o)?,
             _ => {}
         }
         Ok(())
@@ -860,6 +867,94 @@ impl<'a> Reader<'a> {
 
     /// Note something the data has that isn't read: an error on a layer that shows, left out of a
     /// layer that doesn't.
+    /// Non-native art (`/ForeignObject`): the content of a placed PDF, which Illustrator shows but
+    /// doesn't edit. Its `/Data` is that PDF (ASCII85 in comment lines, kept by the `,` handler),
+    /// `/Bounds` the art's box in a space measured from `/Origin` with y down, and `/RTransform`
+    /// maps that space onto the art (both read off files: the box lands on the clipping paths
+    /// around it). The PDF importer reads the PDF, and its art is fitted to the box, as one
+    /// group. Without data, or with a PDF that can't be read, nothing is drawn.
+    fn foreign_art(&mut self, o: &Obj) -> Result<(), String> {
+        let Some(text) = self.foreign_data.take() else { return Ok(()) };
+        // One `%` starts each line; the rest, `_` included, is data.
+        let mut encoded = String::with_capacity(text.len());
+        for line in text.split(|b| matches!(b, b'\r' | b'\n')) {
+            encoded.push_str(&String::from_utf8_lossy(line.strip_prefix(b"%").unwrap_or(line)));
+        }
+        let Some(pdf) = ps::ascii85_decode(&encoded) else {
+            self.warn(NON_NATIVE_ART_UNREAD);
+            return Ok(());
+        };
+        let (rt, origin, bounds) = (o.get("RTransform").map(Self::nums), o.get("Origin").map(Self::nums), o.get("Bounds").map(Self::nums));
+        let (Some([a, b, c, d, tx, ty]), Some([ox, oy]), Some([x0, y0, x1, y1])) = (rt.as_deref(), origin.as_deref(), bounds.as_deref()) else {
+            self.warn(NON_NATIVE_ART_UNREAD);
+            return Ok(());
+        };
+        let place = Affine::new([*a, *b, *c, *d, *tx, *ty]) * Affine::new([1.0, 0.0, 0.0, -1.0, *ox, *oy]);
+        let art_box = place.transform_rect_bbox(Rect::new(*x0, *y0, *x1, *y1));
+        if !sane(art_box) || art_box.width() <= 0.0 || art_box.height() <= 0.0 {
+            self.warn(NON_NATIVE_ART_UNREAD);
+            return Ok(());
+        }
+        let target = self.ensure_space().transform_rect_bbox(art_box);
+        let opts = vectorcraft_pdf::ImportOptions { layers: false, ..Default::default() };
+        let read = match vectorcraft_pdf::import_with_report(&pdf, &opts) {
+            Ok(r) => r,
+            Err(e) => {
+                self.warn(&format!("{NON_NATIVE_ART_UNREAD} ({e})"));
+                return Ok(());
+            }
+        };
+        let mut art: Vec<Node> = read.document.layers.iter().filter_map(|l| l.children()).flatten().map(|n| n.as_ref().clone()).collect();
+        let Some(bbox) = art.iter().fold(None, |acc, n| vectorcraft_geom::union_opt(acc, n.geometric_bounds())) else { return Ok(()) };
+        if bbox.width() <= 0.0 || bbox.height() <= 0.0 {
+            return Ok(());
+        }
+        // The art's own box onto the object's box (its measure of the same art), upside down:
+        // the object's space has y down, and the PDF's content is kept in its own y-up
+        // coordinates, so the PDF's bottom is the top of the box.
+        let fit = Affine::translate((target.x0, target.y1))
+            * Affine::scale_non_uniform(target.width() / bbox.width(), -target.height() / bbox.height())
+            * Affine::translate((-bbox.x0, -bbox.y0));
+        for n in &mut art {
+            n.transform(fit, true);
+            // The PDF's swatches stay with it: its colours keep their values, unlinked.
+            self.reid(n);
+        }
+        for (key, blob) in read.document.images {
+            self.doc.images.entry(key).or_insert(blob);
+        }
+        let gs = self.gs.clone();
+        let mut group = self.new_node(NodeKind::Group { children: art.into_iter().map(Arc::new).collect(), clip: false }, &gs);
+        group.name = Some(NON_NATIVE_ART.into());
+        self.add(group, self.obj_hidden)
+    }
+
+    /// Give `n` and its descendants ids of this document, and unlink their swatches.
+    fn reid(&mut self, n: &mut Node) {
+        n.id = self.doc.alloc_id();
+        for it in &mut n.appearance.items {
+            let p = match it {
+                AppearanceItem::Fill(f) => &mut f.paint,
+                AppearanceItem::Stroke(s) => &mut s.paint,
+            };
+            if let Paint::Gradient(g) = p {
+                g.swatch = None;
+            }
+            p.map_links(&mut |_, link, _| {
+                *link = None;
+                true
+            });
+        }
+        if let Some(m) = &mut n.mask {
+            self.reid(Arc::make_mut(&mut m.art));
+        }
+        if let Some(children) = n.children_mut() {
+            for c in children {
+                self.reid(Arc::make_mut(c));
+            }
+        }
+    }
+
     fn unreadable(&mut self, what: &str) {
         if self.shown() {
             self.unsupported.insert(what.into());

crates/eps/src/import/ai/tests.rs

@@ -5,7 +5,7 @@ use std::sync::Arc;
 use kurbo::{Point, Rect};
 use vectorcraft_color::gradient::GradientKind;
 use vectorcraft_color::{BlendMode, Color, Paint};
-use vectorcraft_doc::{ColorMode, Document, Knockout, LayerColor, LineCap, LineJoin, Node, NodeKind};
+use vectorcraft_doc::{AppearanceItem, ColorMode, Document, Knockout, LayerColor, LineCap, LineJoin, Node, NodeKind};
 
 use super::*;
 
@@ -332,13 +332,102 @@ fn sections_it_skips() {
 /// Non-native art (a placed PDF's content) keeps its PDF as ASCII85 in comment lines after
 /// `/Data ,`. ASCII85 has `_`, so a line may start with `%_`, which otherwise reads as hidden
 /// tokens, and a `(` in it would swallow the rest of the layer (a letterhead saved by Illustrator
-/// 28 wouldn't open: "it ends inside a layer"). The art is left out with a note.
+/// 28 wouldn't open: "it ends inside a layer"). Data that isn't a PDF is left out with a note.
 #[test]
-fn non_native_art_is_skipped_with_its_ascii85_data() {
-    let foreign = "/ForeignObject :\n1 /Version ,\n2 0 0 2 10 20  /RTransform ,\n/Data ,\n%,u@!!/MSk8%41#ocdN=10d&\n%_O%*mGn(RduU.]4`[u[.f\n%_;(]<[%\n%1H@<P2`V<S,pbuU7L]\\~>\n;\n0 0 Xd\n6 () XW\n0 Ae\n";
+fn non_native_art_that_is_not_a_pdf_is_left_out_with_a_note() {
+    let foreign = "/ForeignObject :\n1 /Version ,\n2 0 0 2 10 20  /RTransform ,\n10 20  /Origin ,\n1 2 5 8  /Bounds ,\n/Data ,\n%,u@!!/MSk8%41#ocdN=10d&\n%_O%*mGn(RduU.]4`[u[.f\n%_;(]<[%\n%1H@<P2`V<S,pbuU7L]\\~>\n;\n0 0 Xd\n6 () XW\n0 Ae\n";
     let s = read_ok(&layer(&format!("{foreign}{}", rect_path(0.0, 0.0, 1.0, 1.0) + "f\n")));
     assert_eq!(art(&s.doc).len(), 1);
-    assert!(s.warnings.iter().any(|w| w == read::NON_NATIVE_ART), "{:?}", s.warnings);
+    assert!(s.warnings.iter().any(|w| w.starts_with(read::NON_NATIVE_ART_UNREAD)), "{:?}", s.warnings);
+}
+
+/// A PDF with one page of `size` points holding `content`.
+fn tiny_pdf(size: f64, content: &str) -> Vec<u8> {
+    let objects = [
+        "<< /Type /Catalog /Pages 2 0 R >>".to_string(),
+        "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(),
+        format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 {size} {size}] /Contents 4 0 R >>"),
+        format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len() + 1),
+    ];
+    let mut pdf = b"%PDF-1.4\n".to_vec();
+    let mut offsets = vec![];
+    for (i, o) in objects.iter().enumerate() {
+        offsets.push(pdf.len());
+        pdf.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes());
+    }
+    let xref = pdf.len();
+    pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).as_bytes());
+    for o in offsets {
+        pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
+    }
+    pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objects.len() + 1).as_bytes());
+    pdf
+}
+
+/// `bytes` as ASCII85 in comment lines of 60 characters, ending with `~>`.
+fn ascii85_comment_lines(bytes: &[u8]) -> String {
+    let mut text = String::new();
+    for chunk in bytes.chunks(4) {
+        let mut word = [0u8; 4];
+        word[..chunk.len()].copy_from_slice(chunk);
+        let mut v = u32::from_be_bytes(word);
+        let mut digits = [0u8; 5];
+        for d in digits.iter_mut().rev() {
+            *d = (v % 85) as u8 + b'!';
+            v /= 85;
+        }
+        text.extend(digits.iter().take(chunk.len() + 1).map(|d| char::from(*d)));
+    }
+    let mut lines = String::new();
+    for line in text.as_bytes().chunks(60) {
+        lines.push('%');
+        lines.push_str(std::str::from_utf8(line).unwrap());
+        lines.push('\n');
+    }
+    lines.push_str("%~>\n");
+    lines
+}
+
+/// The PDF of non-native art is drawn by the PDF importer, its art fitted to the object's box:
+/// `/Bounds` in a space measured from `/Origin` with y down, then `/RTransform`. Two squares at
+/// 1..5 × 2..8 of a 10-point page, blue below red: bounds (1, 2)–(5, 8) → from the origin
+/// (10, 20), y down, (11, 18)–(15, 12) → twice that plus (100, 50): x 122..130, y 74..86 in art
+/// space, which is y 14..26 in the document (crop marks 100 high). The space has y down, so the
+/// PDF's bottom (blue) is the top of the box.
+#[test]
+fn non_native_art_is_drawn_from_its_pdf_fitted_to_its_bounds() {
+    let pdf = tiny_pdf(10.0, "0 0 1 rg 1 2 4 3 re f 1 0 0 rg 1 5 4 3 re f");
+    let foreign = format!(
+        "q\n/ForeignObject :\n1 /Version ,\n2 0 0 2 100 50  /RTransform ,\n10 20  /Origin ,\n1 2 5 8  /Bounds ,\n/Data ,\n{};\n0 0 Xd\n6 () XW\n0 Ae\n{}h\nW\nn\nQ\n",
+        ascii85_comment_lines(&pdf),
+        rect_path(120.0, 70.0, 130.0, 90.0)
+    );
+    let s = read_ok(&layer(&foreign));
+    assert!(s.warnings.is_empty(), "{:?}", s.warnings);
+    let objects = art(&s.doc);
+    assert_eq!(objects.len(), 1, "{objects:?}");
+    // A clipping group: the clip path first, then the non-native art's group.
+    let NodeKind::Group { children, clip: true } = &objects[0].kind else { panic!("{:?}", objects[0].kind) };
+    assert_eq!(children.len(), 2);
+    let group = &children[1];
+    assert_eq!(group.name.as_deref(), Some(read::NON_NATIVE_ART));
+    let bounds = group.geometric_bounds().unwrap();
+    for (got, want) in [(bounds.x0, 122.0), (bounds.y0, 14.0), (bounds.x1, 130.0), (bounds.y1, 26.0)] {
+        assert!((got - want).abs() < 1e-6, "{bounds:?}");
+    }
+    let paths = group.children().unwrap();
+    assert_eq!(paths.len(), 2);
+    let fill =
+        |n: &Node| n.appearance.items.iter().find_map(|it| if let AppearanceItem::Fill(f) = it { Some(f.paint.clone()) } else { None }).unwrap();
+    let (blue, red) = (fill(&paths[0]), fill(&paths[1]));
+    assert!(matches!(&blue, Paint::Solid { color, swatch: None, .. } if color.to_rgba8(1.0) == [0, 0, 255, 255]), "{blue:?}");
+    assert!(matches!(&red, Paint::Solid { color, swatch: None, .. } if color.to_rgba8(1.0) == [255, 0, 0, 255]), "{red:?}");
+    let (b, r) = (paths[0].geometric_bounds().unwrap(), paths[1].geometric_bounds().unwrap());
+    assert!((b.y0 - 14.0).abs() < 1e-6 && (b.y1 - 20.0).abs() < 1e-6, "blue {b:?}");
+    assert!((r.y0 - 20.0).abs() < 1e-6 && (r.y1 - 26.0).abs() < 1e-6, "red {r:?}");
+    // Its ids are this document's.
+    assert_ne!(group.id, paths[0].id);
+    assert_ne!(paths[0].id, paths[1].id);
 }
 
 #[test]

docs/mcp.md

@@ -521,9 +521,10 @@ CMYK too.
 PostScript files (`.eps`, and `.ai` files saved in older formats or without PDF compatibility) open through the EPS
 reader (see EPS and PostScript import). An `.ai` saved without PDF compatibility (its PDF part is only a placeholder
 page) opens from its editing data alone: its type is made from the file's text document where it can be (point type,
-area type and type on a path), and what can't be is left out with a warning; so is non-native art (the content of a
-placed PDF, which Illustrator shows but doesn't edit and keeps as a PDF inside the editing data); without editing data
-it says it can't be opened.
+area type and type on a path), and what can't be is left out with a warning. Non-native art (the content of a placed
+PDF, which Illustrator shows but doesn't edit and keeps as a PDF inside the editing data) is drawn by the PDF importer,
+as a group named "Non-native art" fitted to the object's box; one whose PDF can't be read is left out with a warning.
+Without editing data it says it can't be opened.
 
 What a PDF holds comes in as editable art: soft masks become opacity masks (an alpha mask as a white copy of its art;
 the backdrop colour gives Clip, an inverting transfer function Invert), transparency groups keep isolation and knockout,

xtask/src/layers.rs

@@ -75,6 +75,8 @@ pub const INTRA_LAYER_ORDER: &[&[&str]] = &[
     &["text", "effects"],
     &["render", "eps"],
     &["metafile", "eps"],
+    // Non-native art in Illustrator editing data is a PDF, drawn by the PDF importer.
+    &["pdf", "eps"],
     &["plugins", "effects"],
 ];