Keyboard Shortcuts: shortcut set reading and matching stay bounded on hostile files

A set whose entries sit in long, deeply nested sections kept every section's names again for each entry (up to hundreds of megabytes from a 4 MiB file); the names kept over all entries now stop at 8 MiB. Name matching compares character pairs in linear time, so a 20,000-entry set of long names is matched in about 2 s instead of 17 s.

crates/engine/src/cmd/shortcutset.rs

@@ -26,6 +26,9 @@ const MAX_DEPTH: usize = 16;
 const MAX_ENTRIES: usize = 20_000;
 /// The longest name kept (bytes).
 const MAX_NAME: usize = 1024;
+/// The most bytes of names kept over all entries, sections included: each entry repeats the names
+/// of the blocks around it, so a small file could otherwise fill hundreds of megabytes.
+const MAX_KEPT: usize = 2 * MAX_BYTES;
 /// The most tokens listed as unread.
 const MAX_UNREAD: usize = 1000;
 /// The longest unread token shown (characters).
@@ -43,7 +46,7 @@ pub fn specs() -> Vec<CommandSpec> {
         "Read Shortcut Set File",
         [],
         None,
-        "{path? | data?: the file's text | dataBase64?, name?: the file's name (default: the path's)} read a shortcut set another app saved as text (.kys, starting \"/Menus {\") without changing anything: each entry is a block that sets /Key, with the names of the blocks around it as its section → {set: the file's name without its extension, entries: [{section: [names], name, key, modifiers, context, represent, line, target?, chord?}] (codes as the file writes them; null where the entry doesn't set one; target: what importMatches says the name stands for; chord: the shortcut its codes stand for, as Edit › Keyboard Shortcuts writes it, when importMatches decodes them, an entry without /Modifiers taking none), codes: {keys: [{value, count, key?}], modifiers: [{value, count, mods?}]} (key and mods from importMatches), unread: [{line, text}] (tokens that aren't read, at most 1000), truncated: true when a cap stopped the reading (4 MiB, 16 open blocks, 20,000 entries; names past 1,024 bytes are cut)}. A file in another form (such as XML) is an error",
+        "{path? | data?: the file's text | dataBase64?, name?: the file's name (default: the path's)} read a shortcut set another app saved as text (.kys, starting \"/Menus {\") without changing anything: each entry is a block that sets /Key, with the names of the blocks around it as its section → {set: the file's name without its extension, entries: [{section: [names], name, key, modifiers, context, represent, line, target?, chord?}] (codes as the file writes them; null where the entry doesn't set one; target: what importMatches says the name stands for; chord: the shortcut its codes stand for, as Edit › Keyboard Shortcuts writes it, when importMatches decodes them, an entry without /Modifiers taking none), codes: {keys: [{value, count, key?}], modifiers: [{value, count, mods?}]} (key and mods from importMatches), unread: [{line, text}] (tokens that aren't read, at most 1000), truncated: true when a cap stopped the reading (4 MiB, 16 open blocks, 20,000 entries, 8 MiB of names over the entries and their sections; names past 1,024 bytes are cut)}. A file in another form (such as XML) is an error",
         always,
         read
     )]
@@ -214,6 +217,8 @@ pub fn read_kys(bytes: &[u8]) -> std::result::Result<KysSet, String> {
     let mut pending: Option<(usize, Vec<u8>)> = None;
     // Blocks past [`MAX_DEPTH`] being skipped.
     let mut skipped = 0usize;
+    // The bytes of names the entries keep so far ([`MAX_KEPT`]).
+    let mut kept = 0usize;
     while let Some((line, token)) = tokens.next_token() {
         if skipped > 0 {
             match token {
@@ -266,7 +271,8 @@ pub fn read_kys(bytes: &[u8]) -> std::result::Result<KysSet, String> {
                 };
                 let [key, modifiers, context, represent] = block.fields;
                 if key.is_some() {
-                    if set.entries.len() >= MAX_ENTRIES {
+                    kept += block.name.len() + stack.iter().map(|b| b.name.len()).sum::<usize>();
+                    if set.entries.len() >= MAX_ENTRIES || kept > MAX_KEPT {
                         set.truncated = true;
                         break;
                     }

crates/engine/src/namematch.rs

@@ -90,25 +90,29 @@ fn pairs(s: &str) -> Vec<(char, char)> {
     c.windows(2).filter_map(|w| Some((*w.first()?, *w.get(1)?))).collect()
 }
 
-/// How many character pairs `a` and `b` share, as a Dice coefficient (0 to 1): for names written
-/// without spaces.
+/// How many character pairs `a` and `b` (each sorted) share, as a Dice coefficient (0 to 1): for
+/// names written without spaces. Linear in their lengths, so long names in a file stay quick.
 fn pair_similarity(a: &[(char, char)], b: &[(char, char)]) -> f64 {
     if a.is_empty() || b.is_empty() {
         return 0.0;
     }
-    let mut rest: Vec<&(char, char)> = b.iter().collect();
-    let mut shared = 0usize;
-    for p in a {
-        if let Some(i) = rest.iter().position(|q| *q == p) {
-            rest.swap_remove(i);
-            shared += 1;
+    let (mut i, mut j, mut shared) = (0usize, 0usize, 0usize);
+    while let (Some(p), Some(q)) = (a.get(i), b.get(j)) {
+        match p.cmp(q) {
+            std::cmp::Ordering::Less => i += 1,
+            std::cmp::Ordering::Greater => j += 1,
+            std::cmp::Ordering::Equal => {
+                shared += 1;
+                i += 1;
+                j += 1;
+            }
         }
     }
     (2 * shared) as f64 / (a.len() + b.len()) as f64
 }
 
 /// A name or label as the scores compare it: its letters and digits (lowercased), its words and the
-/// character pairs of the former.
+/// character pairs of the former (sorted).
 struct Text {
     folded: String,
     words: Vec<String>,
@@ -118,7 +122,9 @@ struct Text {
 impl Text {
     fn of(s: &str) -> Self {
         let folded = folded(s);
-        Self { pairs: pairs(&folded), words: words(s), folded }
+        let mut pairs = pairs(&folded);
+        pairs.sort_unstable();
+        Self { pairs, words: words(s), folded }
     }
 }
 
@@ -210,7 +216,8 @@ impl Matcher {
             .get(&text.folded)
             .into_iter()
             .chain(text.words.iter().filter_map(|w| self.by_word.get(w)))
-            .chain(text.pairs.iter().filter_map(|p| self.by_pair.get(p)));
+            // Each pair looked up once (they are sorted).
+            .chain(text.pairs.chunk_by(|a, b| a == b).filter_map(|c| self.by_pair.get(c.first()?)));
         let mut scored: Vec<(usize, f64)> = vec![];
         for &i in lists.flatten() {
             if let Some(s) = seen.get_mut(i)

crates/engine/src/tests_shortcutset.rs

@@ -113,6 +113,12 @@ fn caps_nesting_entries_and_names() {
     let big = format!("/Menus {{ /E {{ /Key 1 }} {} }}", " ".repeat(4 << 20));
     let set = read_kys(big.as_bytes()).unwrap();
     assert_eq!((set.entries.len(), set.truncated), (1, true));
+    // Long section names repeated for every entry stop at 8 MiB of names, not hundreds of megabytes.
+    let open: String = (0..15).map(|i| format!("/{}{i} {{ ", "s".repeat(1020))).collect();
+    let wide = format!("{open}{}", "/E{/Key 1}".repeat(19_000));
+    let set = read_kys(wide.as_bytes()).unwrap();
+    let kept: usize = set.entries.iter().map(|e| e.name.len() + e.section.iter().map(String::len).sum::<usize>()).sum();
+    assert!(set.truncated && kept <= 8 << 20 && set.entries.len() > 500, "{} entries, {kept} bytes", set.entries.len());
 }
 
 #[test]

docs/mcp.md

@@ -3447,8 +3447,8 @@ Envelope Options.
 that starts with `/Menus {`) without changing the document or the preferences. Each block that sets `/Key` is an
 entry, listed with its `name`, its `section` (the names of the blocks around it), its codes as the file writes them
 (`key`, `modifiers`, `context`, `represent`) and its `line`. `codes` counts the entries that use each key and modifier
-code, and `unread` lists the tokens the reader skipped, with their lines. The reader stops at 4 MiB, 16 open blocks and
-20,000 entries, cuts names at 1,024 bytes, and sets `truncated` when it reaches one of these limits. An XML shortcut
+code, and `unread` lists the tokens the reader skipped, with their lines. The reader stops at 4 MiB, 16 open blocks,
+20,000 entries and 8 MiB of names over the entries and their sections, cuts names at 1,024 bytes, and sets `truncated` when it reaches one of these limits. An XML shortcut
 file fails with "This shortcut file is in a form VectorCraft can't read."
 
 The import matches record what the names and codes in such files stand for in VectorCraft. They are kept with the