mirror of
https://github.com/storytold/designcraft.git
synced 2026-10-14 19:27:07 -04:00
M2.5: dictionary hyphenation, letter/glyph justification, keeps
- hyphen: bundled en-US dictionary from the public-domain Moby Hyphenator II list (Grady Ward, Gutenberg #3204), front-coded + deflated (433 KB), with case/apostrophe/punctuation/compound handling and inflection inheritance; our own Liang patterns (19.5k, 129 KB) trained by a patgen-style trainer (hyphen::patgen, examples/hyphgen.rs) for unknown words; heuristic last. Discretionary hyphens replace automatic points. - breaker: prioritised word/letter/glyph-scaling glue in Knuth-Plass and layout; uncapped badness; hyphen limit as a hard constraint (hyphen-count states); hyphenation zone for ragged text (both composers); ragged hyphenation measured correctly; optical margin alignment; balance ragged lines; Knuth's node pruning. - compose: Keep Options (keep with next, keep all lines, keep first/last) via paragraph snapshots; desired spacing applied to all alignments; justified last lines shrink instead of overflowing; word shaping and hyphenation caches (300k-char story: ~40 ms warm, ~60 ms cold). - tests: golden corpus, perf (ignored), keeps, balance, optical, zone, limit, letter/glyph scaling, discretionary hyphens. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
9fdd61a390
commit
83915e2fc2
17 files changed
+4032
-327
No files matched your search
@@ -8,6 +8,8 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra
|
||||
|
||||
| Asset | Author | Source | Licence | Notes |
|
||||
|---|---|---|---|---|
|
||||
| `assets/hyphenation/en-us.dic` | Grady Ward (Moby Hyphenator II word list); converted by DesignCraft contributors | https://www.gutenberg.org/ebooks/3204 (file `https://www.gutenberg.org/files/3204/files/mhyph.txt`, SHA-256 `eeb30474c86b8af3469035ec1a0913e35905325ca885290db9dfed9881e230ac`) | Public Domain ("Public Domain material by grant from the author, January, 2001"; Project Gutenberg: "Public domain in the USA") | ~165k words with break points, front-coded + deflate; regenerate with `cargo run --release -p designcraft-compose --example hyphgen -- mhyph.txt assets/hyphenation` |
|
||||
| `assets/hyphenation/en-us.pat` | DesignCraft contributors (generated) | trained from `en-us.dic`'s source list by `crates/compose/src/hyphen/patgen.rs` (`examples/hyphgen.rs`) | Public Domain (CC0-1.0) | Our own Liang hyphenation patterns (no TeX/hyph-utf8 patterns used) |
|
||||
| `assets/fonts/Inter-Medium.ttf` | Rasmus Andersson / The Inter Project Authors | https://github.com/rsms/inter | OFL-1.1 (`assets/fonts/OFL-Inter.txt`) | Open-source typeface |
|
||||
| `assets/fonts/Inter-Regular.ttf` | Rasmus Andersson / The Inter Project Authors | https://github.com/rsms/inter | OFL-1.1 (`assets/fonts/OFL-Inter.txt`) | Open-source typeface |
|
||||
| `assets/fonts/Inter-SemiBold.ttf` | Rasmus Andersson / The Inter Project Authors | https://github.com/rsms/inter | OFL-1.1 (`assets/fonts/OFL-Inter.txt`) | Open-source typeface |
|
||||
|
||||
Generated
+1
@@ -871,6 +871,7 @@ dependencies = [
|
||||
"designcraft-fonts",
|
||||
"designcraft-geom",
|
||||
"log",
|
||||
"miniz_oxide 0.8.9",
|
||||
"proptest",
|
||||
]
|
||||
|
||||
|
||||
@@ -7,6 +7,9 @@ This product bundles third-party material under the following licences. Per-file
|
||||
- Source Sans 3 and Source Serif 4 (assets/fonts/Source*) — SIL Open Font License 1.1. See assets/fonts/OFL-SourceSans3.txt, OFL-SourceSerif4.txt.
|
||||
- Inter (assets/fonts/Inter-*.ttf) — SIL Open Font License 1.1. See assets/fonts/OFL-Inter.txt.
|
||||
- JetBrains Mono (assets/fonts/JetBrainsMono-*.ttf) — SIL Open Font License 1.1. See assets/fonts/OFL-JetBrainsMono.txt.
|
||||
- English hyphenation word list (assets/hyphenation/en-us.dic) — derived from the Moby Hyphenator II
|
||||
list by Grady Ward, public domain (Project Gutenberg eBook #3204). The hyphenation patterns
|
||||
(assets/hyphenation/en-us.pat) are trained from it by DesignCraft and released as CC0-1.0.
|
||||
- All UI icons are drawn in code (crates/ui-egui/src/icons.rs) and are original work, MIT OR Apache-2.0.
|
||||
|
||||
Rust dependencies (vello_cpu, kurbo, skrifa, harfrust, image, egui, eframe, …) are permissively
|
||||
|
||||
+2
-2
@@ -6,7 +6,7 @@ DesignCraft aims at full Adobe InDesign parity — and to be better: faster, ope
|
||||
|
||||
**Working today**
|
||||
- Document model: facing/non-facing spreads, parent pages with page-number markers, sections, layers, frames (text / graphic / unassigned), threaded stories, paragraph/character/object styles with based-on, swatches (process/spot/tints/gradients), corner options, text wrap, drop shadow, opacity/blend.
|
||||
- Text engine: Knuth–Plass paragraph composer and single-line composer, hyphenation, justification, columns, threading across frames and spreads, tabs, bullets/numbering, rules, shading, baseline grid, first-baseline options, vertical justification, wrap exclusions, overset detection, caret/hit testing.
|
||||
- Text engine: Knuth–Plass paragraph composer and single-line composer, dictionary hyphenation (public-domain Moby list + our own trained Liang patterns), justification with word/letter/glyph-scaling ranges, keep options (keep with next, keep lines together, widows/orphans), balance ragged lines, optical margin alignment, hyphenation zone/limit, columns, threading across frames and spreads, tabs, bullets/numbering, rules, shading, baseline grid, first-baseline options, vertical justification, wrap exclusions, overset detection, caret/hit testing.
|
||||
- Rendering: vello_cpu (SIMD, multithreaded) — pages, items, images with mip levels, gradients, composed text.
|
||||
- UI (egui): application bar + menus, Control panel (object and text modes), Tools panel with flyouts, document tabs, rulers, pasteboard, guides (margins, columns, bleed, baseline grid), frame edges, selection handles, text ports and threads, Properties / Pages / Layers dock, Swatches, Styles, Character, Paragraph, Stroke, Text Wrap panels, New Document dialog, ⌘K palette, 4 brightness themes.
|
||||
- Pen tool, Direct Selection anchor/handle editing, rotate from corners, Rotate/Scale/Shear tools, Eyedropper, snapping & smart guides, Align/Distribute, Find/Change (text + GREP), Story Editor, Paragraph Style Options, Effects panel, hidden characters, native macOS menu bar, measured InDesign 2026 look (Medium Dark).
|
||||
@@ -17,7 +17,7 @@ DesignCraft aims at full Adobe InDesign parity — and to be better: faster, ope
|
||||
- MCP server (`designcraft-cli mcp [--connect PORT]`, docs/mcp.md): headless engine or the running app; commands, batch, document/story inspection, page renders as images, window screenshots, pointer/keyboard/dialog input.
|
||||
- Web build (`apps/designcraft-web`, trunk): the same UI on WebGPU with a WebGL2 fallback; open/place via the browser file picker or drag-and-drop, save/export as downloads.
|
||||
|
||||
**Next (in order):** IDML import/export · dictionary hyphenation · native format as zip · tables · PDF/X-4 output intent, bookmarks, tagged PDF · Links panel + relink · Preflight panel · TOC/index/footnotes · EPUB.
|
||||
**Next (in order):** IDML import/export · native format as zip · tables · PDF/X-4 output intent, bookmarks, tagged PDF · Links panel + relink · Preflight panel · TOC/index/footnotes · EPUB.
|
||||
|
||||
## Milestones
|
||||
|
||||
|
||||
Binary file not shown.
File diff suppressed because it is too large.
Load diff
@@ -14,6 +14,7 @@ designcraft-color = { workspace = true }
|
||||
designcraft-doc = { workspace = true }
|
||||
designcraft-fonts = { workspace = true }
|
||||
log = { workspace = true }
|
||||
miniz_oxide = "0.8"
|
||||
|
||||
[dev-dependencies]
|
||||
proptest = { workspace = true }
|
||||
@@ -0,0 +1,174 @@
|
||||
//! Regenerates the bundled English hyphenation data from the Moby Hyphenator II word list.
|
||||
//!
|
||||
//! ```text
|
||||
//! curl -LO https://www.gutenberg.org/files/3204/files/mhyph.txt # Project Gutenberg #3204, public domain
|
||||
//! cargo run --release -p designcraft-compose --example hyphgen -- mhyph.txt assets/hyphenation [--validate]
|
||||
//! ```
|
||||
//!
|
||||
//! Writes `en-us.dic` (the word list, front-coded and deflated; see `hyphen::dict`) and
|
||||
//! `en-us.pat` (Liang patterns trained from the list by `hyphen::patgen`). `--validate` first
|
||||
//! trains on 90% of the words and reports accuracy on the held-out 10%.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::path::PathBuf;
|
||||
|
||||
use designcraft_compose::hyphen::dict::{Dictionary, hyphenated};
|
||||
use designcraft_compose::hyphen::patgen::{self, DEFAULT_LEVELS};
|
||||
|
||||
/// Mac OS Roman, 0x80..=0xFF (the list's encoding; 0xA5 `•` marks breaks).
|
||||
const MAC_ROMAN_HIGH: &str =
|
||||
"ÄÅÇÉÑÖÜáàâäãåçéèêëíìîïñóòôöõúùûü†°¢£§•¶ß®©™´¨≠ÆØ∞±≤≥¥µ∂∑∏π∫ªºΩæø¿¡¬√ƒ≈∆«»…\u{A0}ÀÃÕŒœ–—“”‘’÷◊ÿŸ⁄€‹›fifl‡·‚„‰ÂÊÁËÈÍÎÏÌÓÔ\u{F8FF}ÒÚÛÙıˆ˜¯˘˙˚¸˝˛ˇ";
|
||||
|
||||
fn mac_roman(bytes: &[u8]) -> String {
|
||||
let high: Vec<char> = MAC_ROMAN_HIGH.chars().collect();
|
||||
assert_eq!(high.len(), 128);
|
||||
bytes.iter().map(|&b| if b < 0x80 { b as char } else { high[b as usize - 0x80] }).collect()
|
||||
}
|
||||
|
||||
const BREAK: char = '•';
|
||||
|
||||
fn is_word_char(c: char) -> bool {
|
||||
c.is_alphabetic() || c == '\'' || c == BREAK
|
||||
}
|
||||
|
||||
/// `(lowercase word, mask)` of one token like `Aa•ron's`, or None if it isn't a plain word.
|
||||
fn token(t: &str) -> Option<(String, u64)> {
|
||||
let t = t.trim_matches('\'');
|
||||
if t.is_empty() || !t.chars().all(is_word_char) || t.starts_with(BREAK) || t.ends_with(BREAK) {
|
||||
return None;
|
||||
}
|
||||
let mut w = String::new();
|
||||
let mut m = 0u64;
|
||||
let mut n = 0usize;
|
||||
for c in t.chars() {
|
||||
if c == BREAK {
|
||||
if n < 64 {
|
||||
m |= 1 << n;
|
||||
}
|
||||
} else {
|
||||
w.extend(c.to_lowercase());
|
||||
n += 1;
|
||||
}
|
||||
}
|
||||
((1..64).contains(&n) && w.chars().count() == n).then_some((w, m))
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let args: Vec<String> = std::env::args().skip(1).collect();
|
||||
let validate = args.iter().any(|a| a == "--validate");
|
||||
let pos: Vec<&String> = args.iter().filter(|a| !a.starts_with("--")).collect();
|
||||
let Some(src) = pos.first() else {
|
||||
eprintln!("usage: hyphgen <mhyph.txt> [out-dir] [--validate]");
|
||||
std::process::exit(2);
|
||||
};
|
||||
let out_dir = PathBuf::from(pos.get(1).map_or("assets/hyphenation", |s| s.as_str()));
|
||||
let text = mac_roman(&std::fs::read(src).expect("read word list"));
|
||||
|
||||
// key → (lowercase-origin masks, other masks); compound parts only fill gaps.
|
||||
let mut primary: HashMap<String, (Vec<u64>, Vec<u64>)> = HashMap::new();
|
||||
let mut parts: HashMap<String, u64> = HashMap::new();
|
||||
let mut entries = 0usize;
|
||||
for line in text.lines() {
|
||||
let line = line.trim_end_matches('\r').trim();
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
entries += 1;
|
||||
if let Some((w, m)) = token(line) {
|
||||
let lower = !line.chars().any(char::is_uppercase);
|
||||
let e = primary.entry(w).or_default();
|
||||
if lower { e.0.push(m) } else { e.1.push(m) }
|
||||
} else {
|
||||
for t in line.split([' ', '-', '/']) {
|
||||
if let Some((w, m)) = token(t) {
|
||||
parts.entry(w).or_insert(m);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut list: Vec<(String, u64)> = primary
|
||||
.into_iter()
|
||||
.map(|(w, (lower, other))| {
|
||||
// Prefer lowercase entries; where spellings disagree (re•cord / rec•ord) keep only the
|
||||
// breaks they share.
|
||||
let v = if lower.is_empty() { other } else { lower };
|
||||
(w, v.iter().fold(u64::MAX, |a, b| a & b))
|
||||
})
|
||||
.collect();
|
||||
let have: std::collections::HashSet<String> = list.iter().map(|x| x.0.clone()).collect();
|
||||
let extra: Vec<(String, u64)> = parts.into_iter().filter(|(w, _)| !have.contains(w)).collect();
|
||||
println!("{entries} entries → {} words (+{} from compounds)", list.len(), extra.len());
|
||||
list.extend(extra);
|
||||
let dict = Dictionary::from_entries(list);
|
||||
let bytes = dict.to_bytes();
|
||||
std::fs::create_dir_all(&out_dir).expect("mkdir");
|
||||
std::fs::write(out_dir.join("en-us.dic"), &bytes).expect("write en-us.dic");
|
||||
let back = Dictionary::from_bytes(&bytes).expect("roundtrip");
|
||||
assert_eq!(back.len(), dict.len());
|
||||
println!("en-us.dic: {} words, {} bytes", dict.len(), bytes.len());
|
||||
|
||||
// Training data: plain lowercase ASCII words (+ apostrophes).
|
||||
let train: Vec<(Vec<char>, Vec<usize>)> = dict
|
||||
.iter()
|
||||
.filter(|(w, _)| w.chars().all(|c| c.is_ascii_lowercase() || c == '\''))
|
||||
.map(|(w, m)| (w.chars().collect(), (1..w.len().min(64)).filter(|&i| m & (1 << i) != 0).collect()))
|
||||
.collect();
|
||||
println!("training on {} words", train.len());
|
||||
let mut log = |s: String| println!(" {s}");
|
||||
// Optional override for tuning: HYPHGEN_LEVELS="1,4,1,2,20;2,5,2,1,8;…" (min,max,good,bad,threshold).
|
||||
let custom: Option<Vec<patgen::Level>> = std::env::var("HYPHGEN_LEVELS").ok().map(|s| {
|
||||
s.split(';')
|
||||
.map(|l| {
|
||||
let v: Vec<u32> = l.split(',').map(|x| x.trim().parse().expect("number")).collect();
|
||||
patgen::Level { min_len: v[0] as usize, max_len: v[1] as usize, good_weight: v[2], bad_weight: v[3], threshold: v[4] }
|
||||
})
|
||||
.collect()
|
||||
});
|
||||
let levels = custom.as_deref().unwrap_or(DEFAULT_LEVELS);
|
||||
if validate {
|
||||
let (fit, hold): (Vec<_>, Vec<_>) = train.iter().cloned().enumerate().partition(|(i, _)| i % 10 != 0);
|
||||
let fit: Vec<_> = fit.into_iter().map(|x| x.1).collect();
|
||||
let hold: Vec<_> = hold.into_iter().map(|x| x.1).collect();
|
||||
let p = patgen::train(&fit, levels, &mut log);
|
||||
let s = patgen::score(&p, &hold);
|
||||
println!(
|
||||
"held-out: {} words, {} breaks: found {} ({:.1}%), missed {}, wrong {} ({:.2}% of reported)",
|
||||
s.words,
|
||||
s.breaks,
|
||||
s.found,
|
||||
100.0 * s.found as f64 / s.breaks.max(1) as f64,
|
||||
s.missed(),
|
||||
s.wrong,
|
||||
100.0 * s.wrong as f64 / (s.found + s.wrong).max(1) as f64
|
||||
);
|
||||
}
|
||||
if std::env::var_os("HYPHGEN_VALIDATE_ONLY").is_some() {
|
||||
return;
|
||||
}
|
||||
let pats = patgen::train(&train, levels, &mut log);
|
||||
let s = patgen::score(&pats, &train);
|
||||
println!(
|
||||
"training set: found {}/{} ({:.1}%), wrong {} — {} patterns",
|
||||
s.found,
|
||||
s.breaks,
|
||||
100.0 * s.found as f64 / s.breaks.max(1) as f64,
|
||||
s.wrong,
|
||||
pats.len()
|
||||
);
|
||||
let mut out = String::new();
|
||||
out.push_str("% DesignCraft US English hyphenation patterns (Liang/TeX notation).\n");
|
||||
out.push_str("% Generated by crates/compose/examples/hyphgen.rs: trained with DesignCraft's patgen-style trainer\n");
|
||||
out.push_str("% from the Moby Hyphenator II word list by Grady Ward (public domain, Project Gutenberg #3204).\n");
|
||||
out.push_str("% Released into the public domain (CC0-1.0) by the DesignCraft contributors.\n");
|
||||
for line in pats.to_tex().chunks(12) {
|
||||
out.push_str(&line.join(" "));
|
||||
out.push('\n');
|
||||
}
|
||||
std::fs::write(out_dir.join("en-us.pat"), &out).expect("write en-us.pat");
|
||||
println!("en-us.pat: {} patterns, {} bytes", pats.len(), out.len());
|
||||
for w in ["hyphenation", "typography", "composition", "photographer", "unquestionably", "localization", "blockchain", "smartphone"] {
|
||||
let c: Vec<char> = w.chars().collect();
|
||||
let m = pats.points(&c).unwrap_or_default().iter().fold(0u64, |m, &i| m | 1 << i);
|
||||
println!(" patterns: {}", hyphenated(w, m));
|
||||
}
|
||||
}
|
||||
+437
-133
@@ -1,5 +1,12 @@
|
||||
//! Line breaking: the paragraph composer (Knuth–Plass total fit) and the single-line composer
|
||||
//! (greedy first fit). Both work on a paragraph's glyphs and a per-line measure function.
|
||||
//! Line breaking: the paragraph composer (Knuth–Plass total fit), the single-line composer
|
||||
//! (greedy first fit) and balanced ragged lines. All work on a paragraph's glyphs and a per-line
|
||||
//! measure function.
|
||||
//!
|
||||
//! Justification uses *prioritised* elastic material (plan: typography §5.2): word spaces first,
|
||||
//! then letter spacing (every glyph gets elastic width from the paragraph's letter-spacing range),
|
||||
//! then glyph scaling. The adjustment ratio charges letter and glyph adjustments extra
|
||||
//! ([`TIER_COST`]), so the composer prefers word-space changes, then hyphenation, then letter
|
||||
//! spacing, then glyph scaling.
|
||||
|
||||
use crate::shape::{Glyph, SOFT_HYPHEN};
|
||||
|
||||
@@ -15,38 +22,143 @@ pub struct Break {
|
||||
pub forced: bool,
|
||||
}
|
||||
|
||||
/// Spacing parameters (from the paragraph's Justification settings), as fractions.
|
||||
/// Spacing parameters (from the paragraph's Justification and Hyphenation settings), as fractions.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct Spacing {
|
||||
pub justify: bool,
|
||||
pub word_min: f64,
|
||||
pub word_desired: f64,
|
||||
pub word_max: f64,
|
||||
/// Letter spacing, as fractions of the space width added between glyphs.
|
||||
pub letter_min: f64,
|
||||
pub letter_desired: f64,
|
||||
pub letter_max: f64,
|
||||
/// Horizontal glyph scaling (1 = 100%).
|
||||
pub glyph_min: f64,
|
||||
pub glyph_desired: f64,
|
||||
pub glyph_max: f64,
|
||||
/// Penalty for a hyphenated break (0 = free, larger = fewer hyphens).
|
||||
pub hyphen_penalty: f64,
|
||||
/// Maximum consecutive hyphenated lines (0 = unlimited).
|
||||
/// Maximum consecutive hyphenated lines (0 = unlimited) — a hard constraint.
|
||||
pub hyphen_limit: u32,
|
||||
/// Ragged-right stretchability (points) for unjustified paragraphs.
|
||||
pub ragged_stretch: f64,
|
||||
/// Hyphenation zone (points, unjustified text only): a word is not hyphenated when breaking
|
||||
/// before it would leave less than this much space at the end of the line.
|
||||
pub hyph_zone: f64,
|
||||
/// Optical margin alignment: punctuation hangs outside the measure (counted as extra width).
|
||||
pub optical: bool,
|
||||
}
|
||||
|
||||
impl Default for Spacing {
|
||||
fn default() -> Self {
|
||||
Spacing {
|
||||
justify: false,
|
||||
word_min: 0.8,
|
||||
word_desired: 1.0,
|
||||
word_max: 1.33,
|
||||
letter_min: 0.0,
|
||||
letter_desired: 0.0,
|
||||
letter_max: 0.0,
|
||||
glyph_min: 1.0,
|
||||
glyph_desired: 1.0,
|
||||
glyph_max: 1.0,
|
||||
hyphen_penalty: 275.0,
|
||||
hyphen_limit: 3,
|
||||
ragged_stretch: 24.0,
|
||||
hyph_zone: 0.0,
|
||||
optical: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Spacing {
|
||||
/// Per-tier (word, letter, glyph) stretch and shrink of glyph `g` as a box.
|
||||
fn box_elastic(&self, g: &Glyph) -> ([f64; 3], [f64; 3]) {
|
||||
if !self.justify {
|
||||
return ([0.0; 3], [0.0; 3]);
|
||||
}
|
||||
let natural = g.adv / self.glyph_desired.max(0.01);
|
||||
(
|
||||
[0.0, g.space * (self.letter_max - self.letter_desired).max(0.0), natural * (self.glyph_max - self.glyph_desired).max(0.0)],
|
||||
[0.0, g.space * (self.letter_desired - self.letter_min).max(0.0), natural * (self.glyph_desired - self.glyph_min).max(0.0)],
|
||||
)
|
||||
}
|
||||
/// Word-space stretch and shrink of space glyph `g` (only U+0020 is elastic).
|
||||
fn space_elastic(&self, g: &Glyph) -> (f64, f64) {
|
||||
if g.ch != ' ' {
|
||||
return (0.0, 0.0);
|
||||
}
|
||||
(g.space * (self.word_max - self.word_desired).max(0.0), g.space * (self.word_desired - self.word_min).max(0.0))
|
||||
}
|
||||
}
|
||||
|
||||
/// Relative cost of using each tier of elastic material (word, letter, glyph).
|
||||
pub const TIER_COST: [f64; 3] = [1.0, 0.75, 1.0];
|
||||
|
||||
/// Effective adjustment ratio for `d` points of stretch (`d > 0`) or shrink (`d < 0`), consuming
|
||||
/// the tiers in priority order. Returns `(r, feasible)`; shrinking beyond all tiers is infeasible.
|
||||
pub fn tiered_ratio(d: f64, y: [f64; 3], z: [f64; 3]) -> (f64, bool) {
|
||||
const EPS: f64 = 1e-9;
|
||||
if d.abs() < EPS {
|
||||
return (0.0, true);
|
||||
}
|
||||
let (mut rem, cap) = if d > 0.0 { (d, y) } else { (-d, z) };
|
||||
let mut r = 0.0;
|
||||
for t in 0..3 {
|
||||
if cap[t] > EPS && rem > EPS {
|
||||
let take = rem.min(cap[t]);
|
||||
r += TIER_COST[t] * take / cap[t];
|
||||
rem -= take;
|
||||
}
|
||||
}
|
||||
if d < 0.0 {
|
||||
return (-r, rem <= 1e-6);
|
||||
}
|
||||
if rem > 1e-6 {
|
||||
// Beyond every maximum: word spaces grow without bound (an H&J violation).
|
||||
let unit = if cap[0] > EPS { cap[0] } else { cap.iter().sum::<f64>() };
|
||||
r = if unit > EPS { r + rem / unit } else { INF };
|
||||
}
|
||||
(r, true)
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
enum Kind {
|
||||
Box,
|
||||
Glue,
|
||||
Penalty,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
enum Item {
|
||||
/// Unbreakable width (a glyph).
|
||||
Box {
|
||||
w: f64,
|
||||
},
|
||||
/// Space glyph `g`.
|
||||
Glue {
|
||||
w: f64,
|
||||
st: f64,
|
||||
sh: f64,
|
||||
},
|
||||
Penalty {
|
||||
w: f64,
|
||||
p: f64,
|
||||
flagged: bool,
|
||||
},
|
||||
struct Item {
|
||||
kind: Kind,
|
||||
w: f64,
|
||||
st: [f64; 3],
|
||||
sh: [f64; 3],
|
||||
/// Penalty value (penalties only).
|
||||
p: f64,
|
||||
flagged: bool,
|
||||
/// An automatic/discretionary hyphenation point (subject to the hyphenation zone).
|
||||
auto: bool,
|
||||
/// Breaks: optical hang of the glyph ending the line. Boxes: hang of the glyph starting one.
|
||||
hang: f64,
|
||||
/// Auto hyphenation points: index of the break opportunity before the word (or `NONE`).
|
||||
zone_from: usize,
|
||||
}
|
||||
|
||||
const NONE: usize = usize::MAX;
|
||||
|
||||
impl Item {
|
||||
fn boxed(w: f64, st: [f64; 3], sh: [f64; 3]) -> Item {
|
||||
Item { kind: Kind::Box, w, st, sh, p: 0.0, flagged: false, auto: false, hang: 0.0, zone_from: NONE }
|
||||
}
|
||||
fn glue(w: f64, st: f64, sh: f64) -> Item {
|
||||
Item { kind: Kind::Glue, w, st: [st, 0.0, 0.0], sh: [sh, 0.0, 0.0], p: 0.0, flagged: false, auto: false, hang: 0.0, zone_from: NONE }
|
||||
}
|
||||
fn penalty(w: f64, p: f64, flagged: bool) -> Item {
|
||||
Item { kind: Kind::Penalty, w, st: [0.0; 3], sh: [0.0; 3], p, flagged, auto: false, hang: 0.0, zone_from: NONE }
|
||||
}
|
||||
}
|
||||
|
||||
const INF: f64 = 10000.0;
|
||||
@@ -57,63 +169,152 @@ pub fn is_forced(c: char) -> bool {
|
||||
matches!(c, FORCED_LINE_BREAK | COLUMN_BREAK | FRAME_BREAK | PAGE_BREAK)
|
||||
}
|
||||
|
||||
/// Optical margin protrusion of `c` as fractions of its advance: (left edge, right edge).
|
||||
pub fn hang(c: char) -> (f64, f64) {
|
||||
match c {
|
||||
'.' | ',' => (0.0, 1.0),
|
||||
'-' | '\u{2010}' | '\u{2011}' | '\u{AD}' => (0.75, 0.75),
|
||||
'\'' | '"' | '‘' | '’' | '“' | '”' | '‚' | '„' | '‹' | '›' | '«' | '»' => (1.0, 1.0),
|
||||
'\u{2013}' | '\u{2014}' => (0.5, 0.5),
|
||||
':' | ';' => (0.0, 0.5),
|
||||
'!' | '?' => (0.0, 0.2),
|
||||
'*' => (0.3, 0.3),
|
||||
'(' | '[' | '{' => (0.1, 0.0),
|
||||
')' | ']' | '}' => (0.0, 0.1),
|
||||
'A' | 'V' | 'W' | 'Y' | 'v' | 'w' | 'y' => (0.05, 0.05),
|
||||
'T' => (0.05, 0.05),
|
||||
'O' | 'C' | 'G' | 'Q' | 'o' | 'c' => (0.03, 0.03),
|
||||
_ => (0.0, 0.0),
|
||||
}
|
||||
}
|
||||
|
||||
fn hang_left(g: &Glyph) -> f64 {
|
||||
hang(g.ch).0 * g.adv
|
||||
}
|
||||
|
||||
fn hang_right(g: &Glyph) -> f64 {
|
||||
hang(g.ch).1 * g.adv
|
||||
}
|
||||
|
||||
/// Advance of the hyphen glyph that would follow `g` (cached per face).
|
||||
fn hyphen_width(g: &Glyph, cache: &mut Vec<(u32, f64)>) -> f64 {
|
||||
let id = g.face.id();
|
||||
let units = match cache.iter().find(|c| c.0 == id) {
|
||||
Some(c) => c.1,
|
||||
None => {
|
||||
let gid = designcraft_fonts::first_glyph(&g.face, &['-', '\u{2010}']);
|
||||
let u = g.face.advance(gid);
|
||||
cache.push((id, u));
|
||||
u
|
||||
}
|
||||
};
|
||||
units * g.sx
|
||||
}
|
||||
|
||||
/// Push a break inside a word. Ragged text gets the line-end stretch around it
|
||||
/// (`penalty(∞) glue(0, s) penalty glue(0, −s)`), so hyphenated ragged lines are measured like
|
||||
/// lines broken at spaces.
|
||||
fn push_inword(it: &mut Vec<Item>, ig: &mut Vec<usize>, p: Item, g: usize, sp: &Spacing) {
|
||||
if sp.justify {
|
||||
it.push(p);
|
||||
ig.push(g);
|
||||
return;
|
||||
}
|
||||
it.push(Item::penalty(0.0, INF, false));
|
||||
it.push(Item::glue(0.0, sp.ragged_stretch, 0.0));
|
||||
it.push(p);
|
||||
it.push(Item::glue(0.0, -sp.ragged_stretch, 0.0));
|
||||
ig.extend([g; 4]);
|
||||
}
|
||||
|
||||
/// Build Knuth items. `item_glyph[k]` = glyph index of item k (penalties: the glyph *before* which the break happens).
|
||||
fn items(glyphs: &[Glyph], hyph_after: &[bool], sp: &Spacing) -> (Vec<Item>, Vec<usize>) {
|
||||
let mut it = Vec::with_capacity(glyphs.len() * 2 + 2);
|
||||
let mut it: Vec<Item> = Vec::with_capacity(glyphs.len() * 2 + 2);
|
||||
let mut ig = Vec::with_capacity(glyphs.len() * 2 + 2);
|
||||
let n = glyphs.len();
|
||||
let mut hy_cache = Vec::new();
|
||||
let mut last_space = NONE;
|
||||
for (i, g) in glyphs.iter().enumerate() {
|
||||
let prev_hang = if sp.optical && i > 0 { hang_right(&glyphs[i - 1]) } else { 0.0 };
|
||||
if is_forced(g.ch) {
|
||||
it.push(Item::Glue { w: 0.0, st: INF, sh: 0.0 });
|
||||
it.push(Item::glue(0.0, INF, 0.0));
|
||||
ig.push(i);
|
||||
it.push(Item::Penalty { w: 0.0, p: -INF, flagged: false });
|
||||
let mut p = Item::penalty(0.0, -INF, false);
|
||||
p.hang = prev_hang;
|
||||
it.push(p);
|
||||
ig.push(i + 1);
|
||||
last_space = NONE;
|
||||
continue;
|
||||
}
|
||||
if g.is_space() && !g.no_break {
|
||||
let w = g.adv;
|
||||
if sp.justify {
|
||||
let base = w / sp.word_desired.max(1e-6);
|
||||
it.push(Item::Glue { w, st: base * (sp.word_max - sp.word_desired).max(0.0), sh: base * (sp.word_desired - sp.word_min).max(0.0) });
|
||||
let (st, sh) = sp.space_elastic(g);
|
||||
let mut gl = Item::glue(w, st, sh);
|
||||
gl.hang = prev_hang;
|
||||
last_space = it.len();
|
||||
it.push(gl);
|
||||
ig.push(i);
|
||||
} else {
|
||||
// Ragged right (Knuth): glue(0, s, 0) penalty(0) glue(w, -s, 0).
|
||||
it.push(Item::Glue { w: 0.0, st: sp.ragged_stretch, sh: 0.0 });
|
||||
it.push(Item::glue(0.0, sp.ragged_stretch, 0.0));
|
||||
ig.push(i);
|
||||
it.push(Item::Penalty { w: 0.0, p: 0.0, flagged: false });
|
||||
let mut p = Item::penalty(0.0, 0.0, false);
|
||||
p.hang = prev_hang;
|
||||
last_space = it.len();
|
||||
it.push(p);
|
||||
ig.push(i);
|
||||
it.push(Item::Glue { w, st: -sp.ragged_stretch, sh: 0.0 });
|
||||
it.push(Item::glue(w, -sp.ragged_stretch, 0.0));
|
||||
ig.push(i);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if g.ch == SOFT_HYPHEN {
|
||||
let hw = g.size * 0.33;
|
||||
it.push(Item::Penalty { w: hw, p: sp.hyphen_penalty, flagged: true });
|
||||
ig.push(i + 1);
|
||||
let hw = glyphs[..i].iter().rev().find(|g| g.ch != SOFT_HYPHEN).map_or(g.size * 0.33, |p| hyphen_width(p, &mut hy_cache));
|
||||
let mut p = Item::penalty(hw, sp.hyphen_penalty, true);
|
||||
p.auto = true;
|
||||
p.zone_from = last_space;
|
||||
if sp.optical {
|
||||
p.hang = hw * hang('-').1;
|
||||
}
|
||||
push_inword(&mut it, &mut ig, p, i + 1, sp);
|
||||
continue;
|
||||
}
|
||||
it.push(Item::Box { w: g.adv });
|
||||
let (st, sh) = sp.box_elastic(g);
|
||||
let mut bx = Item::boxed(g.adv, st, sh);
|
||||
if sp.optical {
|
||||
bx.hang = hang_left(g);
|
||||
}
|
||||
it.push(bx);
|
||||
ig.push(i);
|
||||
let next_is_space = glyphs.get(i + 1).is_none_or(|n| n.is_space());
|
||||
if i + 1 < n && !next_is_space && !g.no_break {
|
||||
if matches!(g.ch, '-' | '\u{2010}') {
|
||||
it.push(Item::Penalty { w: 0.0, p: 50.0, flagged: true });
|
||||
ig.push(i + 1);
|
||||
let mut p = Item::penalty(0.0, 50.0, true);
|
||||
p.hang = if sp.optical { hang_right(g) } else { 0.0 };
|
||||
push_inword(&mut it, &mut ig, p, i + 1, sp);
|
||||
} else if matches!(g.ch, '\u{2013}' | '\u{2014}' | '/') {
|
||||
it.push(Item::Penalty { w: 0.0, p: 0.0, flagged: false });
|
||||
ig.push(i + 1);
|
||||
let mut p = Item::penalty(0.0, 0.0, false);
|
||||
p.hang = if sp.optical { hang_right(g) } else { 0.0 };
|
||||
push_inword(&mut it, &mut ig, p, i + 1, sp);
|
||||
} else if hyph_after[i] {
|
||||
let hw = g.size * 0.33;
|
||||
it.push(Item::Penalty { w: hw, p: sp.hyphen_penalty, flagged: true });
|
||||
ig.push(i + 1);
|
||||
let hw = hyphen_width(g, &mut hy_cache);
|
||||
let mut p = Item::penalty(hw, sp.hyphen_penalty, true);
|
||||
p.auto = true;
|
||||
p.zone_from = last_space;
|
||||
if sp.optical {
|
||||
p.hang = hw * hang('-').1;
|
||||
}
|
||||
push_inword(&mut it, &mut ig, p, i + 1, sp);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Paragraph end: fill glue + forced break.
|
||||
it.push(Item::Glue { w: 0.0, st: INF, sh: 0.0 });
|
||||
it.push(Item::glue(0.0, INF, 0.0));
|
||||
ig.push(n);
|
||||
it.push(Item::Penalty { w: 0.0, p: -INF, flagged: false });
|
||||
let mut p = Item::penalty(0.0, -INF, false);
|
||||
p.hang = if sp.optical && n > 0 { hang_right(&glyphs[n - 1]) } else { 0.0 };
|
||||
it.push(p);
|
||||
ig.push(n);
|
||||
(it, ig)
|
||||
}
|
||||
@@ -124,8 +325,8 @@ struct Node {
|
||||
line: usize,
|
||||
fitness: u8,
|
||||
tw: f64,
|
||||
ty: f64,
|
||||
tz: f64,
|
||||
ty: [f64; 3],
|
||||
tz: [f64; 3],
|
||||
demerits: f64,
|
||||
prev: Option<usize>,
|
||||
hyphens: u32,
|
||||
@@ -138,7 +339,7 @@ pub fn knuth_plass(glyphs: &[Glyph], hyph_after: &[bool], sp: &Spacing, width: &
|
||||
return vec![Break { start: 0, end: 0, next: 0, hyphen: false, forced: true }];
|
||||
}
|
||||
let (items, ig) = items(glyphs, hyph_after, sp);
|
||||
for tolerance in [3.0, 30.0, f64::INFINITY] {
|
||||
for tolerance in [2.5, 12.0, f64::INFINITY] {
|
||||
if let Some(b) = kp_pass(&items, &ig, glyphs, sp, width, tolerance, tolerance.is_infinite()) {
|
||||
return b;
|
||||
}
|
||||
@@ -146,6 +347,9 @@ pub fn knuth_plass(glyphs: &[Glyph], hyph_after: &[bool], sp: &Spacing, width: &
|
||||
greedy(glyphs, hyph_after, sp, width)
|
||||
}
|
||||
|
||||
/// Upper bound on the hyphen-count states tracked per breakpoint.
|
||||
const MAX_HYPHEN_STATES: usize = 8;
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn kp_pass(
|
||||
items: &[Item],
|
||||
@@ -159,118 +363,153 @@ fn kp_pass(
|
||||
let m = items.len();
|
||||
// Prefix sums (before item i).
|
||||
let mut sw = vec![0.0; m + 1];
|
||||
let mut sy = vec![0.0; m + 1];
|
||||
let mut sz = vec![0.0; m + 1];
|
||||
let mut sy = vec![[0.0; 3]; m + 1];
|
||||
let mut sz = vec![[0.0; 3]; m + 1];
|
||||
for (i, it) in items.iter().enumerate() {
|
||||
let (w, y, z) = match *it {
|
||||
Item::Box { w } => (w, 0.0, 0.0),
|
||||
Item::Glue { w, st, sh } => (w, st, sh),
|
||||
Item::Penalty { .. } => (0.0, 0.0, 0.0),
|
||||
let (w, y, z) = match it.kind {
|
||||
Kind::Box | Kind::Glue => (it.w, it.st, it.sh),
|
||||
Kind::Penalty => (0.0, [0.0; 3], [0.0; 3]),
|
||||
};
|
||||
sw[i + 1] = sw[i] + w;
|
||||
sy[i + 1] = sy[i] + y;
|
||||
sz[i + 1] = sz[i] + z;
|
||||
for t in 0..3 {
|
||||
sy[i + 1][t] = sy[i][t] + y[t];
|
||||
sz[i + 1][t] = sz[i][t] + z[t];
|
||||
}
|
||||
}
|
||||
let first_hang = items.iter().find(|i| i.kind == Kind::Box).map_or(0.0, |b| b.hang);
|
||||
let mut arena: Vec<Node> =
|
||||
vec![Node { pos: 0, line: 0, fitness: 1, tw: 0.0, ty: 0.0, tz: 0.0, demerits: 0.0, prev: None, hyphens: 0, flagged: false }];
|
||||
vec![Node { pos: 0, line: 0, fitness: 1, tw: first_hang, ty: [0.0; 3], tz: [0.0; 3], demerits: 0.0, prev: None, hyphens: 0, flagged: false }];
|
||||
let limit = sp.hyphen_limit as usize;
|
||||
let states = if limit == 0 { 2 } else { (limit + 1).min(MAX_HYPHEN_STATES) };
|
||||
let mut active: Vec<usize> = vec![0];
|
||||
let mut best: Vec<Option<(f64, usize)>> = vec![None; 4 * states];
|
||||
let mut keep = Vec::new();
|
||||
for b in 0..m {
|
||||
let (is_break, pw, pp, flagged) = match items[b] {
|
||||
Item::Penalty { w, p, flagged } if p < INF => (true, w, p, flagged),
|
||||
Item::Glue { .. } if b > 0 && matches!(items[b - 1], Item::Box { .. }) => (true, 0.0, 0.0, false),
|
||||
let it = items[b];
|
||||
let (is_break, pw, pp, flagged) = match it.kind {
|
||||
Kind::Penalty if it.p < INF => (true, it.w, it.p, it.flagged),
|
||||
Kind::Glue if b > 0 && items[b - 1].kind == Kind::Box => (true, 0.0, 0.0, false),
|
||||
_ => (false, 0.0, 0.0, false),
|
||||
};
|
||||
if !is_break {
|
||||
continue;
|
||||
}
|
||||
let forced = pp <= -INF;
|
||||
let mut best: [Option<(f64, usize, f64)>; 4] = [None; 4]; // (demerits, active node, ratio)
|
||||
let mut keep = Vec::with_capacity(active.len());
|
||||
let mut last_removed: Option<(usize, f64)> = None;
|
||||
best.iter_mut().for_each(|x| *x = None);
|
||||
keep.clear();
|
||||
let mut last_removed: Option<usize> = None;
|
||||
for &a in &active {
|
||||
let n = arena[a];
|
||||
let l = sw[b] - n.tw + pw;
|
||||
let l = sw[b] - n.tw + pw - it.hang;
|
||||
let target = width(n.line).max(1.0);
|
||||
let (y, z) = (sy[b] - n.ty, sz[b] - n.tz);
|
||||
let r = if l < target {
|
||||
if y > 1e-9 { (target - l) / y } else { INF }
|
||||
} else if l > target {
|
||||
if z > 1e-9 { (target - l) / z } else { -INF }
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
let deactivate = r < -1.0 || forced;
|
||||
if !deactivate {
|
||||
let mut y = [0.0; 3];
|
||||
let mut z = [0.0; 3];
|
||||
for t in 0..3 {
|
||||
y[t] = sy[b][t] - n.ty[t];
|
||||
z[t] = sz[b][t] - n.tz[t];
|
||||
}
|
||||
let (r, feasible) = tiered_ratio(target - l, y, z);
|
||||
if feasible && !forced {
|
||||
keep.push(a);
|
||||
} else {
|
||||
last_removed = Some((a, r));
|
||||
last_removed = Some(a);
|
||||
}
|
||||
if r >= -1.0 && (r <= tol || forced) {
|
||||
let bad = (100.0 * r.abs().powi(3)).min(INF);
|
||||
let lp = 10.0 + bad;
|
||||
let mut d = if pp >= 0.0 {
|
||||
lp * lp + pp * pp
|
||||
} else if pp > -INF {
|
||||
lp * lp - pp * pp
|
||||
} else {
|
||||
lp * lp
|
||||
};
|
||||
if flagged && n.flagged {
|
||||
d += 3000.0;
|
||||
if !feasible || (r > tol && !forced) {
|
||||
continue;
|
||||
}
|
||||
if flagged {
|
||||
// Hyphen limit: a hard constraint.
|
||||
if limit > 0 && n.hyphens as usize >= limit {
|
||||
continue;
|
||||
}
|
||||
if flagged && sp.hyphen_limit > 0 && n.hyphens >= sp.hyphen_limit {
|
||||
d += 1e7;
|
||||
}
|
||||
let fit = if r < -0.5 {
|
||||
0
|
||||
} else if r <= 0.5 {
|
||||
1
|
||||
} else if r <= 1.0 {
|
||||
2
|
||||
} else {
|
||||
3
|
||||
};
|
||||
if (fit as i32 - n.fitness as i32).abs() > 1 {
|
||||
d += 3000.0;
|
||||
}
|
||||
let total = n.demerits + d;
|
||||
if best[fit as usize].is_none_or(|(bd, _, _)| total < bd) {
|
||||
best[fit as usize] = Some((total, a, r));
|
||||
// Hyphenation zone (unjustified): break before the word instead when that leaves
|
||||
// less than the zone at the line end.
|
||||
if it.auto && !sp.justify && sp.hyph_zone > 0.0 && it.zone_from != NONE && it.zone_from > n.pos {
|
||||
let before = sw[it.zone_from] - n.tw - items[it.zone_from].hang;
|
||||
if target - before < sp.hyph_zone {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
let bad = 100.0 * r.abs().powi(3);
|
||||
let lp = 10.0 + bad;
|
||||
let mut d = if pp >= 0.0 {
|
||||
lp * lp + pp * pp
|
||||
} else if pp > -INF {
|
||||
lp * lp - pp * pp
|
||||
} else {
|
||||
lp * lp
|
||||
};
|
||||
if flagged && n.flagged {
|
||||
d += 3000.0;
|
||||
}
|
||||
let fit = if r < -0.5 {
|
||||
0
|
||||
} else if r <= 0.5 {
|
||||
1
|
||||
} else if r <= 1.0 {
|
||||
2
|
||||
} else {
|
||||
3
|
||||
};
|
||||
if (fit as i32 - n.fitness as i32).abs() > 1 {
|
||||
d += 3000.0;
|
||||
}
|
||||
let hs = if flagged { (n.hyphens as usize + 1).min(states - 1) } else { 0 };
|
||||
let total = n.demerits + d;
|
||||
let slot = &mut best[fit * states + hs];
|
||||
if slot.is_none_or(|(bd, _)| total < bd) {
|
||||
*slot = Some((total, a));
|
||||
}
|
||||
}
|
||||
// Emergency: nothing can reach this break and nothing remains active → accept an overfull line.
|
||||
if emergency
|
||||
&& keep.is_empty()
|
||||
&& best.iter().all(Option::is_none)
|
||||
&& let Some((a, _)) = last_removed
|
||||
&& let Some(a) = last_removed
|
||||
{
|
||||
best[0] = Some((arena[a].demerits + 1e8, a, -1.0));
|
||||
best[0] = Some((arena[a].demerits + 1e8, a));
|
||||
}
|
||||
std::mem::swap(&mut active, &mut keep);
|
||||
if best.iter().all(Option::is_none) {
|
||||
if active.is_empty() {
|
||||
return None;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
active = keep;
|
||||
// Totals after the break: skip glue and penalties up to the next box.
|
||||
let (mut tw, mut ty, mut tz) = (sw[b], sy[b], sz[b]);
|
||||
let mut i = b;
|
||||
while i < m {
|
||||
match items[i] {
|
||||
Item::Box { .. } => break,
|
||||
Item::Glue { w, st, sh } => {
|
||||
tw += w;
|
||||
ty += st;
|
||||
tz += sh;
|
||||
let x = items[i];
|
||||
match x.kind {
|
||||
Kind::Box => {
|
||||
tw += x.hang;
|
||||
break;
|
||||
}
|
||||
Item::Penalty { p, .. } if p <= -INF && i > b => break,
|
||||
Item::Penalty { .. } => {}
|
||||
Kind::Glue => {
|
||||
tw += x.w;
|
||||
for t in 0..3 {
|
||||
ty[t] += x.st[t];
|
||||
tz[t] += x.sh[t];
|
||||
}
|
||||
}
|
||||
Kind::Penalty if x.p <= -INF && i > b => break,
|
||||
Kind::Penalty => {}
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
for (fit, c) in best.iter().enumerate() {
|
||||
if let Some((d, a, _)) = *c {
|
||||
// Knuth's pruning: a class is only worth a node if it is within the fitness demerit of the best.
|
||||
let min_d = best.iter().flatten().map(|x| x.0).fold(f64::INFINITY, f64::min);
|
||||
for (k, c) in best.iter().enumerate() {
|
||||
if let Some((d, a)) = *c
|
||||
&& d <= min_d + 3000.0
|
||||
{
|
||||
let prev = arena[a];
|
||||
arena.push(Node {
|
||||
pos: b,
|
||||
line: prev.line + 1,
|
||||
fitness: fit as u8,
|
||||
fitness: (k / states) as u8,
|
||||
tw,
|
||||
ty,
|
||||
tz,
|
||||
@@ -282,12 +521,6 @@ fn kp_pass(
|
||||
active.push(arena.len() - 1);
|
||||
}
|
||||
}
|
||||
if active.is_empty() {
|
||||
if std::env::var_os("DC_KP_DEBUG").is_some() {
|
||||
eprintln!("kp tol {tol}: no active nodes at item {b}/{m} (glyph {})", ig[b]);
|
||||
}
|
||||
return None;
|
||||
}
|
||||
}
|
||||
// The final forced break: best node at position m-1.
|
||||
let end = active.iter().copied().filter(|&a| arena[a].pos == m - 1).min_by(|&a, &b| arena[a].demerits.total_cmp(&arena[b].demerits))?;
|
||||
@@ -308,15 +541,15 @@ fn breaks_from_positions(items: &[Item], ig: &[usize], glyphs: &[Glyph], chain:
|
||||
let mut out = Vec::with_capacity(chain.len());
|
||||
let mut start = 0;
|
||||
for &b in chain {
|
||||
let (end, hyphen, forced) = match items[b] {
|
||||
Item::Penalty { p, flagged, w } => {
|
||||
let it = items[b];
|
||||
let (end, hyphen, forced) = match it.kind {
|
||||
Kind::Penalty => {
|
||||
let e = ig[b].min(n);
|
||||
// A flagged penalty with width = an inserted hyphen (not an explicit '-').
|
||||
let explicit = e > 0 && matches!(glyphs[e - 1].ch, '-' | '\u{2010}');
|
||||
(e, flagged && w > 0.0 && !explicit || (flagged && e > 0 && glyphs[e - 1].ch == SOFT_HYPHEN), p <= -INF)
|
||||
(e, it.flagged && it.w > 0.0 && !explicit || (it.flagged && e > 0 && glyphs[e - 1].ch == SOFT_HYPHEN), it.p <= -INF)
|
||||
}
|
||||
Item::Glue { .. } => (ig[b], false, false),
|
||||
Item::Box { .. } => (ig[b], false, false),
|
||||
Kind::Glue | Kind::Box => (ig[b], false, false),
|
||||
};
|
||||
// Trim trailing spaces from the visible range; skip leading spaces of the next line.
|
||||
let mut vis_end = end;
|
||||
@@ -345,14 +578,18 @@ pub fn greedy(glyphs: &[Glyph], hyph_after: &[bool], sp: &Spacing, width: &dyn F
|
||||
let mut out = Vec::new();
|
||||
let mut start = 0;
|
||||
let mut line = 0;
|
||||
let mut hyphens = 0u32;
|
||||
let mut hy_cache = Vec::new();
|
||||
if n == 0 {
|
||||
return vec![Break { start: 0, end: 0, next: 0, hyphen: false, forced: true }];
|
||||
}
|
||||
let may_hyphenate = |hyphens: u32| sp.hyphen_limit == 0 || hyphens < sp.hyphen_limit;
|
||||
while start < n {
|
||||
let w = width(line).max(1.0);
|
||||
let mut x = 0.0;
|
||||
let mut spaces_shrink = 0.0;
|
||||
let mut x = if sp.optical { -hang_left(&glyphs[start]) } else { 0.0 };
|
||||
let mut shrink = 0.0;
|
||||
let mut last_ok: Option<(usize, bool)> = None; // (break position = end glyph, hyphen)
|
||||
let mut last_space: Option<(usize, f64)> = None; // (break position, line width there)
|
||||
let mut i = start;
|
||||
let mut brk: Option<(usize, bool, bool)> = None;
|
||||
while i < n {
|
||||
@@ -363,33 +600,45 @@ pub fn greedy(glyphs: &[Glyph], hyph_after: &[bool], sp: &Spacing, width: &dyn F
|
||||
}
|
||||
if g.is_space() && !g.no_break {
|
||||
if sp.justify {
|
||||
spaces_shrink += g.adv * (sp.word_desired - sp.word_min).max(0.0) / sp.word_desired.max(1e-6);
|
||||
shrink += sp.space_elastic(g).1;
|
||||
}
|
||||
last_ok = Some((i, false));
|
||||
last_space = Some((i, x));
|
||||
x += g.adv;
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
let hy = if hyph_after[i] { g.size * 0.33 } else { 0.0 };
|
||||
if x + g.adv > w + spaces_shrink && i > start {
|
||||
let hang_r = if sp.optical { hang_right(g) } else { 0.0 };
|
||||
if x + g.adv - hang_r > w + shrink && i > start {
|
||||
// Overflow: break at the last opportunity, else before this glyph.
|
||||
brk = Some(match last_ok {
|
||||
// Hyphenation zone: break before the word if that leaves little space.
|
||||
Some((_, true)) if !sp.justify && sp.hyph_zone > 0.0 && last_space.is_some_and(|(_, xs)| w - xs < sp.hyph_zone) => {
|
||||
(last_space.map_or(i, |s| s.0), false, false)
|
||||
}
|
||||
Some((p, h)) => (p, h, false),
|
||||
None => (i, false, false),
|
||||
});
|
||||
break;
|
||||
}
|
||||
x += g.adv;
|
||||
if sp.justify {
|
||||
shrink += sp.box_elastic(g).1.iter().sum::<f64>();
|
||||
}
|
||||
if i + 1 < n && !glyphs[i + 1].is_space() && !g.no_break {
|
||||
if matches!(g.ch, '-' | '\u{2010}' | '\u{2013}' | '\u{2014}' | '/') {
|
||||
last_ok = Some((i + 1, false));
|
||||
} else if hyph_after[i] && x + hy <= w + spaces_shrink {
|
||||
last_ok = Some((i + 1, true));
|
||||
} else if hyph_after[i] && may_hyphenate(hyphens) {
|
||||
let hy = hyphen_width(g, &mut hy_cache) * if sp.optical { 1.0 - hang('-').1 } else { 1.0 };
|
||||
if x + hy <= w + shrink {
|
||||
last_ok = Some((i + 1, true));
|
||||
}
|
||||
}
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
let (end, hyphen, forced) = brk.unwrap_or((n, false, true));
|
||||
hyphens = if hyphen { hyphens + 1 } else { 0 };
|
||||
let mut vis_end = end;
|
||||
while vis_end > start && glyphs[vis_end - 1].is_space() {
|
||||
vis_end -= 1;
|
||||
@@ -414,3 +663,58 @@ pub fn greedy(glyphs: &[Glyph], hyph_after: &[bool], sp: &Spacing, width: &dyn F
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Natural width of a broken line (with its inserted hyphen).
|
||||
pub fn line_width(glyphs: &[Glyph], b: &Break) -> f64 {
|
||||
let w: f64 = glyphs[b.start..b.end.max(b.start)].iter().map(|g| g.adv).sum();
|
||||
w + if b.hyphen && b.end > 0 { glyphs[b.end - 1].size * 0.33 } else { 0.0 }
|
||||
}
|
||||
|
||||
/// Balance Ragged Lines: keep the composer's line count but make the lines as even as possible,
|
||||
/// by composing against the narrowest measure that still yields that many lines.
|
||||
pub fn balanced(glyphs: &[Glyph], hyph_after: &[bool], sp: &Spacing, width: &dyn Fn(usize) -> f64) -> Vec<Break> {
|
||||
let base = knuth_plass(glyphs, hyph_after, sp, width);
|
||||
let n = base.len();
|
||||
if n < 2 || glyphs.len() > 3000 || base[..n - 1].iter().any(|b| b.forced) {
|
||||
return base;
|
||||
}
|
||||
let ok = |f: f64| -> Option<Vec<Break>> {
|
||||
let w = |j: usize| width(j) * f;
|
||||
let br = knuth_plass(glyphs, hyph_after, sp, &w);
|
||||
(br.len() == n && br.iter().enumerate().all(|(j, b)| line_width(glyphs, b) <= w(j) + 0.01)).then_some(br)
|
||||
};
|
||||
let (mut lo, mut hi) = (0.3, 1.0);
|
||||
let mut best = base;
|
||||
for _ in 0..14 {
|
||||
let mid = (lo + hi) / 2.0;
|
||||
match ok(mid) {
|
||||
Some(b) => {
|
||||
best = b;
|
||||
hi = mid;
|
||||
}
|
||||
None => lo = mid,
|
||||
}
|
||||
}
|
||||
best
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn tiers_are_used_in_order() {
|
||||
// Word stretch alone.
|
||||
let (r, ok) = tiered_ratio(5.0, [10.0, 10.0, 10.0], [0.0; 3]);
|
||||
assert!(ok && (r - 0.5).abs() < 1e-9);
|
||||
// Word saturated, half the letter tier.
|
||||
let (r, _) = tiered_ratio(15.0, [10.0, 10.0, 10.0], [0.0; 3]);
|
||||
assert!((r - (1.0 + 0.5 * TIER_COST[1])).abs() < 1e-9);
|
||||
// Beyond every tier: unbounded word spacing.
|
||||
let (r, _) = tiered_ratio(40.0, [10.0, 10.0, 10.0], [0.0; 3]);
|
||||
assert!(r > 2.0);
|
||||
// Shrink beyond the tiers is infeasible.
|
||||
assert!(!tiered_ratio(-25.0, [0.0; 3], [10.0, 10.0, 0.0]).1);
|
||||
assert!(tiered_ratio(-15.0, [0.0; 3], [10.0, 10.0, 0.0]).1);
|
||||
}
|
||||
}
|
||||
@@ -1,135 +0,0 @@
|
||||
//! Dictionary-free English hyphenation (ours): breaks between syllables at vowel–consonant
|
||||
//! boundaries (V-CV "ty-po", VC-CV "hap-pen"), respecting onset clusters ("gra-phy").
|
||||
//!
|
||||
//! A Liang pattern engine with loadable pattern sets is planned (see plan/adr); patterns are only
|
||||
//! bundled once their licence is confirmed to fit the asset policy.
|
||||
|
||||
/// Hyphenation limits from the paragraph's Hyphenation settings.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct Limits {
|
||||
pub min_word: usize,
|
||||
pub after_first: usize,
|
||||
pub before_last: usize,
|
||||
pub capitalized: bool,
|
||||
}
|
||||
|
||||
impl Default for Limits {
|
||||
fn default() -> Self {
|
||||
Limits { min_word: 5, after_first: 2, before_last: 2, capitalized: true }
|
||||
}
|
||||
}
|
||||
|
||||
fn is_vowel(c: char) -> bool {
|
||||
matches!(c.to_ascii_lowercase(), 'a' | 'e' | 'i' | 'o' | 'u' | 'y') || (c.is_alphabetic() && !c.is_ascii())
|
||||
}
|
||||
|
||||
fn onset_pair(a: char, b: char) -> bool {
|
||||
let (a, b) = (a.to_ascii_lowercase(), b.to_ascii_lowercase());
|
||||
matches!(
|
||||
(a, b),
|
||||
('c' | 's' | 't' | 'p' | 'w' | 'g', 'h')
|
||||
| ('b' | 'c' | 'd' | 'f' | 'g' | 'k' | 'p' | 't', 'r')
|
||||
| ('b' | 'c' | 'f' | 'g' | 'k' | 'p' | 's', 'l')
|
||||
| ('q', 'u')
|
||||
| ('s', 't' | 'p' | 'c' | 'k')
|
||||
)
|
||||
}
|
||||
|
||||
/// Common English suffixes that form their own syllable (break before them).
|
||||
/// Common English prefixes (break after them).
|
||||
const PREFIXES: &[&str] = &["re", "de", "pre", "un", "dis", "con", "com", "inter", "over", "under", "trans", "sub", "super", "anti", "auto"];
|
||||
|
||||
const SUFFIXES: &[&str] = &["tion", "sion", "ment", "ness", "less", "ful", "able", "ible", "ing", "ly"];
|
||||
|
||||
/// Allowed hyphenation points in `word` as char indices (a hyphen goes before that char).
|
||||
pub fn hyphen_points(word: &str, lim: &Limits) -> Vec<usize> {
|
||||
let c: Vec<char> = word.chars().collect();
|
||||
let n = c.len();
|
||||
let min_before = lim.after_first.max(1);
|
||||
let min_after = lim.before_last.max(1);
|
||||
if n < lim.min_word.max(2) || n < min_before + min_after || !c.iter().all(|c| c.is_alphabetic() || matches!(c, '\'' | '’')) {
|
||||
return vec![];
|
||||
}
|
||||
if c.iter().all(|c| c.is_uppercase()) || (!lim.capitalized && c[0].is_uppercase()) {
|
||||
return vec![];
|
||||
}
|
||||
let v: Vec<bool> = c.iter().map(|&x| is_vowel(x)).collect();
|
||||
let lower: String = word.to_lowercase();
|
||||
let mut out = vec![];
|
||||
for i in min_before..=n - min_after {
|
||||
if !c[i].is_alphabetic() || !c[i - 1].is_alphabetic() {
|
||||
continue;
|
||||
}
|
||||
let suffix_at = SUFFIXES.iter().any(|s| {
|
||||
let sl = s.chars().count();
|
||||
n - i == sl && lower.ends_with(s) && v[..i].iter().any(|x| *x)
|
||||
});
|
||||
let prefix_at = PREFIXES.iter().any(|p| p.chars().count() == i && lower.starts_with(p) && n - i >= 4 && c[i..].iter().any(|x| is_vowel(*x)));
|
||||
let ok = suffix_at
|
||||
|| prefix_at
|
||||
|| if v[i - 1] && !v[i] {
|
||||
(i + 1 < n && v[i + 1]) || (i + 2 < n && onset_pair(c[i], c[i + 1]) && v[i + 2])
|
||||
} else if !v[i - 1] && !v[i] {
|
||||
i >= 2 && v[i - 2] && i + 1 < n && v[i + 1] && !onset_pair(c[i - 1], c[i]) && c[i - 1] != c[i]
|
||||
|| (c[i - 1] == c[i] && i >= 2 && v[i - 2])
|
||||
} else {
|
||||
false
|
||||
};
|
||||
// Both parts need a sounded vowel: a trailing silent `e`, `ed` or `es` doesn't count
|
||||
// ("strai-ned" is wrong, "re-strained" is right).
|
||||
let right: String = c[i..].iter().collect::<String>().to_lowercase();
|
||||
let right_core = right.strip_suffix("ed").or_else(|| right.strip_suffix("es")).or_else(|| right.strip_suffix('e')).unwrap_or(&right);
|
||||
let sounded = right_core.chars().any(is_vowel) && right_core.len() > 1;
|
||||
let ok = ok && sounded && v[..i].iter().any(|x| *x);
|
||||
if ok && !out.contains(&i) {
|
||||
out.push(i);
|
||||
}
|
||||
}
|
||||
out.sort();
|
||||
out
|
||||
}
|
||||
|
||||
/// `word` with `-` at every hyphenation point (tests and diagnostics).
|
||||
pub fn hyphenate_word(word: &str, lim: &Limits) -> String {
|
||||
let pts = hyphen_points(word, lim);
|
||||
let mut s = String::with_capacity(word.len() + pts.len());
|
||||
for (i, ch) in word.chars().enumerate() {
|
||||
if pts.contains(&i) {
|
||||
s.push('-');
|
||||
}
|
||||
s.push(ch);
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn h(w: &str) -> String {
|
||||
hyphenate_word(w, &Limits::default())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn common_words() {
|
||||
assert_eq!(h("happen"), "hap-pen");
|
||||
assert_eq!(h("typography"), "ty-po-gra-phy");
|
||||
assert!(h("composition").contains("com-"));
|
||||
assert_eq!(h("cat"), "cat");
|
||||
assert!(!h("restrained").contains("-ned"), "{}", h("restrained"));
|
||||
assert!(!h("changed").contains('-'));
|
||||
assert!(h("restrained").starts_with("re-"), "{}", h("restrained"));
|
||||
assert_eq!(h("NASA"), "NASA");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn limits_are_respected() {
|
||||
let lim = Limits { min_word: 5, after_first: 3, before_last: 3, capitalized: false };
|
||||
for w in ["typography", "hyphenation", "justification", "Paragraph", "beautiful"] {
|
||||
for p in hyphen_points(w, &lim) {
|
||||
assert!(p >= 3 && p <= w.chars().count() - 3, "{w} {p}");
|
||||
}
|
||||
}
|
||||
assert!(hyphen_points("Paragraph", &lim).is_empty());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
//! Hyphenation exception dictionary: lowercase words with their break points.
|
||||
//!
|
||||
//! On-disk format (`assets/hyphenation/en-us.dic`): raw-deflate of
|
||||
//! `"DCHYPH1\n"` followed by one line per word, sorted by the plain word's bytes. Each line is
|
||||
//! front-coded against the previous one: a byte with the shared prefix length (in bytes of the
|
||||
//! hyphenated form), then the rest of the hyphenated form (`-` at every break), then `\n`.
|
||||
|
||||
/// Sorted word list with a break bitmask per word (bit `i` = hyphen before char `i`).
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct Dictionary {
|
||||
words: String,
|
||||
offs: Vec<u32>,
|
||||
masks: Vec<u64>,
|
||||
}
|
||||
|
||||
const MAGIC: &[u8] = b"DCHYPH1\n";
|
||||
|
||||
impl Dictionary {
|
||||
/// Build from `(lowercase word, break mask)` pairs (sorted and de-duplicated here; the first
|
||||
/// entry of a duplicate wins).
|
||||
pub fn from_entries(mut entries: Vec<(String, u64)>) -> Dictionary {
|
||||
entries.sort_by(|a, b| a.0.as_bytes().cmp(b.0.as_bytes()));
|
||||
entries.dedup_by(|b, a| a.0 == b.0);
|
||||
let mut d = Dictionary { words: String::new(), offs: vec![0], masks: Vec::with_capacity(entries.len()) };
|
||||
for (w, m) in entries {
|
||||
d.words.push_str(&w);
|
||||
d.offs.push(d.words.len() as u32);
|
||||
d.masks.push(m);
|
||||
}
|
||||
d
|
||||
}
|
||||
|
||||
pub fn len(&self) -> usize {
|
||||
self.masks.len()
|
||||
}
|
||||
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.masks.is_empty()
|
||||
}
|
||||
|
||||
fn word(&self, i: usize) -> &str {
|
||||
&self.words[self.offs[i] as usize..self.offs[i + 1] as usize]
|
||||
}
|
||||
|
||||
/// Break mask of an exact (already lowercased) word.
|
||||
pub fn get(&self, word: &str) -> Option<u64> {
|
||||
let (mut lo, mut hi) = (0usize, self.len());
|
||||
while lo < hi {
|
||||
let mid = (lo + hi) / 2;
|
||||
match self.word(mid).as_bytes().cmp(word.as_bytes()) {
|
||||
std::cmp::Ordering::Less => lo = mid + 1,
|
||||
std::cmp::Ordering::Greater => hi = mid,
|
||||
std::cmp::Ordering::Equal => return Some(self.masks[mid]),
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Iterate `(word, mask)`.
|
||||
pub fn iter(&self) -> impl Iterator<Item = (&str, u64)> + '_ {
|
||||
(0..self.len()).map(|i| (self.word(i), self.masks[i]))
|
||||
}
|
||||
|
||||
/// Serialize (deflate-compressed).
|
||||
pub fn to_bytes(&self) -> Vec<u8> {
|
||||
let mut raw = MAGIC.to_vec();
|
||||
let mut prev: Vec<u8> = Vec::new();
|
||||
for (w, m) in self.iter() {
|
||||
let h = hyphenated(w, m).into_bytes();
|
||||
let p = prev.iter().zip(&h).take_while(|(a, b)| a == b).count().min(255);
|
||||
raw.push(p as u8);
|
||||
raw.extend_from_slice(&h[p..]);
|
||||
raw.push(b'\n');
|
||||
prev = h;
|
||||
}
|
||||
miniz_oxide::deflate::compress_to_vec(&raw, 10)
|
||||
}
|
||||
|
||||
/// Deserialize data written by [`Dictionary::to_bytes`].
|
||||
pub fn from_bytes(data: &[u8]) -> Result<Dictionary, String> {
|
||||
let raw = miniz_oxide::inflate::decompress_to_vec(data).map_err(|e| format!("hyphenation dictionary: inflate failed: {e:?}"))?;
|
||||
let body = raw.strip_prefix(MAGIC).ok_or("hyphenation dictionary: bad header")?;
|
||||
let mut d = Dictionary { words: String::with_capacity(body.len()), offs: vec![0], masks: Vec::new() };
|
||||
let mut cur: Vec<u8> = Vec::new();
|
||||
let mut i = 0;
|
||||
while i < body.len() {
|
||||
let p = body[i] as usize;
|
||||
let end = body[i + 1..].iter().position(|&b| b == b'\n').map(|e| i + 1 + e).ok_or("hyphenation dictionary: truncated")?;
|
||||
if p > cur.len() {
|
||||
return Err("hyphenation dictionary: bad prefix".into());
|
||||
}
|
||||
cur.truncate(p);
|
||||
cur.extend_from_slice(&body[i + 1..end]);
|
||||
i = end + 1;
|
||||
let s = std::str::from_utf8(&cur).map_err(|_| "hyphenation dictionary: bad utf-8")?;
|
||||
let (w, m) = parse_hyphenated(s);
|
||||
d.words.push_str(&w);
|
||||
d.offs.push(d.words.len() as u32);
|
||||
d.masks.push(m);
|
||||
}
|
||||
Ok(d)
|
||||
}
|
||||
}
|
||||
|
||||
/// `word` with `-` before every char whose bit is set in `mask`.
|
||||
pub fn hyphenated(word: &str, mask: u64) -> String {
|
||||
let mut s = String::with_capacity(word.len() + 8);
|
||||
for (i, c) in word.chars().enumerate() {
|
||||
if i > 0 && i < 64 && mask & (1 << i) != 0 {
|
||||
s.push('-');
|
||||
}
|
||||
s.push(c);
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
/// Inverse of [`hyphenated`].
|
||||
pub fn parse_hyphenated(s: &str) -> (String, u64) {
|
||||
let mut w = String::with_capacity(s.len());
|
||||
let mut m = 0u64;
|
||||
let mut n = 0;
|
||||
for c in s.chars() {
|
||||
if c == '-' {
|
||||
if n > 0 && n < 64 {
|
||||
m |= 1 << n;
|
||||
}
|
||||
} else {
|
||||
w.push(c);
|
||||
n += 1;
|
||||
}
|
||||
}
|
||||
(w, m)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn roundtrip() {
|
||||
let e: Vec<(String, u64)> = ["ty-pog-ra-phy", "cat", "hap-pen", "o'clock", "naïve-té", "naïve"].iter().map(|s| parse_hyphenated(s)).collect();
|
||||
let d = Dictionary::from_entries(e);
|
||||
let back = Dictionary::from_bytes(&d.to_bytes()).unwrap();
|
||||
assert_eq!(back.len(), 6);
|
||||
assert_eq!(back.get("typography").map(|m| hyphenated("typography", m)), Some("ty-pog-ra-phy".to_string()));
|
||||
assert_eq!(back.get("cat"), Some(0));
|
||||
assert_eq!(back.get("naïveté").map(|m| hyphenated("naïveté", m)), Some("naïve-té".to_string()));
|
||||
assert!(back.get("dog").is_none());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,327 @@
|
||||
//! English hyphenation.
|
||||
//!
|
||||
//! Lookup order for each word:
|
||||
//! 1. the **dictionary** ([`Dictionary::en_us`]) — the public-domain Moby Hyphenator II word list
|
||||
//! by Grady Ward (~160k words), stored compactly in `assets/hyphenation/en-us.dic`; simple
|
||||
//! inflections (`'s`, `-s`, `-es`, `-ed`, `-d`) inherit the breaks of their stem;
|
||||
//! 2. **Liang patterns** ([`Patterns::en_us`]) for words not in the dictionary — our own pattern
|
||||
//! set, trained from the same list by [`patgen`] (`examples/hyphgen.rs` regenerates both files);
|
||||
//! 3. a small syllable **heuristic** for words outside the pattern alphabet.
|
||||
//!
|
||||
//! Then the paragraph's limits ([`Limits`]) are applied.
|
||||
|
||||
pub mod dict;
|
||||
pub mod patgen;
|
||||
pub mod patterns;
|
||||
|
||||
use std::sync::OnceLock;
|
||||
|
||||
pub use dict::Dictionary;
|
||||
pub use patterns::Patterns;
|
||||
|
||||
static EN_US_DIC: &[u8] = include_bytes!("../../../../assets/hyphenation/en-us.dic");
|
||||
static EN_US_PAT: &str = include_str!("../../../../assets/hyphenation/en-us.pat");
|
||||
|
||||
impl Dictionary {
|
||||
/// The bundled US English dictionary (decoded on first use).
|
||||
pub fn en_us() -> &'static Dictionary {
|
||||
static D: OnceLock<Dictionary> = OnceLock::new();
|
||||
D.get_or_init(|| {
|
||||
if EN_US_DIC.is_empty() {
|
||||
return Dictionary::default();
|
||||
}
|
||||
Dictionary::from_bytes(EN_US_DIC).unwrap_or_else(|e| {
|
||||
log::error!("{e}");
|
||||
Dictionary::default()
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl Patterns {
|
||||
/// The bundled US English patterns (parsed on first use).
|
||||
pub fn en_us() -> &'static Patterns {
|
||||
static P: OnceLock<Patterns> = OnceLock::new();
|
||||
P.get_or_init(|| Patterns::parse(EN_US_PAT))
|
||||
}
|
||||
}
|
||||
|
||||
/// Hyphenation limits from the paragraph's Hyphenation settings.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct Limits {
|
||||
pub min_word: usize,
|
||||
pub after_first: usize,
|
||||
pub before_last: usize,
|
||||
pub capitalized: bool,
|
||||
}
|
||||
|
||||
impl Default for Limits {
|
||||
fn default() -> Self {
|
||||
Limits { min_word: 5, after_first: 2, before_last: 2, capitalized: true }
|
||||
}
|
||||
}
|
||||
|
||||
/// Where a word's break points came from.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum Source {
|
||||
Dictionary,
|
||||
Patterns,
|
||||
Heuristic,
|
||||
None,
|
||||
}
|
||||
|
||||
fn is_apostrophe(c: char) -> bool {
|
||||
matches!(c, '\'' | '’' | 'ʼ')
|
||||
}
|
||||
|
||||
fn is_word_char(c: char) -> bool {
|
||||
c.is_alphabetic() || is_apostrophe(c)
|
||||
}
|
||||
|
||||
/// Unrestricted break points of one lowercase core (letters and `'` only, no leading/trailing `'`).
|
||||
fn core_breaks(core: &[char]) -> (Vec<usize>, Source) {
|
||||
let n = core.len();
|
||||
if n < 2 {
|
||||
return (vec![], Source::None);
|
||||
}
|
||||
let dict = Dictionary::en_us();
|
||||
let s: String = core.iter().collect();
|
||||
let from_mask = |m: u64, len: usize| (1..len.min(64)).filter(|&i| m & (1 << i) != 0).collect::<Vec<_>>();
|
||||
if let Some(m) = dict.get(&s) {
|
||||
return (from_mask(m, n), Source::Dictionary);
|
||||
}
|
||||
// Inflections inherit the stem's breaks (the suffix adds no break of its own).
|
||||
for suf in ["'s", "s", "es", "ed", "d"] {
|
||||
if let Some(stem) = s.strip_suffix(suf)
|
||||
&& stem.chars().count() >= 2
|
||||
&& let Some(m) = dict.get(stem)
|
||||
{
|
||||
return (from_mask(m, stem.chars().count()), Source::Dictionary);
|
||||
}
|
||||
}
|
||||
let pats = Patterns::en_us();
|
||||
if !pats.is_empty()
|
||||
&& let Some(p) = pats.points(core)
|
||||
{
|
||||
return (p, Source::Patterns);
|
||||
}
|
||||
(heuristic_points(core), Source::Heuristic)
|
||||
}
|
||||
|
||||
/// Unrestricted break points of `word` (char indices; a hyphen goes before that char), handling
|
||||
/// case, apostrophes (`don't`, `’s`), surrounding punctuation and compounds (each letter run is
|
||||
/// hyphenated separately). No break is ever placed next to an apostrophe or non-letter.
|
||||
pub fn word_breaks(word: &str) -> (Vec<usize>, Source) {
|
||||
let chars: Vec<char> = word.chars().collect();
|
||||
let mut out = Vec::new();
|
||||
let mut src = Source::None;
|
||||
let mut i = 0;
|
||||
while i < chars.len() {
|
||||
if !is_word_char(chars[i]) {
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
let mut j = i;
|
||||
while j < chars.len() && is_word_char(chars[j]) {
|
||||
j += 1;
|
||||
}
|
||||
let (mut a, mut b) = (i, j);
|
||||
while a < b && is_apostrophe(chars[a]) {
|
||||
a += 1;
|
||||
}
|
||||
while b > a && is_apostrophe(chars[b - 1]) {
|
||||
b -= 1;
|
||||
}
|
||||
let core: Vec<char> = chars[a..b].iter().map(|&c| if is_apostrophe(c) { '\'' } else { c.to_lowercase().next().unwrap_or(c) }).collect();
|
||||
let (pts, s) = core_breaks(&core);
|
||||
if src == Source::None {
|
||||
src = s;
|
||||
}
|
||||
out.extend(pts.into_iter().filter(|&p| p > 0 && p < core.len() && core[p - 1] != '\'' && core[p] != '\'').map(|p| a + p));
|
||||
i = j;
|
||||
}
|
||||
(out, src)
|
||||
}
|
||||
|
||||
/// Allowed hyphenation points in `word` as char indices (a hyphen goes before that char), with
|
||||
/// the paragraph limits applied to each letter run.
|
||||
pub fn hyphen_points(word: &str, lim: &Limits) -> Vec<usize> {
|
||||
let chars: Vec<char> = word.chars().collect();
|
||||
let (pts, _) = word_breaks(word);
|
||||
if pts.is_empty() {
|
||||
return pts;
|
||||
}
|
||||
let min_before = lim.after_first.max(1);
|
||||
let min_after = lim.before_last.max(1);
|
||||
// Run bounds of each point's letter run (trimmed of apostrophes).
|
||||
let run_of = |p: usize| {
|
||||
let mut a = p;
|
||||
while a > 0 && is_word_char(chars[a - 1]) {
|
||||
a -= 1;
|
||||
}
|
||||
let mut b = p;
|
||||
while b < chars.len() && is_word_char(chars[b]) {
|
||||
b += 1;
|
||||
}
|
||||
while a < b && is_apostrophe(chars[a]) {
|
||||
a += 1;
|
||||
}
|
||||
while b > a && is_apostrophe(chars[b - 1]) {
|
||||
b -= 1;
|
||||
}
|
||||
(a, b)
|
||||
};
|
||||
pts.into_iter()
|
||||
.filter(|&p| {
|
||||
let (a, b) = run_of(p);
|
||||
let run = &chars[a..b];
|
||||
let letters = run.iter().filter(|c| c.is_alphabetic()).count();
|
||||
if letters < lim.min_word.max(2) {
|
||||
return false;
|
||||
}
|
||||
if letters > 1 && run.iter().filter(|c| c.is_alphabetic()).all(|c| c.is_uppercase()) {
|
||||
return false;
|
||||
}
|
||||
if !lim.capitalized && run[0].is_uppercase() {
|
||||
return false;
|
||||
}
|
||||
let before = chars[a..p].iter().filter(|c| c.is_alphabetic()).count();
|
||||
let after = chars[p..b].iter().filter(|c| c.is_alphabetic()).count();
|
||||
before >= min_before && after >= min_after
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// `word` with `-` at every hyphenation point (tests and diagnostics).
|
||||
pub fn hyphenate_word(word: &str, lim: &Limits) -> String {
|
||||
let pts = hyphen_points(word, lim);
|
||||
let mut s = String::with_capacity(word.len() + pts.len());
|
||||
for (i, ch) in word.chars().enumerate() {
|
||||
if pts.contains(&i) {
|
||||
s.push('-');
|
||||
}
|
||||
s.push(ch);
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
// ---------- heuristic fallback (words outside the pattern alphabet) ----------
|
||||
|
||||
fn is_vowel(c: char) -> bool {
|
||||
matches!(c.to_ascii_lowercase(), 'a' | 'e' | 'i' | 'o' | 'u' | 'y') || (c.is_alphabetic() && !c.is_ascii())
|
||||
}
|
||||
|
||||
fn onset_pair(a: char, b: char) -> bool {
|
||||
let (a, b) = (a.to_ascii_lowercase(), b.to_ascii_lowercase());
|
||||
matches!(
|
||||
(a, b),
|
||||
('c' | 's' | 't' | 'p' | 'w' | 'g', 'h')
|
||||
| ('b' | 'c' | 'd' | 'f' | 'g' | 'k' | 'p' | 't', 'r')
|
||||
| ('b' | 'c' | 'f' | 'g' | 'k' | 'p' | 's', 'l')
|
||||
| ('q', 'u')
|
||||
| ('s', 't' | 'p' | 'c' | 'k')
|
||||
)
|
||||
}
|
||||
|
||||
/// Syllable heuristic: V-CV ("ty-po"), VC-CV ("hap-pen"), keeping onset clusters ("gra-phy").
|
||||
fn heuristic_points(c: &[char]) -> Vec<usize> {
|
||||
let n = c.len();
|
||||
if n < 4 || !c.iter().all(|c| c.is_alphabetic()) {
|
||||
return vec![];
|
||||
}
|
||||
let v: Vec<bool> = c.iter().map(|&x| is_vowel(x)).collect();
|
||||
let mut out = vec![];
|
||||
for i in 1..n {
|
||||
let ok = if v[i - 1] && !v[i] {
|
||||
(i + 1 < n && v[i + 1]) || (i + 2 < n && onset_pair(c[i], c[i + 1]) && v[i + 2])
|
||||
} else if !v[i - 1] && !v[i] {
|
||||
i >= 2 && v[i - 2] && i + 1 < n && v[i + 1] && !onset_pair(c[i - 1], c[i]) && c[i - 1] != c[i] || (c[i - 1] == c[i] && i >= 2 && v[i - 2])
|
||||
} else {
|
||||
false
|
||||
};
|
||||
// Both parts need a sounded vowel: a trailing silent `e`, `ed` or `es` doesn't count.
|
||||
let right: String = c[i..].iter().collect();
|
||||
let core = right.strip_suffix("ed").or_else(|| right.strip_suffix("es")).or_else(|| right.strip_suffix('e')).unwrap_or(&right);
|
||||
if ok && core.chars().any(is_vowel) && core.len() > 1 && v[..i].iter().any(|x| *x) {
|
||||
out.push(i);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn h(w: &str) -> String {
|
||||
hyphenate_word(w, &Limits::default())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bundled_data_loads() {
|
||||
assert!(Dictionary::en_us().len() > 150_000, "{}", Dictionary::en_us().len());
|
||||
assert!(Patterns::en_us().len() > 1000, "{}", Patterns::en_us().len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dictionary_words() {
|
||||
assert_eq!(h("happen"), "hap-pen");
|
||||
assert_eq!(h("typography"), "ty-pog-ra-phy");
|
||||
assert_eq!(h("composition"), "com-po-si-tion");
|
||||
assert_eq!(h("hyphenation"), "hy-phen-a-tion");
|
||||
assert_eq!(h("international"), "in-ter-na-tion-al");
|
||||
assert_eq!(h("cat"), "cat");
|
||||
assert!(!h("changed").contains('-'));
|
||||
assert_eq!(h("NASA"), "NASA");
|
||||
assert_eq!(word_breaks("publication").1, Source::Dictionary);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn case_apostrophes_and_punctuation() {
|
||||
assert_eq!(h("Typography"), "Ty-pog-ra-phy");
|
||||
assert_eq!(h("TYPOGRAPHY"), "TYPOGRAPHY");
|
||||
assert_eq!(h("typography,"), "ty-pog-ra-phy,");
|
||||
assert_eq!(h("“typography”"), "“ty-pog-ra-phy”");
|
||||
assert_eq!(h("typography's"), "ty-pog-ra-phy's");
|
||||
assert_eq!(h("typography’s"), "ty-pog-ra-phy’s");
|
||||
assert_eq!(h("'typography'"), "'ty-pog-ra-phy'");
|
||||
// Compounds: each part separately, never next to the hyphen.
|
||||
assert_eq!(h("typography-composition"), "ty-pog-ra-phy-com-po-si-tion");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inflections_inherit_the_stem() {
|
||||
assert_eq!(h("ripples"), "rip-ples");
|
||||
assert_eq!(h("gradients"), "gra-di-ents");
|
||||
assert_eq!(h("restrained"), "re-strained");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn patterns_cover_unknown_words() {
|
||||
// Not in the word list: the trained patterns decide.
|
||||
let (pts, src) = word_breaks("hyperlocalization");
|
||||
assert_eq!(src, Source::Patterns);
|
||||
assert!(pts.len() >= 4, "{}", h("hyperlocalization"));
|
||||
let (_, src) = word_breaks("zzyzx");
|
||||
assert_ne!(src, Source::Dictionary);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn limits_are_respected() {
|
||||
let lim = Limits { min_word: 5, after_first: 3, before_last: 3, capitalized: false };
|
||||
for w in ["typography", "hyphenation", "justification", "Paragraph", "beautiful"] {
|
||||
for p in hyphen_points(w, &lim) {
|
||||
assert!(p >= 3 && p <= w.chars().count() - 3, "{w} {p}");
|
||||
}
|
||||
}
|
||||
assert!(hyphen_points("Paragraph", &lim).is_empty());
|
||||
let lim = Limits { min_word: 12, ..Limits::default() };
|
||||
assert!(hyphen_points("typography", &lim).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn heuristic_fallback() {
|
||||
let c: Vec<char> = "happen".chars().collect();
|
||||
assert_eq!(heuristic_points(&c), vec![3]);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
//! A small patgen-style trainer (ours): learns Liang hyphenation patterns from a hyphenated word
|
||||
//! list, level by level. Odd levels add hyphenating patterns, even levels add inhibiting ones.
|
||||
//!
|
||||
//! At level `L`, for every pattern length, each inter-letter position whose current value is
|
||||
//! below `L` is classified as *good* (applying value `L` would fix it) or *bad* (it would break a
|
||||
//! position that is currently right). A candidate (letters + value position) is selected when
|
||||
//! `good·good_weight − bad·bad_weight ≥ threshold`; selected patterns are applied to every word
|
||||
//! before the next length is considered. Used offline by `examples/hyphgen.rs`.
|
||||
|
||||
use super::patterns::{Alphabet, FxMap, MAX_PATTERN_LEN, Patterns, apply, pack};
|
||||
|
||||
/// Parameters of one level.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct Level {
|
||||
pub min_len: usize,
|
||||
pub max_len: usize,
|
||||
pub good_weight: u32,
|
||||
pub bad_weight: u32,
|
||||
pub threshold: u32,
|
||||
}
|
||||
|
||||
/// Defaults tuned on the Moby list (see `examples/hyphgen.rs`).
|
||||
pub const DEFAULT_LEVELS: &[Level] = &[
|
||||
Level { min_len: 1, max_len: 4, good_weight: 1, bad_weight: 2, threshold: 20 },
|
||||
Level { min_len: 2, max_len: 5, good_weight: 2, bad_weight: 1, threshold: 8 },
|
||||
Level { min_len: 3, max_len: 6, good_weight: 1, bad_weight: 4, threshold: 7 },
|
||||
Level { min_len: 3, max_len: 7, good_weight: 3, bad_weight: 2, threshold: 2 },
|
||||
Level { min_len: 4, max_len: 8, good_weight: 1, bad_weight: 20, threshold: 6 },
|
||||
Level { min_len: 4, max_len: 8, good_weight: 3, bad_weight: 1, threshold: 2 },
|
||||
];
|
||||
|
||||
struct Word {
|
||||
/// Padded codes: boundary, letters…, boundary.
|
||||
codes: Vec<u8>,
|
||||
/// Correct hyphen before padded char `p`.
|
||||
hyph: Vec<bool>,
|
||||
vals: Vec<u8>,
|
||||
}
|
||||
|
||||
/// Accuracy of a pattern set over a word list.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq)]
|
||||
pub struct Score {
|
||||
pub words: usize,
|
||||
pub breaks: usize,
|
||||
pub found: usize,
|
||||
pub wrong: usize,
|
||||
}
|
||||
|
||||
impl Score {
|
||||
pub fn missed(&self) -> usize {
|
||||
self.breaks - self.found
|
||||
}
|
||||
}
|
||||
|
||||
/// Train patterns from `(lowercase word, break char indices)` pairs. `log` receives progress lines.
|
||||
pub fn train(words: &[(Vec<char>, Vec<usize>)], levels: &[Level], log: &mut dyn FnMut(String)) -> Patterns {
|
||||
let mut alpha = Alphabet::default();
|
||||
let mut data: Vec<Word> = Vec::with_capacity(words.len());
|
||||
'w: for (w, br) in words {
|
||||
let mut codes = Vec::with_capacity(w.len() + 2);
|
||||
codes.push(1);
|
||||
for &c in w {
|
||||
let Some(k) = alpha.intern(c) else { continue 'w };
|
||||
codes.push(k);
|
||||
}
|
||||
codes.push(1);
|
||||
let mut hyph = vec![false; codes.len() + 1];
|
||||
for &b in br {
|
||||
if b > 0 && b < w.len() {
|
||||
hyph[b + 1] = true;
|
||||
}
|
||||
}
|
||||
let vals = vec![0; codes.len() + 1];
|
||||
data.push(Word { codes, hyph, vals });
|
||||
}
|
||||
let mut pats: FxMap<u64, Box<[u8]>> = FxMap::default();
|
||||
let mut max_len = 0usize;
|
||||
for (li, lv) in levels.iter().enumerate() {
|
||||
let level = (li + 1) as u8;
|
||||
let hyphenating = level % 2 == 1;
|
||||
for len in lv.min_len..=lv.max_len.min(MAX_PATTERN_LEN) {
|
||||
// (packed letters | dot) → (good, bad)
|
||||
let mut counts: FxMap<u64, (u32, u32)> = FxMap::default();
|
||||
for w in &data {
|
||||
let n = w.codes.len();
|
||||
// Positions strictly between the word's letters.
|
||||
for p in 2..n - 1 {
|
||||
let v = w.vals[p];
|
||||
if v >= level {
|
||||
continue;
|
||||
}
|
||||
let (good, bad) = if hyphenating {
|
||||
(w.hyph[p] && v % 2 == 0, !w.hyph[p] && v % 2 == 0)
|
||||
} else {
|
||||
(!w.hyph[p] && v % 2 == 1, w.hyph[p] && v % 2 == 1)
|
||||
};
|
||||
if !good && !bad {
|
||||
continue;
|
||||
}
|
||||
for dot in 0..=len {
|
||||
if p < dot || p - dot + len > n {
|
||||
continue;
|
||||
}
|
||||
let s = p - dot;
|
||||
let key = pack(&w.codes[s..s + len]) | ((dot as u64) << 54);
|
||||
let e = counts.entry(key).or_default();
|
||||
if good {
|
||||
e.0 += 1;
|
||||
} else {
|
||||
e.1 += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// Select.
|
||||
let mut new: FxMap<u64, Box<[u8]>> = FxMap::default();
|
||||
let mut selected = 0usize;
|
||||
for (key, (g, b)) in counts {
|
||||
if g > 0 && (g * lv.good_weight) as i64 - (b * lv.bad_weight) as i64 >= lv.threshold as i64 {
|
||||
let dot = ((key >> 54) & 15) as usize;
|
||||
let letters = key & !(15u64 << 54);
|
||||
let e = new.entry(letters).or_insert_with(|| vec![0u8; len + 1].into_boxed_slice());
|
||||
e[dot] = level;
|
||||
selected += 1;
|
||||
}
|
||||
}
|
||||
if selected == 0 {
|
||||
continue;
|
||||
}
|
||||
for w in &mut data {
|
||||
apply(&new, len, &w.codes, &mut w.vals);
|
||||
}
|
||||
for (k, v) in new {
|
||||
let e = pats.entry(k).or_insert_with(|| vec![0u8; len + 1].into_boxed_slice());
|
||||
for (a, b) in e.iter_mut().zip(v.iter()) {
|
||||
*a = (*a).max(*b);
|
||||
}
|
||||
}
|
||||
max_len = max_len.max(len);
|
||||
let sc = score_vals(&data);
|
||||
log(format!("level {level} len {len}: +{selected} patterns ({} total); found {}/{} wrong {}", pats.len(), sc.found, sc.breaks, sc.wrong));
|
||||
}
|
||||
}
|
||||
// Re-express through the public pattern type (TeX strings) so the alphabet matches at runtime.
|
||||
let tmp = Patterns::from_parts(alpha, pats, max_len);
|
||||
let mut out = Patterns::default();
|
||||
out.add_letters(words.iter().flat_map(|w| w.0.iter().copied()));
|
||||
for t in tmp.to_tex() {
|
||||
out.insert(&t);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn score_vals(data: &[Word]) -> Score {
|
||||
let mut s = Score { words: data.len(), ..Default::default() };
|
||||
for w in data {
|
||||
for p in 2..w.codes.len() - 1 {
|
||||
let on = w.vals[p] % 2 == 1;
|
||||
if w.hyph[p] {
|
||||
s.breaks += 1;
|
||||
if on {
|
||||
s.found += 1;
|
||||
}
|
||||
} else if on {
|
||||
s.wrong += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
/// Score `pats` against a word list.
|
||||
pub fn score(pats: &Patterns, words: &[(Vec<char>, Vec<usize>)]) -> Score {
|
||||
let mut s = Score::default();
|
||||
for (w, br) in words {
|
||||
let Some(pts) = pats.points(w) else { continue };
|
||||
s.words += 1;
|
||||
s.breaks += br.len();
|
||||
for p in &pts {
|
||||
if br.contains(p) {
|
||||
s.found += 1;
|
||||
} else {
|
||||
s.wrong += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn w(s: &str) -> (Vec<char>, Vec<usize>) {
|
||||
let mut chars = Vec::new();
|
||||
let mut br = Vec::new();
|
||||
for c in s.chars() {
|
||||
if c == '-' {
|
||||
br.push(chars.len());
|
||||
} else {
|
||||
chars.push(c);
|
||||
}
|
||||
}
|
||||
(chars, br)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn learns_a_tiny_list() {
|
||||
let list: Vec<_> = [
|
||||
"hap-pen",
|
||||
"hap-pens",
|
||||
"hap-pened",
|
||||
"pen-cil",
|
||||
"pen-cils",
|
||||
"rab-bit",
|
||||
"rab-bits",
|
||||
"kit-ten",
|
||||
"kit-tens",
|
||||
"but-ter",
|
||||
"bet-ter",
|
||||
"let-ter",
|
||||
"lit-ter",
|
||||
"sit-ter",
|
||||
]
|
||||
.iter()
|
||||
.map(|s| w(s))
|
||||
.collect();
|
||||
let levels = [
|
||||
Level { min_len: 1, max_len: 4, good_weight: 1, bad_weight: 1, threshold: 2 },
|
||||
Level { min_len: 1, max_len: 5, good_weight: 1, bad_weight: 1, threshold: 1 },
|
||||
];
|
||||
let p = train(&list, &levels, &mut |_| {});
|
||||
let sc = score(&p, &list);
|
||||
assert_eq!(sc.breaks, list.len());
|
||||
assert!(sc.found >= sc.breaks - 1, "{sc:?}");
|
||||
assert!(sc.wrong <= 1, "{sc:?}");
|
||||
// Generalises to an unseen word with the same shape.
|
||||
assert_eq!(p.points(&"bitter".chars().collect::<Vec<_>>()).unwrap(), vec![3]);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,239 @@
|
||||
//! Liang hyphenation patterns (TeX notation, e.g. `.ab1c`, `4t1ion`): parsing and application.
|
||||
//!
|
||||
//! The bundled English set is *ours*: trained from the public-domain Moby Hyphenator word list by
|
||||
//! [`super::patgen`] (see `examples/hyphgen.rs`). Patterns are stored in a hash map keyed by the
|
||||
//! packed letter string, so applying them to a word costs `O(len × max_pattern_len)` lookups.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::hash::{BuildHasherDefault, Hasher};
|
||||
|
||||
/// Word-boundary marker in patterns.
|
||||
pub const DOT: char = '.';
|
||||
/// Longest pattern the packed key supports.
|
||||
pub const MAX_PATTERN_LEN: usize = 9;
|
||||
|
||||
/// Small, fast multiplicative hasher for packed `u64` keys (FxHash-style).
|
||||
#[derive(Default, Clone, Copy)]
|
||||
pub struct FxHasher(u64);
|
||||
|
||||
impl Hasher for FxHasher {
|
||||
fn finish(&self) -> u64 {
|
||||
self.0
|
||||
}
|
||||
fn write(&mut self, bytes: &[u8]) {
|
||||
for &b in bytes {
|
||||
self.write_u64(b as u64);
|
||||
}
|
||||
}
|
||||
fn write_u64(&mut self, i: u64) {
|
||||
self.0 = (self.0.rotate_left(5) ^ i).wrapping_mul(0x51_7c_c1_b7_27_22_0a_95);
|
||||
}
|
||||
}
|
||||
|
||||
pub type FxMap<K, V> = HashMap<K, V, BuildHasherDefault<FxHasher>>;
|
||||
|
||||
/// Maps characters to 6-bit codes (1 = word boundary, 2.. = letters); 0 = not in the alphabet.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Alphabet {
|
||||
ascii: [u8; 128],
|
||||
other: Vec<(char, u8)>,
|
||||
next: u8,
|
||||
}
|
||||
|
||||
impl Default for Alphabet {
|
||||
fn default() -> Self {
|
||||
let mut a = Alphabet { ascii: [0; 128], other: Vec::new(), next: 2 };
|
||||
a.ascii[DOT as usize] = 1;
|
||||
a
|
||||
}
|
||||
}
|
||||
|
||||
impl Alphabet {
|
||||
pub fn code(&self, c: char) -> u8 {
|
||||
if (c as u32) < 128 { self.ascii[c as usize] } else { self.other.iter().find(|x| x.0 == c).map_or(0, |x| x.1) }
|
||||
}
|
||||
/// Code for `c`, adding it if new (None when the 6-bit alphabet is full).
|
||||
pub fn intern(&mut self, c: char) -> Option<u8> {
|
||||
let k = self.code(c);
|
||||
if k != 0 {
|
||||
return Some(k);
|
||||
}
|
||||
if self.next >= 63 {
|
||||
return None;
|
||||
}
|
||||
let k = self.next;
|
||||
self.next += 1;
|
||||
if (c as u32) < 128 {
|
||||
self.ascii[c as usize] = k;
|
||||
} else {
|
||||
self.other.push((c, k));
|
||||
}
|
||||
Some(k)
|
||||
}
|
||||
}
|
||||
|
||||
/// Packed key of a code string (≤ [`MAX_PATTERN_LEN`] codes).
|
||||
#[inline]
|
||||
pub fn pack(codes: &[u8]) -> u64 {
|
||||
let mut k = 0u64;
|
||||
for &c in codes {
|
||||
k = (k << 6) | c as u64;
|
||||
}
|
||||
k | ((codes.len() as u64) << 58)
|
||||
}
|
||||
|
||||
/// A compiled pattern set.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct Patterns {
|
||||
alpha: Alphabet,
|
||||
/// Packed letters → inter-letter values (`len + 1` entries).
|
||||
map: FxMap<u64, Box<[u8]>>,
|
||||
max_len: usize,
|
||||
}
|
||||
|
||||
impl Patterns {
|
||||
/// Parse TeX-style patterns: whitespace-separated, `%`/`#` start a comment line.
|
||||
pub fn parse(text: &str) -> Patterns {
|
||||
let mut p = Patterns::default();
|
||||
for line in text.lines() {
|
||||
let line = line.trim();
|
||||
if line.is_empty() || line.starts_with('%') || line.starts_with('#') {
|
||||
continue;
|
||||
}
|
||||
for tok in line.split_whitespace() {
|
||||
p.insert(tok);
|
||||
}
|
||||
}
|
||||
p
|
||||
}
|
||||
|
||||
/// Add one pattern like `a1b`; returns false if it can't be represented.
|
||||
pub fn insert(&mut self, pat: &str) -> bool {
|
||||
let mut codes = Vec::with_capacity(pat.len());
|
||||
let mut vals = vec![0u8];
|
||||
for c in pat.chars() {
|
||||
if let Some(d) = c.to_digit(10) {
|
||||
*vals.last_mut().unwrap_or(&mut 0) = d as u8;
|
||||
} else {
|
||||
let Some(k) = self.alpha.intern(c) else { return false };
|
||||
codes.push(k);
|
||||
vals.push(0);
|
||||
}
|
||||
}
|
||||
if codes.is_empty() || codes.len() > MAX_PATTERN_LEN {
|
||||
return false;
|
||||
}
|
||||
self.max_len = self.max_len.max(codes.len());
|
||||
self.map.insert(pack(&codes), vals.into_boxed_slice());
|
||||
true
|
||||
}
|
||||
|
||||
pub(crate) fn from_parts(alpha: Alphabet, map: FxMap<u64, Box<[u8]>>, max_len: usize) -> Patterns {
|
||||
Patterns { alpha, map, max_len }
|
||||
}
|
||||
|
||||
/// Make `letters` part of the alphabet even if no pattern mentions them.
|
||||
pub fn add_letters(&mut self, letters: impl IntoIterator<Item = char>) {
|
||||
for c in letters {
|
||||
let _ = self.alpha.intern(c);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn len(&self) -> usize {
|
||||
self.map.len()
|
||||
}
|
||||
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.map.is_empty()
|
||||
}
|
||||
|
||||
/// Inter-letter values for a lowercase word: `out[i]` is the value *before* char `i`
|
||||
/// (`out.len() == word.len() + 1`). None if a char is outside the pattern alphabet.
|
||||
pub fn values(&self, word: &[char]) -> Option<Vec<u8>> {
|
||||
let mut codes = Vec::with_capacity(word.len() + 2);
|
||||
codes.push(1);
|
||||
for &c in word {
|
||||
let k = self.alpha.code(c);
|
||||
if k == 0 {
|
||||
return None;
|
||||
}
|
||||
codes.push(k);
|
||||
}
|
||||
codes.push(1);
|
||||
let mut vals = vec![0u8; codes.len() + 1];
|
||||
apply(&self.map, self.max_len, &codes, &mut vals);
|
||||
// Padded position p sits before padded char p; word char i is padded char i + 1.
|
||||
Some(vals[1..codes.len()].to_vec())
|
||||
}
|
||||
|
||||
/// Break points (char indices, a hyphen goes before that char) of a lowercase word.
|
||||
pub fn points(&self, word: &[char]) -> Option<Vec<usize>> {
|
||||
let v = self.values(word)?;
|
||||
Some((1..word.len()).filter(|&i| v[i] % 2 == 1).collect())
|
||||
}
|
||||
|
||||
/// The patterns in TeX notation (sorted by letters), e.g. for writing a pattern file.
|
||||
pub fn to_tex(&self) -> Vec<String> {
|
||||
let mut rev: Vec<(u8, char)> =
|
||||
(0u8..128).filter(|&c| self.alpha.ascii[c as usize] != 0).map(|c| (self.alpha.ascii[c as usize], c as char)).collect();
|
||||
rev.extend(self.alpha.other.iter().map(|&(c, k)| (k, c)));
|
||||
let ch = |k: u8| rev.iter().find(|x| x.0 == k).map_or('?', |x| x.1);
|
||||
let mut out: Vec<(String, String)> = self
|
||||
.map
|
||||
.iter()
|
||||
.map(|(&key, vals)| {
|
||||
let n = (key >> 58) as usize;
|
||||
let letters: Vec<char> = (0..n).map(|i| ch(((key >> (6 * (n - 1 - i))) & 63) as u8)).collect();
|
||||
let mut s = String::new();
|
||||
for (i, l) in letters.iter().enumerate() {
|
||||
if vals[i] > 0 {
|
||||
s.push(char::from(b'0' + vals[i]));
|
||||
}
|
||||
s.push(*l);
|
||||
}
|
||||
if vals[n] > 0 {
|
||||
s.push(char::from(b'0' + vals[n]));
|
||||
}
|
||||
(letters.into_iter().collect(), s)
|
||||
})
|
||||
.collect();
|
||||
out.sort();
|
||||
out.into_iter().map(|x| x.1).collect()
|
||||
}
|
||||
}
|
||||
|
||||
/// Apply `map` to padded `codes`, raising `vals` (len = codes.len() + 1).
|
||||
#[inline]
|
||||
pub fn apply(map: &FxMap<u64, Box<[u8]>>, max_len: usize, codes: &[u8], vals: &mut [u8]) {
|
||||
let n = codes.len();
|
||||
for s in 0..n {
|
||||
let mut k = 0u64;
|
||||
for len in 1..=max_len.min(n - s) {
|
||||
k = (k << 6) | codes[s + len - 1] as u64;
|
||||
if let Some(v) = map.get(&(k | ((len as u64) << 58))) {
|
||||
for (j, &x) in v.iter().enumerate() {
|
||||
if x > vals[s + j] {
|
||||
vals[s + j] = x;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn liang_example() {
|
||||
// The classic example from Liang's thesis: "hy-phen-ation".
|
||||
let p = Patterns::parse("hy3ph he2n hena4 hen5at 1na n2at 1tio 2io o2n");
|
||||
let w: Vec<char> = "hyphenation".chars().collect();
|
||||
assert_eq!(p.points(&w).unwrap(), vec![2, 6]);
|
||||
let mut tex = p.to_tex();
|
||||
tex.sort();
|
||||
assert!(tex.contains(&"hen5at".to_string()));
|
||||
assert!(p.values(&['h', 'é']).is_none());
|
||||
}
|
||||
}
|
||||
+407
-44
@@ -14,8 +14,9 @@ mod cache;
|
||||
pub mod hyphen;
|
||||
pub mod shape;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::ops::Range;
|
||||
use std::sync::Arc;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use designcraft_doc::{
|
||||
Align, Composer, Document, FirstBaseline, GridAlign, ItemId, ParaProps, SpanColumns, StartParagraph, Story, StoryId, TabAlign, TextFrameOptions,
|
||||
@@ -246,8 +247,22 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
let cols: Vec<Vec<Rect>> = frames.iter().map(FrameSpec::columns).collect();
|
||||
let mut cur = Cursor { fi: 0, col: 0, last_baseline: None, last_descent: 0.0, pending: 0.0 };
|
||||
let para_ranges = story.para_ranges();
|
||||
let np = para_ranges.len();
|
||||
let mut list_counter: u32 = 0;
|
||||
'paras: for (pi, prange) in para_ranges.iter().enumerate() {
|
||||
// Keep options: paragraphs are re-laid from a snapshot when a keep is violated, either forced
|
||||
// into the next column or with a cap on the lines set before moving on (widow control).
|
||||
let mut snaps: Vec<Snapshot> = Vec::with_capacity(np);
|
||||
let mut info: Vec<ParaInfo> = Vec::with_capacity(np);
|
||||
let mut force_col = vec![false; np];
|
||||
let mut line_cap: Vec<Option<usize>> = vec![None; np];
|
||||
let mut restores = 0usize;
|
||||
let mut pi = 0;
|
||||
'paras: while pi < np {
|
||||
let prange = para_ranges[pi].clone();
|
||||
let snap = Snapshot::take(&out, &cur, list_counter);
|
||||
snaps.truncate(pi);
|
||||
snaps.push(snap);
|
||||
info.truncate(pi);
|
||||
let pf = &story.paras[pi];
|
||||
let (pp, base_chars) = doc.styles.resolve_para(pf);
|
||||
// Bullets & numbering: generated prefix (shaped as its own glyphs, mapped to the paragraph start).
|
||||
@@ -269,23 +284,19 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
}
|
||||
designcraft_doc::ListType::None => list_counter = 0,
|
||||
}
|
||||
let glyphs = sp.glyphs;
|
||||
let mut glyphs = sp.glyphs;
|
||||
apply_desired_spacing(&mut glyphs, &pp);
|
||||
let hyph_after = hyphenation_points(&story.text, &glyphs, &pp);
|
||||
let base_size = base_chars.size;
|
||||
let base_leading = match base_chars.leading {
|
||||
designcraft_doc::Leading::Auto => base_size * pp.auto_leading,
|
||||
designcraft_doc::Leading::Points(v) => v,
|
||||
};
|
||||
let spacing = Spacing {
|
||||
justify: pp.align.is_justified(),
|
||||
word_min: pp.word_space_min,
|
||||
word_desired: pp.word_space_desired,
|
||||
word_max: pp.word_space_max,
|
||||
hyphen_penalty: 50.0 + 450.0 * pp.hyph_weight,
|
||||
hyphen_limit: pp.hyph_limit,
|
||||
ragged_stretch: base_size * 2.0,
|
||||
};
|
||||
let spacing = spacing_for(&pp, base_size);
|
||||
// Paragraph start options.
|
||||
if force_col[pi] {
|
||||
cur.next_column(&cols);
|
||||
}
|
||||
if cur.last_baseline.is_some() {
|
||||
match pp.start_paragraph {
|
||||
StartParagraph::NextColumn => cur.next_column(&cols),
|
||||
@@ -296,9 +307,13 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
if cur.last_baseline.is_some() {
|
||||
cur.pending += pp.space_before;
|
||||
}
|
||||
let at_top = cur.last_baseline.is_none();
|
||||
let start_at = (cur.fi, cur.col);
|
||||
// Rule above / shading track the paragraph's first line.
|
||||
let mut g0 = 0usize;
|
||||
let mut line_no = 0usize;
|
||||
// Paragraph line number of the first line in the current column.
|
||||
let mut col_first_line = 0usize;
|
||||
let has_tabs = glyphs.iter().any(|g| g.ch == '\t');
|
||||
let mut first_line_rect: Option<(usize, f64, f64, f64)> = None; // frame, baseline, ascent, x-span
|
||||
loop {
|
||||
@@ -327,6 +342,8 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
let rest_h = &hyph_after[g0..];
|
||||
let breaks: Vec<Break> = if pp.composer == Composer::SingleLine || has_tabs || rest.len() > 4000 {
|
||||
breaker::greedy(rest, rest_h, &spacing, &width)
|
||||
} else if pp.balance_ragged && !spacing.justify {
|
||||
breaker::balanced(rest, rest_h, &spacing, &width)
|
||||
} else {
|
||||
breaker::knuth_plass(rest, rest_h, &spacing, &width)
|
||||
};
|
||||
@@ -365,20 +382,54 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
}
|
||||
}
|
||||
}
|
||||
let fits = baseline + desc <= col.y1 + 0.01;
|
||||
let capped = line_cap[pi] == Some(line_no) && line_no > col_first_line;
|
||||
let fits = baseline + desc <= col.y1 + 0.01 && !capped;
|
||||
if !fits {
|
||||
if !capped {
|
||||
let ctx = KeepCtx {
|
||||
pi,
|
||||
line_no,
|
||||
in_col: line_no - col_first_line,
|
||||
started_here: col_first_line == 0,
|
||||
at_top,
|
||||
remaining: breaks.len() - k,
|
||||
here: (cur.fi, cur.col),
|
||||
};
|
||||
let action = if restores < 4 * np + 64 { keep_violation(&ctx, &pp, &info, &force_col, &line_cap) } else { None };
|
||||
if let Some(action) = action {
|
||||
restores += 1;
|
||||
let j = match action {
|
||||
KeepAction::Force(j) => {
|
||||
force_col[j] = true;
|
||||
j
|
||||
}
|
||||
KeepAction::Cap(j, n) => {
|
||||
line_cap[j] = Some(n);
|
||||
j
|
||||
}
|
||||
};
|
||||
for k in j + 1..np {
|
||||
force_col[k] = false;
|
||||
line_cap[k] = None;
|
||||
}
|
||||
snaps[j].restore(&mut out, &mut cur, &mut list_counter);
|
||||
pi = j;
|
||||
continue 'paras;
|
||||
}
|
||||
}
|
||||
// Next column / frame; re-break the rest of the paragraph there.
|
||||
g0 = s;
|
||||
cur.next_column(&cols);
|
||||
col_first_line = line_no;
|
||||
moved = true;
|
||||
let _ = k;
|
||||
break;
|
||||
}
|
||||
let ind_l = pp.left_indent + if line_no == 0 { pp.first_line_indent } else { 0.0 };
|
||||
let lx0 = x0 + ind_l;
|
||||
let lx1 = x1 - pp.right_indent;
|
||||
let last = k + 1 == breaks.len();
|
||||
let (placed, end_x, ratio) = layout_line(&glyphs, s, e, b.hyphen, lx0, lx1, col.x0, &pp, last, b.forced && !last, f.left_page);
|
||||
let (placed, end_x, ratio) =
|
||||
layout_line(&glyphs, s, e, b.hyphen, lx0, lx1, col.x0, &pp, &spacing, last, b.forced && !last, f.left_page);
|
||||
let range_end = if last { prange.end } else { glyphs.get(g0 + b.next).map(|g| g.byte).unwrap_or(prange.end) };
|
||||
let range_start = glyphs.get(s).map(|g| g.byte).unwrap_or(prange.start).min(range_end);
|
||||
let range_start = if line_no == 0 { prange.start } else { range_start };
|
||||
@@ -411,8 +462,14 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
if b.forced && e < glyphs.len() + 1 {
|
||||
let brk = glyphs.get(g0 + b.next.saturating_sub(1)).map(|g| g.ch);
|
||||
match brk {
|
||||
Some(story::COLUMN_BREAK) => cur.next_column(&cols),
|
||||
Some(story::FRAME_BREAK) | Some(story::PAGE_BREAK) => cur.next_frame(),
|
||||
Some(story::COLUMN_BREAK) => {
|
||||
cur.next_column(&cols);
|
||||
col_first_line = line_no;
|
||||
}
|
||||
Some(story::FRAME_BREAK) | Some(story::PAGE_BREAK) => {
|
||||
cur.next_frame();
|
||||
col_first_line = line_no;
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
@@ -421,6 +478,18 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
break;
|
||||
}
|
||||
}
|
||||
info.push(ParaInfo {
|
||||
start: start_at,
|
||||
at_top,
|
||||
end: (cur.fi, cur.col),
|
||||
lines: line_no,
|
||||
lines_in_end_col: line_no - col_first_line,
|
||||
keep_with_next: pp.keep_with_next,
|
||||
keep_together: pp.keep_lines_together,
|
||||
keep_all: pp.keep_all_lines,
|
||||
keep_first: pp.keep_first as usize,
|
||||
keep_last: pp.keep_last as usize,
|
||||
});
|
||||
// Rules and shading for the paragraph.
|
||||
if let Some((fi, bl, asc, _)) = first_line_rect {
|
||||
let ft = &mut out.frames[fi];
|
||||
@@ -458,6 +527,7 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
ft.decos.push(Deco { rect: Rect::new(x0 + r.left_indent, y, x1 - r.right_indent, y + r.weight), color: r.color.clone(), tint: r.tint });
|
||||
}
|
||||
cur.pending += pp.space_after;
|
||||
pi += 1;
|
||||
}
|
||||
// Ranges, content heights and vertical justification.
|
||||
for (fi, ft) in out.frames.iter_mut().enumerate() {
|
||||
@@ -485,10 +555,185 @@ pub fn compose(doc: &Document, story: &Story, frames: &[FrameSpec], opts: &Compo
|
||||
out
|
||||
}
|
||||
|
||||
/// Breaker parameters from the paragraph's settings.
|
||||
fn spacing_for(pp: &ParaProps, base_size: f64) -> Spacing {
|
||||
Spacing {
|
||||
justify: pp.align.is_justified(),
|
||||
word_min: pp.word_space_min,
|
||||
word_desired: pp.word_space_desired,
|
||||
word_max: pp.word_space_max,
|
||||
letter_min: pp.letter_space_min.min(pp.letter_space_desired),
|
||||
letter_desired: pp.letter_space_desired,
|
||||
letter_max: pp.letter_space_max.max(pp.letter_space_desired),
|
||||
glyph_min: pp.glyph_scale_min.min(pp.glyph_scale_desired).max(0.01),
|
||||
glyph_desired: pp.glyph_scale_desired.max(0.01),
|
||||
glyph_max: pp.glyph_scale_max.max(pp.glyph_scale_desired),
|
||||
hyphen_penalty: 50.0 + 450.0 * pp.hyph_weight,
|
||||
hyphen_limit: pp.hyph_limit,
|
||||
ragged_stretch: base_size * 2.0,
|
||||
hyph_zone: if pp.align.is_justified() { 0.0 } else { pp.hyph_zone },
|
||||
optical: pp.optical_margin,
|
||||
}
|
||||
}
|
||||
|
||||
/// Desired word spacing, letter spacing and glyph scaling apply to every line (any alignment).
|
||||
fn apply_desired_spacing(glyphs: &mut [Glyph], pp: &ParaProps) {
|
||||
let ws = pp.word_space_desired;
|
||||
let ls = pp.letter_space_desired;
|
||||
let gs = pp.glyph_scale_desired.max(0.01);
|
||||
if (ws - 1.0).abs() < 1e-9 && ls.abs() < 1e-9 && (gs - 1.0).abs() < 1e-9 {
|
||||
return;
|
||||
}
|
||||
for g in glyphs {
|
||||
if g.ch == ' ' {
|
||||
g.adv += g.space * (ws - 1.0);
|
||||
} else if !g.is_space() && g.adv > 0.0 {
|
||||
g.adv = g.adv * gs + ls * g.space;
|
||||
g.sx *= gs;
|
||||
g.dx *= gs;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Lengths of the composed output and the cursor before a paragraph (keep resolution re-lays
|
||||
/// paragraphs from here). Only the cursor's frame and later ones can change afterwards.
|
||||
struct Snapshot {
|
||||
cur: Cursor,
|
||||
list_counter: u32,
|
||||
lines: usize,
|
||||
decos: usize,
|
||||
}
|
||||
|
||||
impl Snapshot {
|
||||
fn take(out: &ComposedStory, cur: &Cursor, list_counter: u32) -> Snapshot {
|
||||
let (lines, decos) = out.frames.get(cur.fi).map_or((0, 0), |f| (f.lines.len(), f.decos.len()));
|
||||
Snapshot { cur: cur.clone(), list_counter, lines, decos }
|
||||
}
|
||||
fn restore(&self, out: &mut ComposedStory, cur: &mut Cursor, list_counter: &mut u32) {
|
||||
let fi = self.cur.fi;
|
||||
for (i, f) in out.frames.iter_mut().enumerate().skip(fi) {
|
||||
let (l, d) = if i == fi { (self.lines, self.decos) } else { (0, 0) };
|
||||
f.lines.truncate(l);
|
||||
f.decos.truncate(d);
|
||||
}
|
||||
*cur = self.cur.clone();
|
||||
*list_counter = self.list_counter;
|
||||
out.overset_at = None;
|
||||
}
|
||||
}
|
||||
|
||||
/// Where a placed paragraph sits (for keep resolution).
|
||||
#[derive(Clone, Debug)]
|
||||
struct ParaInfo {
|
||||
start: (usize, usize),
|
||||
/// Started at the top of a column (moving it can't help).
|
||||
at_top: bool,
|
||||
end: (usize, usize),
|
||||
lines: usize,
|
||||
lines_in_end_col: usize,
|
||||
keep_with_next: u32,
|
||||
keep_together: bool,
|
||||
keep_all: bool,
|
||||
keep_first: usize,
|
||||
keep_last: usize,
|
||||
}
|
||||
|
||||
/// The paragraph being placed when a line doesn't fit its column.
|
||||
struct KeepCtx {
|
||||
pi: usize,
|
||||
/// Paragraph line number of the line that doesn't fit.
|
||||
line_no: usize,
|
||||
/// Lines of this paragraph already in the column.
|
||||
in_col: usize,
|
||||
/// The paragraph started in this column.
|
||||
started_here: bool,
|
||||
at_top: bool,
|
||||
/// Lines left to set, including the one that doesn't fit.
|
||||
remaining: usize,
|
||||
here: (usize, usize),
|
||||
}
|
||||
|
||||
enum KeepAction {
|
||||
/// Re-lay from paragraph `j`, starting it in the next column.
|
||||
Force(usize),
|
||||
/// Re-lay from paragraph `j`, moving to the next column after `n` of its lines.
|
||||
Cap(usize, usize),
|
||||
}
|
||||
|
||||
/// Keep Options (plan: typography §8): keep with next, keep all lines together, keep first/last
|
||||
/// lines. Returns how to re-lay, or None to just continue in the next column (also when a keep
|
||||
/// can't be satisfied, e.g. a paragraph already at the top of a column).
|
||||
fn keep_violation(ctx: &KeepCtx, pp: &ParaProps, info: &[ParaInfo], force_col: &[bool], line_cap: &[Option<usize>]) -> Option<KeepAction> {
|
||||
// The first paragraph of the chain of keep-with-next paragraphs ending in `pi` that sit in this column.
|
||||
let chain_start = |pi: usize| {
|
||||
let mut j = pi;
|
||||
while j > 0 {
|
||||
let p = &info[j - 1];
|
||||
let starts_here = if j == ctx.pi { ctx.started_here } else { info[j].start == ctx.here };
|
||||
if p.keep_with_next > 0 && p.end == ctx.here && starts_here {
|
||||
j -= 1;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
j
|
||||
};
|
||||
let force = |j: usize| -> Option<KeepAction> {
|
||||
let top = if j == ctx.pi { ctx.at_top } else { info[j].at_top };
|
||||
let starts_here = if j == ctx.pi { ctx.started_here } else { info[j].start == ctx.here };
|
||||
(!(force_col[j] || top && starts_here)).then_some(KeepAction::Force(j))
|
||||
};
|
||||
let total_lines = ctx.line_no + ctx.remaining;
|
||||
// Keep with next: the previous paragraph's last line must share a column with our first lines.
|
||||
if ctx.started_here && ctx.pi > 0 {
|
||||
let prev = &info[ctx.pi - 1];
|
||||
let need = (prev.keep_with_next as usize).min(total_lines);
|
||||
if prev.keep_with_next > 0 && prev.end == ctx.here && ctx.line_no < need {
|
||||
let j = chain_start(ctx.pi);
|
||||
if j == ctx.pi - 1 && prev.lines_in_end_col > 1 && !(prev.keep_together && prev.keep_all) && line_cap[j].is_none() {
|
||||
// Move only the previous paragraph's last lines (respecting its widow control).
|
||||
let keep_last = if prev.keep_together { prev.keep_last.max(1) } else { 1 };
|
||||
let stay = prev.lines_in_end_col.saturating_sub(keep_last);
|
||||
let min_stay = if prev.start == ctx.here { if prev.keep_together { prev.keep_first.max(1) } else { 1 } } else { 1 };
|
||||
if stay >= min_stay {
|
||||
return Some(KeepAction::Cap(j, prev.lines - keep_last));
|
||||
}
|
||||
}
|
||||
if let Some(a) = force(j) {
|
||||
return Some(a);
|
||||
}
|
||||
}
|
||||
}
|
||||
if !pp.keep_lines_together || ctx.in_col == 0 {
|
||||
return None;
|
||||
}
|
||||
if pp.keep_all_lines {
|
||||
return if ctx.started_here { force(chain_start(ctx.pi)) } else { None };
|
||||
}
|
||||
// Orphan control: at least `keep_first` lines at the bottom of the column.
|
||||
if ctx.started_here && ctx.in_col < (pp.keep_first as usize).max(1) {
|
||||
return force(chain_start(ctx.pi));
|
||||
}
|
||||
// Widow control: at least `keep_last` lines at the top of the next column.
|
||||
let keep_last = (pp.keep_last as usize).max(1);
|
||||
if ctx.remaining < keep_last {
|
||||
let need = keep_last - ctx.remaining;
|
||||
let min_stay = if ctx.started_here { (pp.keep_first as usize).max(1) } else { 1 };
|
||||
if ctx.in_col >= need + min_stay && line_cap[ctx.pi].is_none() {
|
||||
return Some(KeepAction::Cap(ctx.pi, ctx.line_no - need));
|
||||
}
|
||||
if ctx.started_here {
|
||||
return force(chain_start(ctx.pi));
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn ft_prev_end(overset: &Option<usize>, len: usize) -> usize {
|
||||
overset.unwrap_or(len)
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct Cursor {
|
||||
fi: usize,
|
||||
col: usize,
|
||||
@@ -620,6 +865,10 @@ fn estimate_slots(f: &FrameSpec, col: Rect, first: f64, lead: f64, size: f64, gl
|
||||
}
|
||||
|
||||
/// Position glyphs `s..e` within `[x0, x1]`; returns (glyphs, end x, word-space ratio).
|
||||
///
|
||||
/// Justified lines distribute the difference to the measure in priority order: word spaces up to
|
||||
/// their limit, then letter spacing, then glyph scaling, and anything left over to the word spaces
|
||||
/// (an H&J violation). With optical margin alignment, edge punctuation hangs outside `[x0, x1]`.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn layout_line(
|
||||
glyphs: &[Glyph],
|
||||
@@ -630,6 +879,7 @@ fn layout_line(
|
||||
x1: f64,
|
||||
tab_origin: f64,
|
||||
pp: &ParaProps,
|
||||
sp: &Spacing,
|
||||
last: bool,
|
||||
forced_mid: bool,
|
||||
left_page: bool,
|
||||
@@ -673,10 +923,20 @@ fn layout_line(
|
||||
x += line[i].adv;
|
||||
i += 1;
|
||||
}
|
||||
let has_tab = line.iter().any(|g| g.ch == '\t' || g.ch == story::RIGHT_INDENT_TAB);
|
||||
// Optical margin alignment: the measure grows by the hang of the edge glyphs.
|
||||
let (mut x0, mut measure) = (x0, measure);
|
||||
if sp.optical && !has_tab {
|
||||
let first = line.iter().find(|g| g.adv > 0.0 && !g.is_space());
|
||||
let lastg = line.iter().rev().find(|g| g.adv > 0.0 && !g.is_space());
|
||||
let hl = first.map_or(0.0, |g| breaker::hang(g.ch).0 * g.adv);
|
||||
let hr = lastg.map_or(0.0, |g| breaker::hang(g.ch).1 * g.adv);
|
||||
x0 -= hl;
|
||||
measure += hl + hr;
|
||||
}
|
||||
let natural: f64 = line.iter().map(|g| g.adv).sum();
|
||||
let extra = measure - natural;
|
||||
let spaces: Vec<usize> = line.iter().enumerate().filter(|(_, g)| g.is_space() && !g.no_break).map(|(i, _)| i).collect();
|
||||
let has_tab = line.iter().any(|g| g.ch == '\t' || g.ch == story::RIGHT_INDENT_TAB);
|
||||
let align = match pp.align {
|
||||
Align::TowardsSpine => {
|
||||
if left_page {
|
||||
@@ -695,26 +955,22 @@ fn layout_line(
|
||||
a => a,
|
||||
};
|
||||
let justify_this = align.is_justified() && (!last || align == Align::FullyJustified || forced_mid) && !has_tab;
|
||||
let mut per_space = 0.0;
|
||||
// A justified paragraph's last line may have been composed with shrunk spaces: shrink it too.
|
||||
let squeeze_last = align.is_justified() && !justify_this && !has_tab && extra < 0.0 && !spaces.is_empty();
|
||||
// Extra advance per glyph (word spaces and letter gaps) and horizontal scale per glyph.
|
||||
let mut add = vec![0.0; line.len()];
|
||||
let mut scale = vec![1.0; line.len()];
|
||||
let mut offset = 0.0;
|
||||
let mut ratio = 1.0;
|
||||
if justify_this && !spaces.is_empty() {
|
||||
per_space = extra / spaces.len() as f64;
|
||||
let space_w: f64 = spaces.iter().map(|&i| line[i].adv).sum::<f64>() / spaces.len() as f64;
|
||||
ratio = if space_w > 0.0 { (space_w + per_space) / space_w * pp.word_space_desired } else { 1.0 };
|
||||
if (justify_this || squeeze_last) && !spaces.is_empty() {
|
||||
distribute(&line, &spaces, extra, sp, &mut add, &mut scale);
|
||||
} else if justify_this && spaces.is_empty() && line.len() > 1 && !last {
|
||||
// Single word: Single Word Justification.
|
||||
match pp.single_word_justify {
|
||||
Align::FullyJustified => {
|
||||
let gaps = (line.len() - 1) as f64;
|
||||
let per = extra / gaps;
|
||||
let mut x = x0;
|
||||
let mut out = Vec::with_capacity(line.len());
|
||||
for (i, g) in line.iter().enumerate() {
|
||||
out.push(place(g, x));
|
||||
x += g.adv + if i + 1 < line.len() { per } else { 0.0 };
|
||||
let per = extra / (line.len() - 1) as f64;
|
||||
for a in &mut add[..line.len() - 1] {
|
||||
*a = per;
|
||||
}
|
||||
return (out, x, 1.0);
|
||||
}
|
||||
Align::Center => offset = extra / 2.0,
|
||||
Align::Right => offset = extra,
|
||||
@@ -738,19 +994,103 @@ fn layout_line(
|
||||
offset = 0.0;
|
||||
}
|
||||
}
|
||||
// Word-space ratio actually used (vs the space width), for H&J highlighting.
|
||||
let mut ratio_sum = 0.0;
|
||||
let mut ratio_n = 0usize;
|
||||
for &i in &spaces {
|
||||
let g = &line[i];
|
||||
if g.ch == ' ' && g.space > 0.0 {
|
||||
ratio_sum += (g.adv + add[i]) / g.space;
|
||||
ratio_n += 1;
|
||||
}
|
||||
}
|
||||
let ratio = if ratio_n > 0 { ratio_sum / ratio_n as f64 } else { 1.0 };
|
||||
let mut out = Vec::with_capacity(line.len());
|
||||
let mut x = x0 + offset;
|
||||
for (i, g) in line.iter().enumerate() {
|
||||
out.push(place(g, x));
|
||||
x += g.adv;
|
||||
if per_space != 0.0 && spaces.binary_search(&i).is_ok() {
|
||||
x += per_space;
|
||||
if let Some(p) = out.last_mut() {
|
||||
p.adv += per_space;
|
||||
let mut p = place(g, x);
|
||||
if scale[i] != 1.0 {
|
||||
p.sx *= scale[i];
|
||||
p.x = x + g.dx * scale[i];
|
||||
p.adv *= scale[i];
|
||||
}
|
||||
p.adv += add[i];
|
||||
x += p.adv;
|
||||
out.push(p);
|
||||
}
|
||||
(out, x, ratio)
|
||||
}
|
||||
|
||||
/// Share `extra` points among word spaces, letter gaps and glyph scaling (see [`layout_line`]).
|
||||
fn distribute(line: &[Glyph], spaces: &[usize], extra: f64, sp: &Spacing, add: &mut [f64], scale: &mut [f64]) {
|
||||
let stretch = extra >= 0.0;
|
||||
let word: Vec<f64> = spaces
|
||||
.iter()
|
||||
.map(|&i| {
|
||||
let g = &line[i];
|
||||
if g.ch != ' ' {
|
||||
0.0
|
||||
} else if stretch {
|
||||
g.space * (sp.word_max - sp.word_desired).max(0.0)
|
||||
} else {
|
||||
g.space * (sp.word_desired - sp.word_min).max(0.0)
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
// Letter gaps: between visible glyphs (not after the line's last glyph).
|
||||
let last_box = line.iter().rposition(|g| !g.is_space() && g.adv > 0.0).unwrap_or(0);
|
||||
let is_box = |i: usize, g: &Glyph| i < last_box && !g.is_space() && g.adv > 0.0;
|
||||
let letter: Vec<f64> = line
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, g)| {
|
||||
if !is_box(i, g) {
|
||||
0.0
|
||||
} else if stretch {
|
||||
g.space * (sp.letter_max - sp.letter_desired).max(0.0)
|
||||
} else {
|
||||
g.space * (sp.letter_desired - sp.letter_min).max(0.0)
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
let glyph: Vec<f64> = line
|
||||
.iter()
|
||||
.map(|g| {
|
||||
if g.is_space() || g.adv <= 0.0 {
|
||||
0.0
|
||||
} else {
|
||||
let natural = g.adv / sp.glyph_desired.max(0.01);
|
||||
natural * if stretch { (sp.glyph_max - sp.glyph_desired).max(0.0) } else { (sp.glyph_desired - sp.glyph_min).max(0.0) }
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
let (yw, yl, yg) = (word.iter().sum::<f64>(), letter.iter().sum::<f64>(), glyph.iter().sum::<f64>());
|
||||
let mut rem = extra.abs();
|
||||
let tw = rem.min(yw);
|
||||
rem -= tw;
|
||||
let tl = rem.min(yl);
|
||||
rem -= tl;
|
||||
let tg = rem.min(yg);
|
||||
rem -= tg;
|
||||
let sign = if stretch { 1.0 } else { -1.0 };
|
||||
for (k, &i) in spaces.iter().enumerate() {
|
||||
let share = if yw > 1e-9 { tw * word[k] / yw } else { 0.0 };
|
||||
// Left over beyond every limit: shared equally by the word spaces.
|
||||
add[i] += sign * (share + rem / spaces.len() as f64);
|
||||
}
|
||||
if yl > 1e-9 && tl > 0.0 {
|
||||
for (i, l) in letter.iter().enumerate() {
|
||||
add[i] += sign * tl * l / yl;
|
||||
}
|
||||
}
|
||||
if yg > 1e-9 && tg > 0.0 {
|
||||
for (i, gcap) in glyph.iter().enumerate() {
|
||||
if *gcap > 0.0 {
|
||||
let w = line[i].adv;
|
||||
scale[i] = (w + sign * tg * gcap / yg) / w;
|
||||
}
|
||||
}
|
||||
}
|
||||
(out, x, ratio)
|
||||
}
|
||||
|
||||
fn place(g: &Glyph, x: f64) -> PlacedGlyph {
|
||||
@@ -802,7 +1142,10 @@ fn prepend_label(
|
||||
*glyphs = pre;
|
||||
}
|
||||
|
||||
/// Mark glyphs after which a hyphen may be inserted.
|
||||
type LimitsKey = (usize, usize, usize, bool);
|
||||
|
||||
/// Mark glyphs after which a hyphen may be inserted (dictionary/pattern points within the
|
||||
/// paragraph's limits; words with discretionary hyphens break only there).
|
||||
fn hyphenation_points(text: &str, glyphs: &[Glyph], pp: &ParaProps) -> Vec<bool> {
|
||||
let mut out = vec![false; glyphs.len()];
|
||||
if !pp.hyphenate {
|
||||
@@ -814,6 +1157,24 @@ fn hyphenation_points(text: &str, glyphs: &[Glyph], pp: &ParaProps) -> Vec<bool>
|
||||
before_last: pp.hyph_before_last as usize,
|
||||
capitalized: pp.hyph_capitalized,
|
||||
};
|
||||
// Words repeat a lot: remember their points per set of limits.
|
||||
static CACHE: Mutex<Vec<(LimitsKey, HashMap<Box<str>, Box<[usize]>>)>> = Mutex::new(Vec::new());
|
||||
let key = (lim.min_word, lim.after_first, lim.before_last, lim.capitalized);
|
||||
let mut guard = CACHE.lock().unwrap_or_else(|e| e.into_inner());
|
||||
let idx = match guard.iter().position(|e| e.0 == key) {
|
||||
Some(i) => i,
|
||||
None => {
|
||||
if guard.len() > 32 {
|
||||
guard.clear();
|
||||
}
|
||||
guard.push((key, HashMap::new()));
|
||||
guard.len() - 1
|
||||
}
|
||||
};
|
||||
let cache = &mut guard[idx].1;
|
||||
if cache.len() > 100_000 {
|
||||
cache.clear();
|
||||
}
|
||||
let mut i = 0;
|
||||
while i < glyphs.len() {
|
||||
if !glyphs[i].is_letter() || glyphs[i].len == 0 {
|
||||
@@ -821,17 +1182,19 @@ fn hyphenation_points(text: &str, glyphs: &[Glyph], pp: &ParaProps) -> Vec<bool>
|
||||
continue;
|
||||
}
|
||||
let mut j = i;
|
||||
while j < glyphs.len() && (glyphs[j].is_letter() || glyphs[j].len == 0) && !glyphs[j].no_break {
|
||||
while j < glyphs.len() && (glyphs[j].is_letter() || glyphs[j].len == 0 || glyphs[j].ch == shape::SOFT_HYPHEN) && !glyphs[j].no_break {
|
||||
j += 1;
|
||||
}
|
||||
// A discretionary hyphen in a word (or right before it) replaces automatic hyphenation.
|
||||
let discretionary = glyphs[i..j].iter().any(|g| g.ch == shape::SOFT_HYPHEN) || (i > 0 && glyphs[i - 1].ch == shape::SOFT_HYPHEN);
|
||||
let (a, b) = (glyphs[i].byte, glyphs[j - 1].byte + glyphs[j - 1].len);
|
||||
if b > a && b <= text.len() && text.is_char_boundary(a) && text.is_char_boundary(b) {
|
||||
if !discretionary && b > a && b <= text.len() && text.is_char_boundary(a) && text.is_char_boundary(b) {
|
||||
let word = &text[a..b];
|
||||
// Do not hyphenate the paragraph's last word unless allowed.
|
||||
let is_last_word = !pp.hyph_last_word && glyphs[j..].iter().all(|g| !g.is_letter());
|
||||
if !is_last_word {
|
||||
let pts = hyphen::hyphen_points(word, &lim);
|
||||
for p in pts {
|
||||
let pts = cache.entry(word.into()).or_insert_with(|| hyphen::hyphen_points(word, &lim).into_boxed_slice());
|
||||
for &p in pts.iter() {
|
||||
// char index p → byte → glyph whose cluster ends at that byte.
|
||||
let byte = a + word.char_indices().nth(p).map(|(bi, _)| bi).unwrap_or(word.len());
|
||||
if let Some(k) = (i..j).find(|&k| glyphs[k].byte + glyphs[k].len == byte && glyphs[k].len > 0) {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! Paragraph → styled, positioned glyphs (before line breaking).
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use designcraft_doc::{Capitalization, CharProps, Kerning, Leading, Position, Story, Styles, story};
|
||||
use designcraft_fonts::{Feature, FontDb, FontFace, ShapedGlyph, feature};
|
||||
@@ -37,6 +38,9 @@ pub struct Glyph {
|
||||
pub style: u32,
|
||||
/// Unbreakable (No Break).
|
||||
pub no_break: bool,
|
||||
/// Width of a word space (U+0020) in this glyph's font, size and horizontal scale: the unit of
|
||||
/// word and letter spacing in justification.
|
||||
pub space: f64,
|
||||
}
|
||||
|
||||
impl Glyph {
|
||||
@@ -224,6 +228,54 @@ fn shape_run(
|
||||
flush(seg_start, range.end, &seg_face, out);
|
||||
}
|
||||
|
||||
type WordMap = HashMap<Box<str>, Arc<[ShapedGlyph]>>;
|
||||
|
||||
/// Shaped words (font units) by (face, caps, features), then text.
|
||||
static WORD_CACHE: Mutex<Vec<((u32, bool, String), WordMap)>> = Mutex::new(Vec::new());
|
||||
const WORD_CACHE_MAX: usize = 50_000;
|
||||
|
||||
/// Shape `src` word by word through a cache (like a browser's word cache): the text is split
|
||||
/// after each U+0020 so repeated words are shaped once. Shaping does not cross word spaces in the
|
||||
/// scripts we lay out, and clusters stay byte offsets into `src`.
|
||||
fn shape_cached(face: &FontFace, src: &str, feats: &[Feature], caps: bool) -> Vec<ShapedGlyph> {
|
||||
let map = |c: char| if caps { c.to_uppercase().next().unwrap_or(c) } else { c };
|
||||
if src.len() < 2 || !src.contains(' ') {
|
||||
return designcraft_fonts::shape(face, src, feats, map);
|
||||
}
|
||||
let fkey = format!("{feats:?}");
|
||||
let mut out = Vec::with_capacity(src.len());
|
||||
let mut guard = WORD_CACHE.lock().unwrap_or_else(|e| e.into_inner());
|
||||
let key = (face.id(), caps, fkey);
|
||||
let idx = match guard.iter().position(|e| e.0 == key) {
|
||||
Some(i) => i,
|
||||
None => {
|
||||
if guard.len() > 64 {
|
||||
guard.clear();
|
||||
}
|
||||
guard.push((key, HashMap::new()));
|
||||
guard.len() - 1
|
||||
}
|
||||
};
|
||||
let cache = &mut guard[idx].1;
|
||||
if cache.len() > WORD_CACHE_MAX {
|
||||
cache.clear();
|
||||
}
|
||||
let mut start = 0;
|
||||
for piece in src.split_inclusive(' ') {
|
||||
let glyphs = match cache.get(piece) {
|
||||
Some(g) => g.clone(),
|
||||
None => {
|
||||
let g: Arc<[ShapedGlyph]> = designcraft_fonts::shape(face, piece, feats, map).into();
|
||||
cache.insert(piece.into(), g.clone());
|
||||
g
|
||||
}
|
||||
};
|
||||
out.extend(glyphs.iter().map(|g| ShapedGlyph { cluster: g.cluster + start, ..*g }));
|
||||
start += piece.len();
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn metrics(face: &FontFace, p: &CharProps, auto_leading: f64) -> (f64, f64, f64, f64, f64, f64, f64, f64) {
|
||||
let (size, shift) = effective_size(p);
|
||||
let k = size / face.upem;
|
||||
@@ -266,6 +318,7 @@ fn control_glyph(face: &Arc<FontFace>, p: &CharProps, auto_leading: f64, style:
|
||||
size,
|
||||
style,
|
||||
no_break: p.no_break,
|
||||
space: face.advance(face.glyph_for(' ')) * k * p.h_scale,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -289,7 +342,8 @@ fn shape_segment(
|
||||
let src = replacement.unwrap_or(&text[range.clone()]);
|
||||
let caps = p.capitalization == Capitalization::AllCaps;
|
||||
let feats = features_for(p);
|
||||
let shaped: Vec<ShapedGlyph> = designcraft_fonts::shape(face, src, &feats, |c| if caps { c.to_uppercase().next().unwrap_or(c) } else { c });
|
||||
let space = face.advance(face.glyph_for(' ')) * k * hs;
|
||||
let shaped: Vec<ShapedGlyph> = shape_cached(face, src, &feats, caps);
|
||||
let n = shaped.len();
|
||||
for (gi, sg) in shaped.iter().enumerate() {
|
||||
let (byte, len, ch) = if replacement.is_some() {
|
||||
@@ -328,6 +382,7 @@ fn shape_segment(
|
||||
size,
|
||||
style,
|
||||
no_break: p.no_break,
|
||||
space,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
+359
-11
@@ -178,24 +178,372 @@ fn tabs_align_to_stops() {
|
||||
assert!((l.end_x - 100.0).abs() < 0.5, "{}", l.end_x);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn justified_narrow_column_has_no_extreme_lines() {
|
||||
let text = "Color holds it all together. A restrained palette of two or three swatches, applied consistently to headlines, rules and backgrounds, gives a publication its voice. Spot inks, tints and gradients are all just named swatches, so a single change ripples through every page.";
|
||||
let para = ParaAttrs { align: Some(Align::LeftJustified), first_line_indent: Some(12.0), hyph_min_word: Some(6), ..Default::default() };
|
||||
const SWATCH_TEXT: &str = "Color holds it all together. A restrained palette of two or three swatches, applied consistently to headlines, rules and backgrounds, gives a publication its voice. Spot inks, tints and gradients are all just named swatches, so a single change ripples through every page.";
|
||||
|
||||
/// Worst word-space ratio (excluding last lines) of `text` in a narrow justified column.
|
||||
fn narrow_worst(para: ParaAttrs) -> f64 {
|
||||
let para = ParaAttrs { align: Some(Align::LeftJustified), first_line_indent: Some(12.0), hyph_min_word: Some(6), ..para };
|
||||
let mut d = Document::new(&NewDocument::default());
|
||||
let lid = d.default_layer();
|
||||
let (_, sid) =
|
||||
d.add_text_frame(SpreadRef::Doc(0), Rect::new(0.0, 0.0, 170.67, 2000.0), lid, text, ParaFormat { para, ..Default::default() }).unwrap();
|
||||
d.story_mut(sid).unwrap().format_chars(0..text.len(), |f| f.over.size = Some(9.75));
|
||||
let (_, sid) = d
|
||||
.add_text_frame(SpreadRef::Doc(0), Rect::new(0.0, 0.0, 170.67, 2000.0), lid, SWATCH_TEXT, ParaFormat { para, ..Default::default() })
|
||||
.unwrap();
|
||||
d.story_mut(sid).unwrap().format_chars(0..SWATCH_TEXT.len(), |f| f.over.size = Some(9.75));
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
let mut worst = 0.0f64;
|
||||
for l in all_lines(&cs) {
|
||||
let s: String = d.story(sid).unwrap().text[l.range.clone()].to_string();
|
||||
eprintln!("{:5.2} {}", l.spacing, s);
|
||||
eprintln!("{:5.2} {}", l.spacing, &SWATCH_TEXT[l.range.clone()]);
|
||||
assert!(l.end_x <= l.x1 + 0.5, "overfull line");
|
||||
if !l.last_in_para {
|
||||
assert!((l.end_x - l.x1).abs() < 0.6, "justified line ends at {} not {}", l.end_x, l.x1);
|
||||
worst = worst.max(l.spacing);
|
||||
}
|
||||
}
|
||||
// Regression bound; tighter once dictionary hyphenation lands (plan: Moby PD word list).
|
||||
assert!(worst < 8.0, "loose line ratio {worst}");
|
||||
worst
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn justified_narrow_column_has_no_extreme_lines() {
|
||||
// Word spacing only (InDesign's defaults: letter spacing 0%, glyph scaling 100%). The first
|
||||
// line can't do better than ~2.7: "…together. A re-" is the longest first line that fits.
|
||||
let words_only = narrow_worst(ParaAttrs::default());
|
||||
assert!(words_only < 4.5, "loose line ratio {words_only}");
|
||||
// A typical magazine body setup: letter spacing −5…10% and glyph scaling 97…103% absorb the rest.
|
||||
let full = narrow_worst(ParaAttrs {
|
||||
letter_space_min: Some(-0.05),
|
||||
letter_space_max: Some(0.10),
|
||||
glyph_scale_min: Some(0.97),
|
||||
glyph_scale_max: Some(1.03),
|
||||
..Default::default()
|
||||
});
|
||||
assert!(full < 2.0, "loose line ratio {full}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn glyph_scaling_and_letter_spacing_are_applied() {
|
||||
let para = ParaAttrs {
|
||||
align: Some(Align::LeftJustified),
|
||||
hyphenate: Some(false),
|
||||
letter_space_min: Some(-0.05),
|
||||
letter_space_max: Some(0.2),
|
||||
glyph_scale_min: Some(0.95),
|
||||
glyph_scale_max: Some(1.05),
|
||||
..Default::default()
|
||||
};
|
||||
let text = CORPUS.join(" ");
|
||||
let (d, sid, _) = doc_with(&text, Rect::new(0.0, 0.0, 150.0, 20000.0), para.clone());
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
let lines = all_lines(&cs);
|
||||
let mut scaled = false;
|
||||
let mut sum = 0.0;
|
||||
for l in &lines[..lines.len() - 1] {
|
||||
assert!((l.end_x - l.x1).abs() < 0.6, "justified line ends at {} not {}", l.end_x, l.x1);
|
||||
sum += l.spacing;
|
||||
let k = 12.0 / l.glyphs[0].face.upem;
|
||||
scaled |= l.glyphs.iter().any(|g| g.visible && (g.sx / k - 1.0).abs() > 1e-3);
|
||||
}
|
||||
assert!(scaled, "some line uses glyph scaling");
|
||||
// Word spaces stay closer to their desired width than with word spacing alone.
|
||||
let plain = ParaAttrs { align: Some(Align::LeftJustified), hyphenate: Some(false), ..Default::default() };
|
||||
let (d, sid, _) = doc_with(&text, Rect::new(0.0, 0.0, 150.0, 20000.0), plain);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
let pl = all_lines(&cs);
|
||||
let plain_avg = pl[..pl.len() - 1].iter().map(|l| l.spacing).sum::<f64>() / (pl.len() - 1) as f64;
|
||||
let avg = sum / (lines.len() - 1) as f64;
|
||||
assert!(avg < plain_avg - 0.1, "with tiers {avg:.3}, words only {plain_avg:.3}");
|
||||
// Desired values apply to every line, including ragged text.
|
||||
let wide = |ls: f64| {
|
||||
let (d, sid, _) =
|
||||
doc_with("Hello world", Rect::new(0.0, 0.0, 300.0, 100.0), ParaAttrs { letter_space_desired: Some(ls), ..Default::default() });
|
||||
compose_story(&d, sid, &ComposeOptions::default()).frames[0].lines[0].end_x
|
||||
};
|
||||
assert!(wide(0.5) > wide(0.0) + 10.0 * 0.5 * 2.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hyphen_limit_is_a_hard_constraint() {
|
||||
let text = CORPUS.join(" ");
|
||||
for limit in [1u32, 2] {
|
||||
let para = ParaAttrs { align: Some(Align::LeftJustified), hyph_limit: Some(limit), hyph_weight: Some(0.0), ..Default::default() };
|
||||
let (d, sid, _) = doc_with(&text, Rect::new(0.0, 0.0, 130.0, 100_000.0), para);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
let mut run = 0;
|
||||
let mut any = false;
|
||||
for l in all_lines(&cs) {
|
||||
run = if l.hyphenated { run + 1 } else { 0 };
|
||||
any |= l.hyphenated;
|
||||
assert!(run <= limit, "{run} consecutive hyphens with limit {limit}");
|
||||
}
|
||||
assert!(any);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hyphenation_zone_limits_ragged_hyphens() {
|
||||
let text = CORPUS.join(" ");
|
||||
let count = |zone: f64, composer: designcraft_doc::Composer| {
|
||||
let para =
|
||||
ParaAttrs { align: Some(Align::Left), hyph_zone: Some(zone), composer: Some(composer), hyph_weight: Some(0.0), ..Default::default() };
|
||||
let (d, sid, _) = doc_with(&text, Rect::new(0.0, 0.0, 110.0, 100_000.0), para);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
all_lines(&cs).iter().filter(|l| l.hyphenated).count()
|
||||
};
|
||||
for c in [designcraft_doc::Composer::Paragraph, designcraft_doc::Composer::SingleLine] {
|
||||
let (free, zoned, huge) = (count(0.0, c), count(36.0, c), count(1000.0, c));
|
||||
// A huge zone leaves only words that start a line (no space to break at) hyphenable.
|
||||
assert!(free > 0 && zoned <= free && huge * 4 <= free, "{c:?}: {free} {zoned} {huge}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn discretionary_hyphen_replaces_automatic_points() {
|
||||
let text = "aaaa bbbb cccc extraordi\u{AD}narily dddd eeee";
|
||||
let para = ParaAttrs { align: Some(Align::Left), ..Default::default() };
|
||||
let (d, sid, _) = doc_with(text, Rect::new(0.0, 0.0, 120.0, 1000.0), para);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
let soft = text.find('\u{AD}').unwrap();
|
||||
for l in all_lines(&cs) {
|
||||
if l.hyphenated {
|
||||
let last = l.glyphs.iter().rfind(|g| g.len > 0).unwrap();
|
||||
assert!(last.byte + last.len >= soft && last.byte <= soft + 2, "{:?}", &text[l.range.clone()]);
|
||||
}
|
||||
}
|
||||
// Hyphenation points of the word itself: none besides the discretionary one.
|
||||
let lines = all_lines(&cs);
|
||||
assert!(lines.iter().filter(|l| l.hyphenated).count() <= 1);
|
||||
}
|
||||
|
||||
/// Story of `n` one-line filler paragraphs followed by `extra` paragraphs, in a 2-column frame.
|
||||
fn keep_doc(fillers: usize, extra: &[&str], height: f64) -> (Document, StoryId, Vec<usize>) {
|
||||
let mut parts: Vec<String> = (0..fillers).map(|k| format!("Filler {k}")).collect();
|
||||
parts.extend(extra.iter().map(|s| s.to_string()));
|
||||
let text = parts.join("\n");
|
||||
let (mut d, sid, fid) = doc_with(&text, Rect::new(0.0, 0.0, 400.0, height), ParaAttrs::default());
|
||||
d.item_mut(fid).unwrap().text_frame_mut().unwrap().options.columns = 2;
|
||||
(d, sid, (0..parts.len()).collect())
|
||||
}
|
||||
|
||||
fn column_of_para(cs: &ComposedStory, pi: usize) -> Vec<u32> {
|
||||
all_lines(cs).iter().filter(|l| l.para == pi).map(|l| l.column).collect()
|
||||
}
|
||||
|
||||
/// Lines that fit in one column of the keep test frame.
|
||||
fn lines_per_column(height: f64) -> usize {
|
||||
let (d, sid, _) = keep_doc(60, &[], height);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
all_lines(&cs).iter().filter(|l| l.column == 0).count()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn keep_with_next_moves_a_heading() {
|
||||
let h = 200.0;
|
||||
let n = lines_per_column(h);
|
||||
let body = "Body text that follows the heading.";
|
||||
let (mut d, sid, _) = keep_doc(n - 1, &["Heading", body], h);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
assert_eq!(column_of_para(&cs, n - 1), vec![0], "without keeps the heading ends column 0");
|
||||
d.story_mut(sid).unwrap().paras[n - 1].para.keep_with_next = Some(1);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
assert_eq!(column_of_para(&cs, n - 1), vec![1]);
|
||||
assert_eq!(column_of_para(&cs, n), vec![1]);
|
||||
assert_eq!(column_of_para(&cs, n - 2), vec![0]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn keep_lines_together_moves_the_paragraph() {
|
||||
let h = 200.0;
|
||||
let n = lines_per_column(h);
|
||||
let long = [LOREM, LOREM].join(" ");
|
||||
let (mut d, sid, _) = keep_doc(n - 3, &[&long], h);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
assert!(column_of_para(&cs, n - 3).contains(&0));
|
||||
d.story_mut(sid).unwrap().paras[n - 3].para.keep_lines_together = Some(true);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
assert!(column_of_para(&cs, n - 3).iter().all(|&c| c == 1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn keep_first_and_last_lines() {
|
||||
let h = 200.0;
|
||||
let n = lines_per_column(h);
|
||||
// Orphan: one line at the bottom of column 0 → the paragraph moves.
|
||||
let (mut d, sid, _) = keep_doc(n - 1, &[LOREM], h);
|
||||
{
|
||||
let p = &mut d.story_mut(sid).unwrap().paras[n - 1].para;
|
||||
p.keep_lines_together = Some(true);
|
||||
p.keep_all_lines = Some(false);
|
||||
}
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
let cols = column_of_para(&cs, n - 1);
|
||||
assert!(cols.iter().all(|&c| c == 1), "{cols:?}");
|
||||
// Widow: a paragraph whose last line would sit alone in column 1 gives it a companion.
|
||||
let para_lines = {
|
||||
let (d, sid, _) = keep_doc(0, &[LOREM], 2000.0);
|
||||
compose_story(&d, sid, &ComposeOptions::default()).line_count()
|
||||
};
|
||||
assert!(para_lines >= 4);
|
||||
let start = n - (para_lines - 1);
|
||||
let (mut d, sid, _) = keep_doc(start, &[LOREM], h);
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
assert_eq!(column_of_para(&cs, start).iter().filter(|&&c| c == 1).count(), 1, "one widowed line without keeps");
|
||||
{
|
||||
let p = &mut d.story_mut(sid).unwrap().paras[start].para;
|
||||
p.keep_lines_together = Some(true);
|
||||
p.keep_all_lines = Some(false);
|
||||
}
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
let cols = column_of_para(&cs, start);
|
||||
assert_eq!(cols.iter().filter(|&&c| c == 1).count(), 2, "{cols:?}");
|
||||
assert!(cols.iter().filter(|&&c| c == 0).count() >= 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn balance_ragged_lines_evens_a_headline() {
|
||||
let text = "Balanced headlines read better than a stub";
|
||||
let widths = |balance: bool| {
|
||||
let para = ParaAttrs { balance_ragged: Some(balance), hyphenate: Some(false), ..Default::default() };
|
||||
let (mut d, sid, _) = doc_with(text, Rect::new(0.0, 0.0, 300.0, 500.0), para);
|
||||
d.story_mut(sid).unwrap().format_chars(0..text.len(), |f| f.over.size = Some(18.0));
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
all_lines(&cs).iter().map(|l| l.end_x - l.x0).collect::<Vec<_>>()
|
||||
};
|
||||
let plain = widths(false);
|
||||
let bal = widths(true);
|
||||
assert_eq!(plain.len(), bal.len());
|
||||
assert!(plain.len() >= 2);
|
||||
let spread = |w: &[f64]| w.iter().cloned().fold(f64::MIN, f64::max) - w.iter().cloned().fold(f64::MAX, f64::min);
|
||||
assert!(spread(&bal) < spread(&plain), "{bal:?} vs {plain:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn optical_margin_hangs_punctuation() {
|
||||
let text = "Hanging punctuation, quietly. “Quotes” sit outside the measure, as do commas, periods and hyphens.";
|
||||
let run = |optical: bool| {
|
||||
let para = ParaAttrs { align: Some(Align::LeftJustified), optical_margin: Some(optical), ..Default::default() };
|
||||
let (d, sid, _) = doc_with(text, Rect::new(0.0, 0.0, 120.0, 1000.0), para);
|
||||
compose_story(&d, sid, &ComposeOptions::default())
|
||||
};
|
||||
let plain = run(false);
|
||||
let hung = run(true);
|
||||
for l in all_lines(&plain) {
|
||||
assert!(l.end_x <= l.x1 + 0.5);
|
||||
}
|
||||
// Some justified line ends with punctuation that now protrudes past the right edge.
|
||||
let lines = all_lines(&hung);
|
||||
assert!(lines[..lines.len() - 1].iter().any(|l| l.end_x > l.x1 + 0.5), "no line hangs");
|
||||
assert!(lines.iter().all(|l| l.end_x <= l.x1 + 12.0));
|
||||
}
|
||||
|
||||
// ---------- golden paragraph corpus ----------
|
||||
|
||||
/// Original prose (written for this corpus) with a realistic mix of short and long words.
|
||||
const CORPUS: &[&str] = &[
|
||||
"Every printed page is a negotiation between the designer and the reader. The designer proposes an order: a column of type, a picture that interrupts it, a caption that explains the picture. The reader accepts that order only when nothing on the page calls attention to itself for the wrong reasons.",
|
||||
"Justified text is especially unforgiving. When the measure is narrow, the composer must choose between loose lines with rivers of white running down the column, tight lines in which words collide, and hyphenated lines that interrupt the rhythm of reading. Good composition balances all three considerations across the whole paragraph instead of one line at a time.",
|
||||
"Hyphenation dictionaries record where conventional syllable divisions fall in thousands of ordinary words: international, responsibility, characteristically, misunderstanding, photographer, extraordinarily, administration, comprehensive, establishment, unquestionably and their many relatives.",
|
||||
"Magazines often set body copy in two or three columns on a page, with generous gutters and a baseline grid that keeps neighbouring lines aligned. Headlines span the columns, pull quotes break into them, and captions sit beside photographs in a smaller, contrasting typeface.",
|
||||
"The typesetter of the nineteenth century worked with metal sorts, composing sticks and wedges of spacing material. Many of the conventions we take for granted today, such as the preference for consistent word spacing over consistent line endings, were established by those craftsmen long before computers automated their decisions.",
|
||||
"Readers rarely notice good typography, but they immediately feel the discomfort of bad typography: uneven spacing, awkward breaks, widowed lines stranded at the top of a column, and orphaned headings left at the bottom of a page without the paragraph that follows them.",
|
||||
];
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct CorpusStats {
|
||||
lines: usize,
|
||||
overfull: usize,
|
||||
hyphens: usize,
|
||||
sum_spacing: f64,
|
||||
worst: f64,
|
||||
loose: usize,
|
||||
}
|
||||
|
||||
fn corpus_stats(align: Align, measure: f64, size: f64) -> CorpusStats {
|
||||
let text = CORPUS.join("\n");
|
||||
let para = ParaAttrs { align: Some(align), ..Default::default() };
|
||||
let mut d = Document::new(&NewDocument::default());
|
||||
let lid = d.default_layer();
|
||||
let (_, sid) =
|
||||
d.add_text_frame(SpreadRef::Doc(0), Rect::new(0.0, 0.0, measure, 100_000.0), lid, &text, ParaFormat { para, ..Default::default() }).unwrap();
|
||||
d.story_mut(sid).unwrap().format_chars(0..text.len(), |f| f.over.size = Some(size));
|
||||
let cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
assert!(!cs.is_overset());
|
||||
let mut st = CorpusStats::default();
|
||||
for l in all_lines(&cs) {
|
||||
if l.end_x > l.x1 + 0.5 {
|
||||
st.overfull += 1;
|
||||
}
|
||||
if l.hyphenated {
|
||||
st.hyphens += 1;
|
||||
}
|
||||
if !l.last_in_para {
|
||||
st.lines += 1;
|
||||
st.sum_spacing += l.spacing;
|
||||
st.worst = st.worst.max(l.spacing);
|
||||
if l.spacing > 1.33 + 1e-6 {
|
||||
st.loose += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
st
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn golden_corpus_spacing() {
|
||||
let mut total = CorpusStats::default();
|
||||
for &(measure, size) in &[(120.0, 9.0), (170.67, 9.75), (240.0, 10.0), (340.0, 11.0)] {
|
||||
for align in [Align::LeftJustified, Align::Left] {
|
||||
let st = corpus_stats(align, measure, size);
|
||||
let avg = st.sum_spacing / st.lines.max(1) as f64;
|
||||
eprintln!(
|
||||
"{align:?} measure {measure:6.2} size {size:5.2}: lines {:3} hyphens {:2} avg {avg:.3} worst {:.3} loose {:2} overfull {}",
|
||||
st.lines, st.hyphens, st.worst, st.loose, st.overfull
|
||||
);
|
||||
assert_eq!(st.overfull, 0, "overfull lines at {measure}");
|
||||
if align == Align::LeftJustified && measure >= 170.0 {
|
||||
assert!(st.worst < 3.5, "worst justified line {} at {measure}", st.worst);
|
||||
assert!(avg < 1.65, "average justified word-space ratio {avg} at {measure}");
|
||||
}
|
||||
if align == Align::LeftJustified {
|
||||
total.lines += st.lines;
|
||||
total.sum_spacing += st.sum_spacing;
|
||||
total.worst = total.worst.max(st.worst);
|
||||
total.loose += st.loose;
|
||||
}
|
||||
}
|
||||
}
|
||||
let avg = total.sum_spacing / total.lines.max(1) as f64;
|
||||
eprintln!("justified total: lines {} avg {avg:.3} worst {:.3} loose {}", total.lines, total.worst, total.loose);
|
||||
assert!(avg < 1.9, "average justified word-space ratio {avg}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "perf: run with --release -- --ignored --nocapture"]
|
||||
fn perf_compose_300k_story() {
|
||||
let mut text = String::new();
|
||||
let mut k = 0;
|
||||
while text.len() < 300_000 {
|
||||
text.push_str(CORPUS[k % CORPUS.len()]);
|
||||
text.push('\n');
|
||||
k += 1;
|
||||
}
|
||||
let para = ParaAttrs { align: Some(Align::LeftJustified), ..Default::default() };
|
||||
let mut d = Document::new(&NewDocument::default());
|
||||
let lid = d.default_layer();
|
||||
let (_, sid) =
|
||||
d.add_text_frame(SpreadRef::Doc(0), Rect::new(0.0, 0.0, 240.0, 3_000_000.0), lid, &text, ParaFormat { para, ..Default::default() }).unwrap();
|
||||
// Cold run (loads hyphenation data, fills the word caches), then the best of a few warm runs.
|
||||
let t = std::time::Instant::now();
|
||||
let _ = compose_story(&d, sid, &ComposeOptions::default());
|
||||
let cold = t.elapsed().as_secs_f64() * 1000.0;
|
||||
let mut ms = f64::INFINITY;
|
||||
let mut cs = ComposedStory::default();
|
||||
for _ in 0..5 {
|
||||
let t = std::time::Instant::now();
|
||||
cs = compose_story(&d, sid, &ComposeOptions::default());
|
||||
ms = ms.min(t.elapsed().as_secs_f64() * 1000.0);
|
||||
}
|
||||
eprintln!("composed {} chars, {} lines: cold {cold:.1} ms, warm {ms:.1} ms", text.len(), cs.line_count());
|
||||
assert!(!cs.is_overset());
|
||||
if !cfg!(debug_assertions) {
|
||||
assert!(ms < 300.0, "composition took {ms:.1} ms");
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user