mirror of
https://github.com/storytold/effectcraft.git
synced 2026-10-14 19:27:05 -04:00
M13.36: face tracking trained model — MediaPipe Face Landmarker on a pure-Rust TFLite interpreter
- effectcraft-segment: a FaceModel interface (find a face in a region, follow it; a Topology maps the model's points to the tracker's landmarks and outline), a TensorFlow Lite reader and interpreter (float ops, float16 weights folded; matches ai-edge-litert to ~1e-6), and MediaPipe Face Landmarker (Google, Apache-2.0: BlazeFace + Face Mesh V2, 3.8 MB, pinned SHA-256). The registry gains a task and authors per model; installed weights get a NOTICE. - effectcraft-track: FaceTracker::new_with uses the model, falls back to the classical fitter when it finds no face; chin and jaw come from the outline either way so measurements agree. - Engine: roto_models becomes models (both tasks); face.models / face.model.* commands, Settings ▸ Face Tracking, `wait` on select/install/download for scripts; track.mask reports faceModel. - Credits: ATTRIBUTION.md lists the optional models and the Apache-2.0 specs the code follows. - Docs: README, ROADMAP (G7), gaps, parity (TRK-3 score unchanged pending G1), agents, preferences, architecture, segment README. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
4624c00308
commit
e43bf66775
33 files changed
+2298
-485
No files matched your search
@@ -22,7 +22,7 @@ cpal or muda. Everything in L0 to L4, the egui UI and the web app also build for
|
||||
| L0 | `av1enc` | AV1 encoder from the AV1 bitstream specification: Main profile 8/10-bit 4:2:0, key + inter frames (quarter-pel motion), deblocking, for MP4 (`av01`) and WebM export |
|
||||
| L1 | `raster` | Premultiplied float images, sampling, affine and projective warps, blurs, compositing (parallel with rayon) |
|
||||
| L1 | `keyframe` | Animated values, keyframes with temporal ease and spatial Bezier, roving, hold, velocity |
|
||||
| L1 | `segment` | Swappable trained segmentation models for Roto Brush: the `MaskModel` interface, a registry of open-licensed models (licence, URL, size, SHA-256), a PyTorch checkpoint reader, SHA-256, and MobileSAM (TinyViT + Segment Anything decoder) on rayon and ndarray's safe GEMM |
|
||||
| L1 | `segment` | Swappable trained models for Roto Brush and face tracking: the `MaskModel` and `FaceModel` interfaces, a registry of open-licensed models (authors, licence, URL, size, SHA-256), PyTorch checkpoint and TensorFlow Lite readers (with an interpreter), SHA-256, MobileSAM (TinyViT + Segment Anything decoder) and MediaPipe Face Landmarker (BlazeFace + Face Mesh V2) on rayon and ndarray's safe GEMM |
|
||||
| L1 | `psd` | Photoshop PSD/PSB reader (layers, groups, masks, blend modes, text, layer effects, adjustment layers; 8/16/32-bit; RGB/CMYK/Gray/Lab) and a minimal writer, from Adobe's published format specification |
|
||||
| L1 | `path` | Bezier paths, path operators (trim, offset, round corners, zig zag, twist, merge…), stroking, coverage masks |
|
||||
| L2 | `project` | The document: items, compositions, layers, the property tree, render queue model, `.ecproj` serde |
|
||||
@@ -335,6 +335,10 @@ components inside it, and an active-shape point distribution model trained on sy
|
||||
shapes (our own generator, no external weights) that fills and checks the landmarks. The outline
|
||||
keys the Mask Path; Detailed Features keys a Face Track Points effect, and
|
||||
`track.extractFaceMeasurements` derives a keyed Face Measurements effect (and copies its keys).
|
||||
With a trained face model chosen (`effectcraft_segment::face::FaceModel`, Settings ▸ Face
|
||||
Tracking; `Session::models` hands it to the mask track), `FaceTracker::new_with` lets the model
|
||||
find the face in the mask and follow it; the model's outline and named points replace the
|
||||
classical steps, and chin and jaw still come from the outline so the measurements agree.
|
||||
|
||||
**Warp Stabilizer** (`effects::warp_stab`, `engine::warp`): `warp.analyze` renders the layer's
|
||||
input to the effect (`Renderer::layer_input`: source, masks and the effects above it) for every
|
||||
|
||||
Reference in new issue
Block a user