M13.2: FilmCraft web app (apps/filmcraft-web) with Blob media, OPFS recovery, WebAudio, WebCodecs

- eframe web runner on WebGPU (WebGL2 fallback; GPU compositor on WebGPU, CPU on WebGL2).
- fs: virtual file table of File/Blob handles, 1 MiB chunked async range reads with an
  LRU cache and read-ahead (pending + retry instead of blocking), downloads for written
  files, /opfs/ paths written to OPFS.
- import: File System Access picker / <input type=file>, drag-and-drop anywhere,
  container index prefetch (moov wherever it is), retried imports; project open/save.
- recovery: OPFS auto-save snapshot while dirty, media copies, reopened on next visit.
- audio: AudioWorklet output fed ahead from the UI thread; played-frame audio clock.
- webcodecs: H.264/HEVC/VP9/AV1 MP4/MOV decoded by the browser VideoDecoder via a
  reader opener (fallback to our decoders on any error), probed with isConfigSupported.
- api: window.filmcraft (execute/request/commands/screenshot/importFiles/importUrl/
  openProject/files/readFile/info).
- cargo xtask web [--dev] [--serve PORT] (wasm-bindgen CLI); xtask wasm also checks
  filmcraft-ui-egui and filmcraft-web; layer L6 for the web app.
- Original web icon (favicon.svg) with attribution sidecar; docs/web.md; headless-Chrome
  smoke test (tests/smoke.mjs).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Brandon ThomasandClaude Opus 5.5 committed 2026-10-01 14:52:35 -04:00
1 parent 243e95ab24
commit 5f613aff25
20 files changed
+2664 -7

No files matched your search

+1
View File
@@ -25,6 +25,7 @@ iconography, images or other Adobe assets; open licences only; every asset attri
| `docs/images/filmcraft-scopes.png` | FilmCraft contributors | Original work: FilmCraft UI crop (scopes of a Charade (1963) frame, public domain) | MIT OR Apache-2.0 |
| `docs/images/filmcraft-effects.png` | FilmCraft contributors | Original work: FilmCraft screenshot crop; NASA Earth Views (US Government work, public domain) | MIT OR Apache-2.0; footage public domain |
| `docs/images/filmcraft-timeline.png` | FilmCraft contributors | Original work: FilmCraft UI crop; thumbnails from Night of the Living Dead (1968) and Carnival of Souls (1962), CC0 Chopin waveform | MIT OR Apache-2.0; film frames public domain |
| `apps/filmcraft-web/web/favicon.svg` | FilmCraft contributors | Original work: web app icon (disc, sprocket ring, play triangle) drawn from basic SVG shapes | MIT OR Apache-2.0 |
## Golden test images
+81
View File
@@ -0,0 +1,81 @@
[package]
name = "filmcraft-web"
description = "FilmCraft in the browser (wasm32): eframe web runner, WebGPU/WebGL2, Blob media, OPFS, WebAudio, WebCodecs"
version.workspace = true
edition.workspace = true
license.workspace = true
publish = false
[lints]
workspace = true
[lib]
crate-type = ["cdylib", "rlib"]
# Everything is wasm32-only: on other targets this crate is empty (so `cargo test --workspace`
# and clippy stay cheap). Build with `cargo xtask web` (see docs/web.md).
[target.'cfg(target_arch = "wasm32")'.dependencies]
filmcraft-engine = { workspace = true }
filmcraft-ui-egui = { workspace = true }
filmcraft-media = { workspace = true }
filmcraft-codecs = { workspace = true }
filmcraft-frame = { workspace = true }
filmcraft-color = { workspace = true }
filmcraft-isobmff = { workspace = true }
filmcraft-format = { workspace = true }
filmcraft-time = { workspace = true }
egui = { workspace = true }
eframe = { workspace = true, features = ["default_fonts", "wgpu"] }
serde_json = { workspace = true }
log = { workspace = true }
wasm-bindgen = "=0.2.129"
wasm-bindgen-futures = "0.4"
js-sys = "0.3"
web-time = "1"
web-sys = { version = "0.3", features = [
"AudioContext",
"AudioContextOptions",
"AudioContextState",
"AudioDestinationNode",
"AudioNode",
"AudioWorklet",
"AudioWorkletNode",
"AudioWorkletNodeOptions",
"BaseAudioContext",
"Blob",
"BlobPropertyBag",
"CssStyleDeclaration",
"DataTransfer",
"Document",
"DragEvent",
"Element",
"Event",
"EventTarget",
"File",
"FileList",
"FileSystemDirectoryHandle",
"FileSystemFileHandle",
"FileSystemGetDirectoryOptions",
"FileSystemGetFileOptions",
"FileSystemHandle",
"FileSystemHandleKind",
"FileSystemWritableFileStream",
"HtmlAnchorElement",
"HtmlCanvasElement",
"HtmlElement",
"HtmlInputElement",
"ImageData",
"Location",
"MessageEvent",
"MessagePort",
"Navigator",
"OffscreenCanvas",
"OffscreenCanvasRenderingContext2d",
"Performance",
"Response",
"StorageManager",
"Url",
"UrlSearchParams",
"Window",
"WritableStream",
] }
+10
View File
@@ -0,0 +1,10 @@
# filmcraft-web
FilmCraft in the browser (wasm32): the engine and egui UI on eframe's web runner, with browser
implementations of the host services (Blob media reads, downloads, OPFS auto-save and recovery,
WebAudio output, WebCodecs decoding) and the `window.filmcraft` agent API.
Build with `cargo xtask web [--serve PORT]`. Toolchain, architecture and the API: [docs/web.md](../../docs/web.md).
`web/` holds the static files copied next to the generated `filmcraft_web.js` / `.wasm`;
`tests/smoke.mjs` is a dependency-free headless-Chrome smoke test.
+218
View File
@@ -0,0 +1,218 @@
//! `window.filmcraft`: the control channel / agent API for the page.
//!
//! The same JSON methods as the desktop control channel (`docs/control-protocol.md`) and the
//! same engine commands as MCP, as promises:
//!
//! ```js
//! await filmcraft.execute("sequence.razor", {seconds: 2}) // engine command → its result
//! await filmcraft.request("ui.inspect") // control method → its result
//! await filmcraft.commands() // command list
//! const {pngBase64} = await filmcraft.screenshot() // the canvas as PNG (base64)
//! await filmcraft.importUrl("/clip.mp4") // fetch + import (tests)
//! await filmcraft.importFiles(input.files) // File / FileList / File[]
//! await filmcraft.openProject(file) // .fcproj File or URL
//! filmcraft.files(); await filmcraft.readFile("/exports/a.mp4"); filmcraft.info()
//! ```
//!
//! Results resolve with the method's `result`; errors reject with the error message.
use std::cell::RefCell;
use std::sync::mpsc::{Receiver, Sender, TryRecvError};
use filmcraft_ui_egui::ControlRequest;
use serde_json::{Value, json};
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::{JsFuture, future_to_promise};
type Waiting = (Receiver<Value>, js_sys::Function, js_sys::Function);
thread_local! {
static TX: RefCell<Option<Sender<ControlRequest>>> = const { RefCell::new(None) };
static WAITING: RefCell<Vec<Waiting>> = const { RefCell::new(Vec::new()) };
}
pub fn set_sender(tx: Sender<ControlRequest>) {
TX.with(|t| *t.borrow_mut() = Some(tx));
}
pub fn to_js(v: &Value) -> JsValue {
js_sys::JSON::parse(&v.to_string()).unwrap_or(JsValue::NULL)
}
pub fn from_js(v: &JsValue) -> Value {
if v.is_undefined() || v.is_null() {
return Value::Null;
}
js_sys::JSON::stringify(v).ok().and_then(|s| s.as_string()).and_then(|s| serde_json::from_str(&s).ok()).unwrap_or(Value::Null)
}
/// Resolve the promises of answered control requests (called every frame).
pub fn poll_replies() {
WAITING.with(|w| {
w.borrow_mut().retain(|(rx, resolve, reject)| match rx.try_recv() {
Ok(v) => {
if v.get("ok").and_then(Value::as_bool) == Some(true) {
let _ = resolve.call1(&JsValue::NULL, &to_js(v.get("result").unwrap_or(&Value::Null)));
} else {
let msg = v.get("error").and_then(Value::as_str).unwrap_or("error").to_string();
let _ = reject.call1(&JsValue::NULL, &js_sys::Error::new(&msg));
}
false
}
Err(TryRecvError::Empty) => true,
Err(TryRecvError::Disconnected) => {
let _ = reject.call1(&JsValue::NULL, &js_sys::Error::new("request dropped"));
false
}
});
});
}
/// Send a control request; the promise settles when the UI thread answers it.
pub fn request(method: &str, params: Value) -> js_sys::Promise {
let method = method.to_string();
let p = js_sys::Promise::new(&mut |resolve, reject| {
let (req, rx) = ControlRequest::new(method.clone(), params.clone());
let sent = TX.with(|t| t.borrow().as_ref().map(|tx| tx.send(req).is_ok()).unwrap_or(false));
if sent {
WAITING.with(|w| w.borrow_mut().push((rx, resolve, reject)));
} else {
let _ = reject.call1(&JsValue::NULL, &js_sys::Error::new("FilmCraft is not running yet"));
}
});
crate::repaint();
p
}
fn set_fn(obj: &js_sys::Object, name: &str, f: JsValue) -> Result<(), JsValue> {
js_sys::Reflect::set(obj, &name.into(), &f).map(|_| ())
}
fn files_of(v: &JsValue) -> Vec<web_sys::File> {
if let Some(f) = v.dyn_ref::<web_sys::File>() {
return vec![f.clone()];
}
if let Some(list) = v.dyn_ref::<web_sys::FileList>() {
return (0..list.length()).filter_map(|i| list.get(i)).collect();
}
if js_sys::Array::is_array(v) {
return js_sys::Array::from(v).iter().filter_map(|x| x.dyn_into::<web_sys::File>().ok()).collect();
}
Vec::new()
}
/// Fetch a URL into a `File` named after it (or `name`).
pub async fn fetch_file(url: &str, name: Option<String>) -> Result<web_sys::File, JsValue> {
let w = web_sys::window().ok_or("no window")?;
let resp: web_sys::Response = JsFuture::from(w.fetch_with_str(url)).await?.dyn_into()?;
if !resp.ok() {
return Err(format!("{url}: HTTP {}", resp.status()).into());
}
let blob: web_sys::Blob = JsFuture::from(resp.blob()?).await?.dyn_into()?;
let name = name.unwrap_or_else(|| url.split(['?', '#']).next().unwrap_or(url).rsplit('/').next().unwrap_or("media").to_string());
let parts = js_sys::Array::of1(&blob);
web_sys::File::new_with_blob_sequence(&parts, &name)
}
/// Install `window.filmcraft`.
pub fn install() -> Result<(), JsValue> {
let obj = js_sys::Object::new();
set_fn(
&obj,
"request",
Closure::<dyn Fn(String, JsValue) -> js_sys::Promise>::new(|method: String, params: JsValue| {
let p = from_js(&params);
request(&method, if p.is_null() { json!({}) } else { p })
})
.into_js_value(),
)?;
set_fn(
&obj,
"execute",
Closure::<dyn Fn(String, JsValue) -> js_sys::Promise>::new(|command: String, params: JsValue| {
let p = from_js(&params);
request("engine.execute", json!({"command": command, "params": if p.is_null() { json!({}) } else { p }}))
})
.into_js_value(),
)?;
set_fn(&obj, "commands", Closure::<dyn Fn() -> js_sys::Promise>::new(|| request("engine.commands", json!({}))).into_js_value())?;
set_fn(&obj, "inspect", Closure::<dyn Fn() -> js_sys::Promise>::new(|| request("ui.inspect", json!({}))).into_js_value())?;
set_fn(
&obj,
"screenshot",
Closure::<dyn Fn(JsValue) -> js_sys::Promise>::new(|params: JsValue| {
let p = from_js(&params);
request("ui.screenshot", if p.is_null() { json!({}) } else { p })
})
.into_js_value(),
)?;
set_fn(
&obj,
"info",
Closure::<dyn Fn() -> JsValue>::new(|| {
let mut v = crate::info();
v["webcodecsStats"] = crate::webcodecs::stats();
v["fetchesInFlight"] = json!(crate::fs::inflight());
to_js(&v)
})
.into_js_value(),
)?;
set_fn(
&obj,
"files",
Closure::<dyn Fn() -> JsValue>::new(|| {
to_js(&Value::Array(crate::fs::list().into_iter().map(|(p, n, k)| json!({"path": p, "size": n, "kind": k})).collect()))
})
.into_js_value(),
)?;
set_fn(
&obj,
"readFile",
Closure::<dyn Fn(String) -> js_sys::Promise>::new(|path: String| {
future_to_promise(async move {
let blob = crate::fs::blob(&path).ok_or_else(|| JsValue::from_str(&format!("{path}: not found")))?;
let buf = JsFuture::from(blob.array_buffer()).await?;
Ok(js_sys::Uint8Array::new(&buf).into())
})
})
.into_js_value(),
)?;
set_fn(
&obj,
"importFiles",
Closure::<dyn Fn(JsValue) -> js_sys::Promise>::new(|files: JsValue| {
let files = files_of(&files);
future_to_promise(async move { Ok(to_js(&crate::import::import_files(files).await)) })
})
.into_js_value(),
)?;
set_fn(
&obj,
"importUrl",
Closure::<dyn Fn(String, JsValue) -> js_sys::Promise>::new(|url: String, name: JsValue| {
future_to_promise(async move {
let f = fetch_file(&url, name.as_string()).await?;
Ok(to_js(&crate::import::import_files(vec![f]).await))
})
})
.into_js_value(),
)?;
set_fn(
&obj,
"openProject",
Closure::<dyn Fn(JsValue) -> js_sys::Promise>::new(|src: JsValue| {
future_to_promise(async move {
let file = match src.as_string() {
Some(url) => fetch_file(&url, None).await?,
None => files_of(&src).into_iter().next().ok_or("need a File or URL")?,
};
crate::import::open_project_file(file).await.map(|v| to_js(&v)).map_err(|e| JsValue::from(js_sys::Error::new(&e)))
})
})
.into_js_value(),
)?;
let w = web_sys::window().ok_or("no window")?;
js_sys::Reflect::set(&w, &"filmcraft".into(), &obj)?;
Ok(())
}
+203
View File
@@ -0,0 +1,203 @@
//! WebAudio output: an `AudioWorklet` plays blocks that the UI thread mixes ahead.
//!
//! The mixer runs on the UI thread (wasm has no audio thread of its own here): every frame
//! [`Handle::pump`] mixes until ~250 ms are queued and posts the blocks to the worklet
//! (`web/audio-worklet.js`). The worklet counts the frames it actually played and reports them
//! with its `currentTime`; [`AudioOut::played_frames`] extrapolates from the last report on the
//! context's clock and never runs past what was queued. So, as on the desktop, playback follows
//! the audio clock: if the worklet starves, the playhead waits for it.
use std::cell::RefCell;
use std::rc::Rc;
use filmcraft_ui_egui::AudioOut;
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;
/// Seconds of audio kept queued ahead of the worklet.
const AHEAD_S: f64 = 0.25;
/// Frames per posted block.
const BLOCK: usize = 2048;
struct State {
ctx: web_sys::AudioContext,
node: Option<web_sys::AudioWorkletNode>,
fill: Option<Box<dyn FnMut(&mut [f32], usize) + Send>>,
generation: u32,
/// Frames posted since `start`.
queued: u64,
/// Last report from the worklet: frames played and its `currentTime`.
played: u64,
played_at: f64,
reported: bool,
}
/// Shared handle: one is the app's [`AudioOut`], one is pumped by the host every frame.
#[derive(Clone)]
pub struct Handle(Rc<RefCell<State>>);
impl Handle {
/// Create the audio context and load the worklet (asynchronously). None without WebAudio.
pub fn new() -> Option<Self> {
let opts = web_sys::AudioContextOptions::new();
opts.set_latency_hint(&"interactive".into());
let ctx = web_sys::AudioContext::new_with_context_options(&opts).ok()?;
let h = Handle(Rc::new(RefCell::new(State {
ctx: ctx.clone(),
node: None,
fill: None,
generation: 0,
queued: 0,
played: 0,
played_at: 0.0,
reported: false,
})));
let weak = Rc::downgrade(&h.0);
wasm_bindgen_futures::spawn_local(async move {
let r: Result<web_sys::AudioWorkletNode, JsValue> = async {
JsFuture::from(ctx.audio_worklet()?.add_module("audio-worklet.js")?).await?;
let o = web_sys::AudioWorkletNodeOptions::new();
o.set_number_of_inputs(0);
o.set_number_of_outputs(1);
o.set_output_channel_count(&js_sys::Array::of1(&2.into()));
let node = web_sys::AudioWorkletNode::new_with_options(&ctx, "filmcraft-output", &o)?;
node.connect_with_audio_node(&ctx.destination())?;
Ok(node)
}
.await;
let Some(st) = weak.upgrade() else { return };
match r {
Ok(node) => {
let w = Rc::downgrade(&st);
let on_msg = Closure::<dyn FnMut(web_sys::MessageEvent)>::new(move |e: web_sys::MessageEvent| {
let Some(st) = w.upgrade() else { return };
let d = e.data();
let num = |k: &str| js_sys::Reflect::get(&d, &k.into()).ok().and_then(|v| v.as_f64()).unwrap_or(0.0);
let mut s = st.borrow_mut();
if num("gen") as u32 == s.generation {
s.played = num("played") as u64;
s.played_at = num("time");
s.reported = true;
}
});
if let Ok(port) = node.port() {
port.set_onmessage(Some(on_msg.as_ref().unchecked_ref()));
}
on_msg.forget();
st.borrow_mut().node = Some(node);
crate::set_info("audioWorklet", serde_json::json!(true));
}
Err(e) => {
log::warn!("audio worklet unavailable: {e:?}");
crate::set_info("audioWorklet", serde_json::json!(false));
}
}
});
// Browsers start audio contexts suspended until the user interacts with the page.
if let Some(doc) = web_sys::window().and_then(|w| w.document()) {
let c = h.0.borrow().ctx.clone();
let resume = Closure::<dyn FnMut()>::new(move || {
if c.state() != web_sys::AudioContextState::Running {
let _ = c.resume();
}
});
for ev in ["pointerdown", "keydown"] {
let _ = doc.add_event_listener_with_callback_and_bool(ev, resume.as_ref().unchecked_ref(), true);
}
resume.forget();
}
Some(h)
}
fn post(node: &web_sys::AudioWorkletNode, msg: &JsValue, transfer: Option<&JsValue>) {
if let Ok(port) = node.port() {
let _ = match transfer {
Some(t) => port.post_message_with_transferable(msg, &js_sys::Array::of1(t)),
None => port.post_message(msg),
};
}
}
fn estimate(s: &State) -> u64 {
if !s.reported {
return 0;
}
let sr = s.ctx.sample_rate() as f64;
let ahead = ((s.ctx.current_time() - s.played_at).max(0.0) * sr) as u64;
(s.played + ahead).min(s.queued)
}
/// Mix and post audio until [`AHEAD_S`] is queued (called every frame).
pub fn pump(&self) {
let mut s = self.0.borrow_mut();
let Some(node) = s.node.clone() else { return };
if s.fill.is_none() {
return;
}
let target = (AHEAD_S * s.ctx.sample_rate() as f64) as u64;
let mut guard = 0;
while s.queued.saturating_sub(Self::estimate(&s)) < target && guard < 16 {
guard += 1;
let mut buf = vec![0f32; BLOCK * 2];
if let Some(f) = s.fill.as_mut() {
f(&mut buf, 2);
}
let arr = js_sys::Float32Array::from(&buf[..]);
let msg = js_sys::Object::new();
let _ = js_sys::Reflect::set(&msg, &"type".into(), &"data".into());
let _ = js_sys::Reflect::set(&msg, &"gen".into(), &s.generation.into());
let _ = js_sys::Reflect::set(&msg, &"samples".into(), &arr);
Self::post(&node, &msg, Some(&arr.buffer()));
s.queued += BLOCK as u64;
}
}
fn reset(s: &mut State) {
s.generation = s.generation.wrapping_add(1);
s.queued = 0;
s.played = 0;
s.reported = false;
if let Some(node) = &s.node {
let msg = js_sys::Object::new();
let _ = js_sys::Reflect::set(&msg, &"type".into(), &"reset".into());
let _ = js_sys::Reflect::set(&msg, &"gen".into(), &s.generation.into());
Self::post(node, &msg, None);
}
}
}
impl AudioOut for Handle {
fn start(&mut self, fill: Box<dyn FnMut(&mut [f32], usize) + Send>) -> Result<u32, String> {
{
let mut s = self.0.borrow_mut();
if s.node.is_none() {
return Err("audio worklet not loaded".into());
}
if s.ctx.state() != web_sys::AudioContextState::Running {
let _ = s.ctx.resume();
// without a user gesture the context stays suspended: use the wall clock
return Err("audio context suspended (no user gesture yet)".into());
}
Self::reset(&mut s);
s.fill = Some(fill);
}
self.pump();
Ok(self.sample_rate())
}
fn stop(&mut self) {
let mut s = self.0.borrow_mut();
s.fill = None;
Self::reset(&mut s);
}
fn sample_rate(&self) -> u32 {
self.0.borrow().ctx.sample_rate() as u32
}
fn played_frames(&self) -> Option<u64> {
let s = self.0.borrow();
s.fill.as_ref().map(|_| Self::estimate(&s))
}
}
+391
View File
@@ -0,0 +1,391 @@
//! Browser-backed host services: a virtual file table and the engine's [`Services`].
//!
//! - Media the user opens (`<input type=file>`, drag-and-drop, File System Access pickers, files
//! kept in OPFS) are registered as `File`/`Blob` entries under a virtual path (`/files/<name>`).
//! They are never copied whole: [`BlobReader`] serves the demuxers' range reads from a cache of
//! 1 MiB chunks (LRU, [`CACHE_BUDGET`]), fetched asynchronously with `Blob.slice().arrayBuffer()`.
//! A read whose chunks are not cached yet starts the fetch (plus read-ahead) and fails with
//! `WouldBlock`, marking [`filmcraft_media::pending`]: the frame server and imports retry once
//! the bytes have arrived (the repaint hook wakes the UI).
//! - Files the app writes (saved projects, exports, captions) are kept in memory under their
//! path and offered as a browser download; paths under `/opfs/` are written to the Origin
//! Private File System instead (auto-save / crash recovery).
use std::cell::RefCell;
use std::collections::{HashMap, HashSet};
use std::io;
use std::rc::Rc;
use std::sync::Arc;
use filmcraft_engine::Services;
use filmcraft_media::reader::{ByteReader, MemReader, SharedReader};
use wasm_bindgen::JsCast;
use wasm_bindgen_futures::JsFuture;
/// Chunk size of Blob range reads.
pub const CHUNK: u64 = 1 << 20;
/// Chunks fetched past the end of a read (sequential playback, demuxer scans).
const READ_AHEAD: u64 = 3;
/// Bytes of cached chunks kept (least recently used evicted).
pub const CACHE_BUDGET: usize = 384 << 20;
/// Largest Blob `read_file` will load whole (captions, interchange documents, projects).
const WHOLE_FILE_MAX: u64 = 256 << 20;
enum Entry {
Blob { id: u32, size: u64 },
Mem(Arc<[u8]>),
}
#[derive(Default)]
struct Fs {
entries: HashMap<String, Entry>,
blobs: HashMap<u32, web_sys::Blob>,
next_id: u32,
chunks: HashMap<(u32, u64), (Arc<[u8]>, u64)>,
bytes: usize,
clock: u64,
inflight: HashSet<(u32, u64)>,
failed: HashMap<(u32, u64), String>,
/// Called when fetched bytes arrive (repaint the UI so pending frames are retried).
on_data: Option<Rc<dyn Fn()>>,
/// Called after a file was written (path, bytes): the app offers it as a download.
on_write: Option<Rc<dyn Fn(&str, &[u8])>>,
}
thread_local! {
static FS: RefCell<Fs> = RefCell::new(Fs::default());
}
/// Wake-up hook for arriving data.
pub fn set_on_data(f: impl Fn() + 'static) {
FS.with(|fs| fs.borrow_mut().on_data = Some(Rc::new(f)));
}
/// Hook for written files (downloads).
pub fn set_on_write(f: impl Fn(&str, &[u8]) + 'static) {
FS.with(|fs| fs.borrow_mut().on_write = Some(Rc::new(f)));
}
fn file_name(path: &str) -> &str {
path.rsplit('/').next().unwrap_or(path)
}
/// Register a browser `File` (or `Blob` with a name) and return its virtual path. A file with
/// the same name and size as an existing entry replaces it (the same file picked again keeps
/// its path, so projects referencing it come back online); otherwise a numbered name is used.
pub fn register_blob(name: &str, blob: web_sys::Blob) -> String {
let size = blob.size() as u64;
let clean: String = name.chars().map(|c| if c == '/' || c == '\\' { '_' } else { c }).collect();
FS.with(|fs| {
let mut fs = fs.borrow_mut();
let (stem, ext) = match clean.rfind('.') {
Some(i) if i > 0 => (clean[..i].to_string(), clean[i..].to_string()),
_ => (clean.clone(), String::new()),
};
let mut n = 1;
let path = loop {
let p = if n == 1 { format!("/files/{clean}") } else { format!("/files/{stem} ({n}){ext}") };
match fs.entries.get(&p) {
None => break p,
Some(Entry::Blob { size: s, .. }) if *s == size => break p,
Some(Entry::Mem(b)) if b.len() as u64 == size => break p,
_ => n += 1,
}
};
if let Some(Entry::Blob { id, .. }) = fs.entries.get(&path) {
let id = *id;
fs.blobs.remove(&id);
fs.chunks.retain(|k, _| k.0 != id);
}
let id = fs.next_id;
fs.next_id += 1;
fs.blobs.insert(id, blob);
fs.entries.insert(path.clone(), Entry::Blob { id, size });
fs.recount();
path
})
}
/// Put bytes at a path (no download).
pub fn put(path: &str, data: Vec<u8>) {
FS.with(|fs| fs.borrow_mut().entries.insert(path.to_string(), Entry::Mem(data.into())));
}
/// (path, size, kind) of every file.
pub fn list() -> Vec<(String, u64, &'static str)> {
FS.with(|fs| {
let fs = fs.borrow();
let mut v: Vec<_> = fs
.entries
.iter()
.map(|(p, e)| match e {
Entry::Blob { size, .. } => (p.clone(), *size, "blob"),
Entry::Mem(b) => (p.clone(), b.len() as u64, "memory"),
})
.collect();
v.sort();
v
})
}
/// The browser `Blob` behind a path (in-memory files are wrapped in a new Blob).
pub fn blob(path: &str) -> Option<web_sys::Blob> {
FS.with(|fs| {
let fs = fs.borrow();
match fs.entries.get(path)? {
Entry::Blob { id, .. } => fs.blobs.get(id).cloned(),
Entry::Mem(b) => bytes_to_blob(b, "application/octet-stream").ok(),
}
})
}
pub fn bytes_to_blob(b: &[u8], mime: &str) -> Result<web_sys::Blob, wasm_bindgen::JsValue> {
let arr = js_sys::Array::new();
arr.push(&js_sys::Uint8Array::from(b));
let opts = web_sys::BlobPropertyBag::new();
opts.set_type(mime);
web_sys::Blob::new_with_u8_array_sequence_and_options(&arr, &opts)
}
/// Fetches still running.
pub fn inflight() -> usize {
FS.with(|fs| fs.borrow().inflight.len())
}
/// Wait until no fetch is running.
pub async fn idle() {
while inflight() > 0 {
sleep(4).await;
}
}
/// Resolve after `ms` milliseconds.
pub async fn sleep(ms: i32) {
let p = js_sys::Promise::new(&mut |resolve, _| {
if let Some(w) = web_sys::window() {
let _ = w.set_timeout_with_callback_and_timeout_and_arguments_0(&resolve, ms);
}
});
let _ = JsFuture::from(p).await;
}
/// Load `[a, b)` of a registered Blob into the chunk cache (import prewarm: container index).
pub async fn prefetch(path: &str, a: u64, b: u64) {
let Some((id, size)) = FS.with(|fs| match fs.borrow().entries.get(path) {
Some(Entry::Blob { id, size }) => Some((*id, *size)),
_ => None,
}) else {
return;
};
let (c0, c1) = (a / CHUNK, b.min(size).div_ceil(CHUNK));
for c in c0..c1 {
request(id, c);
// keep a bounded number of reads in flight
while inflight() >= 8 {
sleep(2).await;
}
}
idle().await;
}
impl Fs {
fn recount(&mut self) {
self.bytes = self.chunks.values().map(|(d, _)| d.len()).sum();
}
fn evict(&mut self) {
if self.bytes <= CACHE_BUDGET {
return;
}
let mut all: Vec<((u32, u64), u64, usize)> = self.chunks.iter().map(|(k, (d, t))| (*k, *t, d.len())).collect();
all.sort_unstable_by_key(|x| x.1);
for (k, _, n) in all {
if self.bytes <= CACHE_BUDGET * 9 / 10 {
break;
}
self.chunks.remove(&k);
self.bytes -= n;
}
}
}
/// Start fetching chunk `c` of blob `id` unless it is cached or already on its way.
fn request(id: u32, c: u64) {
let blob = FS.with(|fs| {
let mut fs = fs.borrow_mut();
if fs.chunks.contains_key(&(id, c)) || fs.inflight.contains(&(id, c)) {
return None;
}
let blob = fs.blobs.get(&id)?.clone();
fs.failed.remove(&(id, c));
fs.inflight.insert((id, c));
Some(blob)
});
let Some(blob) = blob else { return };
let size = blob.size() as u64;
let (a, b) = (c * CHUNK, ((c + 1) * CHUNK).min(size));
let fut = blob.slice_with_f64_and_f64(a as f64, b as f64).map(|s| JsFuture::from(s.array_buffer()));
wasm_bindgen_futures::spawn_local(async move {
let r = match fut {
Ok(f) => f.await.map(|buf| js_sys::Uint8Array::new(&buf).to_vec()),
Err(e) => Err(e),
};
let wake = FS.with(|fs| {
let mut fs = fs.borrow_mut();
fs.inflight.remove(&(id, c));
// a blob replaced or dropped meanwhile: discard
if fs.blobs.contains_key(&id) {
match r {
Ok(v) => {
fs.clock += 1;
let t = fs.clock;
fs.bytes += v.len();
fs.chunks.insert((id, c), (v.into(), t));
fs.evict();
}
Err(e) => {
fs.failed.insert((id, c), format!("{e:?}"));
}
}
}
fs.on_data.clone()
});
if let Some(w) = wake {
w();
}
});
}
/// Random-access reads of a registered Blob through the chunk cache.
pub struct BlobReader {
id: u32,
size: u64,
}
impl ByteReader for BlobReader {
fn len(&self) -> u64 {
self.size
}
fn read_at(&self, offset: u64, buf: &mut [u8]) -> io::Result<()> {
if buf.is_empty() {
return Ok(());
}
let end = offset
.checked_add(buf.len() as u64)
.filter(|e| *e <= self.size)
.ok_or_else(|| io::Error::new(io::ErrorKind::UnexpectedEof, "read past end of file"))?;
let (c0, c1) = (offset / CHUNK, (end - 1) / CHUNK);
let last = (self.size.max(1) - 1) / CHUNK;
let missing: Vec<u64> = FS.with(|fs| {
let fs = fs.borrow();
(c0..=c1).filter(|c| !fs.chunks.contains_key(&(self.id, *c))).collect()
});
if let Some(e) = FS.with(|fs| missing.iter().find_map(|c| fs.borrow().failed.get(&(self.id, *c)).cloned())) {
return Err(io::Error::other(format!("file read failed: {e}")));
}
if !missing.is_empty() {
for c in missing.iter().copied().chain(c1 + 1..=(c1 + READ_AHEAD).min(last)) {
request(self.id, c);
}
return Err(filmcraft_media::pending::would_block());
}
FS.with(|fs| {
let mut fs = fs.borrow_mut();
fs.clock += 1;
let t = fs.clock;
let mut pos = 0usize;
for c in c0..=c1 {
let (d, used) = fs.chunks.get_mut(&(self.id, c)).expect("checked above");
*used = t;
let a = if c == c0 { (offset - c * CHUNK) as usize } else { 0 };
let n = (d.len() - a).min(buf.len() - pos);
buf[pos..pos + n].copy_from_slice(&d[a..a + n]);
pos += n;
}
});
// read-ahead for sequential access
for c in c1 + 1..=(c1 + READ_AHEAD).min(last) {
request(self.id, c);
}
Ok(())
}
}
fn not_found(path: &str) -> io::Error {
io::Error::new(io::ErrorKind::NotFound, format!("{path}: not found (files from an earlier browser session must be imported again)"))
}
/// The engine's host services on the web.
pub struct WebServices;
impl WebServices {
fn entry_reader(path: &str) -> io::Result<SharedReader> {
FS.with(|fs| match fs.borrow().entries.get(path) {
Some(Entry::Blob { id, size }) => Ok(Arc::new(BlobReader { id: *id, size: *size }) as SharedReader),
Some(Entry::Mem(b)) => Ok(Arc::new(MemReader(b.clone())) as SharedReader),
None => Err(not_found(path)),
})
}
}
impl Services for WebServices {
fn read_file(&self, path: &str) -> io::Result<Vec<u8>> {
let r = Self::entry_reader(path)?;
if r.len() > WHOLE_FILE_MAX {
return Err(io::Error::other(format!("{path}: too large to read whole in the browser")));
}
filmcraft_media::reader::read_range(&*r, 0, r.len() as usize)
}
fn write_file(&self, path: &str, data: &[u8]) -> io::Result<()> {
if let Some(rel) = path.strip_prefix("/opfs/") {
crate::opfs::write(rel.to_string(), data.to_vec());
}
let hook = FS.with(|fs| {
let mut fs = fs.borrow_mut();
fs.entries.insert(path.to_string(), Entry::Mem(Arc::from(data)));
fs.on_write.clone()
});
if !path.starts_with("/opfs/")
&& let Some(h) = hook
{
h(path, data);
}
Ok(())
}
fn file_size(&self, path: &str) -> io::Result<u64> {
Self::entry_reader(path).map(|r| r.len())
}
fn read_range(&self, path: &str, offset: u64, len: usize) -> io::Result<Vec<u8>> {
filmcraft_media::reader::read_range(&*Self::entry_reader(path)?, offset, len)
}
fn reader(&self, path: &str) -> Option<io::Result<SharedReader>> {
Some(Self::entry_reader(path))
}
fn export_in_memory(&self) -> bool {
true
}
}
/// Offer bytes as a browser download named `name`.
pub fn download(name: &str, data: &[u8]) -> Result<(), wasm_bindgen::JsValue> {
let doc = web_sys::window().and_then(|w| w.document()).ok_or("no document")?;
let blob = bytes_to_blob(data, "application/octet-stream")?;
let url = web_sys::Url::create_object_url_with_blob(&blob)?;
let a: web_sys::HtmlAnchorElement = doc.create_element("a")?.dyn_into()?;
a.set_href(&url);
a.set_download(file_name(name));
a.style().set_property("display", "none")?;
doc.body().ok_or("no body")?.append_child(&a)?;
a.click();
a.remove();
wasm_bindgen_futures::spawn_local(async move {
sleep(60_000).await;
let _ = web_sys::Url::revoke_object_url(&url);
});
Ok(())
}
+234
View File
@@ -0,0 +1,234 @@
//! Getting files in and out: pickers (File System Access API where available, else
//! `<input type=file>`), drag-and-drop, imports, project open/save.
//!
//! An import registers each `File` in [`crate::fs`] (no copy), prefetches the container index
//! (an MP4's `moov`, a small Matroska file whole, the head and tail of a large one), then runs
//! the engine's `file.import`. A read that has to wait for the browser makes the import retry
//! once the bytes are there, so imports never block the UI thread.
use std::cell::RefCell;
use std::rc::Rc;
use filmcraft_engine::Services;
use filmcraft_ui_egui::FilmcraftApp;
use serde_json::{Value, json};
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;
use crate::fs;
/// Run `f` on the UI thread (next frame) and wait for its result.
pub async fn on_ui<T: 'static>(f: impl FnOnce(&mut crate::WebApp, &egui::Context) -> T + 'static) -> T {
let slot: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None));
let s = slot.clone();
crate::post(move |w, c| *s.borrow_mut() = Some(f(w, c)));
loop {
if let Some(v) = slot.borrow_mut().take() {
return v;
}
fs::sleep(5).await;
}
}
fn be32(b: &[u8]) -> u64 {
u32::from_be_bytes([b[0], b[1], b[2], b[3]]) as u64
}
/// Load what opening the file will read first: the container index.
async fn prewarm(path: &str, size: u64) {
let head_len = size.min(64 << 10);
fs::prefetch(path, 0, head_len).await;
let Ok(head) = fs::WebServices.read_range(path, 0, head_len as usize) else { return };
if filmcraft_codecs::mp4::sniff(&head) {
// walk the top-level boxes to the `moov`
let mut off = 0u64;
for _ in 0..256 {
if off + 8 > size {
break;
}
fs::prefetch(path, off, off + 16).await;
let Ok(h) = fs::WebServices.read_range(path, off, 16) else { break };
if h.len() < 8 {
break;
}
let len = match be32(&h) {
1 if h.len() >= 16 => (be32(&h[8..]) << 32) | be32(&h[12..]),
0 => size - off,
n => n,
};
if &h[4..8] == b"moov" {
fs::prefetch(path, off, off + len).await;
break;
}
if len < 8 {
break;
}
off += len;
}
} else if head.starts_with(&[0x1A, 0x45, 0xDF, 0xA3]) {
// Matroska: opening scans every cluster header
if size <= 192 << 20 {
fs::prefetch(path, 0, size).await;
} else {
fs::prefetch(path, 0, 8 << 20).await;
fs::prefetch(path, size.saturating_sub(8 << 20), size).await;
}
} else if size <= 256 << 20 {
// stills, WAV, compressed audio, captions: decoded/parsed whole
fs::prefetch(path, 0, size).await;
}
}
/// `file.import` one path, retried while its bytes are loading.
async fn import_path(path: String) -> Result<Value, String> {
for _ in 0..2000 {
let p = path.clone();
let (r, pending) = on_ui(move |w, _| {
let _ = filmcraft_media::pending::take();
let r = w.app.session.execute("file.import", json!({"paths": [p]})).map_err(|e| e.to_string());
(r, filmcraft_media::pending::take())
})
.await;
let failed = match &r {
Ok(v) => v.get("errors").and_then(Value::as_array).is_some_and(|e| !e.is_empty()),
Err(_) => true,
};
if failed && pending {
fs::idle().await;
continue;
}
return r;
}
Err(format!("{path}: timed out loading the file"))
}
/// Import browser files (no copies): returns `{items, errors, paths}`.
pub async fn import_files(files: Vec<web_sys::File>) -> Value {
let mut items = Vec::new();
let mut errors = Vec::new();
let mut paths = Vec::new();
for f in files {
let name = f.name();
let size = f.size() as u64;
let path = fs::register_blob(&name, f.clone().into());
prewarm(&path, size).await;
match import_path(path.clone()).await {
Ok(v) => {
items.extend(v.get("items").and_then(Value::as_array).cloned().unwrap_or_default());
errors.extend(v.get("errors").and_then(Value::as_array).cloned().unwrap_or_default());
crate::recovery::keep_media(&path, &f);
}
Err(e) => errors.push(json!(e)),
}
paths.push(json!(path));
}
if !errors.is_empty() {
let msg = errors.iter().filter_map(Value::as_str).collect::<Vec<_>>().join("; ");
crate::post(move |w, _| w.app.status(msg));
}
json!({"items": items, "errors": errors, "paths": paths})
}
/// Open a `.fcproj` file picked or dropped by the user.
pub async fn open_project_file(file: web_sys::File) -> Result<Value, String> {
let buf = JsFuture::from(file.array_buffer()).await.map_err(|e| format!("{e:?}"))?;
let path = format!("/projects/{}", file.name());
fs::put(&path, js_sys::Uint8Array::new(&buf).to_vec());
on_ui(move |w, _| w.app.session.execute("file.open", json!({"path": path})).map_err(|e| e.to_string())).await
}
/// Handle files from a picker or a drop: projects open, everything else is imported.
pub fn take_files(files: Vec<web_sys::File>) {
wasm_bindgen_futures::spawn_local(async move {
let (projects, media): (Vec<_>, Vec<_>) = files.into_iter().partition(|f| f.name().to_ascii_lowercase().ends_with(".fcproj"));
for p in projects {
if let Err(e) = open_project_file(p).await {
crate::post(move |w, _| w.app.status(e));
}
}
if !media.is_empty() {
import_files(media).await;
}
});
}
fn has_fsa_picker() -> bool {
web_sys::window().is_some_and(|w| js_sys::Reflect::has(&w, &"showOpenFilePicker".into()).unwrap_or(false))
}
/// Let the user pick files: `showOpenFilePicker` when available, else a file input.
pub fn pick(exts: &[&str], multiple: bool) {
let accept: Vec<String> = exts.iter().map(|e| format!(".{e}")).collect();
if has_fsa_picker() {
let opts = js_sys::Object::new();
let _ = js_sys::Reflect::set(&opts, &"multiple".into(), &multiple.into());
wasm_bindgen_futures::spawn_local(async move {
let Some(w) = web_sys::window() else { return };
let Ok(f) = js_sys::Reflect::get(&w, &"showOpenFilePicker".into()).and_then(|f| f.dyn_into::<js_sys::Function>()) else { return };
let Ok(p) = f.call1(&w, &opts).and_then(|p| p.dyn_into::<js_sys::Promise>()) else { return };
// rejected when the user cancels
let Ok(handles) = JsFuture::from(p).await else { return };
let mut files = Vec::new();
for h in js_sys::Array::from(&handles).iter() {
if let Ok(h) = h.dyn_into::<web_sys::FileSystemFileHandle>()
&& let Ok(f) = JsFuture::from(h.get_file()).await
&& let Ok(f) = f.dyn_into::<web_sys::File>()
{
files.push(f);
}
}
take_files(files);
});
return;
}
let Some(doc) = web_sys::window().and_then(|w| w.document()) else { return };
let Ok(input) = doc.create_element("input").and_then(|e| e.dyn_into::<web_sys::HtmlInputElement>().map_err(Into::into)) else { return };
input.set_type("file");
input.set_multiple(multiple);
input.set_accept(&accept.join(","));
let inp = input.clone();
let on_change = Closure::<dyn FnMut()>::new(move || {
if let Some(list) = inp.files() {
take_files((0..list.length()).filter_map(|i| list.get(i)).collect());
}
});
input.set_onchange(Some(on_change.as_ref().unchecked_ref()));
on_change.forget();
input.click();
}
/// Drop files anywhere on the page to import them (or open a project).
pub fn install_drop_handlers() -> Result<(), JsValue> {
let w = web_sys::window().ok_or("no window")?;
let over = Closure::<dyn FnMut(web_sys::DragEvent)>::new(|e: web_sys::DragEvent| e.prevent_default());
w.add_event_listener_with_callback_and_bool("dragover", over.as_ref().unchecked_ref(), true)?;
over.forget();
let drop = Closure::<dyn FnMut(web_sys::DragEvent)>::new(|e: web_sys::DragEvent| {
e.prevent_default();
e.stop_propagation();
if let Some(list) = e.data_transfer().and_then(|d| d.files()) {
take_files((0..list.length()).filter_map(|i| list.get(i)).collect());
}
});
w.add_event_listener_with_callback_and_bool("drop", drop.as_ref().unchecked_ref(), true)?;
drop.forget();
Ok(())
}
/// Wire the UI's file dialogs to the browser.
pub fn install_hooks(app: &mut FilmcraftApp) {
// Pickers are asynchronous: the dialog hook starts the picker and returns nothing; the files
// are imported when the user has chosen them.
app.hooks.pick_files = Some(Box::new(|exts: &[&str]| {
pick(exts, true);
Vec::new()
}));
app.hooks.pick_open_project = Some(Box::new(|| {
pick(&["fcproj"], false);
None
}));
// Saving writes the file in memory and offers it as a download (`fs::WebServices`).
app.hooks.pick_save = Some(Box::new(|name: &str| Some(format!("/projects/{name}"))));
app.hooks.pick_save_as = Some(Box::new(|_filter: &str, _exts: &[&str], name: &str| Some(format!("/exports/{name}"))));
}
+208
View File
@@ -0,0 +1,208 @@
//! FilmCraft in the browser.
//!
//! The same engine and egui UI as the desktop app, compiled to `wasm32-unknown-unknown` and run
//! by eframe's web runner on WebGPU (WebGL2 fallback). What the desktop shell gets from the OS,
//! this crate gets from the browser:
//!
//! | Desktop | Web ([module]) |
//! |---|---|
//! | file system ([`filmcraft_engine::FsServices`]) | virtual file table of `File`/`Blob`s with chunked range reads, downloads ([`fs`]) |
//! | file dialogs (rfd) | `<input type=file>` / File System Access pickers, drag-and-drop ([`import`]) |
//! | auto-save + recovery journal (data dir) | OPFS snapshot + media copies ([`recovery`], [`opfs`]) |
//! | cpal audio output | WebAudio `AudioWorklet`, played-frame clock ([`audio`]) |
//! | VideoToolbox… | WebCodecs `VideoDecoder` ([`webcodecs`]) |
//! | frame worker threads | cooperative frame queue pumped between UI frames (`FrameServer::pump`) |
//! | TCP control channel / MCP | `window.filmcraft` JavaScript API ([`api`]) |
//!
//! Everything here is `wasm32`-only; on other targets the crate is empty.
#![cfg(target_arch = "wasm32")]
pub mod api;
pub mod audio;
pub mod fs;
pub mod import;
pub mod opfs;
pub mod recovery;
pub mod webcodecs;
use std::cell::RefCell;
use std::sync::Arc;
use filmcraft_engine::Session;
use filmcraft_ui_egui::FilmcraftApp;
use serde_json::json;
use wasm_bindgen::prelude::*;
/// Work for the UI thread with access to the app (posted by async tasks and the JS API).
pub type Action = Box<dyn FnOnce(&mut WebApp, &egui::Context)>;
thread_local! {
static ACTIONS: RefCell<Vec<Action>> = const { RefCell::new(Vec::new()) };
static CTX: RefCell<Option<egui::Context>> = const { RefCell::new(None) };
static INFO: RefCell<serde_json::Value> = const { RefCell::new(serde_json::Value::Null) };
}
/// Run `f` on the UI thread at the next frame.
pub fn post(f: impl FnOnce(&mut WebApp, &egui::Context) + 'static) {
ACTIONS.with(|a| a.borrow_mut().push(Box::new(f)));
repaint();
}
/// Ask for a UI frame.
pub fn repaint() {
CTX.with(|c| {
if let Some(c) = c.borrow().as_ref() {
c.request_repaint();
}
});
}
/// Environment facts reported by `filmcraft.info()` (backend, isolation, codecs…).
pub fn set_info(key: &str, v: serde_json::Value) {
INFO.with(|i| {
let mut i = i.borrow_mut();
if !i.is_object() {
*i = json!({});
}
i[key] = v;
});
}
pub fn info() -> serde_json::Value {
INFO.with(|i| i.borrow().clone())
}
/// The eframe app: the shared FilmCraft UI plus the web host's per-frame duties.
pub struct WebApp {
pub app: FilmcraftApp,
pub audio: Option<audio::Handle>,
pub autosave: recovery::Autosave,
}
impl eframe::App for WebApp {
fn raw_input_hook(&mut self, ctx: &egui::Context, raw_input: &mut egui::RawInput) {
self.app.raw_input_hook(ctx, raw_input);
}
fn logic(&mut self, ctx: &egui::Context, frame: &mut eframe::Frame) {
let actions = ACTIONS.with(|a| std::mem::take(&mut *a.borrow_mut()));
for f in actions {
f(self, ctx);
}
self.app.logic(ctx, frame);
api::poll_replies();
// exports advance between frames (no threads)
if self.app.session.pump_jobs(std::time::Duration::from_millis(30)) {
ctx.request_repaint();
}
if let Some(a) = &self.audio {
a.pump();
}
self.autosave.tick(&self.app.session, ctx);
}
fn ui(&mut self, ui: &mut egui::Ui, frame: &mut eframe::Frame) {
self.app.ui(ui, frame);
}
}
fn query_flag(name: &str) -> bool {
web_sys::window().and_then(|w| w.location().search().ok()).and_then(|s| web_sys::UrlSearchParams::new_with_str(&s).ok()).is_some_and(|p| p.has(name))
}
/// Entry point (called by the page's bootstrap script once the wasm module is instantiated).
#[wasm_bindgen]
pub async fn start(canvas_id: String) -> Result<(), JsValue> {
eframe::WebLogger::init(log::LevelFilter::Info).ok();
let t0 = web_time::Instant::now();
let doc = web_sys::window().and_then(|w| w.document()).ok_or("no document")?;
let canvas: web_sys::HtmlCanvasElement = doc.get_element_by_id(&canvas_id).ok_or("no canvas")?.dyn_into()?;
let isolated = js_sys::Reflect::get(&js_sys::global(), &"crossOriginIsolated".into()).ok().and_then(|v| v.as_bool()).unwrap_or(false);
// Frame rendering runs cooperatively on this thread. wasm threads need a cross-origin
// isolated page *and* a build with atomics (nightly `build-std`); this build has none.
set_info("crossOriginIsolated", json!(isolated));
set_info("threads", json!(false));
set_info("frameWorkers", json!("cooperative"));
set_info("opfs", json!(opfs::available()));
// Media kept in OPFS by earlier sessions come back under their paths (recovery).
let restored = if opfs::available() && !query_flag("fresh") { recovery::restore_media().await } else { 0 };
let recovered = if query_flag("fresh") || query_flag("norecover") { None } else { recovery::load_snapshot().await };
webcodecs::probe().await;
api::install()?;
import::install_drop_handlers()?;
let demo = !query_flag("empty");
let web_options = eframe::WebOptions::default();
let runner = eframe::WebRunner::new();
runner
.start(
canvas,
web_options,
Box::new(move |cc| {
CTX.with(|c| *c.borrow_mut() = Some(cc.egui_ctx.clone()));
let ctx = cc.egui_ctx.clone();
fs::set_on_data(move || ctx.request_repaint());
fs::set_on_write(|path, data| {
if let Err(e) = fs::download(path, data) {
log::warn!("download {path}: {e:?}");
}
});
webcodecs::install();
let mut session = Session::new(Arc::new(fs::WebServices));
let mut toast = None;
if let Some((name, bytes)) = recovered {
let path = format!("/recovered/{name}");
fs::put(&path, bytes);
match session.execute("file.open", json!({"path": path})) {
Ok(_) => {
// the recovered changes are unsaved
session.saved_revision = 0;
toast = Some(format!("Recovered unsaved changes to {name}"));
}
Err(e) => log::warn!("recovery failed: {e}"),
}
} else if demo {
let _ = session.execute("file.openDemoProject", json!({}));
}
let mut app = FilmcraftApp::new(session);
import::install_hooks(&mut app);
let (tx, rx) = std::sync::mpsc::channel();
api::set_sender(tx);
app = app.with_control(rx);
if let Some(t) = toast {
app.status(t);
}
let backend = cc.wgpu_render_state.as_ref().map(|rs| rs.adapter.get_info().backend);
set_info("backend", json!(backend.map(|b| format!("{b:?}"))));
// The GPU compositor needs float render targets everywhere: WebGPU only (WebGL2
// composites on the CPU).
if let Some(rs) = cc.wgpu_render_state.clone()
&& backend == Some(eframe::wgpu::Backend::BrowserWebGpu)
&& !query_flag("cpu")
{
app.set_wgpu(rs);
set_info("compositor", json!("gpu"));
} else {
set_info("compositor", json!("cpu"));
}
let audio = audio::Handle::new();
if let Some(a) = &audio {
app.audio = Some(Box::new(a.clone()));
}
set_info("audio", json!(audio.is_some()));
set_info("restoredMedia", json!(restored));
Ok(Box::new(WebApp { app, audio, autosave: recovery::Autosave::default() }))
}),
)
.await?;
set_info("startupMs", json!(t0.elapsed().as_secs_f64() * 1000.0));
// keep the runner alive for the page's lifetime
std::mem::forget(runner);
if let Some(el) = doc.get_element_by_id("loading") {
el.remove();
}
Ok(())
}
+145
View File
@@ -0,0 +1,145 @@
//! Origin Private File System (OPFS): auto-save / crash recovery and the media copies that let a
//! recovered project find its media after a reload.
//!
//! Writes are queued and coalesced per path (the newest bytes win) and run one at a time.
use std::cell::RefCell;
use std::collections::BTreeMap;
use wasm_bindgen::{JsCast, JsValue};
use wasm_bindgen_futures::JsFuture;
use web_sys::{FileSystemDirectoryHandle, FileSystemFileHandle};
thread_local! {
static QUEUE: RefCell<(BTreeMap<String, Vec<u8>>, bool)> = const { RefCell::new((BTreeMap::new(), false)) };
}
/// Whether this browser has OPFS (`navigator.storage.getDirectory`).
pub fn available() -> bool {
web_sys::window().is_some_and(|w| {
let storage = js_sys::Reflect::get(&w.navigator(), &"storage".into()).unwrap_or(JsValue::UNDEFINED);
!storage.is_undefined() && js_sys::Reflect::has(&storage, &"getDirectory".into()).unwrap_or(false)
})
}
async fn root() -> Result<FileSystemDirectoryHandle, JsValue> {
if !available() {
return Err("OPFS not available".into());
}
let w = web_sys::window().ok_or("no window")?;
JsFuture::from(w.navigator().storage().get_directory()).await?.dyn_into()
}
/// The directory holding `rel` (created when `create`) and the file name.
async fn parent(rel: &str, create: bool) -> Result<(FileSystemDirectoryHandle, String), JsValue> {
let mut dir = root().await?;
let parts: Vec<&str> = rel.split('/').filter(|p| !p.is_empty()).collect();
let (name, dirs) = parts.split_last().ok_or("empty path")?;
for d in dirs {
let o = web_sys::FileSystemGetDirectoryOptions::new();
o.set_create(create);
dir = JsFuture::from(dir.get_directory_handle_with_options(d, &o)).await?.dyn_into()?;
}
Ok((dir, name.to_string()))
}
async fn file_handle(rel: &str, create: bool) -> Result<FileSystemFileHandle, JsValue> {
let (dir, name) = parent(rel, create).await?;
let o = web_sys::FileSystemGetFileOptions::new();
o.set_create(create);
JsFuture::from(dir.get_file_handle_with_options(&name, &o)).await?.dyn_into()
}
async fn write_now(rel: &str, data: &[u8]) -> Result<(), JsValue> {
let fh = file_handle(rel, true).await?;
let w: web_sys::FileSystemWritableFileStream = JsFuture::from(fh.create_writable()).await?.dyn_into()?;
JsFuture::from(w.write_with_u8_array(data)?).await?;
JsFuture::from(w.close()).await?;
Ok(())
}
/// Queue a write of `data` to `rel` (relative to the OPFS root).
pub fn write(rel: String, data: Vec<u8>) {
let start = QUEUE.with(|q| {
let mut q = q.borrow_mut();
q.0.insert(rel, data);
!std::mem::replace(&mut q.1, true)
});
if !start {
return;
}
wasm_bindgen_futures::spawn_local(async {
loop {
let next = QUEUE.with(|q| {
let mut q = q.borrow_mut();
let k = q.0.keys().next().cloned();
match k {
Some(k) => q.0.remove_entry(&k),
None => {
q.1 = false;
None
}
}
});
let Some((rel, data)) = next else { break };
if let Err(e) = write_now(&rel, &data).await {
log::warn!("OPFS write {rel}: {e:?}");
}
}
});
}
/// Read a file (None when missing).
pub async fn read(rel: &str) -> Option<Vec<u8>> {
let fh = file_handle(rel, false).await.ok()?;
let file: web_sys::File = JsFuture::from(fh.get_file()).await.ok()?.dyn_into().ok()?;
let buf = JsFuture::from(file.array_buffer()).await.ok()?;
Some(js_sys::Uint8Array::new(&buf).to_vec())
}
/// Remove a file (ignored when missing).
pub async fn remove(rel: &str) {
if let Ok((dir, name)) = parent(rel, false).await {
let _ = JsFuture::from(dir.remove_entry(&name)).await;
}
}
/// Copy a Blob into OPFS (streamed by the browser, never through wasm memory).
pub async fn store_blob(rel: &str, blob: &web_sys::Blob) -> Result<(), JsValue> {
let fh = file_handle(rel, true).await?;
let w: web_sys::FileSystemWritableFileStream = JsFuture::from(fh.create_writable()).await?.dyn_into()?;
JsFuture::from(w.write_with_blob(blob)?).await?;
JsFuture::from(w.close()).await?;
Ok(())
}
/// The files in an OPFS directory: (name, File).
pub async fn files_in(dir_rel: &str) -> Vec<(String, web_sys::File)> {
let mut out = Vec::new();
let Ok(dir) = (async {
let mut d = root().await?;
for p in dir_rel.split('/').filter(|p| !p.is_empty()) {
d = JsFuture::from(d.get_directory_handle(p)).await?.dyn_into()?;
}
Ok::<_, JsValue>(d)
})
.await
else {
return out;
};
let it = dir.values();
while let Ok(next) = it.next() {
let Ok(r) = JsFuture::from(next).await else { break };
let r: js_sys::IteratorNext = r.unchecked_into();
if r.done() {
break;
}
let Ok(h) = r.value().dyn_into::<FileSystemFileHandle>() else { continue };
if let Ok(f) = JsFuture::from(h.get_file()).await
&& let Ok(f) = f.dyn_into::<web_sys::File>()
{
out.push((f.name(), f));
}
}
out
}
+91
View File
@@ -0,0 +1,91 @@
//! Auto-save and crash recovery in OPFS (the desktop recovery journal, mapped to the browser).
//!
//! - While the project has unsaved changes, a snapshot is written every few seconds to
//! `recovery/snapshot.fcproj` with `recovery/meta.json` saying it is dirty; once the project is
//! saved (downloaded) the meta is marked clean.
//! - Imported media are copied to `media/<name>` (the browser streams the copy; nothing goes
//! through wasm memory), so the snapshot's `/files/<name>` references resolve after a reload.
//! - At start-up a dirty snapshot is reopened (`?norecover` skips it, `?fresh` also skips the media).
use filmcraft_engine::Session;
use serde_json::{Value, json};
use crate::opfs;
const SNAPSHOT: &str = "recovery/snapshot.fcproj";
const META: &str = "recovery/meta.json";
/// Seconds between snapshots of a changing project.
const INTERVAL_S: f64 = 5.0;
/// Media larger than this are not copied to OPFS.
const KEEP_MAX: f64 = 4.0 * 1024.0 * 1024.0 * 1024.0;
#[derive(Default)]
pub struct Autosave {
last_revision: u64,
last_time: f64,
last_dirty: Option<bool>,
}
impl Autosave {
/// Called every frame.
pub fn tick(&mut self, s: &Session, ctx: &egui::Context) {
if !opfs::available() {
return;
}
let now = ctx.input(|i| i.time);
let dirty = s.revision != s.saved_revision;
if dirty {
if s.revision != self.last_revision && now - self.last_time >= INTERVAL_S {
opfs::write(SNAPSHOT.into(), filmcraft_format::encode(&s.project, false));
let meta = json!({"dirty": true, "name": s.project.name, "revision": s.revision, "time": js_sys::Date::now()});
opfs::write(META.into(), meta.to_string().into_bytes());
self.last_revision = s.revision;
self.last_time = now;
} else if s.revision != self.last_revision {
// the UI may go idle: wake up when the snapshot is due
ctx.request_repaint_after(std::time::Duration::from_secs_f64((INTERVAL_S - (now - self.last_time)).max(0.05)));
}
} else if self.last_dirty != Some(false) {
opfs::write(META.into(), json!({"dirty": false}).to_string().into_bytes());
}
self.last_dirty = Some(dirty);
}
}
/// The unsaved snapshot of a session that ended without saving: (file name, bytes).
pub async fn load_snapshot() -> Option<(String, Vec<u8>)> {
if !opfs::available() {
return None;
}
let meta: Value = serde_json::from_slice(&opfs::read(META).await?).ok()?;
if meta.get("dirty").and_then(Value::as_bool) != Some(true) {
return None;
}
let bytes = opfs::read(SNAPSHOT).await?;
let name = meta.get("name").and_then(Value::as_str).unwrap_or("Recovered").to_string();
Some((format!("{name}.fcproj"), bytes))
}
/// Keep a copy of an imported file in OPFS (background).
pub fn keep_media(path: &str, file: &web_sys::File) {
if !opfs::available() || file.size() > KEEP_MAX {
return;
}
let Some(name) = path.strip_prefix("/files/").map(str::to_string) else { return };
let blob: web_sys::Blob = file.clone().into();
wasm_bindgen_futures::spawn_local(async move {
if let Err(e) = opfs::store_blob(&format!("media/{name}"), &blob).await {
log::warn!("keeping {name} in OPFS: {e:?}");
}
});
}
/// Register the media kept in OPFS under their `/files/` paths; returns how many.
pub async fn restore_media() -> usize {
let files = opfs::files_in("media").await;
let n = files.len();
for (name, f) in files {
crate::fs::register_blob(&name, f.into());
}
n
}
+529
View File
@@ -0,0 +1,529 @@
//! WebCodecs (hardware) video decoding for MP4/MOV, with our own decoders as the fallback.
//!
//! WebCodecs decodes asynchronously, and our [`VideoDecoder`] trait is synchronous, so the
//! WebCodecs path plugs in one level up, as a media source: [`reader_opener`] (registered ahead of
//! the built-in openers with `filmcraft_codecs::register_reader_opener`) opens H.264, HEVC, VP9 and
//! AV1 MP4/MOV files as a [`WcSource`]. Its audio and media info come from our own
//! [`filmcraft_codecs::Mp4Source`]; its video frames from a browser `VideoDecoder`:
//!
//! - A frame request finds the frame in the decoded-frame cache, or (re)starts a decode session at
//! the sample's sync sample, feeds samples up to a little past it (read through the Blob chunk
//! cache), marks the request [`filmcraft_media::pending`] and returns; the frame server retries
//! the job on a later frame, when the decoder's output callback has delivered the picture.
//! - A session that has not delivered the frame it was started for is not restarted by other
//! requests (thumbnails and the monitor would otherwise keep resetting each other).
//! - Outputs are converted to RGBA by drawing the `VideoFrame` on an `OffscreenCanvas` (the
//! browser applies the stream's colour description).
//! - Any decoder error or unsupported configuration switches the source to our decoder.
//!
//! `?nowebcodecs` disables it. Support is probed at start-up with `VideoDecoder.isConfigSupported`.
//!
//! [`VideoDecoder`]: filmcraft_codecs::VideoDecoder
use std::cell::{Cell, RefCell};
use std::collections::{BTreeMap, HashMap};
use std::rc::Rc;
use std::sync::{Arc, Mutex};
use filmcraft_frame::{AudioBuffer, VideoFrame};
use filmcraft_isobmff::{CodecConfig, Mp4File, TrackKind};
use filmcraft_media::{FrameRequest, MediaError, MediaInfo, MediaSource, SharedReader, SharedSource};
use filmcraft_time::Tick;
use serde_json::json;
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;
/// Samples fed past the wanted one (decoders hold pictures for reordering).
const LOOKAHEAD: usize = 8;
/// Decoded frames kept per source.
const CACHE_FRAMES: usize = 40;
/// Chunks allowed in the decoder's queue before feeding pauses.
const MAX_QUEUE: u32 = 24;
/// A session that delivered nothing for this long may be restarted by another request.
const STALE_MS: f64 = 2000.0;
thread_local! {
/// Codec families the browser decodes ("avc", "hevc", "vp9", "av1").
static SUPPORTED: RefCell<Vec<&'static str>> = const { RefCell::new(Vec::new()) };
static SESSIONS: RefCell<HashMap<u32, Session>> = RefCell::new(HashMap::new());
static NEXT_ID: Cell<u32> = const { Cell::new(1) };
static CANVAS: RefCell<Option<(web_sys::OffscreenCanvas, web_sys::OffscreenCanvasRenderingContext2d)>> = const { RefCell::new(None) };
static DECODED: Cell<u64> = const { Cell::new(0) };
static FALLBACKS: Cell<u64> = const { Cell::new(0) };
static SOURCES: Cell<u64> = const { Cell::new(0) };
}
fn now_ms() -> f64 {
js_sys::Date::now()
}
fn get(o: &JsValue, k: &str) -> JsValue {
js_sys::Reflect::get(o, &k.into()).unwrap_or(JsValue::UNDEFINED)
}
fn call(o: &JsValue, method: &str, args: &[&JsValue]) -> Result<JsValue, JsValue> {
let f: js_sys::Function = get(o, method).dyn_into()?;
let a = js_sys::Array::new();
for x in args {
a.push(x);
}
f.apply(o, &a)
}
fn ctor(name: &str) -> Option<js_sys::Function> {
get(&js_sys::global(), name).dyn_into().ok()
}
fn obj(pairs: &[(&str, JsValue)]) -> JsValue {
let o = js_sys::Object::new();
for (k, v) in pairs {
let _ = js_sys::Reflect::set(&o, &(*k).into(), v);
}
o.into()
}
/// WebCodecs codec string and decoder `description` for a sample entry.
pub fn codec_string(c: &CodecConfig, fourcc: &str) -> Option<(&'static str, String, Option<Vec<u8>>)> {
match c {
CodecConfig::Avc(a) => Some(("avc", format!("avc1.{:02x}{:02x}{:02x}", a.profile, a.compatibility, a.level), Some(a.to_bytes()))),
CodecConfig::Hevc(h) => {
let space = ["", "A", "B", "C"][(h.general_profile_space & 3) as usize];
let compat = h.general_profile_compatibility_flags.reverse_bits();
let mut s = format!(
"{}.{space}{}.{compat:x}.{}{}",
if fourcc == "hev1" { "hev1" } else { "hvc1" },
h.general_profile_idc,
if h.general_tier_flag { 'H' } else { 'L' },
h.general_level_idc
);
let bytes = h.general_constraint_indicator_flags.to_be_bytes();
let cons = &bytes[2..8];
let last = cons.iter().rposition(|b| *b != 0);
if let Some(l) = last {
for b in &cons[..=l] {
s.push_str(&format!(".{b:x}"));
}
}
Some(("hevc", s, Some(h.to_bytes())))
}
CodecConfig::Vp9(v) => Some(("vp9", format!("vp09.{:02}.{:02}.{:02}", v.profile, v.level, v.bit_depth.max(8)), None)),
CodecConfig::Av1(a) => {
let depth = if a.twelve_bit {
12
} else if a.high_bitdepth {
10
} else {
8
};
Some(("av1", format!("av01.{}.{:02}{}.{depth:02}", a.seq_profile, a.seq_level_idx_0, if a.seq_tier_0 { 'H' } else { 'M' }), Some(a.to_bytes())))
}
_ => None,
}
}
/// Probe which codec families the browser decodes (sets `filmcraft.info().webcodecs`).
pub async fn probe() {
let Some(vd) = ctor("VideoDecoder") else {
crate::set_info("webcodecs", json!(false));
return;
};
if web_sys::window().and_then(|w| w.location().search().ok()).is_some_and(|s| s.contains("nowebcodecs")) {
crate::set_info("webcodecs", json!("disabled"));
return;
}
let mut ok = Vec::new();
for (family, codec) in [("avc", "avc1.640028"), ("hevc", "hvc1.1.6.L120.90"), ("vp9", "vp09.00.40.08"), ("av1", "av01.0.08M.08")] {
let cfg = obj(&[("codec", codec.into()), ("codedWidth", 1920.into()), ("codedHeight", 1080.into())]);
let Ok(p) = call(&vd, "isConfigSupported", &[&cfg]).and_then(|p| p.dyn_into::<js_sys::Promise>()) else { continue };
if let Ok(r) = JsFuture::from(p).await
&& get(&r, "supported").as_bool() == Some(true)
{
ok.push(family);
}
}
crate::set_info("webcodecs", json!(ok));
SUPPORTED.with(|s| *s.borrow_mut() = ok);
}
/// Register the WebCodecs opener (when the browser decodes anything).
pub fn install() {
if SUPPORTED.with(|s| !s.borrow().is_empty()) {
filmcraft_codecs::register_reader_opener(reader_opener);
}
}
/// Live counters for `filmcraft.info()`.
pub fn stats() -> serde_json::Value {
let sessions: Vec<serde_json::Value> = SESSIONS.with(|ss| {
ss.borrow()
.iter()
.map(|(id, s)| {
json!({"id": id, "start": s.start, "next": s.next, "target": s.target, "needsKey": s.needs_key, "flushing": s.flushing.get(),
"errored": s.errored.get(), "queue": s.queue(), "idleMs": now_ms() - s.last_output.get()})
})
.collect()
});
json!({"sources": SOURCES.with(Cell::get), "framesDecoded": DECODED.with(Cell::get), "fallbacks": FALLBACKS.with(Cell::get), "sessions": sessions})
}
/// A [`SharedReader`] as the demuxer's byte source.
struct ReaderSrc(SharedReader);
impl filmcraft_isobmff::ByteSource for ReaderSrc {
fn len(&self) -> u64 {
self.0.len()
}
fn read_at(&self, offset: u64, buf: &mut [u8]) -> std::io::Result<()> {
self.0.read_at(offset, buf)
}
}
/// MP4/MOV with a WebCodecs-decodable video track → [`WcSource`].
pub fn reader_opener(name: &str, head: &[u8], reader: &SharedReader) -> Option<Result<SharedSource, MediaError>> {
if !filmcraft_codecs::mp4::sniff(head) {
return None;
}
let inner: SharedSource = match filmcraft_codecs::Mp4Source::open_reader(name, reader.clone()) {
Ok(s) => Arc::new(s),
Err(e) => return Some(Err(e.into())),
};
let file = match filmcraft_isobmff::open(ReaderSrc(reader.clone())) {
Ok(f) => f,
Err(_) => return Some(Ok(inner)),
};
let Some(track) = file.tracks.iter().position(|t| t.kind == TrackKind::Video && !t.samples.is_empty()) else { return Some(Ok(inner)) };
let entry = &file.tracks[track].entries[0];
let fourcc = entry.format.to_string();
let Some((family, codec, description)) = codec_string(&entry.codec, &fourcc) else { return Some(Ok(inner)) };
if !SUPPORTED.with(|s| s.borrow().contains(&family)) {
return Some(Ok(inner));
}
let info = inner.info().clone();
let (w, h) = info.video.as_ref().map(|v| (v.width, v.height)).unwrap_or((0, 0));
let id = NEXT_ID.with(|n| {
let id = n.get();
n.set(id + 1);
id
});
SOURCES.with(|c| c.set(c.get() + 1));
log::info!("{name}: video decoded with WebCodecs ({codec})");
Some(Ok(Arc::new(WcSource {
inner,
info,
reader: reader.clone(),
file,
track,
id,
config: Config { codec, description, width: w, height: h },
state: Mutex::new(Cache::default()),
})))
}
struct Config {
codec: String,
description: Option<Vec<u8>>,
width: u32,
height: u32,
}
#[derive(Default)]
struct Cache {
frames: BTreeMap<i64, Arc<VideoFrame>>,
failed: bool,
}
/// The browser decoder of one source (UI thread only; JS objects are not `Send`).
struct Session {
decoder: JsValue,
/// Sync sample the session started at, next sample to feed.
start: usize,
next: usize,
/// The decoder must restart at a key frame (new, or after a flush).
needs_key: bool,
flushing: Rc<Cell<bool>>,
errored: Rc<Cell<bool>>,
/// The frame (pts) this session was started for, and when it last delivered anything.
target: i64,
last_output: Rc<Cell<f64>>,
/// Highest presentation time output since the session (re)started.
out_max: Rc<Cell<i64>>,
out: Rc<RefCell<Vec<(i64, VideoFrame)>>>,
_closures: (Closure<dyn FnMut(JsValue)>, Closure<dyn FnMut(JsValue)>),
}
/// Draw a WebCodecs `VideoFrame` and read it back as RGBA.
fn to_rgba(frame: &JsValue) -> Option<VideoFrame> {
let w = get(frame, "displayWidth").as_f64()? as u32;
let h = get(frame, "displayHeight").as_f64()? as u32;
CANVAS.with(|c| {
let mut c = c.borrow_mut();
if c.is_none() {
let canvas = web_sys::OffscreenCanvas::new(w.max(1), h.max(1)).ok()?;
let opts = obj(&[("willReadFrequently", true.into())]);
let ctx = canvas.get_context_with_context_options("2d", &opts).ok()??.dyn_into::<web_sys::OffscreenCanvasRenderingContext2d>().ok()?;
*c = Some((canvas, ctx));
}
let (canvas, ctx) = c.as_ref()?;
if canvas.width() != w || canvas.height() != h {
canvas.set_width(w);
canvas.set_height(h);
}
call(ctx, "drawImage", &[frame, &0.into(), &0.into(), &w.into(), &h.into()]).ok()?;
let img = ctx.get_image_data(0.0, 0.0, w as f64, h as f64).ok()?;
Some(VideoFrame::rgba8(w, h, img.data().0))
})
}
impl Session {
fn new(cfg: &Config) -> Result<Session, JsValue> {
let vd = ctor("VideoDecoder").ok_or("no VideoDecoder")?;
let out: Rc<RefCell<Vec<(i64, VideoFrame)>>> = Rc::default();
let errored = Rc::new(Cell::new(false));
let last_output = Rc::new(Cell::new(now_ms()));
let out_max = Rc::new(Cell::new(i64::MIN));
let (o, lo, om) = (out.clone(), last_output.clone(), out_max.clone());
let on_output = Closure::<dyn FnMut(JsValue)>::new(move |frame: JsValue| {
let ts = get(&frame, "timestamp").as_f64().unwrap_or(0.0) as i64;
om.set(om.get().max(ts));
if let Some(f) = to_rgba(&frame) {
o.borrow_mut().push((ts, f));
DECODED.with(|c| c.set(c.get() + 1));
}
let _ = call(&frame, "close", &[]);
lo.set(now_ms());
crate::repaint();
});
let e = errored.clone();
let on_error = Closure::<dyn FnMut(JsValue)>::new(move |err: JsValue| {
log::warn!("WebCodecs decoder error: {err:?}");
e.set(true);
crate::repaint();
});
let init = obj(&[("output", on_output.as_ref().clone()), ("error", on_error.as_ref().clone())]);
let decoder = js_sys::Reflect::construct(&vd, &js_sys::Array::of1(&init))?;
let s = Session {
decoder,
start: 0,
next: 0,
needs_key: true,
flushing: Rc::new(Cell::new(false)),
errored,
target: i64::MIN,
last_output,
out_max,
out,
_closures: (on_output, on_error),
};
s.configure(cfg)?;
Ok(s)
}
fn configure(&self, cfg: &Config) -> Result<(), JsValue> {
let mut pairs = vec![
("codec", JsValue::from_str(&cfg.codec)),
("codedWidth", cfg.width.into()),
("codedHeight", cfg.height.into()),
("optimizeForLatency", true.into()),
];
if let Some(d) = &cfg.description {
pairs.push(("description", js_sys::Uint8Array::from(&d[..]).into()));
}
call(&self.decoder, "configure", &[&obj(&pairs)]).map(|_| ())
}
fn queue(&self) -> u32 {
get(&self.decoder, "decodeQueueSize").as_f64().unwrap_or(0.0) as u32
}
}
/// An MP4/MOV source whose video is decoded by the browser.
pub struct WcSource {
inner: SharedSource,
info: MediaInfo,
reader: SharedReader,
file: Mp4File,
track: usize,
id: u32,
config: Config,
state: Mutex<Cache>,
}
enum Drive {
/// Work is under way (or bytes are loading): retry later.
Waiting,
/// The decoder failed: use our own.
Failed,
}
impl WcSource {
/// Make progress towards sample `i` (presentation `want`).
fn drive(&self, i: usize, want: i64) -> Drive {
let t = &self.file.tracks[self.track];
let n = t.samples.len();
SESSIONS.with(|ss| {
let mut ss = ss.borrow_mut();
let s = match ss.entry(self.id) {
std::collections::hash_map::Entry::Occupied(o) => o.into_mut(),
std::collections::hash_map::Entry::Vacant(v) => match Session::new(&self.config) {
Ok(s) => v.insert(s),
Err(e) => {
log::warn!("WebCodecs: {e:?}");
return Drive::Failed;
}
},
};
if s.errored.get() {
return Drive::Failed;
}
if s.flushing.get() {
return Drive::Waiting;
}
let key = t.sync_sample_before(i);
// The session reaches the frame by feeding on: it has not output it yet (a frame it
// already output but the cache dropped needs a restart).
let covers = !s.needs_key && s.start <= i && key <= s.next && (i >= s.next || want > s.out_max.get());
if !covers {
// Don't interrupt a session that is still working towards its own frame.
let busy = !s.needs_key && s.target != i64::MIN && s.out_max.get() < s.target && now_ms() - s.last_output.get() < STALE_MS;
if busy {
return Drive::Waiting;
}
if !s.needs_key || s.next > 0 {
let _ = call(&s.decoder, "reset", &[]);
if s.configure(&self.config).is_err() {
return Drive::Failed;
}
}
s.out.borrow_mut().clear();
s.start = key;
s.next = key;
s.needs_key = false;
s.target = want;
s.out_max.set(i64::MIN);
s.last_output.set(now_ms());
} else if s.target == i64::MIN {
s.target = want;
}
let upto = (i + LOOKAHEAD).min(n - 1);
while s.next <= upto && s.queue() < MAX_QUEUE {
let smp = &t.samples[s.next];
let mut data = vec![0u8; smp.size as usize];
match self.reader.read_at(smp.offset, &mut data) {
Ok(()) => {}
// the bytes are being fetched (pending is marked): feed on the next try
Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => return Drive::Waiting,
Err(_) => return Drive::Failed,
}
let Some(chunk_ctor) = ctor("EncodedVideoChunk") else { return Drive::Failed };
let init = obj(&[
("type", (if smp.is_sync || s.next == s.start { "key" } else { "delta" }).into()),
("timestamp", (smp.pts as f64).into()),
("duration", (smp.duration as f64).into()),
("data", js_sys::Uint8Array::from(&data[..]).into()),
]);
let Ok(chunk) = js_sys::Reflect::construct(&chunk_ctor, &js_sys::Array::of1(&init)) else { return Drive::Failed };
if call(&s.decoder, "decode", &[&chunk]).is_err() {
return Drive::Failed;
}
s.next += 1;
}
// End of the stream: pictures held for reordering only come out on a flush.
if s.next >= n && s.queue() == 0 {
s.flushing.set(true);
s.needs_key = true;
if let Ok(p) = call(&s.decoder, "flush", &[]).and_then(|p| p.dyn_into::<js_sys::Promise>()) {
let f = s.flushing.clone();
wasm_bindgen_futures::spawn_local(async move {
let _ = JsFuture::from(p).await;
f.set(false);
crate::repaint();
});
} else {
s.flushing.set(false);
}
}
Drive::Waiting
})
}
/// Move decoded pictures into the cache.
fn collect(&self, c: &mut Cache, want: i64) {
let out = SESSIONS.with(|ss| {
let ss = ss.borrow();
let Some(s) = ss.get(&self.id) else { return Vec::new() };
std::mem::take(&mut *s.out.borrow_mut())
});
for (pts, f) in out {
c.frames.insert(pts, Arc::new(f));
}
while c.frames.len() > CACHE_FRAMES {
let far = c.frames.keys().copied().max_by_key(|p| (p - want).abs()).expect("non-empty");
c.frames.remove(&far);
}
// the session's own frame came out: other requests may restart it now
SESSIONS.with(|ss| {
if let Some(s) = ss.borrow_mut().get_mut(&self.id)
&& s.out_max.get() >= s.target
{
s.target = i64::MIN;
}
});
}
}
impl Drop for WcSource {
fn drop(&mut self) {
SESSIONS.with(|ss| {
if let Ok(mut ss) = ss.try_borrow_mut()
&& let Some(s) = ss.remove(&self.id)
{
let _ = call(&s.decoder, "close", &[]);
}
});
}
}
impl MediaSource for WcSource {
fn info(&self) -> &MediaInfo {
&self.info
}
fn video_frame(&self, req: FrameRequest) -> Result<Arc<VideoFrame>, MediaError> {
let mut c = self.state.lock().unwrap_or_else(|e| e.into_inner());
if c.failed {
drop(c);
return self.inner.video_frame(req);
}
let t = &self.file.tracks[self.track];
let n = t.samples.len();
let target = req.time.max(Tick::ZERO).to_rational_floor(1, t.timescale.max(1) as i64);
let i = t.sample_at_presentation_time(target).unwrap_or(n - 1).min(n - 1);
let want = t.samples[i].pts;
self.collect(&mut c, want);
if let Some(f) = c.frames.get(&want) {
return Ok(f.clone());
}
match self.drive(i, want) {
Drive::Waiting => {
self.collect(&mut c, want);
if let Some(f) = c.frames.get(&want) {
return Ok(f.clone());
}
filmcraft_media::pending::mark();
Err(MediaError::Decode("WebCodecs: decoding".into()))
}
Drive::Failed => {
log::warn!("{}: WebCodecs failed, using FilmCraft's decoder", self.info.name);
FALLBACKS.with(|f| f.set(f.get() + 1));
c.failed = true;
drop(c);
self.inner.video_frame(req)
}
}
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
self.inner.audio(start, frames, sample_rate)
}
}
+162
View File
@@ -0,0 +1,162 @@
// Browser smoke test of the FilmCraft web app over the Chrome DevTools Protocol.
// No npm dependencies (Node ≥ 22: global WebSocket and fetch).
//
// cargo xtask web --serve 8765 & # build + serve target/web/dist
// node apps/filmcraft-web/tests/smoke.mjs --url http://127.0.0.1:8765/ --media clip.mp4 --out /tmp/fc-web
//
// Steps: load (timing), demo project, play 2 s (frames shown/dropped), import the media through
// `filmcraft.importUrl`, put it on a new sequence, export H.264 (download), screenshots of each
// step. `--media` must be reachable from the page (copy it next to index.html). Prints a JSON
// report; exits non-zero on failure.
import { spawn } from "node:child_process";
import { mkdirSync, writeFileSync, readdirSync, statSync, mkdtempSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
const arg = (k, d) => {
const i = process.argv.indexOf(`--${k}`);
return i > 0 ? process.argv[i + 1] : d;
};
const url = arg("url", "http://127.0.0.1:8765/");
const out = arg("out", join(tmpdir(), "filmcraft-web-smoke"));
const media = arg("media", "web-test.mp4");
const chrome = arg("chrome", process.platform === "darwin" ? "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" : "google-chrome");
const port = Number(arg("port", "9333"));
const headless = !process.argv.includes("--headed");
mkdirSync(out, { recursive: true });
const downloads = join(out, "downloads");
mkdirSync(downloads, { recursive: true });
const profile = mkdtempSync(join(tmpdir(), "fc-chrome-"));
const proc = spawn(chrome, [
...(headless ? ["--headless=new"] : []),
`--remote-debugging-port=${port}`,
`--user-data-dir=${profile}`,
"--no-first-run",
"--no-default-browser-check",
"--enable-unsafe-webgpu",
"--autoplay-policy=no-user-gesture-required",
"--window-size=1600,1000",
"about:blank",
], { stdio: "ignore" });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
let ws;
let nextId = 1;
const waiting = new Map();
const logs = [];
async function connect() {
for (let i = 0; i < 100; i++) {
try {
const list = await (await fetch(`http://127.0.0.1:${port}/json/list`)).json();
const page = list.find((t) => t.type === "page");
if (page) {
ws = new WebSocket(page.webSocketDebuggerUrl);
await new Promise((res, rej) => { ws.onopen = res; ws.onerror = rej; });
ws.onmessage = (m) => {
const msg = JSON.parse(m.data);
if (msg.id && waiting.has(msg.id)) {
waiting.get(msg.id)(msg);
waiting.delete(msg.id);
} else if (msg.method === "Runtime.consoleAPICalled") {
logs.push(msg.params.args.map((a) => a.value ?? a.description ?? "").join(" "));
} else if (msg.method === "Runtime.exceptionThrown") {
logs.push("EXCEPTION " + JSON.stringify(msg.params.exceptionDetails.exception?.description ?? msg.params.exceptionDetails.text));
}
};
return;
}
} catch {}
await sleep(100);
}
throw new Error("Chrome did not start");
}
function send(method, params = {}) {
const id = nextId++;
ws.send(JSON.stringify({ id, method, params }));
return new Promise((res, rej) => waiting.set(id, (m) => (m.error ? rej(new Error(`${method}: ${m.error.message}`)) : res(m.result))));
}
async function js(expr, timeout = 120000) {
const r = await send("Runtime.evaluate", { expression: expr, awaitPromise: true, returnByValue: true, timeout });
if (r.exceptionDetails) throw new Error(`${expr}: ${r.exceptionDetails.exception?.description ?? r.exceptionDetails.text}`);
return r.result.value;
}
async function shot(name) {
const r = await send("Page.captureScreenshot", { format: "png" });
const p = join(out, `${name}.png`);
writeFileSync(p, Buffer.from(r.data, "base64"));
return p;
}
async function until(expr, ms = 60000, step = 100) {
const t0 = Date.now();
while (Date.now() - t0 < ms) {
if (await js(expr)) return Date.now() - t0;
await sleep(step);
}
throw new Error(`timeout waiting for ${expr}`);
}
const report = { url, steps: {} };
try {
await connect();
await send("Page.enable");
await send("Runtime.enable");
await send("Browser.setDownloadBehavior", { behavior: "allow", downloadPath: downloads }).catch(() => {});
await send("Emulation.setDeviceMetricsOverride", { width: 1600, height: 1000, deviceScaleFactor: 1, mobile: false });
const t0 = Date.now();
await send("Page.navigate", { url });
await until("!!(window.filmcraftLoad && (window.filmcraftLoad.readyMs || window.filmcraftLoad.error))", 120000);
report.load = await js("window.filmcraftLoad");
report.load.wallMs = Date.now() - t0;
if (report.load.error) throw new Error(report.load.error);
report.resources = await js("performance.getEntriesByType('resource').filter(e => /wasm|\\.js/.test(e.name)).map(e => ({name: e.name.split('/').pop(), kB: Math.round(e.transferSize / 1024), ms: Math.round(e.duration)}))");
report.info = await js("filmcraft.info()");
// let the UI settle (fonts, first frames)
await sleep(1500);
report.steps.demo = { screenshot: await shot("01-demo"), inspect: await js("filmcraft.inspect().then(i => ({window: i.window, activeSequence: i.activeSequence, fps: i.fps}))") };
// play 2 seconds
await js("filmcraft.request('ui.playback', {action: 'play'})");
await sleep(2000);
const pb = await js("filmcraft.inspect().then(i => ({playhead: i.playhead, playback: i.playback, fps: i.fps}))");
report.steps.play = { ...pb, screenshot: await shot("02-playing") };
await js("filmcraft.request('ui.playback', {action: 'stop'})");
// import generated media
const imp = await js(`filmcraft.importUrl(${JSON.stringify(media)})`);
report.steps.import = imp;
if (!imp.items || imp.items.length === 0) throw new Error("import failed: " + JSON.stringify(imp));
const item = imp.items[0];
// a sequence from the clip, then look at its first frames
report.steps.sequence = await js(`filmcraft.execute("file.newSequence", {fromItem: ${item}, name: "Web import"})`).catch((e) => ({ error: String(e) }));
await sleep(1500);
await js("filmcraft.request('ui.playback', {action: 'play'})");
await sleep(1500);
report.steps.importPlay = { ...(await js("filmcraft.inspect().then(i => ({playhead: i.playhead, playback: i.playback}))")), screenshot: await shot("03-imported") };
await js("filmcraft.request('ui.playback', {action: 'stop'})");
// export H.264 (stepped job; the file is offered as a download)
const ex = await js(`filmcraft.execute("file.exportMedia", {format: "h264", path: "/exports/web-export.mp4", bitrateKbps: 4000})`);
const te = Date.now();
await until(`filmcraft.execute("jobs.list").then(js => js.some(j => j.id === ${ex.job} && j.finished))`, 300000, 250);
report.steps.export = { job: ex.job, ms: Date.now() - te, jobs: await js("filmcraft.execute('jobs.list')"), files: await js("filmcraft.files()") };
await sleep(1000);
report.steps.export.downloads = readdirSync(downloads).filter((f) => !f.endsWith(".crdownload")).map((f) => ({ f, bytes: statSync(join(downloads, f)).size }));
report.steps.export.screenshot = await shot("04-exported");
report.ok = true;
} catch (e) {
report.ok = false;
report.error = String(e && e.stack || e);
try { report.failScreenshot = await shot("99-failure"); } catch {}
}
report.console = logs.slice(-60);
writeFileSync(join(out, "report.json"), JSON.stringify(report, null, 2));
console.log(JSON.stringify(report, null, 2));
try { ws && ws.close(); } catch {}
proc.kill();
process.exit(report.ok ? 0 : 1);
+59
View File
@@ -0,0 +1,59 @@
// FilmCraft web audio output (AudioWorklet). The UI thread mixes audio ahead and posts
// interleaved stereo Float32Array blocks; this processor plays them in order and reports how many
// frames it actually played (with its currentTime), which drives the playback clock.
class FilmcraftOutput extends AudioWorkletProcessor {
constructor() {
super();
this.queue = [];
this.offset = 0;
this.gen = 0;
this.played = 0;
this.quanta = 0;
this.port.onmessage = (e) => {
const m = e.data;
if (m.type === "reset") {
this.gen = m.gen;
this.queue = [];
this.offset = 0;
this.played = 0;
this.port.postMessage({ gen: this.gen, played: 0, time: currentTime });
} else if (m.type === "data" && m.gen === this.gen) {
this.queue.push(m.samples);
}
};
}
process(_inputs, outputs) {
const out = outputs[0];
const left = out[0];
const right = out[1] || out[0];
const n = left.length;
let i = 0;
while (i < n && this.queue.length) {
const b = this.queue[0];
const k = Math.min(b.length / 2 - this.offset, n - i);
for (let j = 0; j < k; j++) {
left[i + j] = b[2 * (this.offset + j)];
right[i + j] = b[2 * (this.offset + j) + 1];
}
i += k;
this.offset += k;
if (this.offset * 2 >= b.length) {
this.queue.shift();
this.offset = 0;
}
}
// only real frames advance the clock; a starved queue plays silence and holds it
this.played += i;
for (; i < n; i++) {
left[i] = 0;
right[i] = 0;
}
if (++this.quanta % 4 === 0) {
this.port.postMessage({ gen: this.gen, played: this.played, time: currentTime });
}
return true;
}
}
registerProcessor("filmcraft-output", FilmcraftOutput);
+16
View File
@@ -0,0 +1,16 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64">
<!-- FilmCraft web icon: original work (a film reel ring with a play triangle). -->
<circle cx="32" cy="32" r="30" fill="#14343a"/>
<circle cx="32" cy="32" r="22" fill="none" stroke="#f0a24a" stroke-width="5"/>
<g fill="#14343a">
<circle cx="32" cy="10" r="2.2"/>
<circle cx="32" cy="54" r="2.2"/>
<circle cx="10" cy="32" r="2.2"/>
<circle cx="54" cy="32" r="2.2"/>
<circle cx="16.4" cy="16.4" r="2.2"/>
<circle cx="47.6" cy="47.6" r="2.2"/>
<circle cx="16.4" cy="47.6" r="2.2"/>
<circle cx="47.6" cy="16.4" r="2.2"/>
</g>
<path d="M26 21.5 L44 32 L26 42.5 Z" fill="#f4f1ea"/>
</svg>

After

Width:  |  Height:  |  Size: 689 B

@@ -0,0 +1,9 @@
asset: favicon.svg
title: FilmCraft web app icon (favicon / install icon)
author: FilmCraft contributors
source: original work
license: MIT OR Apache-2.0
added: 2026-10-01 by FilmCraft contributors (Claude agent, M13 web app)
notes: Hand-written SVG: a dark teal disc, an orange ring with eight sprocket dots and a play
triangle, drawn from scratch from basic shapes. Not derived from any other product's
icon.
+41
View File
@@ -0,0 +1,41 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="dark">
<title>FilmCraft</title>
<link rel="icon" href="favicon.svg" type="image/svg+xml">
<style>
html, body { margin: 0; height: 100%; overflow: hidden; background: #1b1c1f; }
canvas { position: absolute; inset: 0; width: 100%; height: 100%; display: block; }
#loading {
position: absolute; inset: 0; display: flex; flex-direction: column; gap: 14px;
align-items: center; justify-content: center; color: #c9c9cc;
font: 14px/1.4 system-ui, sans-serif; pointer-events: none;
}
#loading img { width: 56px; height: 56px; }
#loading .err { color: #ff8a7a; white-space: pre-wrap; max-width: 70ch; pointer-events: auto; }
</style>
</head>
<body>
<canvas id="filmcraft_canvas"></canvas>
<div id="loading"><img src="favicon.svg" alt=""><div id="status">Loading FilmCraft…</div></div>
<script type="module">
import init, { start } from "./filmcraft_web.js";
const t0 = performance.now();
window.filmcraftLoad = {};
try {
await init();
window.filmcraftLoad.wasmMs = performance.now() - t0;
await start("filmcraft_canvas");
window.filmcraftLoad.readyMs = performance.now() - t0;
} catch (e) {
const s = document.getElementById("status");
if (s) { s.className = "err"; s.textContent = "FilmCraft failed to start:\n" + (e && e.stack || e); }
window.filmcraftLoad.error = String(e);
throw e;
}
</script>
</body>
</html>
+5 -3
View File
@@ -20,7 +20,7 @@ Design principles:
## 1. Layers
```text
L6 apps/filmcraft · apps/filmcraft-cli
L6 apps/filmcraft · apps/filmcraft-cli · apps/filmcraft-web
L5 ui-egui · automation
L4 engine
L3 render · gpu · export · golden (test-only)
@@ -65,6 +65,7 @@ and `filmcraft-cli`.
| `automation` | L5 | MCP server (`rmcp`, stdio), headless or bridged to the running app |
| `filmcraft` | L6 | desktop binary: eframe/wgpu window, cpal audio output, file dialogs, native macOS menu, TCP control server |
| `filmcraft-cli` | L6 | headless CLI: `probe`, `commands`, `render`, `run`, `mcp` |
| `filmcraft-web` | L6 | the browser app (wasm32): eframe web runner on WebGPU/WebGL2, Blob-backed services, OPFS recovery, WebAudio, WebCodecs, `window.filmcraft` API ([web.md](web.md)) |
### What `cargo xtask layers` enforces
@@ -80,8 +81,9 @@ are exempt):
| L0 codecs stay standalone | L0 crates other than `time`, `geom`, `color`, `bitstream`, `testkit` may depend on no workspace crate except `filmcraft-bitstream`. External crates such as `thiserror` and `rayon` are allowed. |
| No UI/OS crates below L5 | `egui`, `eframe`, `egui-wgpu`, `winit`, `rfd`, `cpal`, `muda` are allowed only in L5 and L6. |
`cargo xtask wasm` runs `cargo check --target wasm32-unknown-unknown` on every L0–L4 crate, so
everything up to the engine stays web-portable. `unsafe_code = "deny"` applies workspace-wide.
`cargo xtask wasm` runs `cargo check --target wasm32-unknown-unknown` on every L0–L4 crate, the
egui UI and the web app, so everything up to the engine stays web-portable and the web app builds
([web.md](web.md)). `unsafe_code = "deny"` applies workspace-wide.
## 2. Time base
+2 -1
View File
@@ -33,6 +33,7 @@ cargo run --release -p filmcraft -- a.mp4 b.wav # import media (or open a
cargo run --release -p filmcraft-cli -- commands # list engine commands
cargo run --release -p filmcraft-cli -- render --demo --seconds 3 --out frame.png
cargo run --release -p filmcraft-cli -- mcp --demo # headless MCP server on stdio
cargo xtask web --serve 8765 # the web app on http://127.0.0.1:8765/ (docs/web.md)
```
`FILMCRAFT_CONTROL_PORT=9876` works like `--control 9876`, and `FILMCRAFT_CPU_COMPOSITE=1`
@@ -52,7 +53,7 @@ Every commit must pass all of these:
| Tests | `cargo test --workspace` |
| Layering | `cargo xtask layers` |
| Assets | `cargo xtask assets` |
| wasm | `cargo xtask wasm` (checks every L0–L4 crate for `wasm32-unknown-unknown`) |
| wasm | `cargo xtask wasm` (checks every L0–L4 crate, `filmcraft-ui-egui` and `filmcraft-web` for `wasm32-unknown-unknown`) |
| All of the above | `cargo xtask ci` (runs clippy and tests with `--release`) |
`cargo xtask` is an alias in `.cargo/config.toml` for `cargo run -p xtask --`.
+138
View File
@@ -0,0 +1,138 @@
# FilmCraft on the web
`apps/filmcraft-web` runs the same engine and egui UI as the desktop app in the browser: the
workspace compiled to `wasm32-unknown-unknown`, started by eframe's web runner on WebGPU (WebGL2
fallback). It is a static site: one `.wasm`, its `wasm-bindgen` JavaScript glue, `index.html`, an
`AudioWorklet` script and an icon. Nothing is uploaded anywhere; media stay on the user's machine.
## Build and run
| Tool | Version | Install |
|---|---|---|
| Rust target | `wasm32-unknown-unknown` | `rustup target add wasm32-unknown-unknown` |
| wasm-bindgen CLI | exactly the `wasm-bindgen` crate version (0.2.129) | `cargo install wasm-bindgen-cli --version 0.2.129 --locked` |
| wasm-opt (optional) | any recent binaryen | `brew install binaryen`; used by `cargo xtask web` when found |
```sh
cargo xtask web # release build → target/web/dist (index.html, filmcraft_web.js, filmcraft_web_bg.wasm, …)
cargo xtask web --dev # unoptimised build (faster to compile, slow to run)
cargo xtask web --serve 8765 # build, then serve dist on http://127.0.0.1:8765/
```
`--serve` is a tiny localhost-only static server that sends `Cross-Origin-Opener-Policy` /
`Cross-Origin-Embedder-Policy`, so the page is cross-origin isolated like a production deployment
should be. Any static server works (`python3 -m http.server -d target/web/dist 8765`); it must
serve `.wasm` as `application/wasm` for streaming compilation.
The web crate is empty on non-wasm targets, so `cargo test --workspace` and clippy are
unaffected; `cargo xtask wasm` (part of `cargo xtask ci`) checks it, with `filmcraft-ui-egui`, for
`wasm32-unknown-unknown`.
URL flags: `?empty` (no demo project), `?norecover` (don't reopen the auto-saved project),
`?fresh` (also don't restore media kept in OPFS), `?cpu` (CPU compositor), `?nowebcodecs`
(decode with FilmCraft's own decoders only).
## How the desktop pieces map to the browser
| Desktop | Web |
|---|---|
| frame worker threads | `FrameServer::pump`: queued frame jobs run on the UI thread between egui frames, highest priority first, within a time budget (24 ms while playing, 40 ms otherwise) |
| export on a worker thread | `filmcraft_export::Exporter`: the export advances one frame per step from `Session::pump_jobs`, 30 ms per UI frame; progress shows in the header like on the desktop |
| `std::fs` (`FsServices`) | `fs::WebServices`: a virtual file table. Picked/dropped files are `File` handles (never copied); files the app writes are kept in memory and offered as downloads |
| file dialogs | File System Access `showOpenFilePicker` where available, else `<input type=file>`; drop files anywhere on the page |
| crash-recovery journal | OPFS: `recovery/snapshot.fcproj` every 5 s while there are unsaved changes, plus copies of imported media in `media/`; the next visit reopens the snapshot |
| cpal output | WebAudio `AudioWorklet` |
| VideoToolbox & co. | WebCodecs `VideoDecoder` |
| TCP control channel / MCP | `window.filmcraft` (below) |
### Threads
wasm threads need a cross-origin isolated page **and** a wasm build with atomics (a nightly
`build-std` toolchain), so this build is single-threaded everywhere: frame rendering, decoding,
mixing and encoding are cooperative on the UI thread, and `rayon` runs inline.
`filmcraft.info()` reports `crossOriginIsolated` so a threaded build can choose at start-up.
Commands that need a thread on the desktop (render previews, proxies, Project Manager, mask
tracking) report an error on the web for now.
### Media reads
Containers are opened through `Services::reader` (a `filmcraft_media::ByteReader`): MP4/MOV and
Matroska read their index at import and single samples afterwards, so a multi-gigabyte file is
never loaded whole. Browser `Blob` reads are asynchronous, so `BlobReader` keeps a 384 MiB LRU cache
of 1 MiB chunks: a read whose chunks are missing starts fetching them (plus three chunks of
read-ahead) and fails with `WouldBlock`, setting `filmcraft_media::pending`. The frame server then
drops that frame's result and retries the job after the bytes arrive (the fetch wakes the UI);
the stepped exporter re-renders the frame; an import retries `file.import`. Before importing, the
container index is prefetched (an MP4's `moov` box wherever it is, a small Matroska file whole).
Stills, WAV and compressed-audio files are decoded whole, as on the desktop.
Limits: Matroska/WebM files larger than the chunk cache import slowly (opening scans every
cluster header); a file the user picked is only readable while the page is open, unless its OPFS
copy exists (files up to 4 GiB are copied in the background).
### Decoding
FilmCraft's own decoders (H.264, HEVC, VP9, AV1, ProRes, DNxHR, MJPEG, AAC, Opus…) run in the
browser unchanged. When the browser has WebCodecs, H.264, HEVC, VP9 and AV1 video in MP4/MOV is
decoded by the browser's (usually hardware) `VideoDecoder` instead: `webcodecs::reader_opener` is
registered ahead of the built-in openers and opens such files as a `WcSource`, whose audio and
media info come from our `Mp4Source`. WebCodecs is asynchronous, so a frame request feeds samples
to the decoder and returns "pending"; the decoder's output callback converts the `VideoFrame` to
RGBA (drawn on an `OffscreenCanvas`) into a per-source frame cache, and the retried request finds
it. Codec support is probed at start-up with `VideoDecoder.isConfigSupported`; any decoder error
switches that source to our decoder.
### Audio
`audio::Handle` implements `AudioOut`. The UI thread mixes the sequence 250 ms ahead in 2048-frame
blocks and posts them to the worklet (`web/audio-worklet.js`), which counts the frames it really
played and reports them with its `currentTime`. `played_frames` extrapolates from the last report
on the audio context's clock, never past what was queued — so the audio clock is the playback
master as on the desktop, and a starved worklet holds the playhead. Browsers start audio suspended
until the first click or key press; until then playback runs on the wall clock.
### GPU
eframe creates a WebGPU device when the browser has one, else WebGL2. FilmCraft's GPU compositor
(`filmcraft-gpu`) is used on WebGPU; on WebGL2 frames are composited on the CPU. Check
`filmcraft.info().backend` and `.compositor`.
## `window.filmcraft`: the agent / test API
The control channel of the desktop app (`docs/control-protocol.md`) as promises. Results resolve
with the method's `result`, errors reject.
| Call | |
|---|---|
| `filmcraft.execute(command, params)` | run an engine command (same ids and params as MCP / `filmcraft-cli`) |
| `filmcraft.request(method, params)` | any control method: `ui.inspect`, `ui.click`, `ui.key`, `ui.playback`, `ui.menu.invoke`… |
| `filmcraft.commands()` | the command list |
| `filmcraft.inspect()` | `ui.inspect` |
| `filmcraft.screenshot(params)` | `{pngBase64, width, height}` of the canvas (or `{panel}`) |
| `filmcraft.importFiles(files)` | import `File` / `FileList` / `File[]`; resolves `{items, errors, paths}` |
| `filmcraft.importUrl(url, name?)` | fetch a URL and import it (tests) |
| `filmcraft.openProject(fileOrUrl)` | open a `.fcproj` |
| `filmcraft.files()` | the virtual files: `{path, size, kind}` |
| `filmcraft.readFile(path)` | a file's bytes (`Uint8Array`), e.g. an export |
| `filmcraft.info()` | backend, compositor, isolation, OPFS, WebCodecs support and counters, start-up time |
`window.filmcraftLoad` holds `{wasmMs, readyMs}` (module fetch+compile, and until the app runs).
```js
await filmcraft.importUrl("/clip.mp4");
await filmcraft.execute("file.newSequence", {fromItem: 48});
await filmcraft.request("ui.playback", {action: "play"});
const {job} = await filmcraft.execute("file.exportMedia", {format: "h264", path: "/exports/out.mp4"});
```
## Browser test
`apps/filmcraft-web/tests/smoke.mjs` drives headless Chrome over the DevTools protocol (Node ≥ 22,
no npm packages): load, demo playback, import of a generated MP4, playback, H.264 export
(download). It writes screenshots and `report.json`:
```sh
cargo xtask web --serve 8765 &
ffmpeg -f lavfi -i testsrc2=size=640x360:rate=30 -f lavfi -i sine -t 3 -c:v libx264 -pix_fmt yuv420p -c:a aac target/web/dist/web-test.mp4
node apps/filmcraft-web/tests/smoke.mjs --url http://127.0.0.1:8765/ --media web-test.mp4 --out target/web/smoke
```
+121 -3
View File
@@ -2,7 +2,10 @@
//!
//! - `layers`: enforces the dependency layering of `docs/architecture.md` §1 (downward-only edges,
//! listed same-layer edges, L0 codec crates depend on `bitstream` only, no UI/OS crates below L5).
//! - `wasm`: `cargo check --target wasm32-unknown-unknown` for every crate in L0–L4.
//! - `wasm`: `cargo check --target wasm32-unknown-unknown` for every crate in L0–L4, the egui UI
//! and the web app.
//! - `web [--dev] [--serve PORT]`: build the web app (`apps/filmcraft-web`) into
//! `<target>/web/dist` with `wasm-bindgen` (docs/web.md); `--serve` serves it on localhost.
//! - `assets`: every asset file (image, icon, font, LUT, audio, video…) has a complete
//! `<file>.attribution` sidecar and an entry in `ATTRIBUTION.md` (AGENTS.md §1).
//! - `fixtures [crate…]`: pre-generate the ffmpeg fixture matrix of the oracle tests (runs each
@@ -59,6 +62,7 @@ const LAYERS: &[(&str, u8)] = &[
("automation", 5),
("filmcraft", 6),
("cli", 6),
("web", 6),
];
/// L0 crates that are not codecs/containers (no `bitstream`-only restriction).
@@ -224,13 +228,16 @@ fn run(cmd: &mut Command) -> Result<(), String> {
if st.success() { Ok(()) } else { Err(format!("failed: {cmd:?}")) }
}
/// Crates above L4 that must also build for the web.
const WEB_CRATES: &[&str] = &["filmcraft-ui-egui", "filmcraft-web"];
fn wasm() -> Result<(), String> {
let md = metadata()?;
let mut cmd = Command::new(env!("CARGO"));
cmd.args(["check", "--target", "wasm32-unknown-unknown"]);
let mut n = 0;
for (name, _) in workspace_crates(&md) {
if layer_of(&name).is_some_and(|l| l <= 4) {
if layer_of(&name).is_some_and(|l| l <= 4) || WEB_CRATES.contains(&name.as_str()) {
cmd.args(["-p", &name]);
n += 1;
}
@@ -243,6 +250,116 @@ fn wasm() -> Result<(), String> {
Ok(())
}
/// The wasm-bindgen CLI must match the `wasm-bindgen` crate version exactly.
const WASM_BINDGEN: &str = "0.2.129";
fn target_dir() -> std::path::PathBuf {
std::env::var_os("CARGO_TARGET_DIR").map(std::path::PathBuf::from).unwrap_or_else(|| std::path::PathBuf::from("target"))
}
/// Build the web app into `<target>/web/dist` (see docs/web.md).
fn web(args: &[String]) -> Result<(), String> {
let dev = args.iter().any(|a| a == "--dev");
let serve = args.iter().position(|a| a == "--serve").map(|i| args.get(i + 1).and_then(|p| p.parse::<u16>().ok()).unwrap_or(8765));
let profile = if dev { "dev" } else { "release" };
let mut build = Command::new(env!("CARGO"));
build.args(["build", "--target", "wasm32-unknown-unknown", "-p", "filmcraft-web", "--profile", profile]);
run(&mut build)?;
let out = Command::new("wasm-bindgen")
.arg("--version")
.output()
.map_err(|_| format!("wasm-bindgen CLI not found: cargo install wasm-bindgen-cli --version {WASM_BINDGEN} --locked"))?;
let v = String::from_utf8_lossy(&out.stdout);
if !v.contains(WASM_BINDGEN) {
return Err(format!(
"wasm-bindgen CLI is {} but the crate is {WASM_BINDGEN}: cargo install wasm-bindgen-cli --version {WASM_BINDGEN} --locked",
v.trim()
));
}
let dir = if dev { "debug" } else { "release" };
let wasm = target_dir().join("wasm32-unknown-unknown").join(dir).join("filmcraft_web.wasm");
let dist = target_dir().join("web").join("dist");
let _ = std::fs::remove_dir_all(&dist);
std::fs::create_dir_all(&dist).map_err(|e| e.to_string())?;
run(Command::new("wasm-bindgen").args(["--target", "web", "--no-typescript", "--out-dir"]).arg(&dist).arg(&wasm))?;
let bg = dist.join("filmcraft_web_bg.wasm");
if !dev && Command::new("wasm-opt").arg("--version").output().is_ok() {
// optional: smaller and faster (binaryen); skipped when not installed
let opt = dist.join("filmcraft_web_opt.wasm");
run(Command::new("wasm-opt")
.args(["-O2", "--enable-bulk-memory", "--enable-nontrapping-float-to-int", "--enable-sign-ext", "--enable-mutable-globals"])
.arg(&bg)
.arg("-o")
.arg(&opt))?;
std::fs::rename(&opt, &bg).map_err(|e| e.to_string())?;
}
let web = std::path::Path::new("apps/filmcraft-web/web");
for e in std::fs::read_dir(web).map_err(|e| e.to_string())?.flatten() {
let name = e.file_name().to_string_lossy().to_string();
if !name.ends_with(".attribution") {
std::fs::copy(e.path(), dist.join(&name)).map_err(|e| format!("{name}: {e}"))?;
}
}
let size = |p: &std::path::Path| std::fs::metadata(p).map(|m| m.len()).unwrap_or(0);
let total: u64 = std::fs::read_dir(&dist).map_err(|e| e.to_string())?.flatten().map(|e| size(&e.path())).sum();
println!("web: {} ({:.1} MB wasm, {:.1} MB total)", dist.display(), size(&bg) as f64 / 1e6, total as f64 / 1e6);
if let Some(port) = serve {
serve_dir(&dist, port)?;
}
Ok(())
}
/// A tiny static file server for the web build (localhost only). Sends the cross-origin
/// isolation headers so `crossOriginIsolated` is true, like a production deployment would.
fn serve_dir(dir: &std::path::Path, port: u16) -> Result<(), String> {
use std::io::{BufRead, BufReader, Write};
let listener = std::net::TcpListener::bind(("127.0.0.1", port)).map_err(|e| format!("127.0.0.1:{port}: {e}"))?;
println!("serving {} on http://127.0.0.1:{port}/", dir.display());
for stream in listener.incoming().flatten() {
let dir = dir.to_path_buf();
std::thread::spawn(move || {
let mut stream = stream;
let mut line = String::new();
let mut reader = BufReader::new(match stream.try_clone() {
Ok(s) => s,
Err(_) => return,
});
if reader.read_line(&mut line).is_err() {
return;
}
loop {
let mut h = String::new();
if reader.read_line(&mut h).map(|n| n == 0).unwrap_or(true) || h.trim().is_empty() {
break;
}
}
let path = line.split_whitespace().nth(1).unwrap_or("/").split(['?', '#']).next().unwrap_or("/").to_string();
let rel = if path == "/" { "index.html".to_string() } else { path.trim_start_matches('/').replace("..", "") };
let file = dir.join(&rel);
let (status, body) = match std::fs::read(&file) {
Ok(b) => ("200 OK", b),
Err(_) => ("404 Not Found", b"not found".to_vec()),
};
let mime = match file.extension().and_then(|e| e.to_str()).unwrap_or("") {
"html" => "text/html; charset=utf-8",
"js" => "text/javascript",
"wasm" => "application/wasm",
"svg" => "image/svg+xml",
"json" => "application/json",
"mp4" => "video/mp4",
"webm" => "video/webm",
_ => "application/octet-stream",
};
let head = format!(
"HTTP/1.1 {status}\r\nContent-Type: {mime}\r\nContent-Length: {}\r\nCross-Origin-Opener-Policy: same-origin\r\nCross-Origin-Embedder-Policy: require-corp\r\nCache-Control: no-cache\r\nConnection: close\r\n\r\n",
body.len()
);
let _ = stream.write_all(head.as_bytes()).and_then(|_| stream.write_all(&body));
});
}
Ok(())
}
/// (crate, test target) whose ignored `generate_fixtures` test builds that crate's fixture matrix.
const FIXTURE_GENERATORS: &[(&str, &str)] = &[
("h264", "conformance"),
@@ -316,7 +433,8 @@ fn main() -> ExitCode {
"fixtures" => fixtures(&std::env::args().skip(2).collect::<Vec<_>>()),
"ci" => ci(),
"bench-playback" => bench_playback(),
_ => Err("usage: cargo xtask <layers|assets|wasm|fixtures [crate…]|ci|bench-playback [args]>".into()),
"web" => web(&std::env::args().skip(2).collect::<Vec<_>>()),
_ => Err("usage: cargo xtask <layers|assets|wasm|web [--dev] [--serve PORT]|fixtures [crate…]|ci|bench-playback [args]>".into()),
};
match r {
Ok(()) => ExitCode::SUCCESS,