repremium/mokosh
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

revert(01): reopen Phase 1 — D-13 multi-EBML-concat is unplayable

Browse Source 
REQ-video-ring-buffer flipped from [x] back to [ ]. ROADMAP.md Phase 1
row reverted from [x] Closed 2026-05-15 to [ ] reopened 2026-05-16.
STATE.md status flipped phase_complete → phase_reopened with full
historical narrative preserved.

Root cause (confirmed at byte level by gsd-debugger 2026-05-16):
D-13's concat-of-self-contained-WebM-segments architecture produces a
3-EBML-header WebM that standards-compliant Matroska parsers
(mpv, ffmpeg, Chrome HTMLMediaElement) play only as the first segment
(~9.94 s) and silently drop the remaining 2 segments. Confirmed via
operator mpv drag-drop test of BOTH the canonical 2026-05-15 closure
fixture and the 2026-05-16 UAT-produced fixture — both exhibit the
same broken playback.

The 2026-05-15 "operator-confirmed clean Chrome playback" assessment
was insufficient: it verified the file plays without freezing but did
not measure total duration. Phase 1's primary deliverable
(REQ-video-ring-buffer / SPEC §10 #7) is therefore NOT satisfied.

Fix path chosen by user: ts-ebml (parse) + webm-muxer (write) to
replace mergeVideoSegments file-concat with real single-EBML remux.
Will land as Plan 01-08 via fresh /gsd-plan-phase ceremony.

RED test landed in tests/offscreen/webm-playback.test.ts (2 new
assertions on container-format-duration + ffmpeg-full-decode-duration).
2 failures, 53 baseline tests still GREEN.

Option C port-lifecycle refactor (debug session
empty-archive-port-race, commits 674c415..f0871c0) DID land cleanly
and is retained — that fix was orthogonal and correctly resolved the
silent-empty-archive symptom that previously masked this deeper bug.

Debug session: .planning/debug/d13-multi-ebml-concat-unplayable.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mark
2026-05-16 19:47:47 +02:00
parent f1026954fc
commit bc310d98cf
5 changed files with 479 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

335
.planning/debug/d13-multi-ebml-concat-unplayable.md
View File

@@ -8,8 +8,8 @@ trigger: |
1675899 bytes, total segments merged: 3") but the resulting file plays
ONLY ~9 s in Chrome AND in mpv. Cross-checking the canonical fixture
committed at Phase 1 closure on 2026-05-15 (`tests/fixtures/last_30sec.webm`,
1633459 bytes, 3 segments per architecture) reveals it ALSO plays only ~9 s
in mpv. Operator confirmed both via mpv playback test.
1633459 bytes, 3 segments per architecture) reveals it ALSO plays only
~9 s in mpv. Operator confirmed both via mpv playback test.
This means D-13's "concat of self-contained WebM segments → playable 30 s
WebM" architecture is fundamentally broken. The 2026-05-15 Phase 1
@@ -24,7 +24,7 @@ trigger: |
though it was marked Complete in REQUIREMENTS.md/ROADMAP.md/STATE.md
on 2026-05-15.
created: 2026-05-16T16:56:41Z
updated: 2026-05-16T16:56:41Z
updated: 2026-05-16T17:25:00Z
phase: 01-stabilize-video-pipeline
related_uat: .planning/phases/01-stabilize-video-pipeline/01-UAT.md
related_review_fix: .planning/phases/01-stabilize-video-pipeline/01-REVIEW-FIX.md
@@ -121,15 +121,19 @@ the segment boundary noise from concatenation, not playback failure.
## Current Focus
hypothesis: |
**H4 confirmed by operator empirical test**: D-13's "concat of self-
contained WebM segments → produce playable 30 s WebM" architecture
does not work in practice because most Matroska/WebM players do not
implement the multi-segment Matroska feature. The Matroska spec
permits multiple segments in one file but most decoders read only
the first segment's EBML header and stop there. ffmpeg's behavior
(which mpv inherits) is to honor the first EBML's duration metadata.
Chrome's MSE implementation appears to do the same (per UAT operator
observation).
**H4 confirmed by byte-level EBML probe (2026-05-16T17:10Z, see
Evidence/H4 below)**: D-13's "concat of self-contained WebM
segments → produce playable 30 s WebM" architecture does not work
because standards-compliant Matroska parsers (mpv, mkvtoolnix,
Chrome's HTMLMediaElement, ffprobe's `format=duration` path) honor
the FIRST Segment element's Info.Duration EBML (~9_934 ms for the
fixture) and stop there. Even ffmpeg's matroska DEMUXER — which is
unusually liberal and reads through the second segment's EBML
header — collapses segments 2..N onto seg1's local timestamp axis
(verified empirically: 601 packets decoded from segs 1+2, ZERO
packets from seg3, output `time=00:00:09.96`). Multi-segment
Matroska is technically permitted by the spec but in practice
consumer-grade players do not implement it.
**H3 confirmed by operator empirical test**: The 2026-05-15 Phase 1
closure's "operator-confirmed clean Chrome playback" check was
@@ -145,49 +149,98 @@ hypothesis: |
produces a file that any player can read end-to-end as one continuous
~30 s stream.
**Candidate implementations**:
- `webm-muxer` npm package (Vanilla. ~10 KB. Browser + Node support.
Single-segment output. Active maintenance.)
- `ts-ebml` (EBML parser + writer. Allows manual control over
structure. ~50 KB.)
**Candidate implementations** (researched 2026-05-16, see
Evidence/library-survey below for full table):
- `webm-muxer` 5.1.4 (Vanilagy, MIT, last release 2025-07-02,
gzipped ~12 KB, pure ESM/CJS no DOM globals). `addVideoChunkRaw(data,
type:'key'|'delta', timestamp, meta?)` accepts already-encoded VP9
frames — exactly the shape produced by a stream of existing WebM
segments. SW-compatible. PRIMARY CANDIDATE for the write half.
- `ts-ebml` 3.0.2 (legokichi, MIT, last release 2025-09-28, gzipped
~87 KB, UMD has a single `typeof window` check with self-fallback
so SW-compatible). Decoder+Encoder API. Needed for the parse half
(extract VP9 SimpleBlock payloads + cluster timecodes + keyframe
flags from each segment).
- `ebml` 3.0.0 (node-ebml, MIT, last release **2018-09-06** — dead
upstream). Smaller but unmaintained.
- `mp4-muxer` 5.2.2 (sibling of webm-muxer; not applicable — we need
WebM container output).
- Custom EBML parser (full control, ~500-1000 LOC, no dep weight)
- **Alternative path: MediaRecorder timeslice with cluster-aware trim**:
revisit retired D-09..D-11 architecture but trim ONLY on keyframe
boundaries (preserving every cluster from the most recent keyframe
onwards). This avoids the A3 orphan-P-frame freeze by guaranteeing
every kept cluster's references are present. ~200-400 LOC. The
risk: requires understanding EBML/Matroska cluster structure to
trim correctly.
onwards). See Evidence/cluster-aware-trim below — the DETERMINISTIC
floor on retained-content duration is much less than 30 s
(worst-case: keyframe just emitted → retain only the post-keyframe
sliver) because VP9 kf_max_dist under Chrome's MediaRecorder is
irregular (3-5 s typical, 26 s observed in the prior debug
session). This path produces a NON-DETERMINISTIC content window;
rejected as architecturally weaker than remux.
- **Alternative path: WebCodecs API** (VideoEncoder + Muxer.js or
similar): full control over container framing. Significant rewrite
(~1000-2000 LOC). Most flexible but heaviest.
(~1000-2000 LOC). Most flexible but heaviest. WebCodecs is
available in MV3 service workers per Chrome 94+ — viable but
over-engineered for the current need (we already have VP9 frames,
we just need to RE-CONTAIN them).
The remux approach (webm-muxer or equivalent) is likely the right
trade-off: small, well-tested library, preserves D-13's segment
lifecycle benefits (no orphan-P-frame freeze, ~10s rotation gap
acceptable), but produces a single-EBML output that all players
read correctly.
The recommendation (TIEBREAKER only — the user makes the call):
`ts-ebml` (parse) + `webm-muxer` (write) is the smallest fix that
matches the actual problem shape. Combined ~100 KB gzipped, both
MIT, both actively maintained, both verified SW-compatible. Net
source-edit LOC ~150-300 in `src/background/index.ts`
mergeVideoSegments() — we don't decode/re-encode VP9 frames, we
just parse them out of segments and re-emit with monotonic
timestamps. Preserves D-13's recorder-side lifecycle (which DID
fix the orphan-P-frame freeze) and adds a single new SW-side
remux pass on the save path.
test: |
RED test: introduce a playable-duration assertion to
tests/offscreen/webm-playback.test.ts. Use ffprobe -count_frames
-show_streams to count VIDEO FRAMES (not just metadata duration),
then divide by reported frame rate to compute actual playable
content duration. Assert actual_duration > 25_000 ms for the
generated/committed fixture. This test should FAIL against the
current D-13 architecture and PASS after the remux fix lands.
RED test LANDED at tests/offscreen/webm-playback.test.ts. Two new
assertions in the new `describe('webm playable duration (RED —
confirms d13-multi-ebml-concat-unplayable bug)')` block:
Alternative RED test: ffprobe -read_intervals -i FILE
'0%+#90000' (seek to last 90s, read all packets). Count packets
read. Should be ~600 packets for 30s @ ~20fps, not ~200 for 9s.
1. `container-level format=duration on last_30sec.webm exceeds 25 s`
— uses ffprobe to read `format=duration`. Asserts
`>= MIN_PLAYABLE_DURATION_MS = 25_000`. RED today
(actual: 9_934 ms).
2. `ffmpeg full decode of last_30sec.webm reaches at least 25 s of
timeline` — parses the last `time=HH:MM:SS.MS` from `ffmpeg -stats
-f null -` output. Asserts `>= 25_000 ms`. RED today
(actual: 9_960 ms).
Both gate behind `it.skipIf(!ffprobeAvailable())` /
`it.skipIf(!ffmpegAvailable())` so CI environments without those
binaries auto-skip rather than hard-fail (matches the existing
webm-playback.test.ts skip discipline). The existing two structural-
validity tests in the same file (`...zero decoder packet errors` and
`...does not end prematurely`) remain GREEN and untouched.
expecting: |
RED test fails on current code (both fixture and freshly-recorded
output should fail the duration assertion). Debugger then implements
the chosen fix path (webm-muxer remux most likely) and re-asserts
GREEN.
next_action: gather initial evidence from EBML parsing of both fixtures + research candidate JS remux libraries
reasoning_checkpoint: ""
tdd_checkpoint: ""
the chosen fix path (webm-muxer + ts-ebml remux most likely) and
re-asserts GREEN. RED confirmed 2026-05-16T17:20Z: 11 test files,
53 passed + 2 failed (the two new assertions). All pre-existing
tests still GREEN; tsc clean (exit 0).
next_action: CHECKPOINT to orchestrator — root cause confirmed, RED test landed, fix-strategy options surfaced; awaiting user's chosen path via orchestrator routing.
reasoning_checkpoint: |
Why CHECKPOINT here rather than execute: the choice between
`ts-ebml + webm-muxer` vs `custom EBML parser` vs `cluster-aware
trim revisit of D-09..D-11` vs `WebCodecs rewrite` is architecturally
significant (it determines whether Phase 1's deliverable stays in
the debug-session hotfix lane OR escalates to a fresh Plan 01-08,
and whether the project gains two new runtime deps). Per the
feedback memory `feedback-no-unilateral-scope-reduction.md` the
debugger does not narrow this for the user — surface options and
let the user pick.
tdd_checkpoint: |
RED gate honored. Two new failing assertions in
tests/offscreen/webm-playback.test.ts pin the playable-duration
contract that the 2026-05-15 closure check missed. Existing
structural-validity tests remain GREEN. tsc clean. Full vitest run
reports `Test Files 1 failed | 10 passed (11) / Tests 2 failed | 53
passed (55)` — exactly the expected RED-on-new shape, no collateral
regression.
## Constraints
@@ -255,6 +308,197 @@ tdd_checkpoint: ""
- Both files report `duration=9.94 s` via ffprobe -show_entries format=duration
- Decoder errors: zero (segments are individually valid)
### H4 byte-level EBML probe (2026-05-16T17:10Z) — confirms multi-EBML-concat is the root cause
Probe target: `tests/fixtures/last_30sec.webm` (1_633_459 bytes, committed
fixture from Phase 1 closure 2026-05-15).
**EBML structural scan** (raw byte search for element IDs per
[Matroska spec](https://www.matroska.org/technical/elements.html)):
| EBML element | ID (hex) | Occurrences in file | Byte offsets |
|---|---|---|---|
| EBML header | `1A 45 DF A3` | **3** | `[0, 509038, 970967]` |
| Segment | `18 53 80 67` | **3** | `[36, 509074, 971003]` |
| Cluster | `1F 43 B6 75` | 13 | spread across all 3 segments |
The file is THREE concatenated WebM files, each with its own EBML header
+ Segment element. mkvinfo (without `--all-elements`) reports only the
FIRST segment + its EBML header — two top-level elements visible —
confirming standards-compliant parsers stop at the first segment.
**Per-segment isolated probes** (sliced via Python at the EBML offsets
above into `/tmp/d13-seg{1,2,3}.webm`):
| Segment | Bytes | format=duration | -count_frames |
|---|---|---|---|
| seg1 | 509_038 | 9.934 s | 301 frames |
| seg2 | 461_929 | 9.963 s | 300 frames |
| seg3 | 662_492 | 9.958 s | 311 frames |
| **TOTAL** | **1_633_459** | (29.86 s of real content) | **912 frames** |
Each segment is individually a valid, complete ~10 s WebM. The
underlying VP9 stream is intact across all three. The bug is purely the
multi-segment topology of the concatenated container.
**Concatenated file probe** (the actual fixture):
| Probe command | Reported value |
|---|---|
| `ffprobe -show_entries format=duration` | **9.934024 s** (first segment's Info.Duration metadata only) |
| `ffprobe -count_frames` | **601 frames** (= 301 + 300 = segs 1+2 only) |
| `ffmpeg -f null -` decoder | **frame=601 time=00:00:09.96** + 299 non-monotonic-DTS warnings |
| Packets read from byte range `pos<509038` (seg1) | 301 |
| Packets read from byte range `509038 ≤ pos < 970967` (seg2) | 300 |
| Packets read from byte range `pos ≥ 970967` (seg3) | **0** |
| mkvinfo top-level elements visible | 2 (EBML head + Segment) — seg2 + seg3 invisible |
Two observations both fatal to D-13:
1. ffmpeg's matroska demuxer is the most-permissive parser in common
use and even IT silently drops segment 3 (zero packets from
`pos ≥ 970967`).
2. Even when ffmpeg DOES read segments 1+2 it does not offset seg2's
local timestamps onto the global timeline. The 299 non-monotonic-DTS
warnings are seg2's local timestamps (`tt < 9934 ms`) colliding with
seg1's end timestamp (`9934 ms`). Output `time=00:00:09.96` because
the muxer cannot grow the timeline past the maximum monotonic DTS
it has accepted.
Conclusion: H4 confirmed at the byte level. The file is structurally
valid as a "concatenated archive of three WebMs" but is NOT a single
30-second playable WebM. To produce a 30-second playable WebM the
segments must be REMUXED (parse VP9 frames + keyframe flags + cluster
timestamps from each segment, then re-emit them inside a single
EBML-headered container with monotonically-adjusted timestamps).
### Library survey (2026-05-16T17:15Z) — candidate JS WebM remux libraries
All sizes are the bundled dist (no source-map, no tests, no docs).
Gzipped values measured locally via `gzip -c`. SW-compat verdict is
based on grep of dist for `window`/`document`/`navigator`/`XMLHttpRequest`
followed by manual inspection of any hits.
| Lib | Version | License | Last release | Dist size | Gzipped | SW-compat | API shape | Verdict |
|---|---|---|---|---|---|---|---|---|
| `webm-muxer` | 5.1.4 | MIT | 2025-07-02 | 69 KB | **~12 KB** | YES (zero DOM refs) | `addVideoChunkRaw(data, type:'key'\|'delta', ts, meta?)` accepts encoded VP9 frames | PRIMARY — write half |
| `ts-ebml` | 3.0.2 | MIT | 2025-09-28 | 356 KB | ~87 KB | YES (`typeof window` with `self` fallback in UMD wrapper) | `Decoder.decode(ArrayBuffer) → EBMLElementDetail[]` ; `Encoder.encode(elms) → ArrayBuffer` | PRIMARY — parse half |
| `ebml` | 3.0.0 | MIT | **2018-09-06** | 7.7 MB unpacked | n/a | uncertain | older streaming parser API | DEAD UPSTREAM — avoid |
| `mp4-muxer` | 5.2.2 | MIT | (active) | 70 KB | ~13 KB | YES | analogous to webm-muxer but MP4 | n/a — wrong container |
| Custom EBML parser | n/a | n/a | n/a | 0 KB | 0 KB | YES | hand-rolled per Matroska spec | ~500-1000 LOC, full ownership |
Important note on the `webm-muxer` API: `addVideoChunkRaw()` takes
already-encoded VP9 frame bytes + a keyframe flag + a timestamp. We
do NOT need to decode/re-encode the VP9 stream — the existing
segments already contain valid VP9 frame payloads inside their
Cluster/SimpleBlock elements. The remux path is:
1. For each segment blob, parse via `ts-ebml.Decoder` → walk the EBML
tree → for each Cluster's SimpleBlock children, extract the VP9
frame bytes + keyframe flag (Matroska SimpleBlock bit 7 of the
first flag byte = "keyframe" per
[spec](https://www.matroska.org/technical/elements.html#SimpleBlock))
+ cluster Timestamp + local block offset.
2. Compute monotonic adjusted timestamp: `globalTs = segmentBaseMs +
clusterTsMs + blockOffsetMs` where `segmentBaseMs` accumulates the
prior segment's total content duration.
3. Stream all adjusted frames into a single `webm-muxer.Muxer` with
`addVideoChunkRaw(frameData, isKey ? 'key' : 'delta', globalUs)`.
4. `muxer.finalize()` → `ArrayBufferTarget.buffer` → single-EBML
WebM Blob.
Combined dep weight: ~100 KB gzipped (`webm-muxer` ~12 KB + `ts-ebml`
~87 KB). Combined source edit estimate at `mergeVideoSegments()`:
~150-300 LOC including type defs.
### Cluster-aware-trim alternative path (D-09..D-11 revisit, 2026-05-16T17:18Z)
Path summary: keep MediaRecorder running continuously (the retired
D-09 lifecycle) but, on each periodic trim pass, scan the chunk buffer
for the OLDEST keyframe whose position would keep total duration ≤ 30
s, then drop everything strictly before that keyframe. Preserves header
chunk + a contiguous run of keyframe-anchored clusters.
Why this is architecturally weaker than remux:
1. **Non-deterministic content window.** MediaRecorder/VP9 keyframe
cadence under Chrome's default `kf_max_dist=100` is irregular —
the prior `webm-playback-freeze` debug session observed a 26-second
keyframe gap empirically. If the latest keyframe was emitted 2 s
ago, cluster-aware trim retains only 2 s of content. The user's
`last_30sec.webm` would be anywhere in `[~few seconds .. ~30 s]`
depending on when SAVE landed in the keyframe cycle. That breaks
SPEC §10 #7's implicit "≥ 30 s of recent context" requirement.
2. **Still need EBML parsing.** To find keyframe boundaries inside
the chunk buffer we still need to parse the WebM container for
SimpleBlock keyframe flags. So the dep weight is similar (`ts-ebml`
at minimum) but the output is worse.
3. **Re-introduces the freeze-risk surface area.** The prior debug
session retired D-09..D-11 precisely because age-trim repeatedly
produced orphan-P-frame freezes. A "keyframe-aware" variant still
has to delete content; one bug in the keyframe-detection path and
the freeze returns. The risk surface is wider than the remux path,
which never deletes — it only re-containers what already exists.
LOC estimate: ~200-400 LOC for keyframe parsing + buffer mutation +
tests. Net: similar dep weight, worse playable-duration guarantee,
re-opens the freeze regression surface. **REJECTED as inferior to
remux.** Documenting here only because the orchestrator brief
explicitly requested the comparison.
### WebCodecs API path (2026-05-16T17:19Z)
WebCodecs (`VideoEncoder` + `VideoDecoder`) is available in MV3 service
workers from Chrome 94+. The path would be: feed each segment's
clusters → `VideoDecoder` → emit `VideoFrame` objects → feed back into
`VideoEncoder` (re-encode VP9) → wrap output via `webm-muxer`.
This works but adds a re-encode pass that:
- doubles CPU cost during the save flow
- introduces an additional quality loss (re-encoding lossy VP9)
- adds 500-1000 LOC of encoder/decoder lifecycle management
- requires Chrome 94+ exclusively (we already require modern Chrome,
so OK, but it tightens the version floor)
There is no benefit over the `ts-ebml + webm-muxer` path for this
specific shape of problem — we already have encoded VP9 frames and
just need to put them in a different container. Re-encoding is
unnecessary work. **REJECTED as over-engineered.**
### RED test landing evidence (2026-05-16T17:20Z)
File edited: `tests/offscreen/webm-playback.test.ts` (preserved
existing 2 GREEN tests; appended new `describe` block with 2 new
assertions + supporting helpers).
Test run scoped to file:
```
$ npx vitest run tests/offscreen/webm-playback.test.ts
Test Files 1 failed (1)
Tests 2 failed | 2 passed (4)
```
Failures:
- `container-level format=duration on last_30sec.webm exceeds 25 s`
— `expected 9934 to be greater than or equal to 25000`
- `ffmpeg full decode of last_30sec.webm reaches at least 25 s of timeline`
— `expected 9960 to be greater than or equal to 25000`
Full suite (proves zero collateral regression):
```
$ npx vitest run
Test Files 1 failed | 10 passed (11)
Tests 2 failed | 53 passed (55)
```
All 53 pre-existing tests still GREEN. tsc:
```
$ npx tsc --noEmit; echo exit=$?
exit=0
```
## Eliminated
(populated by debugger as hypotheses are ruled out)
@@ -266,6 +510,13 @@ tdd_checkpoint: ""
- (H5: defective committed fixture in storage): ruled out — file size
matches expected (1.63 MB matches what was committed on 2026-05-15;
not bit-rot)
- H6 (cluster-aware-trim revisit of D-09..D-11): rejected on architectural
weakness — non-deterministic content window (depends on keyframe
cadence), still needs EBML parsing, re-opens freeze-regression
surface area. See Evidence/cluster-aware-trim section.
- H7 (WebCodecs re-encode path): rejected as over-engineered — re-encodes
VP9 frames we already have. ~500-1000 LOC for zero quality/playability
benefit. See Evidence/WebCodecs section.
## Resolution