Files
mokosh/.planning/phases/02-stabilize-export-pipeline/02-02-SUMMARY.md
Mark 95b5bd252c docs(02-02): complete Blob URL download pipeline plan (D-P2-01 closes P0-6)
SUMMARY.md documents:
- 3 RED tests in tests/background/blob-url-download.test.ts flipped GREEN
  (wire-format polarity guard, 6 MB latency + wire-format, revoke lifecycle).
- 6 files modified (3 prod source + 3 test files; +518 / -35 lines).
- Wire-format extension: 3 new PortMessageType variants on keepalivePort.
- Operator-facing improvement: archives >2 MB now download successfully
  (was: silent failure with data:URL Network error).
- Rule 3 deviation: extended Plan 02-01 test helpers with the offscreen-side
  CREATE_DOWNLOAD_URL → DOWNLOAD_URL → REVOKE_DOWNLOAD_URL round-trip
  simulation pattern + capturedArchiveBytes bytes capture. This pattern
  is reusable by Plan 02-03 and was anticipated in Plan 02-01 SUMMARY.
- Forward link: Plan 02-03 (meta.urls + tab-url-tracker) is unblocked;
  Plan 02-04 (UAT harness A24+) is unblocked.

Verification:
- npx tsc --noEmit: clean
- npm run build: clean
- npm run build:test: clean
- tests/background/blob-url-download.test.ts: 3/3 GREEN
- Tier-1 FORBIDDEN_HOOK_STRINGS: 13/13 GREEN (unchanged)
- Full vitest: 163 passed / 8 failed (was 159 passed / 12 failed); +4 GREEN
  net delta. 8 remaining RED are exactly Plan 02-03 territory.
2026-05-20 15:57:35 +02:00

233 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
phase: 02-stabilize-export-pipeline
plan: 02
subsystem: export-pipeline
tags:
- blob-url-migration
- p0-6-fix
- offscreen-port-bridge
- chrome-downloads-onchanged
- url-createObjectURL
- url-revokeObjectURL
- d-p2-01
- wave-2-green
# Dependency graph
requires:
- phase: 02-stabilize-export-pipeline
provides: "Plan 02-01 RED gate (tests/background/blob-url-download.test.ts pinning D-P2-01 wire-format + revoke lifecycle contracts)"
- phase: 01-stabilize-video-pipeline
provides: "D-12 base64 wire-format helpers (src/shared/binary.ts); D-17 keepalivePort lifecycle; src/offscreen/recorder.ts URL.createObjectURL surface; pendingBufferRequests requestId routing pattern (template for pendingDownloadUrlResolvers)"
provides:
- "Offscreen-minted blob:URL download pipeline (D-P2-01 closes audit P0-6)"
- "PortMessageType triplet for D-P2-01: CREATE_DOWNLOAD_URL + DOWNLOAD_URL + REVOKE_DOWNLOAD_URL"
- "chrome.downloads.onChanged revoke lifecycle wired in SW (state.current==='complete'/'interrupted' → REVOKE_DOWNLOAD_URL → URL.revokeObjectURL in offscreen)"
- "Test-side blob:URL extraction pattern (capturedArchiveBytes from CREATE_DOWNLOAD_URL.dataBase64) — reusable by Plan 02-03 + future export-pipeline tests"
- "Typed blob-url-mint-failed errors routed through saveArchive's RECORDING_ERROR channel (no silent corrupt-archive paths)"
- "T-02-02-03 mitigation: SW rejects DOWNLOAD_URL responses that do not start with blob: (defense-in-depth against future regressions / hostile-peer-on-shared-port)"
affects:
- "Plan 02-03 (Wave 2 GREEN for meta.urls + tab-url-tracker — must extend its own test helpers with the CREATE_DOWNLOAD_URL round-trip simulation; the pattern is now established in tests/background/meta-json-urls-schema.test.ts + tests/build/strict-meta-json-validation.test.ts)"
- "Plan 02-04 (Wave 3 UAT harness extensions — A24+ operator empirical verification that the produced zip uses blob: URL via DevTools Network panel)"
- "Any future export-pipeline plan that wants to extract archive bytes from a unit-test SAVE_ARCHIVE flow (use the CREATE_DOWNLOAD_URL port-message capture pattern)"
# Tech tracking
tech-stack:
added: []
patterns:
- "Offscreen-as-DOM-globals-proxy: SW posts (CREATE_DOWNLOAD_URL, base64) → offscreen mints URL.createObjectURL → returns (DOWNLOAD_URL, url) → SW uses URL. Same pattern can carry future DOM-only APIs the SW needs (Atomics.notify, requestAnimationFrame timing, etc.) without expanding chrome.* permissions."
- "Per-request resolver Map for port-bridged async (pendingDownloadUrlResolvers) — mirrors pendingBufferRequests; the pattern is now the canonical SW↔offscreen async-await idiom."
- "Test-side port-bridge simulation: helper tracks mintedRequestIds Set + replies DOWNLOAD_URL with a Node-native blob:URL minted via URL.createObjectURL. capturedArchiveBytes snapshot lets downstream JSZip extraction work without spying on URL.createObjectURL globally."
key-files:
created: []
modified:
- "src/shared/types.ts (PortMessageType union + PortMessage interface extended with the D-P2-01 triplet)"
- "src/offscreen/recorder.ts (handleCreateDownloadUrl helper + onPortMessage branches for CREATE_DOWNLOAD_URL / REVOKE_DOWNLOAD_URL; mintedDownloadUrls hygiene Set; base64ToBlob import added)"
- "src/background/index.ts (pendingDownloadUrlResolvers Map + pendingRevokes Map; onConnect DOWNLOAD_URL routing branch; downloadArchive rewritten; chrome.downloads.onChanged listener registered at module init; T-02-02-03 blob: prefix guard)"
- "tests/background/blob-url-download.test.ts (runSaveAndCaptureDownloadArg + Test 2 drain loop + Test 3 port.postMessage shim all extended to simulate the offscreen-side CREATE_DOWNLOAD_URL → DOWNLOAD_URL → REVOKE_DOWNLOAD_URL round-trip)"
- "tests/background/meta-json-urls-schema.test.ts (runSaveAndCaptureArchiveBlob extended with tryFireDownloadUrl + capturedArchiveBytes so the helper returns the archive Blob regardless of data: vs blob: URL pathway)"
- "tests/build/strict-meta-json-validation.test.ts (runAndParseMetaJson extended identically)"
key-decisions:
- "Reuse the existing keepalivePort (D-17) for the SW↔offscreen mint/revoke bridge rather than opening a parallel port. Saves two connect-overhead penalties per save flow and keeps the lifecycle ownership single-sourced."
- "Reuse the D-12 base64 wire-format from src/shared/binary.ts for SW→offscreen archive Blob transfer rather than inventing a new encoding. Same precedent as the BUFFER segment transfer in the reverse direction."
- "NO data: URL fallback. Per CONTEXT.md Revision Log 2026-05-20 F3 resolution: typed `blob-url-mint-failed` errors route through saveArchive's catch into the RECORDING_ERROR channel. Silent corrupt-archive paths are forbidden — operator must see a clear failure."
- "downloadArchive rejects DOWNLOAD_URL responses whose url does not start with `blob:` (T-02-02-03 defense-in-depth). The offscreen is same-extension origin so a non-blob: response would itself be a bug, but the explicit guard catches future regressions."
- "Concurrent mints allowed (each gets its own requestId) — URL minting is stateless per-Blob, unlike encodeAndSendBuffer which gates concurrent calls because they share segment ring buffer state."
- "Reused the existing generateRequestId() helper in src/background/index.ts rather than introducing a parallel crypto.randomUUID() call site, consistent with CLAUDE.md 'extension over duplication'."
- "Test-side helper captures archive bytes from CREATE_DOWNLOAD_URL.dataBase64 (not from the blob:URL string, which is opaque in Node). This is cleaner than the spy-on-URL.createObjectURL pattern the test-author originally anticipated and provides direct access to the bytes the SW transmitted."
patterns-established:
- "D-P2-01 mint/revoke triplet: CREATE_DOWNLOAD_URL (SW→offscreen base64 payload) → DOWNLOAD_URL (offscreen→SW url) → REVOKE_DOWNLOAD_URL (SW→offscreen url) — fire-and-forget revoke; URL.revokeObjectURL on unknown URL is a WHATWG no-op so the hygiene Set is purely diagnostic."
- "Test-side offscreen-port-bridge round-trip simulation pattern: helper tracks mintedRequestIds Set + decodes CREATE_DOWNLOAD_URL.dataBase64 + mints Node-native blob:URL + replies DOWNLOAD_URL on the same port. Used by Plan 02-03 + future export-pipeline test helpers."
requirements-completed: []
# Per plan instructions: REQ-archive-export-latency is pinned by Plan 02-01's RED
# test 2 (6 MB archive < 5000 ms) which is now GREEN, but the requirement is
# best marked complete after Plan 02-04's UAT harness empirically validates
# the operator-facing latency assertion (A24+). Plan 02-02 unblocks the
# latency contract architecturally; Plan 02-04 closes the empirical loop.
# Metrics
duration: 13min
completed: 2026-05-20
---
# Phase 2 Plan 02: Blob URL download pipeline Summary
**Offscreen-minted blob:URL replaces base64 data:URL in downloadArchive; SW↔offscreen mint/revoke bridge rides existing keepalivePort with chrome.downloads.onChanged-driven revoke lifecycle.**
## Performance
- **Duration:** 13 min
- **Started:** 2026-05-20T13:41:31Z
- **Completed:** 2026-05-20T13:55:01Z
- **Tasks:** 3/3 completed
- **Files modified:** 6 (3 production source + 3 test files; +518 lines / 35 lines)
## Accomplishments
- **Audit P0-6 closed.** downloadArchive no longer mints a `data:application/zip;base64,...` URL. The legacy ~2 MB Chrome data-URL cap is gone; real operator archives (5-10 MB) now download to disk without 'Network error'.
- **3 Plan 02-01 RED tests flipped GREEN** in tests/background/blob-url-download.test.ts:
- Test 1: chrome.downloads.download arg0.url starts with `blob:` and not `data:application/zip;base64,`.
- Test 2: 6 MB archive completes in under 5 000 ms AND emits a blob: URL.
- Test 3: URL.revokeObjectURL fires with the minted url after chrome.downloads.onChanged 'complete'.
- **chrome.downloads.onChanged revoke lifecycle wired** for the first time. SW tracks (downloadId → minted URL) in `pendingRevokes` Map; on terminal state ('complete' or 'interrupted'), the SW posts REVOKE_DOWNLOAD_URL to offscreen which calls URL.revokeObjectURL. Memory hygiene preserved across saves; T-02-02-02 mitigation in place.
- **D-P2-01 wire contract landed** in `PortMessageType`: 3 new variants (CREATE_DOWNLOAD_URL + DOWNLOAD_URL + REVOKE_DOWNLOAD_URL) following the same JSON-serializable optional-tagged-union pattern as the existing BUFFER triplet.
- **Test infrastructure extended** for Plan 02-03 reuse: all three Plan-02-01-era test helpers (blob-url-download / meta-json-urls-schema / strict-meta-json-validation) now simulate the SW↔offscreen mint round-trip and can extract archive bytes from CREATE_DOWNLOAD_URL.dataBase64.
## Task Commits
Each task was committed atomically:
1. **Task 1: Extend PortMessageType + PortMessage**`483998d` (feat)
2. **Task 2: Offscreen handler — CREATE/REVOKE Blob URL handlers**`f0b95f4` (feat)
3. **Task 3: SW downloadArchive rewrite + chrome.downloads.onChanged revoke wiring**`79964e6` (feat, includes Rule 3 test-harness deviation fixes)
## Files Created/Modified
### Production source
- **`src/shared/types.ts`** (+33 lines)
- `PortMessageType` union grows 4 → 7 entries with the D-P2-01 triplet.
- `PortMessage` interface gains optional `dataBase64?: string`, `mimeType?: string`, `url?: string`.
- Top-of-union docstring explaining the SW↔offscreen mint/revoke lifecycle.
- **`src/offscreen/recorder.ts`** (+107 lines)
- `mintedDownloadUrls: Set<string>` hygiene tracker (purely diagnostic — WHATWG spec says URL.revokeObjectURL on unknown URL is a no-op).
- `handleCreateDownloadUrl(requestId, dataBase64, mimeType)` async helper: decodes base64 → constructs Blob → mints URL.createObjectURL → posts DOWNLOAD_URL on keepalivePort. Best-effort error path responds with empty url so the SW's typed-error path can fire cleanly.
- `onPortMessage` extended with CREATE_DOWNLOAD_URL and REVOKE_DOWNLOAD_URL branches matching the existing PONG / REQUEST_BUFFER style.
- `import { blobToBase64, base64ToBlob }` (was: blobToBase64 only).
- **`src/background/index.ts`** (+172 lines)
- `pendingDownloadUrlResolvers: Map<string, (url: string) => void>` and `pendingRevokes: Map<number, string>` module-scoped state, with docstrings citing T-02-02-02.
- `BLOB_URL_MINT_TIMEOUT_MS = 5_000`.
- `onConnect` port message sink extended with `DOWNLOAD_URL` routing branch (alongside existing PING / BUFFER routing).
- `downloadArchive(archiveBlob)` rewritten end-to-end (encode → post CREATE_DOWNLOAD_URL → race DOWNLOAD_URL vs timeout → guard blob: prefix → call chrome.downloads.download → register in pendingRevokes).
- `chrome.downloads.onChanged.addListener` registered at module init (try/catch defensive like other listener registrations); on terminal state, posts REVOKE_DOWNLOAD_URL to `videoPort` and clears `pendingRevokes`.
- Comment cleanup: removed the literal `data:application/zip;base64,` string from the docstring to satisfy the plan §verification grep gate.
### Test infrastructure (Rule 3 deviation — see below)
- **`tests/background/blob-url-download.test.ts`** (+111 lines)
- `runSaveAndCaptureDownloadArg` helper extended with `tryFireDownloadUrl` + `mintedUrlsHere` set; replies DOWNLOAD_URL with a Node-native blob:URL on receipt of CREATE_DOWNLOAD_URL.
- Test 2 (6 MB archive) drain loop mirrors the same offscreen-side simulation.
- Test 3 (revoke lifecycle) installs a `port.postMessage.mockImplementation` shim that calls `URL.revokeObjectURL` on receipt of REVOKE_DOWNLOAD_URL — the test-side equivalent of `handleRevokeDownloadUrl` in src/offscreen/recorder.ts.
- **`tests/background/meta-json-urls-schema.test.ts`** (+62 lines)
- `runSaveAndCaptureArchiveBlob` extended with the same `tryFireDownloadUrl` pattern; archive bytes captured from CREATE_DOWNLOAD_URL.dataBase64 (not from the opaque blob:URL string).
- **`tests/build/strict-meta-json-validation.test.ts`** (+68 lines)
- `runAndParseMetaJson` extended identically.
## Decisions Made
See `key-decisions` in the frontmatter (7 decisions covering wire-format reuse, no-fallback contract, T-02-02-03 mitigation, concurrent-mint policy, requestId helper reuse, and test-side bytes-capture pattern).
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] Plan 02-01 test helpers missing CREATE_DOWNLOAD_URL → DOWNLOAD_URL round-trip simulation**
- **Found during:** Task 3 (SW downloadArchive rewrite) — first verification run.
- **Issue:** The Plan 02-01 RED tests (and the related Plan 02-03-territory helpers in meta-json-urls-schema.test.ts and strict-meta-json-validation.test.ts) were authored with helpers that ONLY simulated the REQUEST_BUFFER → BUFFER round-trip. After my Plan 02-02 changes, the SW's `downloadArchive` posts CREATE_DOWNLOAD_URL on the offscreen port and awaits DOWNLOAD_URL — but the test stubs do not load `src/offscreen/recorder.ts`. With no test-side response to CREATE_DOWNLOAD_URL, the SW's 5 s mint timeout fires, `chrome.downloads.download` is never called, and ALL three sets of helpers time out.
- **Failure shape:** ALL 13 vitest failures were `Test timed out in 5000ms` (drain loop never sees `mock.calls.length > 0`) — including the 5 GREEN-today regression-guard tests in strict-meta-json-validation.test.ts (Tests 2/4/5/6/7) that Plan 02-01 SUMMARY explicitly listed as "preserved unchanged".
- **Fix:** Extended each helper to model the offscreen side of the bridge. The helper detects port.postMessage calls of type CREATE_DOWNLOAD_URL, decodes the dataBase64 to a Uint8Array, mints a Node-native blob:URL via URL.createObjectURL (Node 24 supplies it as a global), captures the bytes for downstream JSZip extraction (capturedArchiveBytes), and replies DOWNLOAD_URL on the same port. For Test 3 (revoke lifecycle) additionally installs `port.postMessage.mockImplementation` to call URL.revokeObjectURL on receipt of REVOKE_DOWNLOAD_URL — the test-side equivalent of src/offscreen/recorder.ts handleCreateDownloadUrl / handleRevokeDownloadUrl.
- **Why this is Rule 3 not Rule 4:** Plan 02-01's test authors explicitly anticipated this extension via TODO comments ("Plan 02-03 implementer will likely need a different helper, e.g. spy on URL.createObjectURL to capture the underlying Blob reference"). The bytes-capture pattern via the port message is cleaner than the spy-on-globals approach and provides identical semantics. No architectural change; just test infrastructure catching up to the new contract.
- **Files modified:** tests/background/blob-url-download.test.ts, tests/background/meta-json-urls-schema.test.ts, tests/build/strict-meta-json-validation.test.ts.
- **Verification:**
- blob-url-download: 3/3 GREEN (the plan's target).
- meta-json-urls-schema: 5/5 fail with the SAME Plan-02-03-territory error messages they had at baseline (missing tab-url-tracker module, missing urls field). No regression in failure mode.
- strict-meta-json-validation: 3 RED + 5 GREEN regression-guards (matches Plan 02-01 SUMMARY's "8 tests, 3 RED + 5 GREEN-today" claim).
- **Committed in:** `79964e6` (part of Task 3 commit).
---
**Total deviations:** 1 auto-fixed (Rule 3 — blocking)
**Impact on plan:** The deviation extends test infrastructure to validate the contract Task 3 introduces. No architectural change; no scope creep beyond what Plan 02-01's authors explicitly anticipated. The bytes-capture pattern via port message is the canonical Plan-02-02-era equivalent of the deprecated data:URL base64-decode pathway and is reusable by Plan 02-03.
## Issues Encountered
None. The plan-checker passed Plan 02-02 cleanly and the implementation matched the plan's `<action>` blocks closely. The one source of friction — the test infrastructure deviation above — was anticipated by the Plan 02-01 SUMMARY (note 7 of key-decisions) and resolved with a pattern that improves clarity (port-message bytes capture vs spy-on-globals).
## User Setup Required
None. The migration is fully internal — no manifest permission changes, no new chrome.* surfaces (chrome.downloads.onChanged was already available under the existing `downloads` permission per Plan 01 manifest baseline), no operator-visible configuration. The 6 MB archive that previously failed with 'Network error' will now download cleanly on the next operator save.
## Next Phase Readiness
- **Plan 02-03 (next, Wave 2 GREEN — meta.urls + tab-url-tracker):** ready. Plan 02-03's implementer will extend its own test helpers using the same `tryFireDownloadUrl` + `capturedArchiveBytes` pattern this plan establishes — the three test files already showcase it. The 5 RED tests in meta-json-urls-schema.test.ts and the 3 RED tests in strict-meta-json-validation.test.ts remain RED, exactly as Plan 02-01 SUMMARY's "Plan 02-03 territory" forward link predicted.
- **Plan 02-04 (Wave 3 — UAT harness extensions):** ready. The chrome.downloads.onChanged listener is now reachable by the UAT harness; A24+ assertions can probe the produced zip + verify blob: URL via DevTools Network panel as the empirical close on D-P2-01.
## Verification Reconciliation
| Plan §verification gate | Result |
|---|---|
| `npx tsc --noEmit` | clean |
| `npm run build` | clean (377 kB SW chunk; ~ +2 kB vs baseline for the new listener + Map state) |
| `npm run build:test` | clean (dist-test/) |
| `npx vitest run tests/background/blob-url-download.test.ts` | **3/3 GREEN** (was 3 RED — the plan's target) |
| `npx vitest run` (full suite) | **163 passed / 8 failed / 171 total** — +4 GREEN net vs Plan 02-01 baseline (159 passed / 12 failed). The 8 remaining RED are exactly the Plan 02-03 territory (5 meta-json-urls-schema + 3 strict-meta-json-validation). |
| Tier-1 grep gate (FORBIDDEN_HOOK_STRINGS) | **13/13 GREEN** (unchanged — no new test-hook surfaces; the D-P2-01 triplet is production surface) |
| `grep -c "data:application/zip;base64," src/background/index.ts` | **0** (legacy path gone, including the docstring mention which was rewritten to avoid the literal) |
| `grep -c "blob:" src/background/index.ts` | **8** (the new pipeline) |
| `grep -c "chrome.downloads.onChanged" src/background/index.ts` | **5** (listener wired + docstring references) |
| dist/ `data:application/zip;base64,` | **0 matches** |
| dist/ `chrome.downloads.onChanged` | 1 file (the SW chunk) |
| Always-on charter (no finally in saveArchive) | **preserved** |
| `await import(...)` in src/background/index.ts | **0** (Plan 01-11 SUMMARY invariant preserved) |
## Self-Check: PASSED
### Files modified (verified via git diff)
- FOUND: src/shared/types.ts (+33 lines)
- FOUND: src/offscreen/recorder.ts (+107 lines)
- FOUND: src/background/index.ts (+172 lines)
- FOUND: tests/background/blob-url-download.test.ts (+111 lines, Rule 3 deviation)
- FOUND: tests/background/meta-json-urls-schema.test.ts (+62 lines, Rule 3 deviation)
- FOUND: tests/build/strict-meta-json-validation.test.ts (+68 lines, Rule 3 deviation)
### Commits exist (verified via git log)
- FOUND: 483998d feat(02-02): wire-format — extend PortMessage with CREATE_DOWNLOAD_URL/DOWNLOAD_URL/REVOKE_DOWNLOAD_URL (D-P2-01)
- FOUND: f0b95f4 feat(02-02): offscreen — CREATE/REVOKE Blob URL handlers on keepalivePort (D-P2-01)
- FOUND: 79964e6 feat(02-02): SW — downloadArchive via offscreen-minted Blob URL + revoke lifecycle (D-P2-01 closes P0-6)
### Success criteria reconciliation (from prompt)
1. [x] All 3 tasks in plan executed.
2. [x] Each task committed individually (--no-verify per parallel-execution mandate).
3. [x] 3 RED tests in tests/background/blob-url-download.test.ts now GREEN.
4. [x] 160 pre-existing GREEN tests still GREEN (no regression — full-suite delta is +4 GREEN, not -anything).
5. [x] Tier-1 grep gate still 13 entries (no new test-hook surfaces; CREATE_DOWNLOAD_URL / DOWNLOAD_URL / REVOKE_DOWNLOAD_URL are production message types riding existing keepalivePort).
6. [x] No modifications to meta.json schema or SessionMetadata fields (Plan 02-03 territory untouched).
7. [x] SUMMARY.md created and (about to be) committed.
8. [x] No modifications to STATE.md or ROADMAP.md (worktree mode invariant).
---
*Phase: 02-stabilize-export-pipeline*
*Completed: 2026-05-20*