From a991e1732adba85a88e5da8535331bf67c2c0c47 Mon Sep 17 00:00:00 2001 From: Mark Date: Wed, 20 May 2026 15:36:09 +0200 Subject: [PATCH] =?UTF-8?q?docs(02-01):=20complete=20RED=20gate=20?= =?UTF-8?q?=E2=80=94=203=20test=20files=20pin=20D-P2-01=20+=20D-P2-02=20+?= =?UTF-8?q?=20D-P2-03=20+=20F2?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan 02-01 Wave 0 RED gate closed. Three failing test files (16 it() blocks total: 11 RED + 5 GREEN regression guards) pin the locked decisions for Phase 2 ahead of Plans 02-02 + 02-03 implementation: - blob-url-download.test.ts (3 RED) — D-P2-01 offscreen Blob URL pipeline (closes audit P0-6: base64 data: URL → blob: URL). - meta-json-urls-schema.test.ts (5 RED) — D-P2-02 meta.url → meta.urls migration + F2 empty-tracker → urls:[] resolution. - strict-meta-json-validation.test.ts (3 RED + 5 GREEN) — D-P2-03 strict 8-field schema validation with EXPECTED_KEYS pin including planner-suggested `schemaVersion` 8th field. Test count delta: 155 GREEN → 159 GREEN + 11 RED (+4 GREEN regression guards, +11 RED test contracts). Vitest reporter: Test Files 4 failed | 27 passed (31) Tests 12 failed | 159 passed (171) (12 failed = 3 + 5 + 3 RED from this plan + 1 pre-existing flaky ffprobe test in webm-remux.test.ts — out of scope; documented in SUMMARY.md Deferred Issues.) Tier-1 grep gate: 13/13 GREEN preserved (this plan touches no production code). Planner-resolved tensions carried forward in SUMMARY.md: - D-P2-03 'non-empty urls[]' vs CONTEXT.md permissive empty-array → F2 resolved in favor of permissive (Test 3 of Task 3 relaxed). - 8th field name `schemaVersion` → tentative planner pick; Plan 02-03 implementer commits to schemaVersion: '2' const. - tab-url-tracker module seam → planner-suggested name `src/background/tab-url-tracker.ts` with getTabUrlsSeen() export. - Plan claim 'ALL 8 fail' reconciled honestly: 3 RED + 5 GREEN regression guards (timestamp/semver/totalEvents/buffer-seconds/ duration-minutes already match current 7-field shape). Plan suggestions reconciled with reality: - vitest env: 'node' not 'jsdom' (Node 24 has URL/Blob/performance globals; jsdom not in devDeps). FileReader polyfill inline. - Task 2 Test 1 source-text scan instead of tsc-compile-failure (vitest.config.ts typecheck:{enabled:false}). Per worktree-mode constraint: STATE.md, ROADMAP.md, REQUIREMENTS.md NOT modified. The orchestrator owns those writes after all worktree agents in Wave 0 complete. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../02-01-SUMMARY.md | 223 ++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 .planning/phases/02-stabilize-export-pipeline/02-01-SUMMARY.md diff --git a/.planning/phases/02-stabilize-export-pipeline/02-01-SUMMARY.md b/.planning/phases/02-stabilize-export-pipeline/02-01-SUMMARY.md new file mode 100644 index 0000000..6e1e87e --- /dev/null +++ b/.planning/phases/02-stabilize-export-pipeline/02-01-SUMMARY.md @@ -0,0 +1,223 @@ +--- +phase: 02-stabilize-export-pipeline +plan: 01 +subsystem: testing +tags: + - tdd + - red-tests + - blob-url-migration + - meta-json-urls-array + - schema-validation + - p0-6 + - p1-10 + - wave-0 + - export-pipeline + +# Dependency graph +requires: + - phase: 01-stabilize-video-pipeline + provides: createArchive + remuxSegments + chrome stub patterns + raw-3ebml-concat.webm fixture (Plan 01-08 Wave 0 RED precedent) +provides: + - "3 failing test files (16 it() blocks total) pinning D-P2-01 + D-P2-02 + D-P2-03 + F2 contracts ahead of Plans 02-02 and 02-03 implementation" + - "Blob URL download wire-format contract (chrome.downloads.download arg0.url starts with blob:, not data:application/zip;base64,)" + - "URL.revokeObjectURL lifecycle contract (revoke fires on chrome.downloads.onChanged 'complete')" + - "SessionMetadata.urls: string[] type-shape contract via source-text scan" + - "createArchive meta.json shape contract (urls Array, no url field)" + - "tab-url-tracker module seam (src/background/tab-url-tracker.ts with getTabUrlsSeen(): string[])" + - "URL filter contract (include https + chrome-extension://; exclude chrome:// + about:)" + - "Empty-tracker F2 contract (meta.urls === [] for whole-desktop-no-tab sessions)" + - "Strict 8-field meta.json schema with EXPECTED_KEYS pin including planner-suggested schemaVersion 8th field" + - "FileReader polyfill pattern for vitest Node-env Blob ingestion through JSZip" +affects: + - Plan 02-02 (offscreen Blob URL pipeline implementation) + - Plan 02-03 (meta.json urls[] migration + tab-url-tracker + schema validation) + - Plan 02-04 (UAT harness extensions A24+ — pre-checkpoint bundle gates) + - All future phases that consume meta.json (downstream UAT harness, server-upload candidates) + +# Tech tracking +tech-stack: + added: + - "FileReader polyfill (Node 24 lacks FileReader; JSZip needs it for Blob ingestion)" + - "vi.doMock for remuxSegments short-circuit (6 MB synthetic stress test)" + - "Source-text scan for type-shape pinning (workaround for vitest.config.ts typecheck:{enabled:false})" + patterns: + - "RED-gate triple-test pattern: WIRE-FORMAT polarity guard + LATENCY budget + LIFECYCLE pin (Test 2's necessary-not-sufficient latency + wire-format combo)" + - "Missing-module RED via dynamic-import-in-try/catch + expect.fail (mirrors Plan 01-08 webm-remux.test.ts precedent)" + - "Three-slice fixture pattern (SEG1/SEG2/SEG3 byte offsets from raw-3ebml-concat.webm) for muxer-safe segmentation" + - "Per-test SAVE_ARCHIVE drain helper: dispatch SAVE_ARCHIVE → resolve REQUEST_BUFFER with synthetic segments → loop until chrome.downloads.download called" + +key-files: + created: + - "tests/background/blob-url-download.test.ts (3 RED tests, ~625 lines; pins D-P2-01)" + - "tests/background/meta-json-urls-schema.test.ts (5 RED tests, ~671 lines; pins D-P2-02 + F2)" + - "tests/build/strict-meta-json-validation.test.ts (3 RED + 5 GREEN regression-guard tests, ~621 lines; pins D-P2-03 + F2)" + modified: [] + +key-decisions: + - "Use vitest default `node` env instead of jsdom: Node 24 ships URL.createObjectURL + URL.revokeObjectURL + performance + Blob as globals. Plan's suggestion to use jsdom was incorrect (vitest.config.ts explicitly sets environment: 'node' and jsdom is not in devDependencies)." + - "FileReader polyfill inline in each test file: Node 24 lacks FileReader; JSZip's Blob ingestion path needs it. Minimal polyfill (delegates to Blob.arrayBuffer()) installed at bootSwInRecState time." + - "vi.doMock('../../src/background/webm-remux') for Test 2's 6 MB scale: production muxer rejects monotonicity-breaking input shapes, so a synthetic 6 MB Blob via vi.doMock is the cleanest way to stress the download wire-format under the data-URL-cap-exceeding scale." + - "Test 1 (SessionMetadata shape pin) uses source-text scan instead of tsc compile failure: vitest.config.ts has typecheck:{enabled:false}, so a tsc-failure-based pin would be a no-op. Source-text regex against src/shared/types.ts is the RED gate that survives the typecheck-disabled config." + - "EXPECTED_KEYS 8th field tentatively `schemaVersion`: marks D-P2-02 url→urls breaking-change cutover. Plan 02-03's implementer commits to adding `schemaVersion: '2'` as a constant. Test file is canonical place to amend if a stronger 8th-field argument surfaces." + - "Tests 2, 4, 5, 6, 7 of Task 3 stay GREEN-today as regression guards: plan's 'ALL 8 fail' claim wasn't achievable (timestamp/semver/totalEvents/videoBufferSeconds/logDurationMinutes already match the current 7-field shape). Documented honestly; the GREEN-today tests pin the post-migration schema for regression-guard duty." + - "tab-url-tracker module seam: src/background/tab-url-tracker.ts with export `getTabUrlsSeen(): string[]`. Tests 3+4+5 of Task 2 use dynamic-import-with-expect.fail to surface the missing module as the RED signal. Plan 02-03 implements the module." + +patterns-established: + - "RED-gate triple-pin pattern: (1) precondition (download was called), (2) polarity guard (forbidden prefix absent), (3) target contract (required prefix present). Test 1 of blob-url-download.test.ts exemplifies." + - "Wire-format-at-scale pattern: synthetic-payload stress test + wire-format guard. The latency check is necessary but not sufficient — the wire-format guard converts a sub-budget spurious pass into a meaningful RED. Test 2 of blob-url-download.test.ts." + - "Source-text type-shape pin: when vitest typecheck is disabled, fall back to a regex scan of the source file body. Captures both presence (new field exists) and absence (old field removed) in one test." + - "Per-test SAVE_ARCHIVE driver helper: factored out runSaveAndCaptureArchiveBlob / runSaveAndCaptureDownloadArg / runAndParseMetaJson with a shared try-fire-REQUEST_BUFFER-loop. Future Phase 2 tests inherit this pattern." + - "GREEN-today regression-guard tests bundled with RED-today migration tests: same describe block, same setup, but the assertions are scoped to pin different layers (migration RED + stability GREEN). Future TDD plans can mix these explicitly." + +requirements-completed: [] +# Per plan instructions: requirements (REQ-archive-export-latency, +# REQ-meta-json-schema) are NOT completed by this plan — they're +# pinned by RED tests here and IMPLEMENTED + GREEN-flipped by Plans +# 02-02 + 02-03. Marking them complete now would be incorrect per +# CLAUDE.md "no premature requirement marking" lesson from Plan 01-01 +# (see STATE.md "[Phase ?]: Reverted premature REQ-video-ring-buffer +# Complete marking left by Plan 01-01"). + +# Metrics +duration: ~20min +completed: 2026-05-20 +--- + +# Phase 02 Plan 01: Wave-0 RED tests for export pipeline migration + +**Three failing test files (16 it() blocks) pin D-P2-01 Blob URL download contract + D-P2-02 meta.json urls[] migration + D-P2-03 strict 8-field schema validation + F2 empty-tracker resolution, ahead of Plan 02-02 + 02-03 implementation.** + +## Performance + +- **Duration:** ~20 min +- **Started:** 2026-05-20T13:13:59Z (baseline test capture) +- **Completed:** 2026-05-20T13:33:38Z (final commit + SUMMARY) +- **Tasks:** 3/3 completed +- **Files created:** 3 test files (1 917 lines total) +- **Files modified:** 0 production-code files (this is the RED gate ONLY) + +## Accomplishments + +### Task 1: tests/background/blob-url-download.test.ts (3 RED) + +Pins D-P2-01 (offscreen-minted Blob URL pipeline; closes audit P0-6). + +- **Test 1 (wire-format polarity guard):** `chrome.downloads.download` is called with a `url` that starts with `blob:` and does NOT start with `data:application/zip;base64,`. RED today; concrete assertion error: `chrome.downloads.download was called with url='data:application/zip;base64,UEsDBAoAAAAAAL1qtFw...' — D-P2-01 forbids data:application/zip;base64, prefix`. +- **Test 2 (latency + wire-format at 6 MB scale):** A 6 MB archive completes in under 5 000 ms AND emits a `blob:` URL. The vi.doMock on remuxSegments short-circuits the muxer to inject a synthetic 6 MB Blob. RED today via the wire-format guard (data: URL prefix); GREEN after Plan 02-02. Elapsed ~1 266 ms (well under budget). +- **Test 3 (revoke lifecycle):** `URL.revokeObjectURL` is scheduled with the same URL passed to `chrome.downloads.download` once `chrome.downloads.onChanged` reports `state.current === 'complete'`. RED today; concrete error: `chrome.downloads.onChanged._callbacks.length === 0 at probe time` (current downloadArchive does not register an onChanged listener). + +**Commit:** [748a81f](#) `test(02-01): RED — pin Blob URL download contract (D-P2-01)` + +### Task 2: tests/background/meta-json-urls-schema.test.ts (5 RED) + +Pins D-P2-02 (meta.json url→urls migration) + F2 empty-tracker resolution. + +- **Test 1 (SessionMetadata type-shape via source-text scan):** `src/shared/types.ts` MUST contain `urls: string[]` AND MUST NOT contain `url:` (singular) inside the SessionMetadata interface body. Workaround for vitest typecheck:{enabled:false}. RED today; concrete error: `'urls: string[]' field not present`. +- **Test 2 (createArchive meta.json shape):** Real createArchive output has `Array.isArray(meta.urls) === true` AND `meta.url === undefined`. RED today; concrete error: `meta.urls is not an Array. Got: undefined`. +- **Tests 3+4+5 (tab-url-tracker module seam):** Each test dynamically imports `src/background/tab-url-tracker` (the planner-suggested module name from Plan 02-03) and `expect.fail`s with a precise GREEN-gate contract message when the module doesn't exist. RED today; module missing. + - Test 3 pins dedup + first-seen-first order: ['A', 'B', 'A'] → ['A', 'B']. + - Test 4 pins URL filter: include https + chrome-extension://; exclude chrome:// + about:. + - Test 5 pins F2 empty-tracker contract: no observations → `getTabUrlsSeen() === []` (NOT undefined, NOT [], NOT null). + +**Commit:** [9e45d33](#) `test(02-01): RED — pin meta.json urls[] schema + dedup/filter + empty-tracker (D-P2-02 + F2)` + +### Task 3: tests/build/strict-meta-json-validation.test.ts (3 RED + 5 GREEN regression guards) + +Pins D-P2-03 strict 8-field meta.json schema validation. + +- **Test 1 (RED): exactly 8 fields.** Current shape has 7. Concrete error: `meta.json has 7 fields; D-P2-03 requires exactly 8. Current keys: [timestamp, url, userAgent, extensionVersion, videoBufferSeconds, logDurationMinutes, totalEvents]`. +- **Test 2 (GREEN regression guard): ISO-8601 Z-suffix timestamp.** Current `new Date().toISOString()` already matches; catches future locale-formatter regressions. +- **Test 3 (RED): urls Array of valid URLs (empty permitted per F2).** Concrete error: `meta.urls is not an Array. Got: undefined`. +- **Test 4 (GREEN regression guard): semver extensionVersion.** Current manifest.version '1.0.0' matches; catches build-hash regressions. +- **Test 5 (GREEN regression guard): non-negative integer totalEvents.** Current sum-of-event-counts matches; catches negative/float regressions. +- **Test 6 (GREEN regression guard): videoBufferSeconds === 30 literal.** CON-video-window pin. +- **Test 7 (GREEN regression guard): logDurationMinutes === 10 literal.** CON-event-log-window pin. +- **Test 8 (RED): no extra fields beyond EXPECTED_KEYS.** Concrete error: `meta.json contains extra (unexpected) fields: ["url"]`. EXPECTED_KEYS = [timestamp, urls, userAgent, extensionVersion, videoBufferSeconds, logDurationMinutes, totalEvents, schemaVersion]. + +**Commit:** [94e0346](#) `test(02-01): RED — pin strict 8-field meta.json schema validation (D-P2-03)` + +## Test count delta + +- **Before:** 155 GREEN (baseline including sw-bundle-import.test.ts after npm run build). +- **After:** 159 GREEN + 11 RED (11 new RED from this plan + the 5 Task-3 GREEN-today regression guards count as +4 net new GREEN since Test 5 was added per F2; see "Plan vs reality" below). +- **Net delta:** +11 RED, +4 GREEN tests (was 155, now 170 vitest-discovered + 1 ffprobe-flaky out-of-scope; see Deferred Issues). + +Vitest reporter final state: +``` +Test Files 4 failed | 27 passed (31) + Tests 12 failed | 159 passed (171) +``` +(12 failed = 3 [Task 1] + 5 [Task 2] + 3 [Task 3 RED] + 1 [pre-existing ffprobe-flaky webm-remux test — out of scope; see Deferred Issues].) + +## Tier-1 grep gate + +`tests/background/no-test-hooks-in-prod-bundle.test.ts` continues to pass with 13/13 GREEN (FORBIDDEN_HOOK_STRINGS at the existing inventory; this plan touches no production code). + +## Plan-vs-reality reconciliation + +### Plan claim: "ALL 8 [strict-validation] tests MUST fail under current HEAD" + +**Reality:** 3 of 8 are RED-today (Tests 1, 3, 8 — the migration-driven assertions). The other 5 are GREEN-today regression guards (timestamp regex, semver, totalEvents, videoBufferSeconds, logDurationMinutes — all of which already match the current 7-field shape). + +**Resolution:** Honored the plan's spirit (8 strict-validation tests pinning the post-migration schema) while being honest about the test states. The 5 GREEN-today tests are not test pollution — they're regression guards that prevent Plan 02-03 from accidentally regressing the unchanged fields when adding `urls` + `schemaVersion`. + +### Plan suggestion: "Use jsdom env (default in vitest.config.ts)" + +**Reality:** vitest.config.ts explicitly sets `environment: 'node'` and jsdom is not in devDependencies. Node 24 ships URL.createObjectURL + URL.revokeObjectURL + performance + Blob as globals (verified at land time), so jsdom is not required. FileReader is the only missing global; a minimal polyfill inline in each test file delegates to Blob.arrayBuffer(). + +### Plan suggestion: "Test 1 fails at the TypeScript-compile layer" + +**Reality:** vitest.config.ts has `typecheck: { enabled: false }`. A tsc-failure-based pin would be a no-op. Replaced with a source-text regex scan of `src/shared/types.ts` that asserts both presence of `urls: string[]` AND absence of `url:` inside the SessionMetadata interface body. + +## Planner-resolved tensions (carried forward into Plans 02-02 + 02-03) + +- **D-P2-03 "non-empty urls[]" vs CONTEXT.md `` permissive empty-array clause:** resolved in favor of the permissive clause (F2 — empty IS PERMITTED for whole-desktop-no-tab sessions). Test 3 of Task 3 enforces the relaxed contract. +- **8th field name `schemaVersion`:** TENTATIVE PLANNER PICK to mark the D-P2-02 url→urls breaking-change cutover. EXPECTED_KEYS constant in `tests/build/strict-meta-json-validation.test.ts` pins this choice; Plan 02-03's implementer COMMITS to adding `schemaVersion: '2'` as a constant in `src/background/index.ts`. If a stronger argument for a different 8th-field name surfaces, this test file is the canonical place to amend. +- **tab-url-tracker module name `src/background/tab-url-tracker.ts`:** PLANNER-SUGGESTED. Plan 02-03's implementer commits to this name; if renaming is desired, all three Task 2 dynamic-import strings need to update lockstep. +- **`_resetForTesting` / `_observeForTesting` ergonomic hooks on tab-url-tracker:** OPTIONAL contract the test file documents. Plan 02-03 implementer SHOULD expose these for Tests 3+4+5 ergonomic wiring; if absent, the tests need to wire chrome.tabs.onUpdated callbacks directly. + +## Forward links + +- **Plan 02-02 (Wave 1 GREEN):** flips Task 1's 3 RED tests by minting the Blob URL in the offscreen document via `URL.createObjectURL`, transferring it to the SW via a new port-bridge message, and registering the `chrome.downloads.onChanged` listener for `URL.revokeObjectURL` lifecycle. +- **Plan 02-03 (Wave 1 GREEN):** flips Task 2's 5 RED tests + Task 3's 3 RED tests by (a) amending `src/shared/types.ts` to swap `url: string` for `urls: string[]`, (b) implementing `src/background/tab-url-tracker.ts` with `getTabUrlsSeen()` fed by chrome.tabs.onUpdated + chrome.tabs.onActivated listeners (DEC-011 Amendment 1 `tabs` permission), (c) amending `createArchive` in `src/background/index.ts` to call `snapshotOpenTabs() + getTabUrlsSeen()` and write `urls` + `schemaVersion: '2'` to meta.json (removing the old `url:` field). +- **Plan 02-04 (Wave 2 UAT extensions):** harness A24+ assertions read the GREEN-flipped meta.json. A28 (added per F1) pins REQ-archive-layout strict zip-layout (5 entries exact). + +## Deferred Issues + +### Pre-existing flaky test in tests/background/webm-remux.test.ts + +The `ffprobe -count_frames reports between 905 and 912 frames inclusive` test (Plan 01-08 Task 2) fails when run as part of the full-suite parallel runner but passes when run in isolation. The failure surface is intermittent; the input fixture (`raw-3ebml-concat.webm`) and `remuxSegments` output are deterministic. Suspected cause: ffprobe spawnSync timing interaction with parallel vitest workers, or transient tmpdir contention. + +- **Out of scope** for this plan per Plan 02-01 §verification: this plan touches no production code and no Phase 1 test file. +- **Scope boundary**: not directly caused by current plan changes (verified by running the test in isolation post-commit). +- **Recommended owner:** Phase 5 hardening OR a Phase 2 closure debug session if the flake recurs during Plans 02-02 / 02-03 execution. If the flake blocks Phase 2 plan-checker or operator empirical UAT, escalate to a /gsd-debug session. + +## Self-Check: PASSED + +### Files created (verified via stat at SUMMARY commit time) + +- FOUND: tests/background/blob-url-download.test.ts (~625 lines) +- FOUND: tests/background/meta-json-urls-schema.test.ts (~671 lines) +- FOUND: tests/build/strict-meta-json-validation.test.ts (~621 lines) + +### Commits exist (verified via git log) + +- FOUND: 748a81f test(02-01): RED — pin Blob URL download contract (D-P2-01) +- FOUND: 9e45d33 test(02-01): RED — pin meta.json urls[] schema + dedup/filter + empty-tracker (D-P2-02 + F2) +- FOUND: 94e0346 test(02-01): RED — pin strict 8-field meta.json schema validation (D-P2-03) + +### Plan §verification gates + +- Task 1: `npx vitest run tests/background/blob-url-download.test.ts` → 3 failed (3) ✓ +- Task 2: `npx vitest run tests/background/meta-json-urls-schema.test.ts` → 5 failed (5) ✓ +- Task 3: `npx vitest run tests/build/strict-meta-json-validation.test.ts` → 3 failed | 5 passed (8) ✓ (5 GREEN = regression guards per plan-vs-reality reconciliation above) +- Skip discipline: `grep -cE "\.skip\(|\.skip$|\.skip[, ]"` returns 0 for all three files ✓ +- Tier-1 grep gate: `npx vitest run tests/background/no-test-hooks-in-prod-bundle.test.ts` → 13/13 GREEN preserved ✓ + +### Success criteria reconciliation + +1. [x] Three RED test files exist with concrete failure messages under current HEAD (3 + 5 + 3 = 11 RED). +2. [x] `meta-json-urls-schema.test.ts` Test 1 fails (source-text scan; the planner's tsc-compile-failure path was a no-op due to typecheck:{enabled:false}, but the source-text scan is a stronger structural pin that survives the disabled-typecheck config). +3. [x] Tier-1 grep gate (FORBIDDEN_HOOK_STRINGS inventory) remains GREEN (13/13). +4. [x] UAT harness baseline UNCHANGED (test-only changes; this plan touches no production-source code). +5. [x] Commits land with `test(02-01): RED — ...` prefix matching Plan 01-02 / 01-13 RED-test precedent.