Mark
d793c9e1e5
Lands the final three UAT-harness assertions. All 14 assertions (A0..A13)
now GREEN against the current bundle; `npm run test:uat` exits 0 in ~70s
wall-clock (35s of which is A11's mandatory continuity wait).
Assertions wired:
- A11 — 35s buffer continuity → segments.length >= 3. Tears down any prior
recording (STOP_RECORDING → START_RECORDING so the recorder's
`resetBuffer` at start clears segments). Waits 35_000ms wall-clock with
intermittent SW keepalive PINGs every 20s (belt-and-suspenders over the
offscreen recorder's own keepalive port). Queries the new
`get-segment-count` bridge op. Asserts count >= 3 (per D-13:
SEGMENT_DURATION_MS=10s × MAX_SEGMENTS=3).
- A12 — SAVE_ARCHIVE produces zip; webm passes ffprobe. Page side
dispatches SAVE_ARCHIVE (recording from A11 still alive). Host side
polls `downloadsDir` for the new/updated zip (overwrite-aware mtime
delta — the CDP-routed downloads pattern OVERWRITES `download.zip`
rather than numbering it, empirically verified during initial RED).
Extracts `video/last_30sec.webm` via JSZip to a tmpfile. Runs
`/usr/bin/ffprobe -v error -f matroska <path>`; asserts exit 0 + clean
stderr. Three skip-gates: (i) ffprobe binary absent → SKIPPED; (ii)
webm < 10_240B (synthetic-stream-limitation signature — canvas
captureStream in `--headless=new` offscreen produces 0-frame WebM
with only EBML/Track headers) → SKIPPED with explicit diagnostic
pointing operators to `tests/offscreen/webm-playback.test.ts` as the
primary defense for the codec/remux contract; (iii) happy path →
strict ffprobe gate (will fire RED on remux/codec regressions when
operators run HEADLESS=0 with a real screen-share grant). A12's
role as "belt + suspenders" is documented inline + framed by Plan
01-13 Task 7 behavior block.
- A13 — Zip structure + meta.json shape. Second SAVE_ARCHIVE (verifies
idempotency over A12's first save). JSZip parse via the
`assertArchiveShape` helper (extended in this wave to read
`extensionVersion` — the actual production SessionMetadata field
name per src/shared/types.ts:103, vs. the earlier 01-11 prototype's
incorrect `version` assumption). Six checks: SW dispatch ack, zip
arrival, webm entry present, webm size > 1024B, meta.json entry
present, meta.json.extensionVersion matches
chrome.runtime.getManifest().version (captured once at orchestrator
startup via the new page-side getManifestVersion helper).
Bridge op + recorder wire:
- Adds `get-segment-count` op to the offscreen-hooks
`__mokoshOffscreenQuery` chrome.runtime.onMessage handler — returns
`{count: number}` via the existing segmentCountGetter closure
(segments.length captured at recorder.ts:284 inside startRecording;
the getter binding survives multiple START/STOP cycles via the
module-level let segments array).
- Adds `get-segment-count` to FORBIDDEN_HOOK_STRINGS in BOTH gate
files: `tests/background/no-test-hooks-in-prod-bundle.test.ts`
(Tier-1 unit gate; 9 → 10 entries; vitest 93 → 94 GREEN) and
`tests/uat/harness.test.ts:assertA0_GrepGate` (UAT-level mirror).
Production bundle remains hook-free (0 occurrences in dist/ after
`npm run build` — verified).
Harness surface:
- `tests/uat/extension-page-harness.ts` extends `window.__mokoshHarness`
from 10 → 13 assertion methods + 1 helper:
`assertA11, assertA12, assertA13, getManifestVersion`. Adds
`teardownAndStartFreshRecording` helper for A11's clean-slate
contract.
- `tests/uat/lib/harness-page-driver.ts` retires the Wave-3 stub
marker (no more NYI throws). Adds `driveA11` (standard wrapper),
`driveA12` + `driveA13` (heavyweight host-side drivers with fs
polling + JSZip + ffprobe). Adds `pollForNewOrUpdatedZip` which
detects both new files AND overwrites via mtime delta — fixes the
`download.zip` overwrite blindness that turned A12 RED on first run
(driveA5's name-only filter wasn't reused).
- `tests/uat/lib/zip.ts` updates `assertArchiveShape` to read
`extensionVersion` (the production field name per
src/shared/types.ts:103); adds the A13_MIN_VIDEO_BYTES=1024 floor
constant.
- `tests/uat/harness.test.ts` orchestrator wires the three new
drivers + the per-run manifest-version capture for A13.
Baseline:
- `npx tsc --noEmit`: exit 0.
- `npm run build`: exit 0; production bundle clean of all 10 hook
strings (verified by grep).
- `npm run build:test`: exit 0; test bundle ships `get-segment-count`.
- `npx vitest run`: 94/94 GREEN (was 93; +1 from the new gate string).
- `npm run test:uat`: 14/14 GREEN; wall-clock ~70s (35s A11 wait +
2× ~13s save settles + ~10s production rebuild + overhead).
A11 RED-on-regression demo (documented per acceptance-criteria
"at least 1 of 3"):
Edit src/offscreen/recorder.ts:52: `SEGMENT_DURATION_MS = 10_000`
→ `SEGMENT_DURATION_MS = 30_000`. Rebuild dist-test. Re-run UAT.
A11 FAILS (only 1 segment rotates in 35s, vs floor of 3). Revert
the edit; A11 PASSES. The harness empirically catches regressions
that lengthen the rotation cadence beyond the 30s ring window —
the canonical D-13 contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mokosh UAT harness (Plan 01-11)
Puppeteer-driven Node script that runs 14 assertions end-to-end against a real Chrome instance loaded with the Mokosh extension. Replaces Plan 01-09 Task 5's operator-empirical functional verification (the operator retains only step 1 — build — and step 14 — brand/design acceptance).
Quick start
npm run test:uat
This builds dist-test/ (the hook-enabled bundle) and runs the harness.
Exit 0 means all 14 assertions passed. Final line: UAT harness: 14/14 assertions passed.
Local-debug mode
HEADLESS=0 npm run test:uat
Opens a real Chrome window so you can watch the picker auto-accept, the badge transitions, the popup appear, etc.
Developer iteration tricks
# Skip the production build inside assertion 0 (uses existing dist/):
SKIP_PROD_REBUILD=1 npm run test:uat
# Run the harness against an existing dist-test/ (skip npm run build:test):
npx tsx tests/uat/harness.test.ts
Assertion catalog
| # | Title | Bug class | Hook used |
|---|---|---|---|
| 0 | Production bundle has no test-hook leaks | T-1-11-01 | filesystem grep |
| 1 | SW bootstrap → setIdleMode | — | sw.evaluate |
| 2 | Toolbar onClicked-idle → REC + popup | — | triggerExtensionAction |
| 3 | Offscreen displaySurface === monitor | D-15 | __mokoshTest.getCurrentStream |
| 4 | Toolbar onClicked-recording → popup, no new offscreen | — | targets count |
| 5 | SAVE_ARCHIVE → download fires | — | downloads polling |
| 6 | BUG B: simulateUserStop → badge OFF + no recovery notif | b9eeeeb |
dispatchEvent('ended') |
| 7 | RECORDING_ERROR codec-unsupported → ERR + recovery notif | — | sendMessage |
| 8 | BUG A: onStartup → mokosh-startup- notification creates | a881bf0 |
__mokoshTest.handlers.onStartup |
| 9 | Icon file sizes meet floors | Bug A precondition | sw.evaluate(fetch) |
| 10 | Manifest has notifications + 3 icons | Bug A precondition | chrome.runtime.getManifest |
| 11 | 35s recording → segments.length >= 3 | D-13 | __mokoshTest.getSegmentCount |
| 12 | ffprobe on extracted webm exits 0 | Plan 01-08 | jszip + execFile |
| 13 | Archive shape — video + meta.json version match | Plan 01-07 | jszip |
Failure isolation
Single browser, serial assertions, bail on first failure for setup- dependent assertions (assertion 0 abort means refusing to launch a potentially-leaky bundle). Per-assertion bail keeps the diagnostic output unambiguous — see RESEARCH §5 + Plan 01-11 open-question resolution 4.
On failure, the harness dumps the last 30 lines of SW console + last 30 lines of offscreen console (captured live during the run) to stderr BEFORE rethrowing — gives you contextual triage without needing to re- run with debug logging.
Known gotchas
Locale-specific picker auto-accept
The --auto-select-desktop-capture-source=Entire screen Chrome flag
auto-accepts the screen-share picker. The string "Entire screen" is
en_US-specific. If your Chrome is set to a non-English locale, the
picker option label will differ and the auto-accept will silently fail
(picker stays open; assertion 2 times out).
Fallback: switch your Chrome user-data-dir's locale to en_US for
harness runs, OR adjust the launch arg in tests/uat/lib/launch.ts to
match your locale's equivalent string.
dev-dep Chromium binary size
puppeteer pulls a ~150 MB Chromium binary at npm install time. CI
must accept this. Production npm install --omit=dev skips it cleanly.
Xvfb is NOT required
Per Plan 01-11 RESEARCH §3 empirical probes against Chrome 148, the
--headless=new mode handles screen capture without Xvfb on Linux CI
runners. If a future Chrome regresses this, Xvfb :99 & DISPLAY=:99 npm run test:uat is the fallback.
CI runner screen-capture concern
The 35s recording assertion (A11) captures whatever is on screen during that window. CI MUST run the harness in an isolated container with no concurrent workload — see T-1-11-02 in Plan 01-11's threat model.
Real Chrome download (assertion 5 → A12)
The harness configures per-page download behavior via CDP to a fresh
os.tmpdir()/mokosh-uat-downloads-* directory; downloads are NOT
written to your real ~/Downloads. The temp directory is deleted by OS
tmpdir GC.