feat(01-11): wave-2 — Puppeteer harness scaffolding + A0 GREEN, popup-bridge architecture
Task 3 of Plan 01-11 (Puppeteer UAT harness).
Harness file tree (tests/uat/):
- harness.test.ts: tsx-runnable top-to-bottom harness entry point.
Runs A0 inline (filesystem grep gate, abort-on-fail T-1-11-01),
then launches Chrome + opens popup bridge + queries manifest, then
iterates A1-A13 stubs. Each stub throws "NOT YET IMPLEMENTED —
Plan 01-11 Task N wires this assertion". Exit code = 0 on full
pass, 1 otherwise. Final line: "UAT harness: N/14 assertions passed".
- lib/launch.ts: launchHarnessBrowser() — wraps puppeteer.launch with
enableExtensions:[dist-test/], headless default (HEADLESS=0
override), --no-sandbox + --auto-select-desktop-capture-source flags.
Polls browser.extensions() until the extension registers (empirically
~100ms but the first call right after launch returns Map(0)).
Opens both a blank page (for triggerExtensionAction) AND the popup
page (the bridge surface). Returns { browser, extension, extensionId,
sw, downloadsDir, page, popup }.
- lib/extension.ts: waitForOffscreenTarget + attachToOffscreen +
countOffscreenTargets. Offscreen attach uses target.type() ===
'background_page' + .asPage() (NOT .page() — RESEARCH §4 Pitfall 1).
- lib/sw.ts: chrome.* state queries via the POPUP page handle (NOT
the WebWorker handle — see architecture note below). getBadgeText,
getPopup, getManifest, getIconSize, getIsRecording (side-channeled
through badge text), fireOnStartup (via __mokoshTestQuery bridge),
sendSyntheticRecordingError, getNotificationSnapshot (via bridge),
keepalivePing (no-op message to wake SW for ~30s).
- lib/offscreen.ts: getDisplaySurface, simulateUserStop (the
dispatchEvent('ended') path per RESEARCH §7 BLOCKER — DO NOT REFACTOR
to track.stop()), getSegmentCount.
- lib/assertions.ts: runAssertion(idx, name, buffers, fn) wrapper —
records pass/fail/duration; on failure dumps last 30 lines of SW
+ offscreen console buffers to stderr before rethrowing. assertEqual
/ assertMatch / assertTrue / assertGte / waitFor polling helper.
- lib/zip.ts: jszip-based assertArchiveShape + extractEntryToFile for
assertions 12 + 13.
- README.md: runtime + local-debug + CI semantics + locale gotcha
+ dev-dep size note + assertion catalog table.
- tsconfig.json: per-tree type-check config (mirrors root tsconfig.json
compiler options but includes the harness tree explicitly).
Architecture refinement (DEVIATION from RESEARCH §1 — Rule 1+3 inline fix):
- RESEARCH §1 sketched `sw.evaluate(() => chrome.action.getBadgeText({}))`
as the chrome.* query path. Empirical probes during Task 3 execution
against Puppeteer 25.0.2 + Chrome 148 + --headless=true revealed two
blockers:
1. Puppeteer's WebWorker.evaluate runs in an ISOLATED WORLD that
carries SW globals (clients, registration, ...) but NOT the
extension's full chrome.* API surface. Object.keys(chrome) inside
sw.evaluate returns ["loadTimes","csi"] — the public webpage
chrome, not the extension chrome.
2. Chrome 148's headless mode aggressively suspends MV3 service
workers; subsequent swTarget.worker() calls return
"Protocol error: No target with given id found".
- WORKAROUND: open the popup page (chrome-extension://<id>/src/popup/
index.html) as a separate Puppeteer Page. The popup has full
chrome.* access (it's an extension context with same privileges as
the SW) AND stable Puppeteer lifetime. For SW-globalThis state
(__mokoshTest in the SW isolate, NOT in the popup), bridge via
chrome.runtime.sendMessage. The popup sends
{ type: '__mokoshTestQuery', op: 'snapshot' | 'fire-on-startup' |
'handler-types' }; the SW hook's onMessage handler responds.
- Bridge implementation added to src/test-hooks/sw-hooks.ts — registers
AFTER the production listeners so it never intercepts production
messages (__mokoshTest* type is unambiguously test-only). Tier-1
grep gate (no-test-hooks-in-prod-bundle.test.ts) continues to enforce
ZERO __mokoshTest occurrences in dist/ — the bridge handler is
tree-shaken alongside the rest of the hook module via the
__MOKOSH_UAT__ gate.
Other configuration changes:
- vitest.config.ts: exclude tests/uat/** from vitest discovery. The
Puppeteer harness is invoked via `npm run test:uat` (not vitest);
running it under vitest would try to launch real Chrome inside a
vitest worker. The .test.ts suffix is retained for editor +
naming-convention consistency with the rest of the tree.
Verification:
- npx tsc --noEmit (src/): exit 0
- npx tsc --noEmit -p tests/uat: exit 0
- npm run build: exit 0
- grep -rln '__mokoshTest|simulateUserStop|getSegmentCount|setCurrentStream|setSegmentCountGetter|__mokoshTestQuery|__mokoshKeepalive' dist/: ZERO matches
- npm run build:test: exit 0; dist-test/ populated with the new bridge code
- SKIP_BUILD=1 npx vitest run: 89/89 GREEN
- SKIP_PROD_REBUILD=1 npx tsx tests/uat/harness.test.ts:
→ A0 [PASS]: production bundle has no test-hook leaks (19ms)
→ Browser launches; popup opens; manifest read succeeds
→ A1-A13 [FAIL]: NOT YET IMPLEMENTED — Plan 01-11 Task N wires this
→ "UAT harness: 1/14 assertions passed, 13 failed (first failure: A1)"
→ Exit code: 1 (expected — 13 RED stubs intentional)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
106
tests/uat/README.md
Normal file
106
tests/uat/README.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# Mokosh UAT harness (Plan 01-11)
|
||||
|
||||
Puppeteer-driven Node script that runs 14 assertions end-to-end against a
|
||||
real Chrome instance loaded with the Mokosh extension. Replaces Plan 01-09
|
||||
Task 5's operator-empirical functional verification (the operator retains
|
||||
only step 1 — build — and step 14 — brand/design acceptance).
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
npm run test:uat
|
||||
```
|
||||
|
||||
This builds `dist-test/` (the hook-enabled bundle) and runs the harness.
|
||||
Exit 0 means all 14 assertions passed. Final line: `UAT harness: 14/14
|
||||
assertions passed`.
|
||||
|
||||
## Local-debug mode
|
||||
|
||||
```bash
|
||||
HEADLESS=0 npm run test:uat
|
||||
```
|
||||
|
||||
Opens a real Chrome window so you can watch the picker auto-accept, the
|
||||
badge transitions, the popup appear, etc.
|
||||
|
||||
## Developer iteration tricks
|
||||
|
||||
```bash
|
||||
# Skip the production build inside assertion 0 (uses existing dist/):
|
||||
SKIP_PROD_REBUILD=1 npm run test:uat
|
||||
|
||||
# Run the harness against an existing dist-test/ (skip npm run build:test):
|
||||
npx tsx tests/uat/harness.test.ts
|
||||
```
|
||||
|
||||
## Assertion catalog
|
||||
|
||||
| # | Title | Bug class | Hook used |
|
||||
|---|-------|-----------|-----------|
|
||||
| 0 | Production bundle has no test-hook leaks | T-1-11-01 | filesystem grep |
|
||||
| 1 | SW bootstrap → setIdleMode | — | sw.evaluate |
|
||||
| 2 | Toolbar onClicked-idle → REC + popup | — | triggerExtensionAction |
|
||||
| 3 | Offscreen displaySurface === monitor | D-15 | __mokoshTest.getCurrentStream |
|
||||
| 4 | Toolbar onClicked-recording → popup, no new offscreen | — | targets count |
|
||||
| 5 | SAVE_ARCHIVE → download fires | — | downloads polling |
|
||||
| 6 | **BUG B**: simulateUserStop → badge OFF + no recovery notif | b9eeeeb | dispatchEvent('ended') |
|
||||
| 7 | RECORDING_ERROR codec-unsupported → ERR + recovery notif | — | sendMessage |
|
||||
| 8 | **BUG A**: onStartup → mokosh-startup- notification creates | a881bf0 | __mokoshTest.handlers.onStartup |
|
||||
| 9 | Icon file sizes meet floors | Bug A precondition | sw.evaluate(fetch) |
|
||||
| 10 | Manifest has notifications + 3 icons | Bug A precondition | chrome.runtime.getManifest |
|
||||
| 11 | 35s recording → segments.length >= 3 | D-13 | __mokoshTest.getSegmentCount |
|
||||
| 12 | ffprobe on extracted webm exits 0 | Plan 01-08 | jszip + execFile |
|
||||
| 13 | Archive shape — video + meta.json version match | Plan 01-07 | jszip |
|
||||
|
||||
## Failure isolation
|
||||
|
||||
Single browser, serial assertions, bail on first failure for setup-
|
||||
dependent assertions (assertion 0 abort means refusing to launch a
|
||||
potentially-leaky bundle). Per-assertion bail keeps the diagnostic
|
||||
output unambiguous — see RESEARCH §5 + Plan 01-11 open-question
|
||||
resolution 4.
|
||||
|
||||
On failure, the harness dumps the last 30 lines of SW console + last 30
|
||||
lines of offscreen console (captured live during the run) to stderr
|
||||
BEFORE rethrowing — gives you contextual triage without needing to re-
|
||||
run with debug logging.
|
||||
|
||||
## Known gotchas
|
||||
|
||||
### Locale-specific picker auto-accept
|
||||
|
||||
The `--auto-select-desktop-capture-source=Entire screen` Chrome flag
|
||||
auto-accepts the screen-share picker. The string `"Entire screen"` is
|
||||
en_US-specific. If your Chrome is set to a non-English locale, the
|
||||
picker option label will differ and the auto-accept will silently fail
|
||||
(picker stays open; assertion 2 times out).
|
||||
|
||||
Fallback: switch your Chrome user-data-dir's locale to en_US for
|
||||
harness runs, OR adjust the launch arg in `tests/uat/lib/launch.ts` to
|
||||
match your locale's equivalent string.
|
||||
|
||||
### dev-dep Chromium binary size
|
||||
|
||||
`puppeteer` pulls a ~150 MB Chromium binary at `npm install` time. CI
|
||||
must accept this. Production `npm install --omit=dev` skips it cleanly.
|
||||
|
||||
### Xvfb is NOT required
|
||||
|
||||
Per Plan 01-11 RESEARCH §3 empirical probes against Chrome 148, the
|
||||
`--headless=new` mode handles screen capture without Xvfb on Linux CI
|
||||
runners. If a future Chrome regresses this, `Xvfb :99 & DISPLAY=:99
|
||||
npm run test:uat` is the fallback.
|
||||
|
||||
### CI runner screen-capture concern
|
||||
|
||||
The 35s recording assertion (A11) captures whatever is on screen during
|
||||
that window. CI MUST run the harness in an isolated container with no
|
||||
concurrent workload — see T-1-11-02 in Plan 01-11's threat model.
|
||||
|
||||
### Real Chrome download (assertion 5 → A12)
|
||||
|
||||
The harness configures per-page download behavior via CDP to a fresh
|
||||
`os.tmpdir()/mokosh-uat-downloads-*` directory; downloads are NOT
|
||||
written to your real ~/Downloads. The temp directory is deleted by OS
|
||||
tmpdir GC.
|
||||
Reference in New Issue
Block a user