repremium/mokosh
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(01-13): wave-2 — launchHarnessBrowser + assertions + harness-page-driver scaffolding

Browse Source 
Build out the Approach-B harness driver utilities atop the Wave 1
production paths. Three new files form the shared scaffold that
Wave 3's 13 assertion drivers (A1-A5, A7-A13) and the eventual
orchestrator (`tests/uat/harness.test.ts`) will all consume. The
standalone A6 driver (`tests/uat/a6.test.ts`) is rewritten to use
the new lib — behavior-preserving: A6 still PASSES 5/5 in ~7s.

New files:

  - tests/uat/lib/launch.ts (~320 LoC)
      `launchHarnessBrowser({ headless?, downloadsDir? }) → HarnessHandles`
      Extracts the Chrome-launch + victim-page + harness-page + console-
      attach pattern from a6.test.ts into a single reusable helper.
      NEW vs prototype: CDP `Browser.setDownloadBehavior` wires
      Chrome's download path to a per-run `mkdtempSync` tmp dir so A5
      (SAVE_ARCHIVE) can poll a known location without colliding with
      the operator's real downloads. Architectural commitments
      enforced (per 01-11-SUMMARY): no `--auto-select-desktop-capture-
      source` flag; victim about:blank brought to front for the
      production `chrome.tabs.query({active:true})` workaround; SW
      console attach best-effort with bounded poll; offscreen console
      attach opportunistic via `targetcreated` listener (offscreen
      target appears later, when the harness page calls
      chrome.offscreen.createDocument).

  - tests/uat/lib/assertions.ts (~210 LoC)
      Host-side assertion primitives:
        * `AssertionRecord`, `CheckRecord`, `ConsoleBuffers` types —
          mirror the page-side shape returned by `assertA*` methods.
        * `runAssertion(name, fn, buffers)` — try/catch wrapper that
          dumps the SW + offscreen console tails (last 100 lines each)
          to stderr on failure, then returns `{passed: false, error}`
          if `fn` throws.
        * `printAssertionResult(result)` — single source of truth for
          the formatted result print. Extracted from the inline
          `printResult` previously in the prototype's a6.test.ts so
          Wave 3's orchestrator can reuse it across all 14 assertions.
        * `assertEqual / assertGte / assertMatch / assertTrue` —
          structured failure messages atop node:assert/strict.
        * `waitFor(probe, predicate, timeoutMs, description)` — host-
          side polling primitive; mirrors the page-side waitFor
          semantics verbatim (they can't share a module: page-side is
          bundled into the harness HTML, host-side runs in Node).
      NO chrome.* helpers here — all chrome.* work happens inside the
      extension-internal harness page. This module is host-side ONLY
      by construction (no chrome global in Node anyway).

  - tests/uat/lib/harness-page-driver.ts (~170 LoC)
      One driver wrapper per assertion (A1..A13). Each wraps a single
      `page.evaluate(() => window.__mokoshHarness.assertXX())`.
      Centralizing this means adding/renaming an assertion = two-file
      edit (extension-page-harness.ts impl + this file) instead of
      touching every test-file caller.
      Wave 2 wires `driveA6` (proven from c647f61). The 12 Wave-3
      drivers (driveA1..A5, A7..A13) are stubbed as
      `throw new Error('NOT YET IMPLEMENTED — Wave 3<X> wires driveXX')`
      so the future orchestrator's `for (const drive of drivers)` loop
      fails cleanly on the first unimplemented one (bail-on-first-
      failure semantics). The `AssertionWithBytes` type is declared
      for A5/A12/A13 which return `bytesBase64` payloads (zip / webm
      bytes that the host side processes after the page-side
      assertion completes).

Rewrite — `tests/uat/a6.test.ts`:
  - Drops ~80 LoC of Chrome-launch + console-attach + result-print
    plumbing now living in lib/launch.ts + lib/assertions.ts.
  - Now ~70 LoC total — pure orchestration of
    launchHarnessBrowser → runAssertion(driveA6) → printAssertionResult
    → browser.close() → exit code.
  - Behavior-preserving: A6 still 5/5 GREEN with the same diagnostic
    output (SETUP, A6.1-A6.4) and the same ~7s end-to-end runtime.

Verification (all GREEN):
  - `npx tsc --noEmit` — exit 0 (root + tests/uat/tsconfig.json).
  - `npx tsx tests/uat/a6.test.ts` — exits 0 with "PASS"; 5 checks
    GREEN (SETUP, A6.1, A6.2, A6.3, A6.4). End-to-end runtime ~7s
    headless on this workstation.
  - `npm run build` — exit 0; Tier-1 grep gate GREEN (production
    bundle contains zero hook strings AND zero lib symbol names —
    the new lib files are test-only and not bundled into dist/).
  - `npm run build:test` — exit 0; dist-test/ still emits the
    extension-page-harness.html harness (lib files are host-side,
    not rollup inputs).
  - `npx vitest run` — 92/92 GREEN.

Wave 3 ready: harness-page-driver.ts has driveA1..A5/A7..A13 stubs
in place; extending requires only:
  1. Add `assertAXX` method to window.__mokoshHarness in
     tests/uat/extension-page-harness.ts.
  2. Replace the corresponding stub body in this file with the
     page.evaluate wrapper.
  3. (Wave 3A) Create tests/uat/harness.test.ts orchestrator that
     iterates over [A0 grep gate, driveA1..A13] with bail-on-fail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mark
2026-05-18 15:21:11 +02:00
parent eb2258a880
commit eb64521321
4 changed files with 871 additions and 283 deletions
Download Patch File Download Diff File Expand all files Collapse all files

316
tests/uat/a6.test.ts
View File

@@ -1,311 +1,61 @@
// tests/uat/a6.test.ts — Plan 01-13 standalone A6 entry point.
//
// Puppeteer-driven single-assertion driver for A6 (Bug B canonical).
// Originally landed as the Plan 01-11 prototype at commit c647f61;
// Plan 01-13 Wave 1 promoted this file from `tests/uat/prototype/` to
// the production path without behavioral change. Wave 2 will refactor
// the launch + console-capture + result-print plumbing into reusable
// lib helpers (`tests/uat/lib/{launch,assertions,harness-page-driver}
// .ts`) and rewrite this driver against them; Wave 3 folds A6 into
// `tests/uat/harness.test.ts` as the assertion of record for `npm run
// test:uat`. This standalone entry is RETAINED throughout for fast
// TDD iteration on the A6 contract (`npx tsx tests/uat/a6.test.ts` —
// ~7s end-to-end vs the orchestrator's ~60-90s for all 14).
// Refactored in Wave 2 to use the shared `tests/uat/lib/` scaffolding
// (`launchHarnessBrowser`, `driveA6`, `runAssertion`, `printAssertionResult`).
// Behavior-preserving: A6 still PASSES 5/5 in ~7s end-to-end. The ~80
// LoC of Chrome-launch + console-attach + result-print plumbing
// previously inlined here now lives in `tests/uat/lib/{launch,assertions,
// harness-page-driver}.ts` — single source of truth for Wave 3's 13
// additional assertions.
//
// Assertion contract — A6 (Bug B canonical): when the offscreen
// recorder fires RECORDING_ERROR{error: 'user-stopped-sharing'}
// (simulated via dispatchEvent('ended') on the active video track per
// 01-11 RESEARCH §7 BLOCKER — track.stop() does NOT fire 'ended' per
// W3C spec), the SW state machine routes through setIdleMode (NOT
// setErrorMode): badge becomes empty, popup empties, isRecording=false,
// NO recovery notification fires. The prototype verified this PASSES
// 5/5 today AND FAILS on local revert of the Bug B fix at
// src/background/index.ts:776 — both halves of the RED-on-regression
// demo land in the Wave 3B commit body as the canonical TDD canon.
// This standalone entry is RETAINED throughout the rest of Plan 01-13
// for fast TDD iteration on the A6 contract:
// `npx tsx tests/uat/a6.test.ts` # headless, ~7s
// `HEADLESS=0 npx tsx tests/uat/a6.test.ts` # interactive debug view
//
// Usage:
// tsx tests/uat/a6.test.ts
// HEADLESS=0 tsx tests/uat/a6.test.ts # debug view
// The orchestrator-level entry `npm run test:uat` (lands in Wave 3A)
// runs all 14 assertions (~60-90s); this single-A6 entry is for the
// inner loop when iterating on Bug B fix verification or harness-page
// surface changes.
//
// Pre-flight: requires `dist-test/` from `npm run build:test`. The test
// will fail loudly if the bundle is missing.
//
// References:
// - eyeo's MV3 testing journey (uses extension-internal test page +
// bidirectional messaging):
// https://developer.chrome.com/blog/eyeos-journey-to-testing-mv3-service%20worker-suspension
// - Chrome MV3 E2E testing official guide:
// https://developer.chrome.com/docs/extensions/mv3/end-to-end-testing/
// Pre-flight: requires `dist-test/` from `npm run build:test`. The
// `assertBundlePresent` call inside `launchHarnessBrowser` fails
// loudly if the bundle is missing.
import { existsSync, statSync } from 'node:fs';
import { dirname, resolve as resolvePath } from 'node:path';
import { fileURLToPath } from 'node:url';
import puppeteer, { type Browser, type Page } from 'puppeteer';
// Plan 01-13 Wave 1: this file lives at `tests/uat/a6.test.ts` (was
// `tests/uat/prototype/a6.test.ts` pre-Wave-1). Repo root is two
// directory levels up — was three pre-Wave-1. The resolvePath chain
// MUST stay in sync with the on-disk location or `DIST_TEST_DIR` will
// resolve to the wrong path and `assertBundlePresent` will throw.
const HARNESS_FILE_DIR = dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = resolvePath(HARNESS_FILE_DIR, '..', '..');
const DIST_TEST_DIR = resolvePath(REPO_ROOT, 'dist-test');
/** Per-check record returned by the harness page. */
interface CheckRecord {
name: string;
expected: unknown;
actual: unknown;
passed: boolean;
}
/** Result returned by `window.__mokoshHarness.assertA6()`. */
interface HarnessAssertionResult {
passed: boolean;
name: string;
checks: CheckRecord[];
diagnostics: string[];
error?: string;
}
import { launchHarnessBrowser } from './lib/launch';
import { driveA6 } from './lib/harness-page-driver';
import { runAssertion, printAssertionResult } from './lib/assertions';
/**
* Verify the test bundle is present; fail loudly if missing.
* Standalone A6 driver entry point.
*
* @throws If dist-test/ is missing or not a directory.
*/
function assertBundlePresent(): void {
if (!existsSync(DIST_TEST_DIR)) {
throw new Error(
`dist-test/ missing at ${DIST_TEST_DIR} — run \`npm run build:test\` first.`,
);
}
if (!statSync(DIST_TEST_DIR).isDirectory()) {
throw new Error(`dist-test/ at ${DIST_TEST_DIR} is not a directory.`);
}
}
/**
* Launch Chrome with the test bundle loaded as an unpacked MV3
* extension. Returns the browser handle + resolved extension id.
*
* Bumps `protocolTimeout` from the default 30s to 90s so the
* end-to-end assertion (which does several sendMessage round-trips
* + waits for badge transitions) has enough headroom on slow CI
* runners without the assertion call itself timing out at the CDP layer.
*
* @returns Browser handle + extension id.
*/
async function launchChrome(): Promise<{
browser: Browser;
extensionId: string;
}> {
const headless = process.env.HEADLESS !== '0';
const browser = await puppeteer.launch({
enableExtensions: [DIST_TEST_DIR],
headless,
pipe: true,
protocolTimeout: 90_000,
args: [
'--no-sandbox',
// We do NOT need --auto-select-desktop-capture-source for the
// prototype because the fake getDisplayMedia bypasses the picker
// entirely. Including it would be a no-op.
],
});
// Resolve extension id. browser.extensions() returns a Map<id, Extension>
// populated asynchronously after the extension's manifest loads. Poll
// for up to 5s with a clear diagnostic on timeout.
const POLL_TIMEOUT_MS = 5_000;
const POLL_INTERVAL_MS = 100;
const pollStart = Date.now();
let extensionsMap = await browser.extensions();
while (extensionsMap.size === 0 && Date.now() - pollStart < POLL_TIMEOUT_MS) {
await new Promise((r) => setTimeout(r, POLL_INTERVAL_MS));
extensionsMap = await browser.extensions();
}
const entries = [...extensionsMap];
if (entries.length === 0) {
await browser.close();
throw new Error(
`No extensions loaded after ${POLL_TIMEOUT_MS}ms — dist-test/ malformed?`,
);
}
const [extensionId] = entries[0];
return { browser, extensionId };
}
/**
* Pretty-print the harness assertion result for stdout.
*
* @param result - The structured result from assertA6().
*/
function printResult(result: HarnessAssertionResult): void {
process.stdout.write('\n');
process.stdout.write('='.repeat(72) + '\n');
process.stdout.write(`A6 result: ${result.passed ? 'PASS' : 'FAIL'}\n`);
process.stdout.write(`Assertion: ${result.name}\n`);
if (result.error !== undefined) {
process.stdout.write(`Top-level error: ${result.error}\n`);
}
process.stdout.write('\nChecks:\n');
for (const check of result.checks) {
const mark = check.passed ? '[PASS]' : '[FAIL]';
process.stdout.write(` ${mark} ${check.name}\n`);
process.stdout.write(` expected: ${JSON.stringify(check.expected)}\n`);
process.stdout.write(` actual: ${JSON.stringify(check.actual)}\n`);
}
process.stdout.write('\nDiagnostics:\n');
for (const diag of result.diagnostics) {
process.stdout.write(` - ${diag}\n`);
}
process.stdout.write('='.repeat(72) + '\n');
}
/**
* Main prototype entry point. Returns the process exit code.
*
* @returns 0 on PASS, 1 on FAIL.
* @returns Process exit code: 0 on PASS, 1 on FAIL.
*/
async function main(): Promise<number> {
process.stdout.write('\nMokosh Plan 01-13 — A6 (Bug B canonical) standalone driver\n');
process.stdout.write('Architecture: extension-internal page + bridge + synthetic stream\n');
process.stdout.write('='.repeat(72) + '\n');
assertBundlePresent();
process.stdout.write(`Bundle: ${DIST_TEST_DIR}\n`);
process.stdout.write('Launching Chrome...\n');
const { browser, extensionId } = await launchChrome();
process.stdout.write(`Extension id: ${extensionId}\n`);
// Diagnostic capture buffers — flushed on result print.
const consoleLines: string[] = [];
const handles = await launchHarnessBrowser();
process.stdout.write(`Extension id: ${handles.extensionId}\n`);
process.stdout.write(`Downloads dir: ${handles.downloadsDir}\n`);
process.stdout.write('Harness page ready; invoking assertA6()...\n\n');
let exitCode = 1;
try {
// Open the prototype harness page. The page lives at the test-build
// path (vite.test.config.ts adds it as a rollup input).
const harnessUrl = `chrome-extension://${extensionId}/tests/uat/extension-page-harness.html`;
process.stdout.write(`Opening: ${harnessUrl}\n`);
// Open a 'victim' page first — production code calls
// chrome.tabs.query({active:true}) and demands a tab with .url
// (the operator's recording-target page). The harness page itself
// is a chrome-extension:// URL which has no .url surfaced (without
// 'tabs' permission). We open a real http URL in a separate tab
// and bring it to front before REQUEST_PERMISSIONS fires.
const victimPage = await browser.newPage();
await victimPage.goto('about:blank');
// about:blank has tab.url === 'about:blank' (truthy), so production
// tab.id + tab.url check passes.
const page: Page = await browser.newPage();
page.on('console', (msg) => {
const line = `[page:${msg.type()}] ${msg.text()}`;
consoleLines.push(line);
process.stderr.write(line + '\n');
});
page.on('pageerror', (err: unknown) => {
const msg = err instanceof Error ? err.message : String(err);
const line = `[page:ERROR] ${msg}`;
consoleLines.push(line);
process.stderr.write(line + '\n');
});
// Also capture SW console logs (where production logger.* writes).
// The SW target appears when the extension loads — wait briefly,
// then attach a worker handle and forward console events.
try {
const swTarget = await browser.waitForTarget(
(t) => t.type() === 'service_worker' && t.url().includes(extensionId),
{ timeout: 10_000 },
);
const sw = await swTarget.worker();
if (sw !== null) {
sw.on('console', (msg) => {
const line = `[sw:${msg.type()}] ${msg.text()}`;
consoleLines.push(line);
process.stderr.write(line + '\n');
});
}
} catch (swAttachErr) {
process.stderr.write(
`(note: SW console attach skipped — ${String(swAttachErr)})\n`,
);
}
await page.goto(harnessUrl, {
waitUntil: 'domcontentloaded',
timeout: 10_000,
});
process.stdout.write('Page loaded; waiting for window.__mokoshHarness...\n');
// The harness page's bundled script installs window.__mokoshHarness
// on module-load. Wait for the bootstrap to land.
await page.waitForFunction(
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- evaluate runs in browser context where window types are loose.
() => (window as any).__mokoshHarness !== undefined,
{ timeout: 5_000 },
const result = await runAssertion(
'A6 — BUG B canonical: user-stopped-sharing routes via setIdleMode',
() => driveA6(handles.harnessPage),
{ swConsole: handles.swConsole, offConsole: handles.offConsole },
);
process.stdout.write('Harness page ready; invoking assertA6()...\n\n');
// Try also attaching to offscreen target console logs once it appears.
let offscreenAttached = false;
browser.on('targetcreated', async (target) => {
if (offscreenAttached) return;
const url = target.url();
if (
target.type() === 'background_page' &&
url.includes(extensionId) &&
url.includes('offscreen')
) {
offscreenAttached = true;
try {
const offPage = await target.asPage();
offPage.on('console', (msg) => {
const line = `[off:${msg.type()}] ${msg.text()}`;
consoleLines.push(line);
process.stderr.write(line + '\n');
});
} catch (offAttachErr) {
process.stderr.write(
`(note: offscreen console attach skipped — ${String(offAttachErr)})\n`,
);
}
}
});
// Bring the victim page to front so chrome.tabs.query({active:true})
// returns it (not the harness page) when production startVideoCapture
// runs. The harness page can still be evaluated against — Puppeteer's
// page handle doesn't care about active-tab state.
await victimPage.bringToFront();
// Run the end-to-end A6 assertion. The page-side code does all the
// orchestration — Puppeteer is just the trigger + result reader.
const result = await page.evaluate(async () => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- evaluate runs in browser context.
const harness = (window as any).__mokoshHarness;
const r = await harness.assertA6();
return r;
}) as HarnessAssertionResult;
printResult(result);
printAssertionResult(result);
exitCode = result.passed ? 0 : 1;
} catch (err) {
process.stderr.write(`\n*** Top-level harness error: ${String(err)}\n`);
if (consoleLines.length > 0) {
process.stderr.write('\nCaptured console (last 50 lines):\n');
for (const line of consoleLines.slice(-50)) {
process.stderr.write(` ${line}\n`);
}
}
exitCode = 1;
} finally {
try {
await browser.close();
await handles.browser.close();
} catch (closeErr) {
process.stderr.write(`(non-fatal: browser close threw: ${String(closeErr)})\n`);
}