feat(01-13): wave-2 — launchHarnessBrowser + assertions + harness-page-driver scaffolding

Build out the Approach-B harness driver utilities atop the Wave 1
production paths. Three new files form the shared scaffold that
Wave 3's 13 assertion drivers (A1-A5, A7-A13) and the eventual
orchestrator (`tests/uat/harness.test.ts`) will all consume. The
standalone A6 driver (`tests/uat/a6.test.ts`) is rewritten to use
the new lib — behavior-preserving: A6 still PASSES 5/5 in ~7s.

New files:

  - tests/uat/lib/launch.ts (~320 LoC)
      `launchHarnessBrowser({ headless?, downloadsDir? }) → HarnessHandles`
      Extracts the Chrome-launch + victim-page + harness-page + console-
      attach pattern from a6.test.ts into a single reusable helper.
      NEW vs prototype: CDP `Browser.setDownloadBehavior` wires
      Chrome's download path to a per-run `mkdtempSync` tmp dir so A5
      (SAVE_ARCHIVE) can poll a known location without colliding with
      the operator's real downloads. Architectural commitments
      enforced (per 01-11-SUMMARY): no `--auto-select-desktop-capture-
      source` flag; victim about:blank brought to front for the
      production `chrome.tabs.query({active:true})` workaround; SW
      console attach best-effort with bounded poll; offscreen console
      attach opportunistic via `targetcreated` listener (offscreen
      target appears later, when the harness page calls
      chrome.offscreen.createDocument).

  - tests/uat/lib/assertions.ts (~210 LoC)
      Host-side assertion primitives:
        * `AssertionRecord`, `CheckRecord`, `ConsoleBuffers` types —
          mirror the page-side shape returned by `assertA*` methods.
        * `runAssertion(name, fn, buffers)` — try/catch wrapper that
          dumps the SW + offscreen console tails (last 100 lines each)
          to stderr on failure, then returns `{passed: false, error}`
          if `fn` throws.
        * `printAssertionResult(result)` — single source of truth for
          the formatted result print. Extracted from the inline
          `printResult` previously in the prototype's a6.test.ts so
          Wave 3's orchestrator can reuse it across all 14 assertions.
        * `assertEqual / assertGte / assertMatch / assertTrue` —
          structured failure messages atop node:assert/strict.
        * `waitFor(probe, predicate, timeoutMs, description)` — host-
          side polling primitive; mirrors the page-side waitFor
          semantics verbatim (they can't share a module: page-side is
          bundled into the harness HTML, host-side runs in Node).
      NO chrome.* helpers here — all chrome.* work happens inside the
      extension-internal harness page. This module is host-side ONLY
      by construction (no chrome global in Node anyway).

  - tests/uat/lib/harness-page-driver.ts (~170 LoC)
      One driver wrapper per assertion (A1..A13). Each wraps a single
      `page.evaluate(() => window.__mokoshHarness.assertXX())`.
      Centralizing this means adding/renaming an assertion = two-file
      edit (extension-page-harness.ts impl + this file) instead of
      touching every test-file caller.
      Wave 2 wires `driveA6` (proven from c647f61). The 12 Wave-3
      drivers (driveA1..A5, A7..A13) are stubbed as
      `throw new Error('NOT YET IMPLEMENTED — Wave 3<X> wires driveXX')`
      so the future orchestrator's `for (const drive of drivers)` loop
      fails cleanly on the first unimplemented one (bail-on-first-
      failure semantics). The `AssertionWithBytes` type is declared
      for A5/A12/A13 which return `bytesBase64` payloads (zip / webm
      bytes that the host side processes after the page-side
      assertion completes).

Rewrite — `tests/uat/a6.test.ts`:
  - Drops ~80 LoC of Chrome-launch + console-attach + result-print
    plumbing now living in lib/launch.ts + lib/assertions.ts.
  - Now ~70 LoC total — pure orchestration of
    launchHarnessBrowser → runAssertion(driveA6) → printAssertionResult
    → browser.close() → exit code.
  - Behavior-preserving: A6 still 5/5 GREEN with the same diagnostic
    output (SETUP, A6.1-A6.4) and the same ~7s end-to-end runtime.

Verification (all GREEN):
  - `npx tsc --noEmit` — exit 0 (root + tests/uat/tsconfig.json).
  - `npx tsx tests/uat/a6.test.ts` — exits 0 with "PASS"; 5 checks
    GREEN (SETUP, A6.1, A6.2, A6.3, A6.4). End-to-end runtime ~7s
    headless on this workstation.
  - `npm run build` — exit 0; Tier-1 grep gate GREEN (production
    bundle contains zero hook strings AND zero lib symbol names —
    the new lib files are test-only and not bundled into dist/).
  - `npm run build:test` — exit 0; dist-test/ still emits the
    extension-page-harness.html harness (lib files are host-side,
    not rollup inputs).
  - `npx vitest run` — 92/92 GREEN.

Wave 3 ready: harness-page-driver.ts has driveA1..A5/A7..A13 stubs
in place; extending requires only:
  1. Add `assertAXX` method to window.__mokoshHarness in
     tests/uat/extension-page-harness.ts.
  2. Replace the corresponding stub body in this file with the
     page.evaluate wrapper.
  3. (Wave 3A) Create tests/uat/harness.test.ts orchestrator that
     iterates over [A0 grep gate, driveA1..A13] with bail-on-fail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-18 15:21:11 +02:00
parent eb2258a880
commit eb64521321
4 changed files with 871 additions and 283 deletions

394
tests/uat/lib/launch.ts Normal file
View File

@@ -0,0 +1,394 @@
// tests/uat/lib/launch.ts — Plan 01-13 Wave 2.
//
// Approach-B harness launch helper. Inherits the Puppeteer launch +
// victim-page-bringToFront + harness-page-open pattern from the proven
// `tests/uat/a6.test.ts` driver (originally landed as Plan 01-11
// prototype at commit c647f61; promoted to production paths by 01-13
// Wave 1). Refactored into a reusable helper so Wave 3's 13 assertion
// drivers share the same setup overhead — one Chrome launch + one
// harness page + one victim page per `npm run test:uat` run.
//
// Architectural commitments (per 01-11-SUMMARY.md, DO NOT REGRESS):
// - Drive Chrome FROM INSIDE: `harnessPage` runs at
// `chrome-extension://<id>/tests/uat/extension-page-harness.html`
// with full chrome.* API access (Approach B; sw.evaluate fallback
// was falsified per SUMMARY §2 — only chrome.{loadTimes,csi}
// surfaced through CDP).
// - `victimPage` is a brought-to-front about:blank tab so the
// production `chrome.tabs.query({active:true})` sees a real tab
// with a `.url` (Plan 01-13 retains the `tabs` permission gap as
// out-of-scope; A2 + similar tests send `START_RECORDING` directly
// to offscreen, bypassing the SW's `startVideoCapture` which needs
// the tabs permission to read `tab.url`). Workaround documented in
// the plan's resolved-open-questions table row 2.
// - Downloads land in a per-run tmp dir (`mkdtempSync`) so A5 polling
// does not collide with operator downloads. Configured via CDP
// `Browser.setDownloadBehavior` on the harness page's CDP session.
// - SW + offscreen consoles forwarded to `swConsole` / `offConsole`
// accumulating string buffers. Offscreen attach via
// `browser.on('targetcreated')` is OPPORTUNISTIC per the prototype
// pattern — offscreen targets appear asynchronously when
// `chrome.offscreen.createDocument` runs from inside the harness
// page; the harness must not block waiting for them.
// - NO `--auto-select-desktop-capture-source` flag: unreliable in
// `--headless=new` per 01-11-SUMMARY falsification 4. The synthetic
// `installFakeDisplayMedia` (offscreen-hooks.ts eager install)
// bypasses Chrome's picker entirely.
//
// References:
// - puppeteer.launch options:
// https://pptr.dev/api/puppeteer.launchoptions
// - puppeteer.Browser.extensions():
// https://pptr.dev/api/puppeteer.browser.extensions
// - CDP Browser.setDownloadBehavior (per-context download path):
// https://chromedevtools.github.io/devtools-protocol/tot/Browser/#method-setDownloadBehavior
// - puppeteer CDP session helper:
// https://pptr.dev/api/puppeteer.cdpsession
// - Node fs.mkdtempSync:
// https://nodejs.org/api/fs.html#fsmkdtempsyncprefix-options
import { existsSync, mkdtempSync, statSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join, resolve as resolvePath } from 'node:path';
import { fileURLToPath } from 'node:url';
import puppeteer, { type Browser, type Page } from 'puppeteer';
/** Repo root resolved from this file's location (tests/uat/lib/launch.ts). */
const HARNESS_FILE_DIR = dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = resolvePath(HARNESS_FILE_DIR, '..', '..', '..');
const DIST_TEST_DIR = resolvePath(REPO_ROOT, 'dist-test');
/** Time bounds for the various polling/attach steps. Keep in sync with
* the comments below — each value has a rationale, not a guess. */
const EXTENSION_ID_POLL_TIMEOUT_MS = 5_000;
const EXTENSION_ID_POLL_INTERVAL_MS = 100;
const HARNESS_BOOTSTRAP_TIMEOUT_MS = 5_000;
const HARNESS_GOTO_TIMEOUT_MS = 10_000;
const SW_TARGET_ATTACH_TIMEOUT_MS = 10_000;
/** Bumped from the puppeteer default 30s to give the assertions
* several sendMessage round-trips of CDP headroom on slow CI runners. */
const PROTOCOL_TIMEOUT_MS = 90_000;
/**
* Handles returned by `launchHarnessBrowser`. The caller owns the
* `browser` and is responsible for calling `browser.close()` in a
* `finally` block. `downloadsDir` is created by this function and is
* deliberately NOT cleaned up automatically — failing tests benefit
* from the operator inspecting the downloads dir post-mortem.
*/
export interface HarnessHandles {
readonly browser: Browser;
readonly extensionId: string;
readonly harnessPage: Page;
readonly victimPage: Page;
readonly downloadsDir: string;
/** Accumulating SW console log lines, format `[sw:<type>] <text>`. */
readonly swConsole: string[];
/** Accumulating offscreen console log lines, format `[off:<type>] <text>`. */
readonly offConsole: string[];
}
/**
* Options for `launchHarnessBrowser`. All fields optional; defaults
* apply (`headless: process.env.HEADLESS !== '0'`; `downloadsDir` ←
* fresh mkdtempSync).
*/
export interface LaunchOptions {
/** Override `--headless=new`; useful for visual debugging. */
readonly headless?: boolean;
/** Override the auto-created downloads dir; useful for cross-run debugging. */
readonly downloadsDir?: string;
}
/**
* Verify the test bundle is present at `dist-test/`; fail loudly with
* an actionable error if missing. The harness cannot launch without
* the bundle so failing early avoids confusing puppeteer errors.
*
* @throws If `dist-test/` is missing or not a directory.
*/
function assertBundlePresent(): void {
if (!existsSync(DIST_TEST_DIR)) {
throw new Error(
`dist-test/ missing at ${DIST_TEST_DIR} — run \`npm run build:test\` first.`,
);
}
if (!statSync(DIST_TEST_DIR).isDirectory()) {
throw new Error(`dist-test/ at ${DIST_TEST_DIR} is not a directory.`);
}
}
/**
* Poll `browser.extensions()` until at least one extension is loaded
* or the timeout elapses. Returns the first extension's id.
*
* @param browser - Puppeteer browser handle.
* @returns The resolved extension id string.
* @throws If no extension loads within `EXTENSION_ID_POLL_TIMEOUT_MS`.
*/
async function resolveExtensionIdWithPolling(browser: Browser): Promise<string> {
const pollStart = Date.now();
let extensionsMap = await browser.extensions();
while (
extensionsMap.size === 0 &&
Date.now() - pollStart < EXTENSION_ID_POLL_TIMEOUT_MS
) {
await new Promise((resolve) => setTimeout(resolve, EXTENSION_ID_POLL_INTERVAL_MS));
extensionsMap = await browser.extensions();
}
const entries = [...extensionsMap];
if (entries.length === 0) {
throw new Error(
`No extensions loaded after ${EXTENSION_ID_POLL_TIMEOUT_MS}ms — dist-test/ malformed?`,
);
}
const [extensionId] = entries[0];
return extensionId;
}
/**
* Attach a SW console listener that forwards every console event to
* the provided buffer (both for in-memory diagnostic capture AND for
* stderr streaming so the operator sees live logs during a hung
* assertion). Best-effort: if the SW target cannot be found inside
* `SW_TARGET_ATTACH_TIMEOUT_MS`, the failure is logged to stderr but
* the harness continues (the assertion may still pass — many
* assertions do not need SW console data).
*
* @param browser - Puppeteer browser handle.
* @param extensionId - The resolved extension id.
* @param swConsole - Accumulating string buffer to push log lines into.
*/
async function attachSwConsoleBestEffort(
browser: Browser,
extensionId: string,
swConsole: string[],
): Promise<void> {
try {
const swTarget = await browser.waitForTarget(
(t) => t.type() === 'service_worker' && t.url().includes(extensionId),
{ timeout: SW_TARGET_ATTACH_TIMEOUT_MS },
);
const sw = await swTarget.worker();
if (sw !== null) {
/**
* Named callback per project style — every chrome.* console event
* formatted with a leading `[sw:<type>]` tag for grep-ability.
*/
const onSwConsole = (msg: { type: () => string; text: () => string }): void => {
const line = `[sw:${msg.type()}] ${msg.text()}`;
swConsole.push(line);
process.stderr.write(line + '\n');
};
sw.on('console', onSwConsole);
}
} catch (swAttachErr) {
process.stderr.write(
`(launch: SW console attach skipped — ${String(swAttachErr)})\n`,
);
}
}
/**
* Register a `targetcreated` listener that lazily attaches the
* offscreen console once it appears. The offscreen target is created
* later (when the harness page calls `chrome.offscreen.createDocument`),
* so we cannot wait for it eagerly; instead we register the listener
* upfront and let it fire when the offscreen target spawns.
*
* Idempotent — only the first matching offscreen target is attached.
*
* @param browser - Puppeteer browser handle.
* @param extensionId - The resolved extension id.
* @param offConsole - Accumulating string buffer for offscreen log lines.
*/
function registerOffscreenConsoleAttach(
browser: Browser,
extensionId: string,
offConsole: string[],
): void {
let offscreenAttached = false;
/**
* Targetcreated handler — checks each new target for the offscreen
* extension URL pattern, attaches the console listener on the first
* match.
*/
const onTargetCreated = async (
target: { type: () => string; url: () => string; asPage: () => Promise<Page> },
): Promise<void> => {
if (offscreenAttached) {
return;
}
const url = target.url();
if (
target.type() === 'background_page' &&
url.includes(extensionId) &&
url.includes('offscreen')
) {
offscreenAttached = true;
try {
const offPage = await target.asPage();
/**
* Per-message callback — same tag format as the SW attach
* (`[off:<type>] <text>`).
*/
const onOffConsole = (msg: { type: () => string; text: () => string }): void => {
const line = `[off:${msg.type()}] ${msg.text()}`;
offConsole.push(line);
process.stderr.write(line + '\n');
};
offPage.on('console', onOffConsole);
} catch (offAttachErr) {
process.stderr.write(
`(launch: offscreen console attach skipped — ${String(offAttachErr)})\n`,
);
}
}
};
browser.on('targetcreated', onTargetCreated);
}
/**
* Configure the harness page's CDP session to use the per-run
* `downloadsDir` so A5 (SAVE_ARCHIVE → chrome.downloads.download) can
* poll a known directory without colliding with the operator's real
* downloads. Uses CDP `Browser.setDownloadBehavior` with
* `behavior: 'allow'` + the explicit path.
*
* @param harnessPage - The opened harness page handle.
* @param downloadsDir - Absolute path to the downloads directory.
*/
async function configureDownloadsDir(
harnessPage: Page,
downloadsDir: string,
): Promise<void> {
const session = await harnessPage.createCDPSession();
await session.send('Browser.setDownloadBehavior', {
behavior: 'allow',
downloadPath: downloadsDir,
});
}
/**
* Launch Chrome with the test bundle as an unpacked MV3 extension,
* open the extension-internal harness page + a victim about:blank
* page, configure downloads, attach SW + offscreen console listeners,
* and return the assembled handles.
*
* Caller MUST close the browser in a `finally` block:
* ```typescript
* const handles = await launchHarnessBrowser();
* try {
* // ... run assertions ...
* } finally {
* await handles.browser.close();
* }
* ```
*
* @param opts - Override headless / downloadsDir.
* @returns Assembled HarnessHandles.
*/
export async function launchHarnessBrowser(
opts: LaunchOptions = {},
): Promise<HarnessHandles> {
assertBundlePresent();
const headless = opts.headless ?? process.env.HEADLESS !== '0';
const downloadsDir = opts.downloadsDir ?? mkdtempSync(join(tmpdir(), 'mokosh-uat-'));
const browser = await puppeteer.launch({
enableExtensions: [DIST_TEST_DIR],
headless,
pipe: true,
protocolTimeout: PROTOCOL_TIMEOUT_MS,
args: [
'--no-sandbox',
// DO NOT add --auto-select-desktop-capture-source — unreliable
// in --headless=new per 01-11-SUMMARY falsification 4; the
// synthetic getDisplayMedia (offscreen-hooks.ts:installFake)
// bypasses Chrome's picker entirely.
],
});
const extensionId = await resolveExtensionIdWithPolling(browser);
// Accumulating console buffers — empty until SW + offscreen attach.
const swConsole: string[] = [];
const offConsole: string[] = [];
// Open the victim page FIRST so it's already in the tab list when
// the harness page opens. About:blank's `tab.url` resolves to
// 'about:blank' (truthy), passing production
// chrome.tabs.query({active:true}) presence checks.
const victimPage = await browser.newPage();
await victimPage.goto('about:blank');
// Open the harness page; attach console + pageerror listeners
// BEFORE the goto so we don't miss bootstrap-time messages.
const harnessPage = await browser.newPage();
/**
* Named callback per project style — forwards all page-side console
* events to stderr with the `[page:<type>]` tag.
*/
const onPageConsole = (msg: { type: () => string; text: () => string }): void => {
const line = `[page:${msg.type()}] ${msg.text()}`;
process.stderr.write(line + '\n');
};
harnessPage.on('console', onPageConsole);
/**
* Named callback — page errors get an explicit `[page:ERROR]` tag
* separate from the console events so the operator can spot them in
* the stderr stream.
*/
const onPageError = (err: unknown): void => {
const msg = err instanceof Error ? err.message : String(err);
const line = `[page:ERROR] ${msg}`;
process.stderr.write(line + '\n');
};
harnessPage.on('pageerror', onPageError);
// Best-effort SW console attach — the SW target is usually ready
// by the time the extension finishes loading, but slow CI may need
// a brief poll (handled inside `attachSwConsoleBestEffort`).
await attachSwConsoleBestEffort(browser, extensionId, swConsole);
// Register the offscreen console attach BEFORE opening the harness
// page so the listener catches the offscreen target whenever it
// spawns (which happens later, when the page calls
// chrome.offscreen.createDocument from inside an assertion).
registerOffscreenConsoleAttach(browser, extensionId, offConsole);
// Configure downloads via CDP. This MUST happen on the harness
// page's CDP session (not the browser's default session) per
// puppeteer's per-page session model.
await configureDownloadsDir(harnessPage, downloadsDir);
// Bring the victim page to front so chrome.tabs.query({active:true})
// returns it (not the harness page) when production startVideoCapture
// runs. The harness page can still be evaluated against — Puppeteer's
// page handle doesn't care about active-tab state.
await victimPage.bringToFront();
// Open the harness page; wait for window.__mokoshHarness to install.
const harnessUrl = `chrome-extension://${extensionId}/tests/uat/extension-page-harness.html`;
await harnessPage.goto(harnessUrl, {
waitUntil: 'domcontentloaded',
timeout: HARNESS_GOTO_TIMEOUT_MS,
});
await harnessPage.waitForFunction(
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- waitForFunction runs in browser context where window types are loose.
() => (window as any).__mokoshHarness !== undefined,
{ timeout: HARNESS_BOOTSTRAP_TIMEOUT_MS },
);
return {
browser,
extensionId,
harnessPage,
victimPage,
downloadsDir,
swConsole,
offConsole,
};
}