Build out the Approach-B harness driver utilities atop the Wave 1
production paths. Three new files form the shared scaffold that
Wave 3's 13 assertion drivers (A1-A5, A7-A13) and the eventual
orchestrator (`tests/uat/harness.test.ts`) will all consume. The
standalone A6 driver (`tests/uat/a6.test.ts`) is rewritten to use
the new lib — behavior-preserving: A6 still PASSES 5/5 in ~7s.
New files:
- tests/uat/lib/launch.ts (~320 LoC)
`launchHarnessBrowser({ headless?, downloadsDir? }) → HarnessHandles`
Extracts the Chrome-launch + victim-page + harness-page + console-
attach pattern from a6.test.ts into a single reusable helper.
NEW vs prototype: CDP `Browser.setDownloadBehavior` wires
Chrome's download path to a per-run `mkdtempSync` tmp dir so A5
(SAVE_ARCHIVE) can poll a known location without colliding with
the operator's real downloads. Architectural commitments
enforced (per 01-11-SUMMARY): no `--auto-select-desktop-capture-
source` flag; victim about:blank brought to front for the
production `chrome.tabs.query({active:true})` workaround; SW
console attach best-effort with bounded poll; offscreen console
attach opportunistic via `targetcreated` listener (offscreen
target appears later, when the harness page calls
chrome.offscreen.createDocument).
- tests/uat/lib/assertions.ts (~210 LoC)
Host-side assertion primitives:
* `AssertionRecord`, `CheckRecord`, `ConsoleBuffers` types —
mirror the page-side shape returned by `assertA*` methods.
* `runAssertion(name, fn, buffers)` — try/catch wrapper that
dumps the SW + offscreen console tails (last 100 lines each)
to stderr on failure, then returns `{passed: false, error}`
if `fn` throws.
* `printAssertionResult(result)` — single source of truth for
the formatted result print. Extracted from the inline
`printResult` previously in the prototype's a6.test.ts so
Wave 3's orchestrator can reuse it across all 14 assertions.
* `assertEqual / assertGte / assertMatch / assertTrue` —
structured failure messages atop node:assert/strict.
* `waitFor(probe, predicate, timeoutMs, description)` — host-
side polling primitive; mirrors the page-side waitFor
semantics verbatim (they can't share a module: page-side is
bundled into the harness HTML, host-side runs in Node).
NO chrome.* helpers here — all chrome.* work happens inside the
extension-internal harness page. This module is host-side ONLY
by construction (no chrome global in Node anyway).
- tests/uat/lib/harness-page-driver.ts (~170 LoC)
One driver wrapper per assertion (A1..A13). Each wraps a single
`page.evaluate(() => window.__mokoshHarness.assertXX())`.
Centralizing this means adding/renaming an assertion = two-file
edit (extension-page-harness.ts impl + this file) instead of
touching every test-file caller.
Wave 2 wires `driveA6` (proven from c647f61). The 12 Wave-3
drivers (driveA1..A5, A7..A13) are stubbed as
`throw new Error('NOT YET IMPLEMENTED — Wave 3<X> wires driveXX')`
so the future orchestrator's `for (const drive of drivers)` loop
fails cleanly on the first unimplemented one (bail-on-first-
failure semantics). The `AssertionWithBytes` type is declared
for A5/A12/A13 which return `bytesBase64` payloads (zip / webm
bytes that the host side processes after the page-side
assertion completes).
Rewrite — `tests/uat/a6.test.ts`:
- Drops ~80 LoC of Chrome-launch + console-attach + result-print
plumbing now living in lib/launch.ts + lib/assertions.ts.
- Now ~70 LoC total — pure orchestration of
launchHarnessBrowser → runAssertion(driveA6) → printAssertionResult
→ browser.close() → exit code.
- Behavior-preserving: A6 still 5/5 GREEN with the same diagnostic
output (SETUP, A6.1-A6.4) and the same ~7s end-to-end runtime.
Verification (all GREEN):
- `npx tsc --noEmit` — exit 0 (root + tests/uat/tsconfig.json).
- `npx tsx tests/uat/a6.test.ts` — exits 0 with "PASS"; 5 checks
GREEN (SETUP, A6.1, A6.2, A6.3, A6.4). End-to-end runtime ~7s
headless on this workstation.
- `npm run build` — exit 0; Tier-1 grep gate GREEN (production
bundle contains zero hook strings AND zero lib symbol names —
the new lib files are test-only and not bundled into dist/).
- `npm run build:test` — exit 0; dist-test/ still emits the
extension-page-harness.html harness (lib files are host-side,
not rollup inputs).
- `npx vitest run` — 92/92 GREEN.
Wave 3 ready: harness-page-driver.ts has driveA1..A5/A7..A13 stubs
in place; extending requires only:
1. Add `assertAXX` method to window.__mokoshHarness in
tests/uat/extension-page-harness.ts.
2. Replace the corresponding stub body in this file with the
page.evaluate wrapper.
3. (Wave 3A) Create tests/uat/harness.test.ts orchestrator that
iterates over [A0 grep gate, driveA1..A13] with bail-on-fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
395 lines
15 KiB
TypeScript
395 lines
15 KiB
TypeScript
// tests/uat/lib/launch.ts — Plan 01-13 Wave 2.
|
|
//
|
|
// Approach-B harness launch helper. Inherits the Puppeteer launch +
|
|
// victim-page-bringToFront + harness-page-open pattern from the proven
|
|
// `tests/uat/a6.test.ts` driver (originally landed as Plan 01-11
|
|
// prototype at commit c647f61; promoted to production paths by 01-13
|
|
// Wave 1). Refactored into a reusable helper so Wave 3's 13 assertion
|
|
// drivers share the same setup overhead — one Chrome launch + one
|
|
// harness page + one victim page per `npm run test:uat` run.
|
|
//
|
|
// Architectural commitments (per 01-11-SUMMARY.md, DO NOT REGRESS):
|
|
// - Drive Chrome FROM INSIDE: `harnessPage` runs at
|
|
// `chrome-extension://<id>/tests/uat/extension-page-harness.html`
|
|
// with full chrome.* API access (Approach B; sw.evaluate fallback
|
|
// was falsified per SUMMARY §2 — only chrome.{loadTimes,csi}
|
|
// surfaced through CDP).
|
|
// - `victimPage` is a brought-to-front about:blank tab so the
|
|
// production `chrome.tabs.query({active:true})` sees a real tab
|
|
// with a `.url` (Plan 01-13 retains the `tabs` permission gap as
|
|
// out-of-scope; A2 + similar tests send `START_RECORDING` directly
|
|
// to offscreen, bypassing the SW's `startVideoCapture` which needs
|
|
// the tabs permission to read `tab.url`). Workaround documented in
|
|
// the plan's resolved-open-questions table row 2.
|
|
// - Downloads land in a per-run tmp dir (`mkdtempSync`) so A5 polling
|
|
// does not collide with operator downloads. Configured via CDP
|
|
// `Browser.setDownloadBehavior` on the harness page's CDP session.
|
|
// - SW + offscreen consoles forwarded to `swConsole` / `offConsole`
|
|
// accumulating string buffers. Offscreen attach via
|
|
// `browser.on('targetcreated')` is OPPORTUNISTIC per the prototype
|
|
// pattern — offscreen targets appear asynchronously when
|
|
// `chrome.offscreen.createDocument` runs from inside the harness
|
|
// page; the harness must not block waiting for them.
|
|
// - NO `--auto-select-desktop-capture-source` flag: unreliable in
|
|
// `--headless=new` per 01-11-SUMMARY falsification 4. The synthetic
|
|
// `installFakeDisplayMedia` (offscreen-hooks.ts eager install)
|
|
// bypasses Chrome's picker entirely.
|
|
//
|
|
// References:
|
|
// - puppeteer.launch options:
|
|
// https://pptr.dev/api/puppeteer.launchoptions
|
|
// - puppeteer.Browser.extensions():
|
|
// https://pptr.dev/api/puppeteer.browser.extensions
|
|
// - CDP Browser.setDownloadBehavior (per-context download path):
|
|
// https://chromedevtools.github.io/devtools-protocol/tot/Browser/#method-setDownloadBehavior
|
|
// - puppeteer CDP session helper:
|
|
// https://pptr.dev/api/puppeteer.cdpsession
|
|
// - Node fs.mkdtempSync:
|
|
// https://nodejs.org/api/fs.html#fsmkdtempsyncprefix-options
|
|
|
|
import { existsSync, mkdtempSync, statSync } from 'node:fs';
|
|
import { tmpdir } from 'node:os';
|
|
import { dirname, join, resolve as resolvePath } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
import puppeteer, { type Browser, type Page } from 'puppeteer';
|
|
|
|
/** Repo root resolved from this file's location (tests/uat/lib/launch.ts). */
|
|
const HARNESS_FILE_DIR = dirname(fileURLToPath(import.meta.url));
|
|
const REPO_ROOT = resolvePath(HARNESS_FILE_DIR, '..', '..', '..');
|
|
const DIST_TEST_DIR = resolvePath(REPO_ROOT, 'dist-test');
|
|
|
|
/** Time bounds for the various polling/attach steps. Keep in sync with
|
|
* the comments below — each value has a rationale, not a guess. */
|
|
const EXTENSION_ID_POLL_TIMEOUT_MS = 5_000;
|
|
const EXTENSION_ID_POLL_INTERVAL_MS = 100;
|
|
const HARNESS_BOOTSTRAP_TIMEOUT_MS = 5_000;
|
|
const HARNESS_GOTO_TIMEOUT_MS = 10_000;
|
|
const SW_TARGET_ATTACH_TIMEOUT_MS = 10_000;
|
|
/** Bumped from the puppeteer default 30s to give the assertions
|
|
* several sendMessage round-trips of CDP headroom on slow CI runners. */
|
|
const PROTOCOL_TIMEOUT_MS = 90_000;
|
|
|
|
/**
|
|
* Handles returned by `launchHarnessBrowser`. The caller owns the
|
|
* `browser` and is responsible for calling `browser.close()` in a
|
|
* `finally` block. `downloadsDir` is created by this function and is
|
|
* deliberately NOT cleaned up automatically — failing tests benefit
|
|
* from the operator inspecting the downloads dir post-mortem.
|
|
*/
|
|
export interface HarnessHandles {
|
|
readonly browser: Browser;
|
|
readonly extensionId: string;
|
|
readonly harnessPage: Page;
|
|
readonly victimPage: Page;
|
|
readonly downloadsDir: string;
|
|
/** Accumulating SW console log lines, format `[sw:<type>] <text>`. */
|
|
readonly swConsole: string[];
|
|
/** Accumulating offscreen console log lines, format `[off:<type>] <text>`. */
|
|
readonly offConsole: string[];
|
|
}
|
|
|
|
/**
|
|
* Options for `launchHarnessBrowser`. All fields optional; defaults
|
|
* apply (`headless: process.env.HEADLESS !== '0'`; `downloadsDir` ←
|
|
* fresh mkdtempSync).
|
|
*/
|
|
export interface LaunchOptions {
|
|
/** Override `--headless=new`; useful for visual debugging. */
|
|
readonly headless?: boolean;
|
|
/** Override the auto-created downloads dir; useful for cross-run debugging. */
|
|
readonly downloadsDir?: string;
|
|
}
|
|
|
|
/**
|
|
* Verify the test bundle is present at `dist-test/`; fail loudly with
|
|
* an actionable error if missing. The harness cannot launch without
|
|
* the bundle so failing early avoids confusing puppeteer errors.
|
|
*
|
|
* @throws If `dist-test/` is missing or not a directory.
|
|
*/
|
|
function assertBundlePresent(): void {
|
|
if (!existsSync(DIST_TEST_DIR)) {
|
|
throw new Error(
|
|
`dist-test/ missing at ${DIST_TEST_DIR} — run \`npm run build:test\` first.`,
|
|
);
|
|
}
|
|
if (!statSync(DIST_TEST_DIR).isDirectory()) {
|
|
throw new Error(`dist-test/ at ${DIST_TEST_DIR} is not a directory.`);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Poll `browser.extensions()` until at least one extension is loaded
|
|
* or the timeout elapses. Returns the first extension's id.
|
|
*
|
|
* @param browser - Puppeteer browser handle.
|
|
* @returns The resolved extension id string.
|
|
* @throws If no extension loads within `EXTENSION_ID_POLL_TIMEOUT_MS`.
|
|
*/
|
|
async function resolveExtensionIdWithPolling(browser: Browser): Promise<string> {
|
|
const pollStart = Date.now();
|
|
let extensionsMap = await browser.extensions();
|
|
while (
|
|
extensionsMap.size === 0 &&
|
|
Date.now() - pollStart < EXTENSION_ID_POLL_TIMEOUT_MS
|
|
) {
|
|
await new Promise((resolve) => setTimeout(resolve, EXTENSION_ID_POLL_INTERVAL_MS));
|
|
extensionsMap = await browser.extensions();
|
|
}
|
|
const entries = [...extensionsMap];
|
|
if (entries.length === 0) {
|
|
throw new Error(
|
|
`No extensions loaded after ${EXTENSION_ID_POLL_TIMEOUT_MS}ms — dist-test/ malformed?`,
|
|
);
|
|
}
|
|
const [extensionId] = entries[0];
|
|
return extensionId;
|
|
}
|
|
|
|
/**
|
|
* Attach a SW console listener that forwards every console event to
|
|
* the provided buffer (both for in-memory diagnostic capture AND for
|
|
* stderr streaming so the operator sees live logs during a hung
|
|
* assertion). Best-effort: if the SW target cannot be found inside
|
|
* `SW_TARGET_ATTACH_TIMEOUT_MS`, the failure is logged to stderr but
|
|
* the harness continues (the assertion may still pass — many
|
|
* assertions do not need SW console data).
|
|
*
|
|
* @param browser - Puppeteer browser handle.
|
|
* @param extensionId - The resolved extension id.
|
|
* @param swConsole - Accumulating string buffer to push log lines into.
|
|
*/
|
|
async function attachSwConsoleBestEffort(
|
|
browser: Browser,
|
|
extensionId: string,
|
|
swConsole: string[],
|
|
): Promise<void> {
|
|
try {
|
|
const swTarget = await browser.waitForTarget(
|
|
(t) => t.type() === 'service_worker' && t.url().includes(extensionId),
|
|
{ timeout: SW_TARGET_ATTACH_TIMEOUT_MS },
|
|
);
|
|
const sw = await swTarget.worker();
|
|
if (sw !== null) {
|
|
/**
|
|
* Named callback per project style — every chrome.* console event
|
|
* formatted with a leading `[sw:<type>]` tag for grep-ability.
|
|
*/
|
|
const onSwConsole = (msg: { type: () => string; text: () => string }): void => {
|
|
const line = `[sw:${msg.type()}] ${msg.text()}`;
|
|
swConsole.push(line);
|
|
process.stderr.write(line + '\n');
|
|
};
|
|
sw.on('console', onSwConsole);
|
|
}
|
|
} catch (swAttachErr) {
|
|
process.stderr.write(
|
|
`(launch: SW console attach skipped — ${String(swAttachErr)})\n`,
|
|
);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Register a `targetcreated` listener that lazily attaches the
|
|
* offscreen console once it appears. The offscreen target is created
|
|
* later (when the harness page calls `chrome.offscreen.createDocument`),
|
|
* so we cannot wait for it eagerly; instead we register the listener
|
|
* upfront and let it fire when the offscreen target spawns.
|
|
*
|
|
* Idempotent — only the first matching offscreen target is attached.
|
|
*
|
|
* @param browser - Puppeteer browser handle.
|
|
* @param extensionId - The resolved extension id.
|
|
* @param offConsole - Accumulating string buffer for offscreen log lines.
|
|
*/
|
|
function registerOffscreenConsoleAttach(
|
|
browser: Browser,
|
|
extensionId: string,
|
|
offConsole: string[],
|
|
): void {
|
|
let offscreenAttached = false;
|
|
/**
|
|
* Targetcreated handler — checks each new target for the offscreen
|
|
* extension URL pattern, attaches the console listener on the first
|
|
* match.
|
|
*/
|
|
const onTargetCreated = async (
|
|
target: { type: () => string; url: () => string; asPage: () => Promise<Page> },
|
|
): Promise<void> => {
|
|
if (offscreenAttached) {
|
|
return;
|
|
}
|
|
const url = target.url();
|
|
if (
|
|
target.type() === 'background_page' &&
|
|
url.includes(extensionId) &&
|
|
url.includes('offscreen')
|
|
) {
|
|
offscreenAttached = true;
|
|
try {
|
|
const offPage = await target.asPage();
|
|
/**
|
|
* Per-message callback — same tag format as the SW attach
|
|
* (`[off:<type>] <text>`).
|
|
*/
|
|
const onOffConsole = (msg: { type: () => string; text: () => string }): void => {
|
|
const line = `[off:${msg.type()}] ${msg.text()}`;
|
|
offConsole.push(line);
|
|
process.stderr.write(line + '\n');
|
|
};
|
|
offPage.on('console', onOffConsole);
|
|
} catch (offAttachErr) {
|
|
process.stderr.write(
|
|
`(launch: offscreen console attach skipped — ${String(offAttachErr)})\n`,
|
|
);
|
|
}
|
|
}
|
|
};
|
|
browser.on('targetcreated', onTargetCreated);
|
|
}
|
|
|
|
/**
|
|
* Configure the harness page's CDP session to use the per-run
|
|
* `downloadsDir` so A5 (SAVE_ARCHIVE → chrome.downloads.download) can
|
|
* poll a known directory without colliding with the operator's real
|
|
* downloads. Uses CDP `Browser.setDownloadBehavior` with
|
|
* `behavior: 'allow'` + the explicit path.
|
|
*
|
|
* @param harnessPage - The opened harness page handle.
|
|
* @param downloadsDir - Absolute path to the downloads directory.
|
|
*/
|
|
async function configureDownloadsDir(
|
|
harnessPage: Page,
|
|
downloadsDir: string,
|
|
): Promise<void> {
|
|
const session = await harnessPage.createCDPSession();
|
|
await session.send('Browser.setDownloadBehavior', {
|
|
behavior: 'allow',
|
|
downloadPath: downloadsDir,
|
|
});
|
|
}
|
|
|
|
/**
|
|
* Launch Chrome with the test bundle as an unpacked MV3 extension,
|
|
* open the extension-internal harness page + a victim about:blank
|
|
* page, configure downloads, attach SW + offscreen console listeners,
|
|
* and return the assembled handles.
|
|
*
|
|
* Caller MUST close the browser in a `finally` block:
|
|
* ```typescript
|
|
* const handles = await launchHarnessBrowser();
|
|
* try {
|
|
* // ... run assertions ...
|
|
* } finally {
|
|
* await handles.browser.close();
|
|
* }
|
|
* ```
|
|
*
|
|
* @param opts - Override headless / downloadsDir.
|
|
* @returns Assembled HarnessHandles.
|
|
*/
|
|
export async function launchHarnessBrowser(
|
|
opts: LaunchOptions = {},
|
|
): Promise<HarnessHandles> {
|
|
assertBundlePresent();
|
|
|
|
const headless = opts.headless ?? process.env.HEADLESS !== '0';
|
|
const downloadsDir = opts.downloadsDir ?? mkdtempSync(join(tmpdir(), 'mokosh-uat-'));
|
|
|
|
const browser = await puppeteer.launch({
|
|
enableExtensions: [DIST_TEST_DIR],
|
|
headless,
|
|
pipe: true,
|
|
protocolTimeout: PROTOCOL_TIMEOUT_MS,
|
|
args: [
|
|
'--no-sandbox',
|
|
// DO NOT add --auto-select-desktop-capture-source — unreliable
|
|
// in --headless=new per 01-11-SUMMARY falsification 4; the
|
|
// synthetic getDisplayMedia (offscreen-hooks.ts:installFake)
|
|
// bypasses Chrome's picker entirely.
|
|
],
|
|
});
|
|
|
|
const extensionId = await resolveExtensionIdWithPolling(browser);
|
|
|
|
// Accumulating console buffers — empty until SW + offscreen attach.
|
|
const swConsole: string[] = [];
|
|
const offConsole: string[] = [];
|
|
|
|
// Open the victim page FIRST so it's already in the tab list when
|
|
// the harness page opens. About:blank's `tab.url` resolves to
|
|
// 'about:blank' (truthy), passing production
|
|
// chrome.tabs.query({active:true}) presence checks.
|
|
const victimPage = await browser.newPage();
|
|
await victimPage.goto('about:blank');
|
|
|
|
// Open the harness page; attach console + pageerror listeners
|
|
// BEFORE the goto so we don't miss bootstrap-time messages.
|
|
const harnessPage = await browser.newPage();
|
|
/**
|
|
* Named callback per project style — forwards all page-side console
|
|
* events to stderr with the `[page:<type>]` tag.
|
|
*/
|
|
const onPageConsole = (msg: { type: () => string; text: () => string }): void => {
|
|
const line = `[page:${msg.type()}] ${msg.text()}`;
|
|
process.stderr.write(line + '\n');
|
|
};
|
|
harnessPage.on('console', onPageConsole);
|
|
/**
|
|
* Named callback — page errors get an explicit `[page:ERROR]` tag
|
|
* separate from the console events so the operator can spot them in
|
|
* the stderr stream.
|
|
*/
|
|
const onPageError = (err: unknown): void => {
|
|
const msg = err instanceof Error ? err.message : String(err);
|
|
const line = `[page:ERROR] ${msg}`;
|
|
process.stderr.write(line + '\n');
|
|
};
|
|
harnessPage.on('pageerror', onPageError);
|
|
|
|
// Best-effort SW console attach — the SW target is usually ready
|
|
// by the time the extension finishes loading, but slow CI may need
|
|
// a brief poll (handled inside `attachSwConsoleBestEffort`).
|
|
await attachSwConsoleBestEffort(browser, extensionId, swConsole);
|
|
|
|
// Register the offscreen console attach BEFORE opening the harness
|
|
// page so the listener catches the offscreen target whenever it
|
|
// spawns (which happens later, when the page calls
|
|
// chrome.offscreen.createDocument from inside an assertion).
|
|
registerOffscreenConsoleAttach(browser, extensionId, offConsole);
|
|
|
|
// Configure downloads via CDP. This MUST happen on the harness
|
|
// page's CDP session (not the browser's default session) per
|
|
// puppeteer's per-page session model.
|
|
await configureDownloadsDir(harnessPage, downloadsDir);
|
|
|
|
// Bring the victim page to front so chrome.tabs.query({active:true})
|
|
// returns it (not the harness page) when production startVideoCapture
|
|
// runs. The harness page can still be evaluated against — Puppeteer's
|
|
// page handle doesn't care about active-tab state.
|
|
await victimPage.bringToFront();
|
|
|
|
// Open the harness page; wait for window.__mokoshHarness to install.
|
|
const harnessUrl = `chrome-extension://${extensionId}/tests/uat/extension-page-harness.html`;
|
|
await harnessPage.goto(harnessUrl, {
|
|
waitUntil: 'domcontentloaded',
|
|
timeout: HARNESS_GOTO_TIMEOUT_MS,
|
|
});
|
|
await harnessPage.waitForFunction(
|
|
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- waitForFunction runs in browser context where window types are loose.
|
|
() => (window as any).__mokoshHarness !== undefined,
|
|
{ timeout: HARNESS_BOOTSTRAP_TIMEOUT_MS },
|
|
);
|
|
|
|
return {
|
|
browser,
|
|
extensionId,
|
|
harnessPage,
|
|
victimPage,
|
|
downloadsDir,
|
|
swConsole,
|
|
offConsole,
|
|
};
|
|
}
|