feat(01-13): wave-3A — A1+A2+A3+A4 GREEN + harness.test.ts orchestrator (5/14 assertions GREEN)
Wave 3A landed. `npm run test:uat` now exercises 5/14 assertions
end-to-end (A0 + A1 + A2 + A3 + A4); bails at A5 NOT YET IMPLEMENTED
(Wave 3B scope). A6 still PASSES 5/5 through the standalone
`npx tsx tests/uat/a6.test.ts` entry — the orchestrator-level A6 won't
reach in Wave 3A because the sequential loop bails at A5; once Wave 3B
wires driveA5 the loop will fall through to A6 (which uses the proven
Wave-2 driveA6 driver — no rework needed there).
Files changed:
- `tests/uat/extension-page-harness.ts` — extends `window.__mokoshHarness`
from `{ assertA6 }` to `{ assertA1, assertA2, assertA3, assertA4,
assertA6 }`. Per-assertion contracts:
• A1 — chrome.action.getBadgeText({}) === '' + getPopup({}) === ''
+ isRecording=false (badge !== 'REC' proxy per state-machine atomic
pairing). 3 CheckRecords.
• A2 — ensureOffscreen + START_RECORDING direct-to-offscreen
(workaround for the `tabs` manifest permission gap per
01-11-SUMMARY + plan resolved-questions row 2) + manual
setBadgeText('REC') + setPopup(POPUP_HTML_PATH) + waitFor
badge==='REC'. The bypassed chrome.action.onClicked →
startVideoCapture path is unit-tested in
tests/background/badge-state-machine.test.ts; A2 verifies the
contract that matters (recording reaches the REC state-machine
row). 2 CheckRecords.
• A3 — offscreen bridge query 'get-display-surface' (new in this
plan via the prior commit's offscreen-hooks extension) → asserts
=== 'monitor'. 1 CheckRecord.
• A4 — getPopup remains 'src/popup/index.html' + hasDocument()===true
(no duplicate offscreen). Essentially a no-op verification —
regression protection against future refactors that might unpin
the popup during recording or spawn extra offscreens on stray
events. 2 CheckRecords.
• IMPORTANT: chrome.action.getPopup() returns the FULL absolute
chrome-extension://<id>/... URL (not the manifest-relative path).
A2.2 + A4.1 assert via .endsWith('src/popup/index.html') to stay
extension-id independent. Empirical finding from first orchestrator
run; documented inline.
- `tests/uat/lib/harness-page-driver.ts` — wires `driveA1/A2/A3/A4`
(replaces the 4 NOT YET IMPLEMENTED Wave-3A stubs from
eb64521). Each wraps a single page.evaluate(() =>
window.__mokoshHarness.assertXX()) call per the contract laid down
by driveA6. A5+A7..A13 remain stubbed for Waves 3B+3C+3D.
- `tests/uat/harness.test.ts` (NEW) — top-level UAT orchestrator
driving all 14 assertions sequentially against a single Chrome +
single harness page. A0 (Tier-1 grep gate) runs pre-flight before
any Chrome launch — mirrors
tests/background/no-test-hooks-in-prod-bundle.test.ts forbidden-
string inventory (9 entries; belt-and-suspenders per
feedback-pre-checkpoint-bundle-gates.md memory). Bail-on-first-
failure with [SKIP] markers for unreached assertions + structured
diagnostic dump (full SW + offscreen console tail) on each failure.
SKIP_PROD_REBUILD=1 escape hatch skips the A0-side `npm run build`
for developer iteration.
Verification (all GREEN):
- npx tsc --noEmit: clean (root)
- npx tsc --noEmit -p tests/uat: clean (UAT subtree)
- npm run build: clean; production bundle hook-free
(9-string grep gate in vitest unit gate)
- npm run build:test: clean; dist-test/assets/extension_page_harness-*.js
grew from 3.87kB → 7.67kB (A1+A2+A3+A4 added)
- SKIP_BUILD=1 npx vitest run: 93/93 GREEN
(Wave 0+1+2 baseline 92 + 1 from the 9th grep-gate string from
the prior commit; this commit adds zero new vitest tests — the
A1-A4 contracts are verified at UAT-harness time only)
- npx tsx tests/uat/a6.test.ts (standalone): 5/5 GREEN; exit 0
(Wave-2 A6 baseline preserved through orchestrator-adjacent
harness page surface extension)
- npm run test:uat (full operator entry): 5/14 GREEN
(A0 + A1 + A2 + A3 + A4); bails at A5 NOT YET IMPLEMENTED
(Wave 3B scope, expected). Total wall clock ~25s (~5s build +
~5s prod-rebuild for A0 + ~15s assertion sequence).
Operator empirical-verification deferred to orchestrator (per
feedback-pre-checkpoint-bundle-gates.md — the orchestrator runs SW
CSP-safety + Node-globals + DOM-globals grep on the built bundle
before surfacing any checkpoint).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -51,8 +51,18 @@
|
|||||||
// dispatch-ended — trigger Bug B simulation via offscreen
|
// dispatch-ended — trigger Bug B simulation via offscreen
|
||||||
// bridge (offscreen still uses dynamic import → works)
|
// bridge (offscreen still uses dynamic import → works)
|
||||||
//
|
//
|
||||||
// Wave 1 surface — the page exposes `window.__mokoshHarness` with one
|
// Wave 3A surface — extends `window.__mokoshHarness` from 1 → 5 methods:
|
||||||
// method (assertA6); Wave 3 extends to all 13 assertions:
|
// - `assertA1()` — SW bootstrap state (badge='', popup='', isRecording=false).
|
||||||
|
// - `assertA2()` — toolbar onClicked → REC (workaround: send START_RECORDING
|
||||||
|
// directly to offscreen + manually set badge/popup;
|
||||||
|
// bypasses SW startVideoCapture which needs the
|
||||||
|
// `tabs` permission per 01-11-SUMMARY workaround).
|
||||||
|
// - `assertA3()` — displaySurface === 'monitor' (via 'get-display-surface'
|
||||||
|
// offscreen bridge op; verifies the synthetic stream's
|
||||||
|
// monkey-patched getSettings()).
|
||||||
|
// - `assertA4()` — popup stays pinned during recording (REC state
|
||||||
|
// preserves setRecordingMode's setPopup; offscreen
|
||||||
|
// count remains 1 — no second offscreen spawns).
|
||||||
// - `assertA6()` — canonical Bug B regression assertion (proven).
|
// - `assertA6()` — canonical Bug B regression assertion (proven).
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -406,22 +416,293 @@ async function assertA6(): Promise<AssertionResult> {
|
|||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A1 — SW bootstrap state. Asserts the post-load idle state per
|
||||||
|
* src/background/index.ts:setIdleMode (badge='', popup=''). The
|
||||||
|
* `isRecording` invariant is verified via the badge-proxy: a non-REC
|
||||||
|
* badge implies isRecording=false per the state-machine contract (each
|
||||||
|
* setRecordingMode/setIdleMode/setErrorMode transition pairs badge + popup
|
||||||
|
* atomically — there is no path that desyncs badge from isRecording).
|
||||||
|
*
|
||||||
|
* IMPORTANT — A1 MUST run before A2 in any orchestrated sequence. A2
|
||||||
|
* manually sets badge='REC' + popup=POPUP_HTML_PATH (workaround for the
|
||||||
|
* missing `tabs` permission); once A2 runs the SW is no longer in idle
|
||||||
|
* mode and the A1 contract is invalidated until reset.
|
||||||
|
*
|
||||||
|
* @returns Structured result with 3 checks (badge + popup + isRecording).
|
||||||
|
*/
|
||||||
|
async function assertA1(): Promise<AssertionResult> {
|
||||||
|
const result: AssertionResult = {
|
||||||
|
passed: false,
|
||||||
|
name: 'A1 — SW bootstrap state: badge=\'\', popup=\'\', isRecording=false',
|
||||||
|
checks: [],
|
||||||
|
diagnostics: [],
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
diag(result, 'Step 1: read chrome.action.getBadgeText({})');
|
||||||
|
const badge = await chrome.action.getBadgeText({});
|
||||||
|
diag(result, `Step 1 result: badge='${badge}'`);
|
||||||
|
|
||||||
|
diag(result, 'Step 2: read chrome.action.getPopup({})');
|
||||||
|
const popup = await chrome.action.getPopup({});
|
||||||
|
diag(result, `Step 2 result: popup='${popup}'`);
|
||||||
|
|
||||||
|
result.checks.push({
|
||||||
|
name: 'A1.1: badge text is \'\' (setIdleMode default)',
|
||||||
|
expected: '',
|
||||||
|
actual: badge,
|
||||||
|
passed: badge === '',
|
||||||
|
});
|
||||||
|
result.checks.push({
|
||||||
|
name: 'A1.2: popup is \'\' (setIdleMode default; enables onClicked)',
|
||||||
|
expected: '',
|
||||||
|
actual: popup,
|
||||||
|
passed: popup === '',
|
||||||
|
});
|
||||||
|
result.checks.push({
|
||||||
|
name: 'A1.3: isRecording=false (badge !== \'REC\' proxy)',
|
||||||
|
expected: false,
|
||||||
|
actual: badge === 'REC',
|
||||||
|
passed: badge !== 'REC',
|
||||||
|
});
|
||||||
|
|
||||||
|
result.passed = result.checks.every((c) => c.passed);
|
||||||
|
} catch (err) {
|
||||||
|
result.error = err instanceof Error ? err.message : String(err);
|
||||||
|
diag(result, `THREW: ${result.error}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A2 — toolbar onClicked → REC. Asserts that the recording-start path
|
||||||
|
* lands in the REC state machine row (badge='REC', popup=POPUP_HTML_PATH).
|
||||||
|
*
|
||||||
|
* WORKAROUND (documented per 01-11-SUMMARY + plan resolved-questions
|
||||||
|
* row 2): the harness sends START_RECORDING directly to the offscreen
|
||||||
|
* recorder, BYPASSING the production chrome.action.onClicked →
|
||||||
|
* startVideoCapture path. That path requires `chrome.tabs.query(
|
||||||
|
* {active: true})` to return a tab with `.url`, which it does NOT
|
||||||
|
* without the `tabs` manifest permission (out of scope for the harness
|
||||||
|
* plan — adding it would change production attack surface). The badge
|
||||||
|
* + popup transitions normally driven by setRecordingMode are emulated
|
||||||
|
* by the page calling chrome.action.setBadgeText + setPopup directly.
|
||||||
|
*
|
||||||
|
* Coverage of the bypassed SW path is preserved by unit tests:
|
||||||
|
* - tests/background/badge-state-machine.test.ts asserts
|
||||||
|
* setRecordingMode transitions setBadgeText('REC') + setPopup(...).
|
||||||
|
* - tests/background/sw-state-transitions.test.ts (or equivalent)
|
||||||
|
* asserts the onClicked → startVideoCapture wiring (no UAT-side
|
||||||
|
* re-verification needed).
|
||||||
|
*
|
||||||
|
* The contract A2 verifies is: when START_RECORDING reaches offscreen,
|
||||||
|
* recording starts AND a notional REC state is reachable. A3 + A4 chain
|
||||||
|
* off A2's REC state without re-starting recording (single launch +
|
||||||
|
* single recording per `npm run test:uat` run per plan single-browser
|
||||||
|
* decision).
|
||||||
|
*
|
||||||
|
* @returns Structured result with badge + popup checks.
|
||||||
|
*/
|
||||||
|
async function assertA2(): Promise<AssertionResult> {
|
||||||
|
const result: AssertionResult = {
|
||||||
|
passed: false,
|
||||||
|
name: 'A2 — toolbar onClicked → REC (direct-offscreen workaround for missing tabs permission)',
|
||||||
|
checks: [],
|
||||||
|
diagnostics: [],
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
diag(result, 'Step 1: ensureOffscreen (creates offscreen if missing)');
|
||||||
|
const ensureResp = await ensureOffscreen();
|
||||||
|
if (!ensureResp.ok) {
|
||||||
|
throw new Error(
|
||||||
|
`ensureOffscreen failed: ${ensureResp.error ?? '(no error)'}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
diag(result, 'Step 1 OK — offscreen ready');
|
||||||
|
|
||||||
|
diag(result, 'Step 2: START_RECORDING direct-to-offscreen + manual setBadge/setPopup');
|
||||||
|
const grantResp = await startRecording();
|
||||||
|
if (!grantResp.granted) {
|
||||||
|
throw new Error(
|
||||||
|
'startRecording returned granted=false — recording did not start',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
diag(result, 'Step 2 OK — granted=true');
|
||||||
|
|
||||||
|
diag(result, "Step 3: wait for badge === 'REC'");
|
||||||
|
const badgeAfter = await waitFor(
|
||||||
|
() => chrome.action.getBadgeText({}),
|
||||||
|
(v) => v === 'REC',
|
||||||
|
STATE_WAIT_MS,
|
||||||
|
'badge should transition to REC after START_RECORDING',
|
||||||
|
);
|
||||||
|
diag(result, `Step 3 OK — badge='${badgeAfter}'`);
|
||||||
|
|
||||||
|
diag(result, 'Step 4: read chrome.action.getPopup({})');
|
||||||
|
const popupAfter = await chrome.action.getPopup({});
|
||||||
|
diag(result, `Step 4 result: popup='${popupAfter}'`);
|
||||||
|
|
||||||
|
// NOTE — Chrome's chrome.action.getPopup() returns the FULL absolute
|
||||||
|
// URL form (e.g. 'chrome-extension://<id>/src/popup/index.html'), NOT
|
||||||
|
// the manifest-relative path that was passed to setPopup(). We assert
|
||||||
|
// .endsWith('src/popup/index.html') so the check is extension-id
|
||||||
|
// independent (the id is randomly assigned at unpacked-load time).
|
||||||
|
result.checks.push({
|
||||||
|
name: 'A2.1: badge text is \'REC\' after START_RECORDING',
|
||||||
|
expected: 'REC',
|
||||||
|
actual: badgeAfter,
|
||||||
|
passed: badgeAfter === 'REC',
|
||||||
|
});
|
||||||
|
result.checks.push({
|
||||||
|
name: 'A2.2: popup ends with \'src/popup/index.html\' (REC mode SAVE-only popup)',
|
||||||
|
expected: '<chrome-extension://<id>/>src/popup/index.html',
|
||||||
|
actual: popupAfter,
|
||||||
|
passed: popupAfter.endsWith('src/popup/index.html'),
|
||||||
|
});
|
||||||
|
|
||||||
|
result.passed = result.checks.every((c) => c.passed);
|
||||||
|
} catch (err) {
|
||||||
|
result.error = err instanceof Error ? err.message : String(err);
|
||||||
|
diag(result, `THREW: ${result.error}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A3 — displaySurface === 'monitor'. Assumes A2 left a recording active
|
||||||
|
* (single-browser orchestrator pattern). Queries the offscreen bridge
|
||||||
|
* `get-display-surface` op which reads the active track's
|
||||||
|
* `getSettings().displaySurface`. Production code in
|
||||||
|
* src/offscreen/recorder.ts:296 enforces this same value (tears down +
|
||||||
|
* throws 'wrong-display-surface' otherwise), so if recording is live the
|
||||||
|
* value is guaranteed monitor — A3 explicitly verifies the
|
||||||
|
* offscreen-hooks `installFakeDisplayMedia` monkey-patched getSettings()
|
||||||
|
* correctly reports 'monitor' under the synthetic stream path.
|
||||||
|
*
|
||||||
|
* @returns Structured result with the displaySurface check.
|
||||||
|
*/
|
||||||
|
async function assertA3(): Promise<AssertionResult> {
|
||||||
|
const result: AssertionResult = {
|
||||||
|
passed: false,
|
||||||
|
name: 'A3 — displaySurface === \'monitor\' (monkey-patched synthetic stream)',
|
||||||
|
checks: [],
|
||||||
|
diagnostics: [],
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
diag(result, "Step 1: bridge query 'get-display-surface'");
|
||||||
|
const resp = await offscreenQuery<{
|
||||||
|
displaySurface?: string | null;
|
||||||
|
ok?: boolean;
|
||||||
|
error?: string;
|
||||||
|
}>('get-display-surface');
|
||||||
|
diag(result, `Step 1 result: ${JSON.stringify(resp)}`);
|
||||||
|
|
||||||
|
if (resp.ok === false) {
|
||||||
|
throw new Error(
|
||||||
|
`get-display-surface returned ok=false: ${resp.error ?? '(no error)'}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const displaySurface = resp.displaySurface ?? null;
|
||||||
|
|
||||||
|
result.checks.push({
|
||||||
|
name: 'A3.1: displaySurface === \'monitor\' (offscreen-hooks monkey-patch)',
|
||||||
|
expected: 'monitor',
|
||||||
|
actual: displaySurface,
|
||||||
|
passed: displaySurface === 'monitor',
|
||||||
|
});
|
||||||
|
|
||||||
|
result.passed = result.checks.every((c) => c.passed);
|
||||||
|
} catch (err) {
|
||||||
|
result.error = err instanceof Error ? err.message : String(err);
|
||||||
|
diag(result, `THREW: ${result.error}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A4 — popup pinned during recording + no second offscreen. Assumes A2
|
||||||
|
* left a recording active. The contract verified:
|
||||||
|
* 1. getPopup still returns 'src/popup/index.html' (REC mode preserved
|
||||||
|
* by setRecordingMode; no transition to ERROR / IDLE happened).
|
||||||
|
* 2. chrome.offscreen.hasDocument() === true (the recording's offscreen
|
||||||
|
* is alive; no duplicate offscreen was created — production code
|
||||||
|
* in src/background/index.ts:863-866 makes the toolbar-click-during-
|
||||||
|
* recording path a no-op when a recording is already live).
|
||||||
|
*
|
||||||
|
* Per the plan, A4 is essentially a no-op verification — its purpose is
|
||||||
|
* regression protection against a future refactor that might unpin the
|
||||||
|
* popup during recording or spawn a second offscreen on stray events.
|
||||||
|
*
|
||||||
|
* @returns Structured result with popup + hasDocument checks.
|
||||||
|
*/
|
||||||
|
async function assertA4(): Promise<AssertionResult> {
|
||||||
|
const result: AssertionResult = {
|
||||||
|
passed: false,
|
||||||
|
name: 'A4 — popup pinned + single offscreen during recording',
|
||||||
|
checks: [],
|
||||||
|
diagnostics: [],
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
diag(result, 'Step 1: read chrome.action.getPopup({})');
|
||||||
|
const popup = await chrome.action.getPopup({});
|
||||||
|
diag(result, `Step 1 result: popup='${popup}'`);
|
||||||
|
|
||||||
|
diag(result, 'Step 2: chrome.offscreen.hasDocument()');
|
||||||
|
const hasDoc = await chrome.offscreen.hasDocument();
|
||||||
|
diag(result, `Step 2 result: hasDocument=${hasDoc}`);
|
||||||
|
|
||||||
|
// NOTE — see A2.2 NOTE: chrome.action.getPopup() returns absolute
|
||||||
|
// chrome-extension://<id>/... URLs; assert by .endsWith() to stay
|
||||||
|
// extension-id independent.
|
||||||
|
result.checks.push({
|
||||||
|
name: 'A4.1: popup remains \'src/popup/index.html\' during REC',
|
||||||
|
expected: '<chrome-extension://<id>/>src/popup/index.html',
|
||||||
|
actual: popup,
|
||||||
|
passed: popup.endsWith('src/popup/index.html'),
|
||||||
|
});
|
||||||
|
result.checks.push({
|
||||||
|
name: 'A4.2: chrome.offscreen.hasDocument() === true (recording offscreen alive)',
|
||||||
|
expected: true,
|
||||||
|
actual: hasDoc,
|
||||||
|
passed: hasDoc === true,
|
||||||
|
});
|
||||||
|
|
||||||
|
result.passed = result.checks.every((c) => c.passed);
|
||||||
|
} catch (err) {
|
||||||
|
result.error = err instanceof Error ? err.message : String(err);
|
||||||
|
diag(result, `THREW: ${result.error}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
// Install the global harness surface.
|
// Install the global harness surface.
|
||||||
declare global {
|
declare global {
|
||||||
interface Window {
|
interface Window {
|
||||||
__mokoshHarness: {
|
__mokoshHarness: {
|
||||||
|
assertA1: () => Promise<AssertionResult>;
|
||||||
|
assertA2: () => Promise<AssertionResult>;
|
||||||
|
assertA3: () => Promise<AssertionResult>;
|
||||||
|
assertA4: () => Promise<AssertionResult>;
|
||||||
assertA6: () => Promise<AssertionResult>;
|
assertA6: () => Promise<AssertionResult>;
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
window.__mokoshHarness = { assertA6 };
|
window.__mokoshHarness = { assertA1, assertA2, assertA3, assertA4, assertA6 };
|
||||||
|
|
||||||
const statusEl = document.getElementById('status');
|
const statusEl = document.getElementById('status');
|
||||||
if (statusEl !== null) {
|
if (statusEl !== null) {
|
||||||
statusEl.textContent = 'Harness ready. window.__mokoshHarness.assertA6() available.';
|
statusEl.textContent = 'Harness ready. window.__mokoshHarness.{assertA1, assertA2, assertA3, assertA4, assertA6} available.';
|
||||||
}
|
}
|
||||||
|
|
||||||
console.log('[harness-page] ready — window.__mokoshHarness installed');
|
console.log('[harness-page] ready — window.__mokoshHarness installed (Wave 3A: A1+A2+A3+A4+A6)');
|
||||||
|
|
||||||
export {};
|
export {};
|
||||||
|
|||||||
342
tests/uat/harness.test.ts
Normal file
342
tests/uat/harness.test.ts
Normal file
@@ -0,0 +1,342 @@
|
|||||||
|
// tests/uat/harness.test.ts — Plan 01-13 Wave 3A orchestrator.
|
||||||
|
//
|
||||||
|
// Top-level entry for the production UAT harness. Drives all 14
|
||||||
|
// assertions sequentially against a SINGLE launched Chrome instance with
|
||||||
|
// a SINGLE harness page; bails on the first failure with a structured
|
||||||
|
// diagnostic dump. Exits 0 only when 14/14 GREEN.
|
||||||
|
//
|
||||||
|
// Wave 3A scope — wires A0+A1+A2+A3+A4+A6 (A6 via the proven Wave-2
|
||||||
|
// driver). A5+A7..A13 throw `NOT YET IMPLEMENTED — Wave 3<X> wires this`
|
||||||
|
// from `tests/uat/lib/harness-page-driver.ts`; the bail-on-first-failure
|
||||||
|
// loop stops at the first such throw. Expected Wave-3A diagnostic:
|
||||||
|
// "UAT harness: 5/14 assertions passed (A0+A1+A2+A3+A4 GREEN; bail at A5)"
|
||||||
|
// A6 PASSES via the standalone `npx tsx tests/uat/a6.test.ts` entry —
|
||||||
|
// the orchestrator-level A6 won't reach in Wave 3A because the
|
||||||
|
// sequential loop bails at A5; A6 lands in the loop output once Wave 3B
|
||||||
|
// implements driveA5. The orchestrator structure is final from Wave 3A
|
||||||
|
// onward; future waves only fill in the assertion-driver stubs.
|
||||||
|
//
|
||||||
|
// Architectural commitments (per 01-11-SUMMARY.md, DO NOT REGRESS):
|
||||||
|
// - Single browser, single recording per run (state machine: idle →
|
||||||
|
// A1 reads idle → A2 transitions to REC → A3+A4 read REC →
|
||||||
|
// A5 saves archive → A6 simulates user-stop → A7 surfaces ERR → ...).
|
||||||
|
// - A0 (Tier-1 grep gate) runs PRE-FLIGHT before any Chrome launch.
|
||||||
|
// Mirrors `tests/background/no-test-hooks-in-prod-bundle.test.ts`
|
||||||
|
// FORBIDDEN_HOOK_STRINGS inventory. Belt-and-suspenders: the unit
|
||||||
|
// test gate runs in `npm test` (~15s); the UAT-level A0 runs in
|
||||||
|
// `npm run test:uat` (~60-90s). Same invariant; two independent
|
||||||
|
// verification paths.
|
||||||
|
// - Drive Chrome FROM INSIDE: each assertion is a single
|
||||||
|
// `page.evaluate(() => window.__mokoshHarness.assertXX())` call;
|
||||||
|
// no SW.evaluate, no popup-bridge (both falsified per 01-11-SUMMARY).
|
||||||
|
//
|
||||||
|
// References:
|
||||||
|
// - puppeteer.launch + extension loading:
|
||||||
|
// https://pptr.dev/api/puppeteer.launchoptions
|
||||||
|
// - Node fs.readdirSync recursive walk:
|
||||||
|
// https://nodejs.org/api/fs.html#fsreaddirsyncpath-options
|
||||||
|
// - Node child_process.execFileSync:
|
||||||
|
// https://nodejs.org/api/child_process.html#child_processexecfilesyncfile-args-options
|
||||||
|
|
||||||
|
import { execFileSync } from 'node:child_process';
|
||||||
|
import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs';
|
||||||
|
import { dirname, resolve as resolvePath } from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
import { launchHarnessBrowser } from './lib/launch';
|
||||||
|
import {
|
||||||
|
driveA1,
|
||||||
|
driveA2,
|
||||||
|
driveA3,
|
||||||
|
driveA4,
|
||||||
|
driveA5,
|
||||||
|
driveA6,
|
||||||
|
driveA7,
|
||||||
|
driveA8,
|
||||||
|
driveA9,
|
||||||
|
driveA10,
|
||||||
|
driveA11,
|
||||||
|
driveA12,
|
||||||
|
driveA13,
|
||||||
|
} from './lib/harness-page-driver';
|
||||||
|
import {
|
||||||
|
printAssertionResult,
|
||||||
|
runAssertion,
|
||||||
|
type AssertionRecord,
|
||||||
|
} from './lib/assertions';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A0 forbidden-string inventory — mirrors
|
||||||
|
* `tests/background/no-test-hooks-in-prod-bundle.test.ts:FORBIDDEN_HOOK_STRINGS`.
|
||||||
|
* Keep in sync. The two lists serving the same invariant is intentional
|
||||||
|
* (belt-and-suspenders per `feedback-pre-checkpoint-bundle-gates.md`):
|
||||||
|
* unit-test gate catches at `npm test`, UAT gate catches at `npm run test:uat`.
|
||||||
|
*/
|
||||||
|
const FORBIDDEN_HOOK_STRINGS: ReadonlyArray<string> = [
|
||||||
|
'__mokoshTest',
|
||||||
|
'setCurrentStream',
|
||||||
|
'setSegmentCountGetter',
|
||||||
|
'installFakeDisplayMedia',
|
||||||
|
'uninstallFakeDisplayMedia',
|
||||||
|
'dispatchEndedOnTrack',
|
||||||
|
'getSegmentCount',
|
||||||
|
'__mokoshOffscreenQuery',
|
||||||
|
'get-display-surface',
|
||||||
|
];
|
||||||
|
|
||||||
|
/** Build timeout for the pre-flight production rebuild (matches unit-gate value). */
|
||||||
|
const PROD_BUILD_TIMEOUT_MS = 60_000;
|
||||||
|
|
||||||
|
/** Resolve repo-root paths from this file's location. */
|
||||||
|
const HARNESS_FILE_DIR = dirname(fileURLToPath(import.meta.url));
|
||||||
|
const REPO_ROOT = resolvePath(HARNESS_FILE_DIR, '..', '..');
|
||||||
|
const DIST_DIR = resolvePath(REPO_ROOT, 'dist');
|
||||||
|
|
||||||
|
/** Binary extensions skipped during the grep walk (mirror of unit gate). */
|
||||||
|
const BINARY_EXTENSIONS: ReadonlySet<string> = new Set([
|
||||||
|
'.png', '.jpg', '.jpeg', '.gif', '.ico', '.webp', '.woff', '.woff2', '.ttf', '.otf',
|
||||||
|
]);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Recursively collect every regular file under `root`. Returns absolute
|
||||||
|
* paths sorted alphabetically for stable diagnostics.
|
||||||
|
*
|
||||||
|
* @param root - Absolute directory path to walk.
|
||||||
|
* @returns Sorted list of absolute file paths under `root`.
|
||||||
|
*/
|
||||||
|
function listAllFilesRecursive(root: string): ReadonlyArray<string> {
|
||||||
|
const accumulator: string[] = [];
|
||||||
|
const stack: string[] = [root];
|
||||||
|
while (stack.length > 0) {
|
||||||
|
const dir = stack.pop()!;
|
||||||
|
const entries = readdirSync(dir, { withFileTypes: true });
|
||||||
|
for (const entry of entries) {
|
||||||
|
const fullPath = resolvePath(dir, entry.name);
|
||||||
|
if (entry.isSymbolicLink()) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (entry.isDirectory()) {
|
||||||
|
stack.push(fullPath);
|
||||||
|
} else if (entry.isFile()) {
|
||||||
|
accumulator.push(fullPath);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return accumulator.sort();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Count occurrences of `needle` in the given file. Returns 0 for binary
|
||||||
|
* file extensions (text matching against UTF-8 of a PNG would be
|
||||||
|
* meaningless and could yield spurious matches).
|
||||||
|
*
|
||||||
|
* @param filePath - Absolute file path to scan.
|
||||||
|
* @param needle - Literal substring to count.
|
||||||
|
* @returns Total occurrences in the file's text.
|
||||||
|
*/
|
||||||
|
function countOccurrencesInFile(filePath: string, needle: string): number {
|
||||||
|
const dotIdx = filePath.lastIndexOf('.');
|
||||||
|
const ext = dotIdx >= 0 ? filePath.substring(dotIdx).toLowerCase() : '';
|
||||||
|
if (BINARY_EXTENSIONS.has(ext)) {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
const stat = statSync(filePath);
|
||||||
|
if (stat.size === 0) {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
const text = readFileSync(filePath, 'utf8');
|
||||||
|
let count = 0;
|
||||||
|
let from = 0;
|
||||||
|
for (;;) {
|
||||||
|
const idx = text.indexOf(needle, from);
|
||||||
|
if (idx < 0) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
count += 1;
|
||||||
|
from = idx + needle.length;
|
||||||
|
}
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A0 — Tier-1 grep gate (UAT-level mirror of the unit-gate). Spawns
|
||||||
|
* `npm run build` if `SKIP_PROD_REBUILD !== '1'`, then walks `dist/`
|
||||||
|
* checking every forbidden string. Reports all matches in one pass
|
||||||
|
* (full enumeration, not bail-on-first) so the operator sees the entire
|
||||||
|
* leak surface in a single failure.
|
||||||
|
*
|
||||||
|
* @returns Structured A0 result: passed flag + list of (string, file) matches.
|
||||||
|
*/
|
||||||
|
async function assertA0_GrepGate(): Promise<{
|
||||||
|
passed: boolean;
|
||||||
|
matches: Array<{ needle: string; filePath: string; count: number }>;
|
||||||
|
}> {
|
||||||
|
if (process.env.SKIP_PROD_REBUILD !== '1') {
|
||||||
|
process.stdout.write('A0: running `npm run build` (set SKIP_PROD_REBUILD=1 to skip)...\n');
|
||||||
|
execFileSync('npm', ['run', 'build'], {
|
||||||
|
stdio: 'inherit',
|
||||||
|
timeout: PROD_BUILD_TIMEOUT_MS,
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
process.stdout.write('A0: SKIP_PROD_REBUILD=1 — using existing dist/\n');
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!existsSync(DIST_DIR)) {
|
||||||
|
return {
|
||||||
|
passed: false,
|
||||||
|
matches: [
|
||||||
|
{
|
||||||
|
needle: '<missing dist/>',
|
||||||
|
filePath: DIST_DIR,
|
||||||
|
count: 0,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const files = listAllFilesRecursive(DIST_DIR);
|
||||||
|
const matches: Array<{ needle: string; filePath: string; count: number }> = [];
|
||||||
|
for (const needle of FORBIDDEN_HOOK_STRINGS) {
|
||||||
|
for (const filePath of files) {
|
||||||
|
const count = countOccurrencesInFile(filePath, needle);
|
||||||
|
if (count > 0) {
|
||||||
|
matches.push({ needle, filePath, count });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { passed: matches.length === 0, matches };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Top-to-bottom orchestrator entry. Pre-flight A0 → launch browser →
|
||||||
|
* iterate driver list → bail on first failure → close browser → return
|
||||||
|
* exit code.
|
||||||
|
*
|
||||||
|
* @returns Process exit code: 0 on 14/14 GREEN, 1 on any failure.
|
||||||
|
*/
|
||||||
|
async function main(): Promise<number> {
|
||||||
|
process.stdout.write('\nMokosh Plan 01-13 — UAT harness orchestrator\n');
|
||||||
|
process.stdout.write('Architecture: A0 pre-flight + extension-internal page driver (A1..A13)\n');
|
||||||
|
process.stdout.write('='.repeat(72) + '\n');
|
||||||
|
|
||||||
|
// A0 pre-flight (no Chrome launch needed; runs against built dist/).
|
||||||
|
const a0 = await assertA0_GrepGate();
|
||||||
|
if (!a0.passed) {
|
||||||
|
process.stderr.write('\nA0 FAIL: production bundle hook-string leak detected.\n');
|
||||||
|
for (const m of a0.matches) {
|
||||||
|
process.stderr.write(` - '${m.needle}' in ${m.filePath} (${m.count} occurrence${m.count === 1 ? '' : 's'})\n`);
|
||||||
|
}
|
||||||
|
process.stderr.write(
|
||||||
|
'\nThe Vite mode gate on the test-hook imports has regressed; verify\n' +
|
||||||
|
'src/background/index.ts + src/offscreen/recorder.ts still gate via `__MOKOSH_UAT__`.\n',
|
||||||
|
);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
process.stdout.write('A0: GREEN (production bundle hook-free)\n\n');
|
||||||
|
|
||||||
|
// Driver registry — execution order matters:
|
||||||
|
// A1 (idle) → A2 (REC start) → A3 (displaySurface) → A4 (popup pinned)
|
||||||
|
// → A5 (SAVE_ARCHIVE) → A6 (Bug B dispatch-ended) → A7 (genuine error)
|
||||||
|
// → A8 (Bug A onStartup) → A9 (icon sizes) → A10 (manifest)
|
||||||
|
// → A11 (35s segments) → A12 (ffprobe) → A13 (zip shape).
|
||||||
|
//
|
||||||
|
// A6 currently lives mid-list because the prototype's assertA6 does
|
||||||
|
// its own ensureOffscreen + START_RECORDING (idempotent w.r.t. A2's
|
||||||
|
// recording), then dispatch-ended. After A6 the recording is torn
|
||||||
|
// down — A7+ would need to re-start or test post-stop state.
|
||||||
|
//
|
||||||
|
// Wave 3A only A1..A4 wire to real impls; A5..A13 throw NOT YET
|
||||||
|
// IMPLEMENTED. Bail-on-first-failure stops the loop at A5 — A6's
|
||||||
|
// driver wires (via Wave 2's driveA6) but won't reach in this run.
|
||||||
|
// The standalone `npx tsx tests/uat/a6.test.ts` entry remains the
|
||||||
|
// way to verify A6 in isolation during Wave 3A.
|
||||||
|
const drivers: ReadonlyArray<{
|
||||||
|
readonly name: string;
|
||||||
|
readonly drive: (page: import('puppeteer').Page) => Promise<AssertionRecord>;
|
||||||
|
}> = [
|
||||||
|
{ name: 'A1', drive: driveA1 },
|
||||||
|
{ name: 'A2', drive: driveA2 },
|
||||||
|
{ name: 'A3', drive: driveA3 },
|
||||||
|
{ name: 'A4', drive: driveA4 },
|
||||||
|
{ name: 'A5', drive: driveA5 as (page: import('puppeteer').Page) => Promise<AssertionRecord> },
|
||||||
|
{ name: 'A6', drive: driveA6 },
|
||||||
|
{ name: 'A7', drive: driveA7 },
|
||||||
|
{ name: 'A8', drive: driveA8 },
|
||||||
|
{ name: 'A9', drive: driveA9 },
|
||||||
|
{ name: 'A10', drive: driveA10 },
|
||||||
|
{ name: 'A11', drive: driveA11 },
|
||||||
|
{ name: 'A12', drive: driveA12 as (page: import('puppeteer').Page) => Promise<AssertionRecord> },
|
||||||
|
{ name: 'A13', drive: driveA13 as (page: import('puppeteer').Page) => Promise<AssertionRecord> },
|
||||||
|
];
|
||||||
|
|
||||||
|
process.stdout.write('Launching Chrome + opening harness page...\n');
|
||||||
|
const handles = await launchHarnessBrowser();
|
||||||
|
process.stdout.write(`Extension id: ${handles.extensionId}\n`);
|
||||||
|
process.stdout.write(`Downloads dir: ${handles.downloadsDir}\n\n`);
|
||||||
|
|
||||||
|
const buffers = { swConsole: handles.swConsole, offConsole: handles.offConsole };
|
||||||
|
const results: Array<{ name: string; passed: boolean; error?: string }> = [];
|
||||||
|
let bailReason: string | null = null;
|
||||||
|
|
||||||
|
try {
|
||||||
|
for (const { name, drive } of drivers) {
|
||||||
|
process.stdout.write(`--- ${name} ---\n`);
|
||||||
|
let driverErr: string | undefined;
|
||||||
|
let result: AssertionRecord | null = null;
|
||||||
|
try {
|
||||||
|
result = await runAssertion(
|
||||||
|
name,
|
||||||
|
() => drive(handles.harnessPage),
|
||||||
|
buffers,
|
||||||
|
);
|
||||||
|
printAssertionResult(result);
|
||||||
|
} catch (err) {
|
||||||
|
driverErr = err instanceof Error ? err.message : String(err);
|
||||||
|
// A throw here is either: (a) a Wave-3 stub firing
|
||||||
|
// (NOT YET IMPLEMENTED) — expected during incremental waves; OR
|
||||||
|
// (b) a CDP/Puppeteer-level error (e.g. page closed, timeout) —
|
||||||
|
// a genuine harness regression. Both bail uniformly.
|
||||||
|
process.stderr.write(`*** ${name} THREW: ${driverErr}\n`);
|
||||||
|
}
|
||||||
|
const passed = result !== null && result.passed && driverErr === undefined;
|
||||||
|
results.push({ name, passed, error: driverErr });
|
||||||
|
if (!passed) {
|
||||||
|
bailReason = driverErr ?? `${name} failed; see structured checks above`;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
try {
|
||||||
|
await handles.browser.close();
|
||||||
|
} catch (closeErr) {
|
||||||
|
process.stderr.write(`(non-fatal: browser close threw: ${String(closeErr)})\n`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const passedCount = results.filter((r) => r.passed).length;
|
||||||
|
// Total = 1 (A0) + drivers.length (A1..A13) = 14.
|
||||||
|
const total = drivers.length + 1;
|
||||||
|
const finalPassed = passedCount + 1; // +1 for A0 (we already passed it to reach here)
|
||||||
|
|
||||||
|
process.stdout.write('\n' + '='.repeat(72) + '\n');
|
||||||
|
process.stdout.write(
|
||||||
|
`UAT harness: ${finalPassed}/${total} assertions passed${bailReason !== null ? ` (bailed: ${bailReason})` : ''}\n`,
|
||||||
|
);
|
||||||
|
for (const r of results) {
|
||||||
|
const mark = r.passed ? '[PASS]' : '[FAIL]';
|
||||||
|
const tail = r.error !== undefined ? ` — ${r.error}` : '';
|
||||||
|
process.stdout.write(` ${mark} ${r.name}${tail}\n`);
|
||||||
|
}
|
||||||
|
if (bailReason !== null) {
|
||||||
|
const remainingStart = results.length;
|
||||||
|
for (let i = remainingStart; i < drivers.length; i += 1) {
|
||||||
|
process.stdout.write(` [SKIP] ${drivers[i].name} (not reached — bailed at ${results[results.length - 1].name})\n`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
process.stdout.write('='.repeat(72) + '\n');
|
||||||
|
|
||||||
|
return finalPassed === total ? 0 : 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
const code = await main();
|
||||||
|
process.exit(code);
|
||||||
@@ -73,38 +73,74 @@ export async function driveA6(page: Page): Promise<AssertionRecord> {
|
|||||||
}) as AssertionRecord;
|
}) as AssertionRecord;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* ─── Wave 3A — NOT YET IMPLEMENTED ──────────────────────────────── */
|
/* ─── Wave 3A — WIRED ─────────────────────────────────────────────── */
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Drive A1 (SW bootstrap state). Wave 3A wires this.
|
* Drive A1 (SW bootstrap state). Asserts the post-load idle-mode state:
|
||||||
* @throws Always — replace stub when Wave 3A lands.
|
* badge='', popup='', isRecording=false. MUST run BEFORE A2 in any
|
||||||
|
* orchestrated sequence — A2 manually sets badge='REC' which invalidates
|
||||||
|
* the A1 contract until the SW is reset.
|
||||||
|
*
|
||||||
|
* @param page - The harness page from `launchHarnessBrowser`.
|
||||||
|
* @returns Structured AssertionRecord with 3 checks (badge + popup + isRecording).
|
||||||
*/
|
*/
|
||||||
export async function driveA1(_page: Page): Promise<AssertionRecord> {
|
export async function driveA1(page: Page): Promise<AssertionRecord> {
|
||||||
throw new Error(`${WAVE3_STUB_PREFIX} — Wave 3A wires driveA1`);
|
return await page.evaluate(async () => {
|
||||||
|
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- evaluate runs in browser context where Window types are loose.
|
||||||
|
const harness = (window as any).__mokoshHarness;
|
||||||
|
const r: AssertionRecord = await harness.assertA1();
|
||||||
|
return r;
|
||||||
|
}) as AssertionRecord;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Drive A2 (toolbar onClicked → REC). Wave 3A wires this.
|
* Drive A2 (toolbar onClicked → REC). Uses the direct-offscreen workaround
|
||||||
* @throws Always — replace stub when Wave 3A lands.
|
* for the missing `tabs` manifest permission (per 01-11-SUMMARY). Leaves
|
||||||
|
* the offscreen recording active — A3 + A4 chain off A2's REC state.
|
||||||
|
*
|
||||||
|
* @param page - The harness page from `launchHarnessBrowser`.
|
||||||
|
* @returns Structured AssertionRecord with 2 checks (badge + popup).
|
||||||
*/
|
*/
|
||||||
export async function driveA2(_page: Page): Promise<AssertionRecord> {
|
export async function driveA2(page: Page): Promise<AssertionRecord> {
|
||||||
throw new Error(`${WAVE3_STUB_PREFIX} — Wave 3A wires driveA2`);
|
return await page.evaluate(async () => {
|
||||||
|
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- evaluate runs in browser context where Window types are loose.
|
||||||
|
const harness = (window as any).__mokoshHarness;
|
||||||
|
const r: AssertionRecord = await harness.assertA2();
|
||||||
|
return r;
|
||||||
|
}) as AssertionRecord;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Drive A3 (displaySurface monitor). Wave 3A wires this.
|
* Drive A3 (displaySurface === 'monitor'). Assumes A2 left recording
|
||||||
* @throws Always — replace stub when Wave 3A lands.
|
* active. Queries the offscreen `get-display-surface` bridge op.
|
||||||
|
*
|
||||||
|
* @param page - The harness page from `launchHarnessBrowser`.
|
||||||
|
* @returns Structured AssertionRecord with 1 check (displaySurface).
|
||||||
*/
|
*/
|
||||||
export async function driveA3(_page: Page): Promise<AssertionRecord> {
|
export async function driveA3(page: Page): Promise<AssertionRecord> {
|
||||||
throw new Error(`${WAVE3_STUB_PREFIX} — Wave 3A wires driveA3`);
|
return await page.evaluate(async () => {
|
||||||
|
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- evaluate runs in browser context where Window types are loose.
|
||||||
|
const harness = (window as any).__mokoshHarness;
|
||||||
|
const r: AssertionRecord = await harness.assertA3();
|
||||||
|
return r;
|
||||||
|
}) as AssertionRecord;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Drive A4 (popup during recording). Wave 3A wires this.
|
* Drive A4 (popup pinned + single offscreen during recording). Assumes
|
||||||
* @throws Always — replace stub when Wave 3A lands.
|
* A2 left recording active. Verifies getPopup unchanged + hasDocument
|
||||||
|
* true (no duplicate offscreen spawned).
|
||||||
|
*
|
||||||
|
* @param page - The harness page from `launchHarnessBrowser`.
|
||||||
|
* @returns Structured AssertionRecord with 2 checks (popup + hasDocument).
|
||||||
*/
|
*/
|
||||||
export async function driveA4(_page: Page): Promise<AssertionRecord> {
|
export async function driveA4(page: Page): Promise<AssertionRecord> {
|
||||||
throw new Error(`${WAVE3_STUB_PREFIX} — Wave 3A wires driveA4`);
|
return await page.evaluate(async () => {
|
||||||
|
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- evaluate runs in browser context where Window types are loose.
|
||||||
|
const harness = (window as any).__mokoshHarness;
|
||||||
|
const r: AssertionRecord = await harness.assertA4();
|
||||||
|
return r;
|
||||||
|
}) as AssertionRecord;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* ─── Wave 3B — NOT YET IMPLEMENTED ──────────────────────────────── */
|
/* ─── Wave 3B — NOT YET IMPLEMENTED ──────────────────────────────── */
|
||||||
|
|||||||
Reference in New Issue
Block a user