Closes the iter-2 BLOCKER 1 resolution end-to-end: the inline-SVG
strategy now has HONEST automated coverage at two layers — source
contract (Task 1 unit tests + the narrowed A17.8 source-bundling
grep) and live-DOM cascade (the NEW host-side A35 harness assertion
that opens welcome.html as a real Puppeteer tab).
- tests/uat/extension-page-harness.ts (A17.8 NARROWED HONESTLY):
swap the data:image/svg+xml URL-grep + .svg filename-grep target
for a raw-source grep — A17.8 now asserts the welcome chunk JS
contains the raw SVG signature `stroke="currentColor"` AND the
canonical `viewBox="0 0 32 32"` (the `?raw` import's output). The
explanatory comment block now DISAVOWS the live-DOM claim and
points at the NEW A35 driver for the runtime injection + cascade
proof. A17.8 is honest source-bundling only.
- tests/uat/lib/harness-page-driver.ts (NEW host-side driveA35):
appended LAST per the iter-2 ADV-2C concern (any driver-pollution
worry is moot since nothing reads A35's return value, AND
welcomePage.close() in finally guarantees no tab leak). driveA35
opens chrome-extension://<id>/src/welcome/welcome.html in a fresh
browser.newPage() tab, waits for the `.welcome-hero__mark svg`
selector at DOMContentLoaded, then runs a single page.evaluate()
that reads four signals: A35.1 inline <svg> present, A35.2
stroke=currentColor, A35.3 getComputedStyle().stroke resolves to
a non-default colour (the real cascade proof), A35.4 no legacy
<img> in the slot. Host-side pattern mirrors driveA32/A33/A34.
- tests/uat/harness.test.ts (orchestrator wiring):
+ driveA35 added to the import block from './lib/harness-page-driver'.
+ driveA35Wrapped closure capturing handles.browser + handles.extensionId
(alongside driveA33Wrapped/driveA34Wrapped).
+ { name: 'A35', drive: driveA35Wrapped } appended as the LAST
entry of the `drivers` array. Total auto-increments via
`drivers.length + 1` (line 580) — no hardcoded count to bump.
+ Architecture banner string (line 283) refreshed with A33, A34,
A35 inline (ADV-2A cosmetic advisory — banner was already stale
pre-04-06; A33+A34 added at the same time).
- .planning/phases/01-stabilize-video-pipeline/01-07-SUMMARY.md
(back-patch, DEFECT 2 resolution):
Flipped 5 lines (22, 47, 82, 135, 205) that carried the now-stale
"deferred to Phase 5" framing for cursor visibility — the
`cursor: 'always'` constraint was opportunistically shipped in
Plan 01-09 (recorder.ts:285) and is verified by Plan 04-06 Task 1
(tests/build/cursor-visibility.test.ts). Each flip is surgical
(single line / single bullet, with explicit "back-patched in
Phase 4 Plan 04-06" citation). Historical commit-description
lines 40, 89, 109, 110 are LEFT unchanged — they describe what
the Phase-1-closure commits literally did at the time, not
forward-looking deferrals.
- .planning/phases/04-harden-clean-up-optional/deferred-items.md
(correction, BLOCKER 2 resolution):
Corrected the misdiagnosed entry from commit 6a989e8. The prior
entry named tests/build/strict-meta-json-validation.test.ts as
failing on a clean tree — that diagnosis was WRONG (the test is
8/8 GREEN in isolation). The real root cause is the pre-existing
04-CONTEXT #9 + #10 parallel-vitest / ffprobe-timeout flake
family (lands non-deterministically on whichever ffprobe / race
test loses the worker race; observed instance this session was
tests/background/webm-remux.test.ts > ffprobe -count_frames,
which is also 5/5 GREEN in isolation). True clean baseline is
184/184 GREEN; 188/188 after Plan 04-06's +4 new tests.
Gates run:
- npx tsc --noEmit exit 0.
- npm run build:test exit 0; dist-test/assets/welcome-CMygHJ_J.js
carries the raw SVG source.
- HEADLESS=1 SKIP_PROD_REBUILD=0 SKIP_LONG_UAT=1 npm run test:uat:
36/36 UAT assertions GREEN (was 35/35; +A35). A17.8 PASS:
currentColorStroke=true, canonicalViewBox=true. A35 live-DOM
probe: svgPresent=true strokeAttr=currentColor
computedStroke="rgb(250, 247, 241)" (linen-50, the
--mks-fg-inverse value flowing through the cascade — the
currentColor strategy WORKS in real Chrome) imgPresent=false.
- All Task 3 acceptance greps PASS: driveA35 count in
harness-page-driver.ts=5, in harness.test.ts=6; name:'A35'=1;
getComputedStyle=6; stroke="currentColor" in
extension-page-harness.ts=4; data:image/svg+xml=0 (grep target
and comment refs both removed).
References:
- 04-06-PLAN.md iter-2 BLOCKER 1 + BLOCKER 2 resolutions.
- .planning/phases/04-harden-clean-up-optional/04-UI-SPEC.md
§"Implementation amendment" (Option A currentColor + inline-SVG).
627 lines
29 KiB
TypeScript
627 lines
29 KiB
TypeScript
// tests/uat/harness.test.ts — Plan 01-13 orchestrator (Wave 3A → Task 9).
|
|
//
|
|
// Top-level entry for the production UAT harness. Drives all 15
|
|
// assertions sequentially against a SINGLE launched Chrome instance with
|
|
// a SINGLE harness page; bails on the first failure with a structured
|
|
// diagnostic dump. Exits 0 only when 15/15 GREEN.
|
|
//
|
|
// Wave 3A scope — wires A0+A1+A2+A3+A4+A6 (A6 via the proven Wave-2
|
|
// driver). A5+A7..A13 threw `NOT YET IMPLEMENTED — Wave 3<X> wires this`
|
|
// from `tests/uat/lib/harness-page-driver.ts`; the bail-on-first-failure
|
|
// loop stopped at the first such throw.
|
|
//
|
|
// Wave 3B wires A5 (SAVE_ARCHIVE → zip on disk) + A7 (genuine
|
|
// RECORDING_ERROR → ERR + recovery notification). Wave 3C wires A8
|
|
// (Bug A canonical onStartup-notification regression rewind) + A9 (icon
|
|
// file sizes meet imageUtil floors) + A10 (manifest shape contract).
|
|
// Wave 3D wires A11+A12+A13 for 14/14 GREEN.
|
|
//
|
|
// Plan 01-13 Task 9 closure (debug 01-09-save-stops-recording) adds A14:
|
|
// post-SAVE auto-stop state check (badge='', popup='', no new
|
|
// mokosh-recovery-*). Chains off A13's SAVE_ARCHIVE — read-only
|
|
// observation, no new dispatch.
|
|
//
|
|
// Plan 01-14 adds A23 as the final functional assertion (post-A14 chain):
|
|
// read-only inspection of the last `getDisplayMedia` constraints from
|
|
// A2's setupFreshRecording; verifies the production call site passes
|
|
// `monitorTypeSurfaces: 'include'` (W3C Screen Capture spec §6.1; Chrome
|
|
// ≥ 119 picker-narrowing semantics — removes the Window + Chrome-Tab
|
|
// panes from the operator's picker dialog). A23 has no side effects
|
|
// (the constraints cell is populated by A2 and read by the bridge op);
|
|
// hence independent of A14's no-side-effects post-SAVE contract.
|
|
// Final target: 16/16 GREEN.
|
|
//
|
|
// The orchestrator structure is final from Wave 3A onward; future waves
|
|
// only fill in the assertion-driver stubs.
|
|
//
|
|
// Architectural commitments (per 01-11-SUMMARY.md, DO NOT REGRESS):
|
|
// - Single browser, single recording per run (state machine: idle →
|
|
// A1 reads idle → A2 transitions to REC → A3+A4 read REC →
|
|
// A5 saves archive → A6 simulates user-stop → A7 surfaces ERR → ...).
|
|
// - A0 (Tier-1 grep gate) runs PRE-FLIGHT before any Chrome launch.
|
|
// Mirrors `tests/background/no-test-hooks-in-prod-bundle.test.ts`
|
|
// FORBIDDEN_HOOK_STRINGS inventory. Belt-and-suspenders: the unit
|
|
// test gate runs in `npm test` (~15s); the UAT-level A0 runs in
|
|
// `npm run test:uat` (~60-90s). Same invariant; two independent
|
|
// verification paths.
|
|
// - Drive Chrome FROM INSIDE: each assertion is a single
|
|
// `page.evaluate(() => window.__mokoshHarness.assertXX())` call;
|
|
// no SW.evaluate, no popup-bridge (both falsified per 01-11-SUMMARY).
|
|
//
|
|
// References:
|
|
// - puppeteer.launch + extension loading:
|
|
// https://pptr.dev/api/puppeteer.launchoptions
|
|
// - Node fs.readdirSync recursive walk:
|
|
// https://nodejs.org/api/fs.html#fsreaddirsyncpath-options
|
|
// - Node child_process.execFileSync:
|
|
// https://nodejs.org/api/child_process.html#child_processexecfilesyncfile-args-options
|
|
|
|
import { execFileSync } from 'node:child_process';
|
|
import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs';
|
|
import { dirname, resolve as resolvePath } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
import { launchHarnessBrowser } from './lib/launch';
|
|
import {
|
|
driveA1,
|
|
driveA2,
|
|
driveA3,
|
|
driveA4,
|
|
driveA5,
|
|
driveA6,
|
|
driveA7,
|
|
driveA8,
|
|
driveA9,
|
|
driveA10,
|
|
driveA11,
|
|
driveA12,
|
|
driveA13,
|
|
driveA14,
|
|
// Plan 01-10 Wave 3 — onboarding + design-swap-readiness
|
|
driveA15,
|
|
driveA16,
|
|
driveA17,
|
|
// Plan 01-12 Wave 6 — design integration assertions
|
|
driveA18,
|
|
driveA19,
|
|
driveA20,
|
|
driveA21,
|
|
driveA22,
|
|
// Plan 01-14 — picker-narrowing constraint
|
|
driveA23,
|
|
// Plan 02-04 Task 1 — D-P2-01 empirical Blob URL verification
|
|
driveA24,
|
|
// Plan 02-04 Task 2 — REQ-archive-export-latency (5s ceiling)
|
|
driveA25,
|
|
// Plan 02-04 Task 3 — meta.json 8-field + multi-tab strict + REQ-archive-layout
|
|
driveA26,
|
|
driveA27,
|
|
driveA28,
|
|
// Plan 03-01 — rrweb DOM verification (SPEC §10 #4 / REQ-rrweb-dom-buffer)
|
|
driveA29,
|
|
// Plan 03-02 — event-log verification (SPEC §10 #5 / REQ-user-event-log)
|
|
driveA30,
|
|
// Plan 03-03 — password-filter PARTIAL (SPEC §10 #8 PARTIAL per D-P3-02)
|
|
driveA31,
|
|
// Plan 03-04 — RAM scaffolding best-effort (SPEC §10 #9 per D-P3-04)
|
|
driveA32,
|
|
// Plan 04-08 — driveA33 SW state persistence (ROADMAP SC #1; methodology
|
|
// reframe per debug session-2 verdict; needs Browser + extensionId for
|
|
// CDP-based SW kill + downloadsDir for host-side JSZip parse).
|
|
driveA33,
|
|
// Plan 04-05 — driveA34 fetch + XHR network_error empirical (ROADMAP SC #2;
|
|
// needs downloadsDir for host-side JSZip parse of logs/events.json).
|
|
driveA34,
|
|
// Plan 04-06 — driveA35 UI-SPEC dark-logo `currentColor` LIVE-DOM proof.
|
|
// Opens welcome.html in a fresh browser.newPage() tab so populateMark()
|
|
// actually runs; reads getComputedStyle().stroke on the injected <svg>
|
|
// to verify the currentColor cascade. Host-side driver — needs Browser +
|
|
// extensionId (mirrors driveA33's Browser+extensionId capture pattern).
|
|
driveA35,
|
|
getManifestVersion,
|
|
} from './lib/harness-page-driver';
|
|
import {
|
|
printAssertionResult,
|
|
runAssertion,
|
|
type AssertionRecord,
|
|
} from './lib/assertions';
|
|
|
|
/**
|
|
* A0 forbidden-string inventory — mirrors
|
|
* `tests/background/no-test-hooks-in-prod-bundle.test.ts:FORBIDDEN_HOOK_STRINGS`.
|
|
* Keep in sync. The two lists serving the same invariant is intentional
|
|
* (belt-and-suspenders per `feedback-pre-checkpoint-bundle-gates.md`):
|
|
* unit-test gate catches at `npm test`, UAT gate catches at `npm run test:uat`.
|
|
*/
|
|
const FORBIDDEN_HOOK_STRINGS: ReadonlyArray<string> = [
|
|
'__mokoshTest',
|
|
'setCurrentStream',
|
|
'setSegmentCountGetter',
|
|
'installFakeDisplayMedia',
|
|
'uninstallFakeDisplayMedia',
|
|
'dispatchEndedOnTrack',
|
|
'getSegmentCount',
|
|
'__mokoshOffscreenQuery',
|
|
'get-display-surface',
|
|
'get-segment-count',
|
|
// Plan 01-14 A23 surface — lockstep with unit-gate inventory at
|
|
// tests/background/no-test-hooks-in-prod-bundle.test.ts:105.
|
|
'lastGetDisplayMediaConstraints',
|
|
'get-last-getDisplayMedia-constraints',
|
|
];
|
|
|
|
/** Build timeout for the pre-flight production rebuild (matches unit-gate value). */
|
|
const PROD_BUILD_TIMEOUT_MS = 60_000;
|
|
|
|
/** Resolve repo-root paths from this file's location. */
|
|
const HARNESS_FILE_DIR = dirname(fileURLToPath(import.meta.url));
|
|
const REPO_ROOT = resolvePath(HARNESS_FILE_DIR, '..', '..');
|
|
const DIST_DIR = resolvePath(REPO_ROOT, 'dist');
|
|
|
|
/** Binary extensions skipped during the grep walk (mirror of unit gate). */
|
|
const BINARY_EXTENSIONS: ReadonlySet<string> = new Set([
|
|
'.png', '.jpg', '.jpeg', '.gif', '.ico', '.webp', '.woff', '.woff2', '.ttf', '.otf',
|
|
]);
|
|
|
|
/**
|
|
* Recursively collect every regular file under `root`. Returns absolute
|
|
* paths sorted alphabetically for stable diagnostics.
|
|
*
|
|
* @param root - Absolute directory path to walk.
|
|
* @returns Sorted list of absolute file paths under `root`.
|
|
*/
|
|
function listAllFilesRecursive(root: string): ReadonlyArray<string> {
|
|
const accumulator: string[] = [];
|
|
const stack: string[] = [root];
|
|
while (stack.length > 0) {
|
|
const dir = stack.pop()!;
|
|
const entries = readdirSync(dir, { withFileTypes: true });
|
|
for (const entry of entries) {
|
|
const fullPath = resolvePath(dir, entry.name);
|
|
if (entry.isSymbolicLink()) {
|
|
continue;
|
|
}
|
|
if (entry.isDirectory()) {
|
|
stack.push(fullPath);
|
|
} else if (entry.isFile()) {
|
|
accumulator.push(fullPath);
|
|
}
|
|
}
|
|
}
|
|
return accumulator.sort();
|
|
}
|
|
|
|
/**
|
|
* Count occurrences of `needle` in the given file. Returns 0 for binary
|
|
* file extensions (text matching against UTF-8 of a PNG would be
|
|
* meaningless and could yield spurious matches).
|
|
*
|
|
* @param filePath - Absolute file path to scan.
|
|
* @param needle - Literal substring to count.
|
|
* @returns Total occurrences in the file's text.
|
|
*/
|
|
function countOccurrencesInFile(filePath: string, needle: string): number {
|
|
const dotIdx = filePath.lastIndexOf('.');
|
|
const ext = dotIdx >= 0 ? filePath.substring(dotIdx).toLowerCase() : '';
|
|
if (BINARY_EXTENSIONS.has(ext)) {
|
|
return 0;
|
|
}
|
|
const stat = statSync(filePath);
|
|
if (stat.size === 0) {
|
|
return 0;
|
|
}
|
|
const text = readFileSync(filePath, 'utf8');
|
|
let count = 0;
|
|
let from = 0;
|
|
for (;;) {
|
|
const idx = text.indexOf(needle, from);
|
|
if (idx < 0) {
|
|
break;
|
|
}
|
|
count += 1;
|
|
from = idx + needle.length;
|
|
}
|
|
return count;
|
|
}
|
|
|
|
/**
|
|
* A0 — Tier-1 grep gate (UAT-level mirror of the unit-gate). Spawns
|
|
* `npm run build` if `SKIP_PROD_REBUILD !== '1'`, then walks `dist/`
|
|
* checking every forbidden string. Reports all matches in one pass
|
|
* (full enumeration, not bail-on-first) so the operator sees the entire
|
|
* leak surface in a single failure.
|
|
*
|
|
* @returns Structured A0 result: passed flag + list of (string, file) matches.
|
|
*/
|
|
async function assertA0_GrepGate(): Promise<{
|
|
passed: boolean;
|
|
matches: Array<{ needle: string; filePath: string; count: number }>;
|
|
}> {
|
|
if (process.env.SKIP_PROD_REBUILD !== '1') {
|
|
process.stdout.write('A0: running `npm run build` (set SKIP_PROD_REBUILD=1 to skip)...\n');
|
|
execFileSync('npm', ['run', 'build'], {
|
|
stdio: 'inherit',
|
|
timeout: PROD_BUILD_TIMEOUT_MS,
|
|
});
|
|
} else {
|
|
process.stdout.write('A0: SKIP_PROD_REBUILD=1 — using existing dist/\n');
|
|
}
|
|
|
|
if (!existsSync(DIST_DIR)) {
|
|
return {
|
|
passed: false,
|
|
matches: [
|
|
{
|
|
needle: '<missing dist/>',
|
|
filePath: DIST_DIR,
|
|
count: 0,
|
|
},
|
|
],
|
|
};
|
|
}
|
|
|
|
const files = listAllFilesRecursive(DIST_DIR);
|
|
const matches: Array<{ needle: string; filePath: string; count: number }> = [];
|
|
for (const needle of FORBIDDEN_HOOK_STRINGS) {
|
|
for (const filePath of files) {
|
|
const count = countOccurrencesInFile(filePath, needle);
|
|
if (count > 0) {
|
|
matches.push({ needle, filePath, count });
|
|
}
|
|
}
|
|
}
|
|
return { passed: matches.length === 0, matches };
|
|
}
|
|
|
|
/**
|
|
* Top-to-bottom orchestrator entry. Pre-flight A0 → launch browser →
|
|
* iterate driver list → bail on first failure → close browser → return
|
|
* exit code.
|
|
*
|
|
* Plan 01-13 Task 9 closure (debug 01-09-save-stops-recording) added A14
|
|
* after A13. The orchestrator now drives 14 page-side assertions
|
|
* (A1..A14) plus the host-side A0 grep gate = 15 total.
|
|
*
|
|
* @returns Process exit code: 0 on 15/15 GREEN, 1 on any failure.
|
|
*/
|
|
async function main(): Promise<number> {
|
|
process.stdout.write('\nMokosh Plan 01-13 + 01-14 + 02-04 — UAT harness orchestrator\n');
|
|
process.stdout.write('Architecture: A0 pre-flight + extension-internal page driver (A1..A14, A15..A17, A18..A22, A23, A24, A25, A26, A27, A28, A29, A30, A31, A32, A33, A34, A35)\n');
|
|
process.stdout.write('='.repeat(72) + '\n');
|
|
|
|
// A0 pre-flight (no Chrome launch needed; runs against built dist/).
|
|
const a0 = await assertA0_GrepGate();
|
|
if (!a0.passed) {
|
|
process.stderr.write('\nA0 FAIL: production bundle hook-string leak detected.\n');
|
|
for (const m of a0.matches) {
|
|
process.stderr.write(` - '${m.needle}' in ${m.filePath} (${m.count} occurrence${m.count === 1 ? '' : 's'})\n`);
|
|
}
|
|
process.stderr.write(
|
|
'\nThe Vite mode gate on the test-hook imports has regressed; verify\n' +
|
|
'src/background/index.ts + src/offscreen/recorder.ts still gate via `__MOKOSH_UAT__`.\n',
|
|
);
|
|
return 1;
|
|
}
|
|
process.stdout.write('A0: GREEN (production bundle hook-free)\n\n');
|
|
|
|
// Driver registry — execution order matters:
|
|
// A1 (idle) → A2 (REC start) → A3 (displaySurface) → A4 (popup pinned)
|
|
// → A5 (SAVE_ARCHIVE) → A6 (Bug B dispatch-ended) → A7 (genuine error)
|
|
// → A8 (Bug A onStartup) → A9 (icon sizes) → A10 (manifest)
|
|
// → A11 (35s segments) → A12 (ffprobe) → A13 (zip shape).
|
|
//
|
|
// A6 currently lives mid-list because the prototype's assertA6 does
|
|
// its own ensureOffscreen + START_RECORDING (idempotent w.r.t. A2's
|
|
// recording), then dispatch-ended. After A6 the recording is torn
|
|
// down — A7+ would need to re-start or test post-stop state.
|
|
//
|
|
// Wave 3C wires A8 + A9 + A10 in addition to A1..A7 — bail-on-first-
|
|
// failure stops at A11 (Wave 3D wires that). Expected diagnostic:
|
|
// "11/14 GREEN: A0+A1+A2+A3+A4+A5+A6+A7+A8+A9+A10; A11..A13 NOT YET IMPLEMENTED".
|
|
// The standalone `npx tsx tests/uat/a6.test.ts` entry remains the
|
|
// way to verify A6 in isolation for inner-loop iteration.
|
|
process.stdout.write('Launching Chrome + opening harness page...\n');
|
|
const handles = await launchHarnessBrowser();
|
|
process.stdout.write(`Extension id: ${handles.extensionId}\n`);
|
|
process.stdout.write(`Downloads dir: ${handles.downloadsDir}\n\n`);
|
|
|
|
// Adapter: driveA5 / driveA12 / driveA13 need `handles.downloadsDir`
|
|
// (host-side fs polling). driveA13 additionally needs the manifest
|
|
// version (read once at orchestrator startup via the page-side
|
|
// `getManifestVersion` helper). All other drivers take only `page`.
|
|
// The driver list is constructed AFTER `launchHarnessBrowser` returns
|
|
// so the closures can capture handles without a TDZ trap.
|
|
const expectedManifestVersion = await getManifestVersion(handles.harnessPage);
|
|
process.stdout.write(`Manifest version (for A13): ${expectedManifestVersion}\n\n`);
|
|
|
|
const driveA5Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA5(page, handles.downloadsDir);
|
|
const driveA12Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA12(page, handles.downloadsDir);
|
|
const driveA13Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA13(page, handles.downloadsDir, expectedManifestVersion);
|
|
// Plan 02-04 Task 2 — driveA25 needs downloadsDir for the host-side
|
|
// dispatch→file-on-disk latency check (mirrors A5/A12/A13 wrapping).
|
|
const driveA25Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA25(page, handles.downloadsDir);
|
|
// Plan 02-04 Task 3 — driveA26/A27/A28 need downloadsDir for host-side
|
|
// zip inspection (JSZip-parse meta.json + zip-layout enumeration). A26
|
|
// chains off A25's zip (no new SAVE); A27 owns its SAVE (multi-tab);
|
|
// A28 chains off A27's zip (no new SAVE).
|
|
const driveA26Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA26(page, handles.downloadsDir);
|
|
const driveA27Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA27(page, handles.downloadsDir);
|
|
const driveA28Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA28(page, handles.downloadsDir);
|
|
// Plan 03-01 — driveA29 needs downloadsDir for host-side JSZip parse
|
|
// of rrweb/session.json from the just-produced zip.
|
|
const driveA29Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA29(page, handles.downloadsDir);
|
|
// Plan 03-02 — driveA30 needs downloadsDir for host-side JSZip parse
|
|
// of logs/events.json from the just-produced zip.
|
|
const driveA30Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA30(page, handles.downloadsDir);
|
|
// Plan 03-03 — driveA31 needs downloadsDir for host-side JSZip
|
|
// negative-assertion against logs/events.json (sentinel absence +
|
|
// password-selector-target absence) + control-sentinel presence
|
|
// (defense-in-depth A31.4).
|
|
const driveA31Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA31(page, handles.downloadsDir);
|
|
// Plan 04-08 — driveA33 needs Browser + extensionId for CDP-based SW kill
|
|
// AND downloadsDir for host-side JSZip parse of post-restart zip.
|
|
const driveA33Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA33(page, handles.browser, handles.extensionId, handles.downloadsDir);
|
|
// Plan 04-05 — driveA34 needs downloadsDir for host-side JSZip parse of
|
|
// logs/events.json (fetch + XHR network_error entry inspection).
|
|
const driveA34Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA34(page, handles.downloadsDir);
|
|
// Plan 04-06 — driveA35 needs Browser + extensionId to open a fresh
|
|
// welcome.html tab via browser.newPage() (UI-SPEC dark-logo LIVE-DOM
|
|
// proof). Mirrors the driveA33 Browser+extensionId capture pattern.
|
|
const driveA35Wrapped: (page: import('puppeteer').Page) => Promise<AssertionRecord> =
|
|
(page) => driveA35(page, handles.browser, handles.extensionId);
|
|
|
|
const drivers: ReadonlyArray<{
|
|
readonly name: string;
|
|
readonly drive: (page: import('puppeteer').Page) => Promise<AssertionRecord>;
|
|
}> = [
|
|
{ name: 'A1', drive: driveA1 },
|
|
{ name: 'A2', drive: driveA2 },
|
|
{ name: 'A3', drive: driveA3 },
|
|
{ name: 'A4', drive: driveA4 },
|
|
{ name: 'A5', drive: driveA5Wrapped },
|
|
{ name: 'A6', drive: driveA6 },
|
|
{ name: 'A7', drive: driveA7 },
|
|
{ name: 'A8', drive: driveA8 },
|
|
{ name: 'A9', drive: driveA9 },
|
|
{ name: 'A10', drive: driveA10 },
|
|
{ name: 'A11', drive: driveA11 },
|
|
{ name: 'A12', drive: driveA12Wrapped },
|
|
{ name: 'A13', drive: driveA13Wrapped },
|
|
// Plan 01-13 Task 9 closure (debug 01-09-save-stops-recording): A14
|
|
// verifies that A13's SAVE_ARCHIVE auto-stopped the recording per
|
|
// SPEC one-shot intent. Read-only assertion on chrome.action +
|
|
// notification ids state; no new SAVE dispatch — A13's already
|
|
// exercised the SAVE path. Recording stays stopped after A14.
|
|
{ name: 'A14', drive: driveA14 },
|
|
// Plan 01-10 Wave 3 — onboarding + design-swap-readiness (read-only;
|
|
// chained AFTER A14 + before A18 so A15/A16/A17 inspect the
|
|
// welcome-page artifacts that A22's skip-gate test (Plan 01-12 Wave 6)
|
|
// previously fell through. With Plan 01-10 landed, A22 no longer
|
|
// skip-gates — it's a substantive token-usage check.
|
|
// A15 — chrome.storage.local 'onboarding-completed' + 'installed-at'
|
|
// A16 — 2s settle: no new welcome tabs spontaneously reappear
|
|
// A17 — welcome.html parse + .welcome-hero + ≥7 mokosh-keyed +
|
|
// welcome.css canonical @import or inlined tokens + zero hex
|
|
// (or canonical resolved) + ≥5 var(--mks-*) + bundled JS
|
|
// has COPY[ or chrome.i18n.getMessage(welcomeHero +
|
|
// getComputedStyle --mks-rec probe resolves
|
|
{ name: 'A15', drive: driveA15 },
|
|
{ name: 'A16', drive: driveA16 },
|
|
{ name: 'A17', drive: driveA17 },
|
|
// Plan 01-12 Wave 6 — design integration assertions (read-only;
|
|
// independent of A14). Chained here so they execute regardless of
|
|
// the recording state machine; they only inspect static brand /
|
|
// i18n / token / icon surfaces.
|
|
// A18 — Lora WOFF2 reachability + size floor
|
|
// A19 — icons NOT the Bug A placeholders
|
|
// A20 — manifest:name resolves via chrome i18n
|
|
// A21 — --mks-font-display resolves to Lora
|
|
// A22 — welcome page tokens.css adoption (CONDITIONAL on 01-10
|
|
// landing; with Plan 01-10 landed it executes the
|
|
// substantive token-usage path rather than skip-gating)
|
|
{ name: 'A18', drive: driveA18 },
|
|
{ name: 'A19', drive: driveA19 },
|
|
{ name: 'A20', drive: driveA20 },
|
|
{ name: 'A21', drive: driveA21 },
|
|
{ name: 'A22', drive: driveA22 },
|
|
// Plan 01-14 A23: read-only inspection of the last getDisplayMedia
|
|
// constraints object captured by A2's setupFreshRecording. Verifies
|
|
// the production call at src/offscreen/recorder.ts:270 passes
|
|
// `monitorTypeSurfaces: 'include'` (W3C Screen Capture spec §6.1;
|
|
// Chrome ≥ 119 picker-narrowing semantics). Independent of A14 —
|
|
// no new getDisplayMedia call, no new state change.
|
|
{ name: 'A23', drive: driveA23 },
|
|
// Plan 02-04 Task 1 A24: D-P2-01 empirical Blob URL verification.
|
|
// Installs chrome.downloads.onCreated listener cross-realm, dispatches
|
|
// SAVE_ARCHIVE, captures the download URL, asserts the `blob:` prefix
|
|
// (closes audit P0-6 end-to-end through a real Chrome instance +
|
|
// the offscreen mint round-trip + chrome.downloads platform call).
|
|
// A24 does its OWN setupFreshRecording + SAVE because the listener
|
|
// must be installed pre-dispatch. After A24 the recording stays alive
|
|
// for any chained Plan 02-04 Tasks 2-3 assertions (Phase 2 closure).
|
|
{ name: 'A24', drive: driveA24 },
|
|
// Plan 02-04 Task 2 A25: REQ-archive-export-latency / SPEC §10 #6.
|
|
// Page-side measures SAVE→ack via performance.now() bookends; host-side
|
|
// adds the dispatch→file-on-disk latency check via downloadsDir
|
|
// polling + mtime delta. Hard ceiling: 5000ms end-to-end. A25 owns
|
|
// its setupFreshRecording (clean latency measurement; not compounded
|
|
// with A24's still-pending state). The 11s segment-settle is NOT
|
|
// counted toward the 5s budget — only the SAVE dispatch.
|
|
{ name: 'A25', drive: driveA25Wrapped },
|
|
// Plan 02-04 Task 3 A26: D-P2-02 + D-P2-03 meta.json 8-field shape.
|
|
// Chains off A25's zip (no new SAVE); host-side JSZip-parse meta.json
|
|
// and asserts the 8-field shape with urls[] + schemaVersion='2'.
|
|
{ name: 'A26', drive: driveA26Wrapped },
|
|
// Plan 02-04 Task 3 A27: STRICT multi-tab urls[] post DEC-011 Amendment 1.
|
|
// Opens 2 tabs sequentially + activates each + 11s settle + SAVE; host-side
|
|
// asserts meta.urls contains BOTH example.com + iana.org (length>=2
|
|
// REQUIRED; FAILS on length<2; no extension-origin sentinels; no
|
|
// chrome-internal URLs). Owns its SAVE dispatch (multi-tab tracker
|
|
// state needs both onActivated events to fire BEFORE the SAVE).
|
|
{ name: 'A27', drive: driveA27Wrapped },
|
|
// Plan 02-04 Task 3 A28: REQ-archive-layout strict 5-entry zip-layout.
|
|
// Chains off A27's zip (no new SAVE); host-side enumerates zip entries
|
|
// and asserts EXACTLY 5 paths: video/last_30sec.webm, rrweb/session.json,
|
|
// logs/events.json, screenshot.png, meta.json (set-equality; no extras).
|
|
{ name: 'A28', drive: driveA28Wrapped },
|
|
// Plan 03-01 A29: rrweb DOM verification (SPEC §10 #4).
|
|
// A29 owns its SAVE because the probe-page DOM mutation must
|
|
// happen between page load and SAVE so rrweb's IncrementalSnapshot
|
|
// fires (RESEARCH Pitfall 1). Host-side driveA29 JSZip-parses
|
|
// rrweb/session.json and asserts the EventType enum surfaces
|
|
// (Meta=4, FullSnapshot=2, IncrementalSnapshot=3) are present.
|
|
{ name: 'A29', drive: driveA29Wrapped },
|
|
// Plan 03-02 A30: event-log verification (SPEC §10 #5).
|
|
// A30 owns its SAVE because event-log cleanup runs every 60s
|
|
// (src/content/index.ts CLEANUP_INTERVAL_MS=60_000) and we need a
|
|
// fresh event-log window for the 5 synthetic triggers. Host-side
|
|
// driveA30 JSZip-parses logs/events.json and asserts presence of
|
|
// each of the 5 UserEvent.type literal values.
|
|
{ name: 'A30', drive: driveA30Wrapped },
|
|
// Plan 03-03 A31: password-filter PARTIAL (SPEC §10 #8 PARTIAL per
|
|
// D-P3-02). Negative-assertion: opens a fresh https://example.com
|
|
// probe tab (Plan 03-02 cs-injection-world precedent), injects a
|
|
// synthetic <input type="password"> + a control <input type="text">
|
|
// via chrome.scripting.executeScript ISOLATED-world, types the
|
|
// sentinels, settles, SAVEs while the probe tab is active, finally-
|
|
// cleanup. Host-side driveA31 inspects logs/events.json and asserts
|
|
// sentinel value absence + password-selector-target absence (proves
|
|
// src/content/index.ts:82 filter fired) + control-sentinel presence
|
|
// (defense-in-depth: proves the listener is alive so A31.2/A31.3
|
|
// mean the filter actually fired rather than the trivial "no
|
|
// events at all" tautology).
|
|
{ name: 'A31', drive: driveA31Wrapped },
|
|
// Plan 03-04 A32: RAM scaffolding (SPEC §10 #9 best-effort per D-P3-04).
|
|
// NOTE — Page.metrics is page-realm only; SW context is a separate
|
|
// Puppeteer target (RESEARCH Pitfall 2). A32 is informational
|
|
// scaffolding; the binding §10 #9 gate lives in Plan 03-05
|
|
// VERIFICATION.md `human_verification` block. No wrapped const
|
|
// needed — driveA32 takes only `page`.
|
|
{ name: 'A32', drive: driveA32 },
|
|
// Plan 04-08 A33: SW state persistence 5-min idle (ROADMAP SC #1).
|
|
// Methodology reframe per debug session-2 — video-file MediaStream
|
|
// replaces the canvas.captureStream invisible-source throttling that
|
|
// produced 8505-byte 0-frames archives under the previous Plan 04-04
|
|
// spike methodology. Architecture (offscreen-RAM segments: Blob[]) is
|
|
// unchanged and canonically correct per debug session-2 segment-count
|
|
// probe evidence (POST-KILL count=3 confirms structural persistence).
|
|
// Forces SW eviction via Puppeteer CDP worker.close() per the canonical
|
|
// Chrome devrel pattern (stopServiceWorker helper from Plan 04-04).
|
|
// Env-gated by SKIP_LONG_UAT for fast per-commit iteration; defaults
|
|
// to RUN for Phase 4 closure + alpha gate.
|
|
{
|
|
name: 'A33',
|
|
drive: process.env.SKIP_LONG_UAT === '1'
|
|
? async (): Promise<AssertionRecord> => ({
|
|
name: 'A33',
|
|
passed: true,
|
|
checks: [],
|
|
diagnostics: ['A33 SKIPPED (SKIP_LONG_UAT=1; unset to run 5-min idle test)'],
|
|
})
|
|
: driveA33Wrapped,
|
|
},
|
|
// Plan 04-05 A34: fetch + XHR network_error empirical (ROADMAP SC #2).
|
|
// Verifies both protocol paths in src/content/index.ts setupNetworkLogging
|
|
// produce events.json entries. Empirically validates Plan 04-01 P1 #11
|
|
// fetch URL extraction fix at the SAVE->archive layer (A34.4 + A34.5).
|
|
// A34 owns its SAVE because event-log cleanup runs every 60s
|
|
// (src/content/index.ts CLEANUP_INTERVAL_MS) and the 2 synthetic
|
|
// failing requests need a fresh event-log window. Opens a fresh
|
|
// https://example.com probe tab + injects fetch(404)+XHR(404) via
|
|
// chrome.scripting.executeScript ISOLATED-world. Runs ~25s (always
|
|
// RUN — not env-gated; the 5-min wait is A33's, not A34's).
|
|
{ name: 'A34', drive: driveA34Wrapped },
|
|
// Plan 04-06 A35: UI-SPEC dark-logo `currentColor` strategy LIVE-DOM
|
|
// proof. Opens welcome.html as a real Puppeteer tab so populateMark()
|
|
// actually runs; reads getComputedStyle().stroke on the injected
|
|
// <svg> to verify the currentColor cascade resolves through
|
|
// .welcome-hero__mark color: var(--mks-fg-inverse) (UI-SPEC Option A).
|
|
// Appended LAST in the drivers array so the new welcome tab cannot
|
|
// pollute any later driver (and welcomePage.close() in finally
|
|
// guarantees no tab leak regardless). Host-side driver — mirrors
|
|
// the driveA32/A33/A34 host-side pattern (NOT a page.evaluate
|
|
// (window.__mokoshHarness) wrapper).
|
|
{ name: 'A35', drive: driveA35Wrapped },
|
|
];
|
|
|
|
const buffers = { swConsole: handles.swConsole, offConsole: handles.offConsole };
|
|
const results: Array<{ name: string; passed: boolean; error?: string }> = [];
|
|
let bailReason: string | null = null;
|
|
|
|
try {
|
|
for (const { name, drive } of drivers) {
|
|
process.stdout.write(`--- ${name} ---\n`);
|
|
let driverErr: string | undefined;
|
|
let result: AssertionRecord | null = null;
|
|
try {
|
|
result = await runAssertion(
|
|
name,
|
|
() => drive(handles.harnessPage),
|
|
buffers,
|
|
);
|
|
printAssertionResult(result);
|
|
} catch (err) {
|
|
driverErr = err instanceof Error ? err.message : String(err);
|
|
// A throw here is either: (a) a Wave-3 stub firing
|
|
// (NOT YET IMPLEMENTED) — expected during incremental waves; OR
|
|
// (b) a CDP/Puppeteer-level error (e.g. page closed, timeout) —
|
|
// a genuine harness regression. Both bail uniformly.
|
|
process.stderr.write(`*** ${name} THREW: ${driverErr}\n`);
|
|
}
|
|
const passed = result !== null && result.passed && driverErr === undefined;
|
|
results.push({ name, passed, error: driverErr });
|
|
if (!passed) {
|
|
bailReason = driverErr ?? `${name} failed; see structured checks above`;
|
|
break;
|
|
}
|
|
}
|
|
} finally {
|
|
try {
|
|
await handles.browser.close();
|
|
} catch (closeErr) {
|
|
process.stderr.write(`(non-fatal: browser close threw: ${String(closeErr)})\n`);
|
|
}
|
|
}
|
|
|
|
const passedCount = results.filter((r) => r.passed).length;
|
|
// Total = 1 (A0) + drivers.length (A1..A14, A23) = 16. Plan 01-14
|
|
// appended A23 after A14 — the running count adapts via `drivers.length`
|
|
// so no manual update is needed when future plans extend the chain.
|
|
const total = drivers.length + 1;
|
|
const finalPassed = passedCount + 1; // +1 for A0 (we already passed it to reach here)
|
|
|
|
process.stdout.write('\n' + '='.repeat(72) + '\n');
|
|
process.stdout.write(
|
|
`UAT harness: ${finalPassed}/${total} assertions passed${bailReason !== null ? ` (bailed: ${bailReason})` : ''}\n`,
|
|
);
|
|
for (const r of results) {
|
|
const mark = r.passed ? '[PASS]' : '[FAIL]';
|
|
const tail = r.error !== undefined ? ` — ${r.error}` : '';
|
|
process.stdout.write(` ${mark} ${r.name}${tail}\n`);
|
|
}
|
|
if (bailReason !== null) {
|
|
const remainingStart = results.length;
|
|
for (let i = remainingStart; i < drivers.length; i += 1) {
|
|
process.stdout.write(` [SKIP] ${drivers[i].name} (not reached — bailed at ${results[results.length - 1].name})\n`);
|
|
}
|
|
}
|
|
process.stdout.write('='.repeat(72) + '\n');
|
|
|
|
return finalPassed === total ? 0 : 1;
|
|
}
|
|
|
|
const code = await main();
|
|
process.exit(code);
|