repremium/mokosh
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore(01-13): wave-0 — clean broken Approach-A artifacts per 01-11-SUMMARY

Browse Source 
Restore a clean baseline before promoting the c647f61 prototype to
production paths (Wave 1) and building out Approach-B driver
scaffolding (Wave 2). All deletions trace back to falsifications
documented in 01-11-SUMMARY.md.

Deleted — broken Approach-A files:
  - src/test-hooks/sw-hooks.ts
      MV3 SW blocks dynamic import (Chromium es_modules.md;
      w3c/webextensions#212). The gated `await import('../test-hooks/
      sw-hooks')` from 01-11 Wave 1 never resolved → SW silently died →
      production listeners never registered. File was dead-on-arrival;
      no fix possible while MV3 SWs disallow dynamic import. Approach-B
      replaces SW-side instrumentation with the extension-internal
      harness page's chrome.action.* + chrome.notifications.* surface
      (full privilege; no monkey-patching needed).

  - tests/uat/lib/{launch,extension,sw,offscreen,assertions}.ts
      Popup-bridge architecture (01-11 dbd977c) — falsification 2 +
      falsification 3 in 01-11-SUMMARY: `sw.evaluate` exposes only
      chrome.{loadTimes,csi}, NOT chrome.action.* / chrome.notifications.*
      / chrome.runtime.sendMessage; setPopup-juggling for extension-id
      resolution turned out to be unnecessary (browser.extensions()
      works directly per the prototype). These files will be reborn in
      Wave 2 around the extension-page architecture.

      Kept: tests/uat/lib/zip.ts (host-side JSZip work — architecture-
      agnostic; A12+A13 still use it) and tests/uat/lib/test-hook-
      contract.d.ts (type mirror — extended in Wave 3 but kept as-is here).

  - tests/uat/prototype/probe_{offscreen,sw,tabs,tabs2}.mjs
      Feasibility-research probes (01-11 spike) that empirically falsified
      the Approach-A hypotheses. The findings are encoded in 01-11-
      SUMMARY.md; the probes themselves are dead code.

  - tests/uat/harness.test.ts
      01-11 Wave 2 popup-bridge orchestrator (dbd977c). Imports the
      now-deleted tests/uat/lib/{assertions,extension,sw,offscreen,launch}
      modules — would not typecheck after this commit. Reborn in
      Wave 3A as the Approach-B orchestrator (extension-internal page
      driver + A0 grep gate + 13 assertion drivers).

Reverted — SW-side dynamic-import gate comment block:
  - src/background/index.ts lines 13-29
      The existing comment block (post-spike) described the SW-side
      gated dynamic import that never landed. Rewritten to cite 01-13
      Approach-B explicitly, link to 01-11-SUMMARY.md falsification,
      and clarify that the Tier-1 grep gate's enduring value is
      catching regressions in the offscreen chunk's __MOKOSH_UAT__
      gate (the SW chunk is hook-free by construction).

Updated — Tier-1 grep gate FORBIDDEN_HOOK_STRINGS inventory:
  - tests/background/no-test-hooks-in-prod-bundle.test.ts
      Removed: `simulateUserStop` (Approach-A naming; replaced by
      Approach-B `dispatchEndedOnTrack` which matches the W3C
      dispatchEvent semantics per RESEARCH §7 BLOCKER — track.stop()
      does NOT fire 'ended' per spec, so the simulation MUST use
      dispatchEvent).
      Added: `installFakeDisplayMedia`, `uninstallFakeDisplayMedia`,
      `dispatchEndedOnTrack`, `__mokoshOffscreenQuery`.
      Total inventory: 8 surface strings (was 5). Each MUST be absent
      from every file under dist/ post-build.

Verification (all GREEN):
  - `npm run build` — exit 0; dist/ populated.
  - `grep -rln <forbidden> dist/` — 0 matches.
  - `npm run build:test` — exit 0; dist-test/ populated; offscreen-hooks
    chunk contains `installFakeDisplayMedia` (gate runs correctly
    against the test build's distinct artifact).
  - `npx tsc --noEmit` — exit 0 (root + tests/uat/tsconfig.json).
  - `npx vitest run` — 92/92 tests passing (was 89; the +3 new tests
    come from the FORBIDDEN_HOOK_STRINGS list expanding 5 → 8 — each
    forbidden string is one parametric `it(...)` block).

Both prior-failing tests now GREEN:
  - tests/background/sw-bundle-import.test.ts (was missing dist/ → 92/92
    requires the test run to have a current dist/; vitest gate test
    rebuilds via execFile when SKIP_BUILD≠1, otherwise relies on prior
    `npm run build`).
  - tests/background/no-test-hooks-in-prod-bundle.test.ts (was failing
    on stale dist; now GREEN against the freshly-rebuilt clean bundle).

Wave 1 (next): promote tests/uat/prototype/{extension-page-harness.html,
extension-page-harness.ts,a6.test.ts} to tests/uat/ via `git mv`;
update vite.test.config.ts rollup input.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mark
2026-05-18 14:54:41 +02:00
parent 70f4f4136a
commit a63066a289
13 changed files with 57 additions and 1991 deletions
Download Patch File Download Diff File Expand all files Collapse all files

499
tests/uat/harness.test.ts
View File

@@ -1,499 +0,0 @@
// tests/uat/harness.test.ts — Plan 01-11 Puppeteer UAT harness entry point.
//
// Runs end-to-end via `npm run test:uat` (build:test + tsx tests/uat/harness.test.ts).
// Top-to-bottom narrative: launch Chrome with dist-test loaded as
// MV3 extension, attach to SW + offscreen, run 14 assertions
// sequentially with bail-on-first-fail semantics + structured
// diagnostic dump on failure (RESEARCH §5 + open-question resolution 4).
//
// Exit code:
// 0 — all 14 assertions passed
// 1 — at least one assertion failed
//
// Local-debug mode: `HEADLESS=0 npm run test:uat` (opens real Chrome)
// Skip prod rebuild: `SKIP_PROD_REBUILD=1` (assertion 0 still verifies
// the EXISTING dist/ rather than spawning npm run build).
//
// Assertion catalog (14 total):
// 0 — Production bundle grep gate (filesystem-only; pre-flight).
// 1 — SW bootstrap → setIdleMode (badge '', popup '', isRecording=false).
// 2 — Toolbar onClicked-idle → badge 'REC' + popup popup.html + isRecording=true.
// 3 — Offscreen displaySurface === 'monitor' (post-grant validation).
// 4 — Toolbar onClicked while recording → popup, NO new offscreen.
// 5 — SAVE_ARCHIVE → download fires + session_report_*.zip appears.
// 6 — BUG B (canonical): simulateUserStop → badge '' + popup '' + NO recovery notif.
// 7 — RECORDING_ERROR codec-unsupported → badge 'ERR' + recovery notif.
// 8 — BUG A (canonical): onStartup → mokosh-startup- notification creates cleanly.
// 9 — Icon file sizes meet floors (16→200, 48→500, 128→1024).
// 10 — Manifest has notifications permission + all three icons declared.
// 11 — 35s recording yields >= 3 segments per D-13.
// 12 — ffprobe -v error -f matroska on extracted webm exits 0.
// 13 — Archive shape (video/last_30sec.webm + meta.json with version match).
import { execFileSync, execSync } from 'node:child_process';
import { existsSync, readdirSync, readFileSync, statSync, mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join, resolve as resolvePath } from 'node:path';
import { fileURLToPath } from 'node:url';
import type { Page } from 'puppeteer';
import {
type AssertionRecord,
type ConsoleBuffers,
assertEqual,
assertGte,
assertMatch,
assertTrue,
runAssertion,
waitFor,
} from './lib/assertions';
import {
attachToOffscreen,
countOffscreenTargets,
waitForOffscreenTarget,
} from './lib/extension';
import {
getDisplaySurface,
getSegmentCount,
simulateUserStop,
} from './lib/offscreen';
import {
fireOnStartup,
getBadgeText,
getIconSize,
getIsRecording,
getManifest,
getNotificationSnapshot,
getPopup,
keepalivePing,
sendSyntheticRecordingError,
} from './lib/sw';
import { assertArchiveShape, extractEntryToFile } from './lib/zip';
import { launchHarnessBrowser, type HarnessHandles } from './lib/launch';
const HARNESS_FILE_DIR = dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = resolvePath(HARNESS_FILE_DIR, '..', '..');
const DIST_DIR = resolvePath(REPO_ROOT, 'dist');
const FFPROBE_BIN = '/usr/bin/ffprobe';
const TOTAL_ASSERTIONS = 14;
/**
* Forbidden hook surface strings — assertion 0 verifies absence
* in production dist/. Mirrors the Tier-1 unit gate's surface list
* (tests/background/no-test-hooks-in-prod-bundle.test.ts) but runs
* against the SAME dist/ as the live harness for E2E parity.
*/
const FORBIDDEN_HOOK_STRINGS: ReadonlyArray<string> = [
'__mokoshTest',
'simulateUserStop',
'getSegmentCount',
'setCurrentStream',
'setSegmentCountGetter',
];
/** Icon-size floors per assertion 9 (per orchestrator brief). */
const ICON_SIZE_FLOORS: ReadonlyArray<readonly [string, number]> = [
['icons/icon16.png', 200],
['icons/icon48.png', 500],
['icons/icon128.png', 1024],
];
/**
* Recursively list all files under a root directory (sync). Used by
* assertion 0 to walk dist/. Symlinks are skipped defensively.
*
* @param root - Absolute directory path.
* @returns Sorted list of absolute file paths.
*/
function listAllFilesRecursive(root: string): ReadonlyArray<string> {
const acc: string[] = [];
const stack: string[] = [root];
while (stack.length > 0) {
const dir = stack.pop()!;
const entries = readdirSync(dir, { withFileTypes: true });
for (const entry of entries) {
const fullPath = resolvePath(dir, entry.name);
if (entry.isSymbolicLink()) continue;
if (entry.isDirectory()) {
stack.push(fullPath);
} else if (entry.isFile()) {
acc.push(fullPath);
}
}
}
return acc.sort();
}
/**
* Grep `needle` across every text-like file under `root`. Returns
* file paths that contain at least one occurrence.
*
* @param root - Absolute directory path.
* @param needle - Literal substring to find.
* @returns Paths containing `needle`.
*/
function grepRecursive(root: string, needle: string): ReadonlyArray<string> {
const binaryExt = new Set(['.png', '.jpg', '.jpeg', '.gif', '.ico', '.webp', '.woff', '.woff2', '.ttf']);
const out: string[] = [];
for (const filePath of listAllFilesRecursive(root)) {
const dotIdx = filePath.lastIndexOf('.');
const ext = dotIdx >= 0 ? filePath.substring(dotIdx).toLowerCase() : '';
if (binaryExt.has(ext)) continue;
if (statSync(filePath).size === 0) continue;
const text = readFileSync(filePath, 'utf8');
if (text.includes(needle)) {
out.push(filePath);
}
}
return out;
}
/**
* Poll `downloadsDir` for any *session_report*.zip file. Returns the
* absolute path of the first match. Used by assertion 5.
*
* @param downloadsDir - Absolute downloads directory path.
* @param timeoutMs - Maximum wait time.
* @returns Absolute path to the matched .zip.
* @throws On timeout.
*/
async function waitForDownloadedZip(
downloadsDir: string,
timeoutMs: number,
): Promise<string> {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const entries = readdirSync(downloadsDir);
for (const name of entries) {
if (name.includes('session_report') && name.endsWith('.zip')) {
const full = join(downloadsDir, name);
// Make sure write completed (size stabilized).
const size1 = statSync(full).size;
await new Promise((r) => setTimeout(r, 200));
const size2 = statSync(full).size;
if (size1 === size2 && size1 > 0) {
return full;
}
}
}
await new Promise((r) => setTimeout(r, 200));
}
throw new Error(
`waitForDownloadedZip: no session_report_*.zip appeared in ${downloadsDir} within ${timeoutMs}ms`,
);
}
/**
* Run a production build of dist/ unless SKIP_PROD_REBUILD=1.
* Assertion 0 reads dist/, so this guarantees the gate runs against
* a fresh artifact.
*/
function ensureProductionBuild(): void {
if (process.env.SKIP_PROD_REBUILD === '1') {
process.stdout.write(' (SKIP_PROD_REBUILD=1 — using existing dist/)\n');
return;
}
process.stdout.write(' Running `npm run build` (assertion 0 pre-flight)...\n');
execFileSync('npm', ['run', 'build'], {
stdio: 'inherit',
cwd: REPO_ROOT,
});
}
/**
* Stub placeholder for assertions Task 4+ wires. Each stub throws so
* the harness exits non-zero today; the diagnostic clearly identifies
* the assertion as un-implemented vs failing-in-production.
*
* @param taskNumber - The plan task number that will wire this assertion.
* @returns A function that always throws.
*/
function notYetImplemented(taskNumber: number): () => Promise<void> {
return async () => {
throw new Error(
`NOT YET IMPLEMENTED — Plan 01-11 Task ${taskNumber} wires this assertion`,
);
};
}
/**
* Main harness entry point. Runs all 14 assertions sequentially with
* bail-on-first-fail semantics for the SETUP-dependent assertions
* (we still record every assertion's outcome — bail only stops
* subsequent FUNCTIONAL assertions from running).
*/
async function main(): Promise<number> {
const results: AssertionRecord[] = [];
const buffers: ConsoleBuffers = { swLines: [], offscreenLines: [] };
let handles: HarnessHandles | null = null;
process.stdout.write('\nMokosh UAT harness — Plan 01-11 Puppeteer-driven 14-assertion suite\n');
process.stdout.write('='.repeat(72) + '\n\n');
try {
// ─── Assertion 0: Pre-flight grep gate ──────────────────────────
process.stdout.write('Assertion 0 (pre-flight, filesystem-only):\n');
ensureProductionBuild();
const a0 = await runAssertion(
0,
'production bundle has no test-hook leaks (T-1-11-01)',
buffers,
async () => {
for (const needle of FORBIDDEN_HOOK_STRINGS) {
const matches = grepRecursive(DIST_DIR, needle);
assertEqual(
matches.length,
0,
`production dist/ contains '${needle}' in: ${JSON.stringify(matches)}`,
);
}
},
);
results.push(a0);
if (!a0.passed) {
// Hook leak is security-critical (T-1-11-01) — abort immediately.
process.stderr.write(
'\n*** ABORT: assertion 0 (hook leak gate) FAILED — refusing to ' +
'continue with potentially-leaky production bundle. ***\n',
);
return 1;
}
// ─── Setup: launch browser, attach to SW + open popup bridge ───
process.stdout.write('\nLaunching Chrome + opening popup bridge...\n');
handles = await launchHarnessBrowser();
const { browser, sw, page, popup, extension, extensionId, downloadsDir } = handles;
process.stdout.write(` extensionId: ${extensionId}\n`);
process.stdout.write(` downloadsDir: ${downloadsDir}\n`);
process.stdout.write(` popup: chrome-extension://${extensionId}/src/popup/index.html\n\n`);
// Wire console buffers. The popup carries the chrome.* queries;
// the SW handle is kept for diagnostic console capture (when the
// SW is alive). Both feed buffers for failure dumps.
const popupPage: Page = popup;
popupPage.on('console', (msg) => {
buffers.swLines.push(`[Popup:${msg.type()}] ${msg.text()}`);
});
sw.on('console', (msg) => {
buffers.swLines.push(`[SW:${msg.type()}] ${msg.text()}`);
});
// Read the manifest version once for assertion 13.
const manifest = await getManifest(popupPage);
const expectedVersion = manifest.version;
// ─── Assertion 1: SW bootstrap → setIdleMode ───────────────────
const a1 = await runAssertion(
1,
'SW bootstrap → setIdleMode (badge="" + popup="" + isRecording=false)',
buffers,
async () => {
// Wake the SW via the keepalive ping first — this ensures the
// SW's initialize() has been observed before we sample state.
// In MV3, the SW may have suspended between launch and now;
// sendMessage wakes it for 30s.
await keepalivePing(popupPage);
// After SW init (production initialize() calls setIdleMode()),
// badge becomes empty + popup becomes empty + isRecording is false.
// Poll for the popup transition because setIdleMode's setPopup
// call is async-propagated through the action API; the popup
// page may briefly observe the manifest default before the
// override lands.
const popupUrl = await waitFor(
() => getPopup(popupPage),
(v) => v === '',
3_000,
'popup should become empty after SW initialize() runs setIdleMode',
100,
);
assertEqual(popupUrl, '', 'popup expected empty in idle mode');
const badge = await getBadgeText(popupPage);
assertEqual(badge, '', 'badge expected empty after SW bootstrap');
const recording = await getIsRecording(popupPage);
assertEqual(recording, false, 'isRecording expected false at bootstrap');
},
);
results.push(a1);
// ─── Assertion 2: Toolbar onClicked-idle → REC + popup ─────────
const a2 = await runAssertion(
2,
'toolbar onClicked-idle → badge "REC" + popup set + isRecording=true',
buffers,
async () => {
// Trigger the toolbar action — the production code's
// chrome.action.onClicked listener calls startVideoCapture(),
// which creates the offscreen + starts recording + transitions
// the badge to REC + sets the popup to popup/index.html.
await page.bringToFront();
await page.triggerExtensionAction(extension);
// Badge transition is async (offscreen create + handshake +
// getDisplayMedia picker + post-grant validation + setBadgeText).
// Poll up to 8s; the auto-select picker shaves most of the time.
const badge = await waitFor(
() => getBadgeText(popupPage),
(v) => v === 'REC',
8_000,
'badge should become REC after toolbar click',
200,
);
assertEqual(badge, 'REC', 'badge after toolbar click');
const popupUrl = await getPopup(popupPage);
assertMatch(
popupUrl,
/src\/popup\/index\.html$/,
'popup should be set to popup/index.html during recording',
);
const recording = await getIsRecording(popupPage);
assertEqual(recording, true, 'isRecording expected true after click');
},
);
results.push(a2);
// ─── Assertion 3: offscreen displaySurface === 'monitor' ───────
const a3 = await runAssertion(
3,
'offscreen getCurrentStream displaySurface === "monitor"',
buffers,
async () => {
// A2 left recording active. The offscreen document was created
// by startVideoCapture; wait for the target + attach.
const offTarget = await waitForOffscreenTarget(browser, extensionId);
const offPage = await attachToOffscreen(offTarget);
offPage.on('console', (msg) => {
buffers.offscreenLines.push(`[OS:${msg.type()}] ${msg.text()}`);
});
const ds = await getDisplaySurface(offPage);
assertEqual(
ds,
'monitor',
'displaySurface expected "monitor" per post-grant validation (D-15)',
);
},
);
results.push(a3);
// ─── Assertion 4: onClicked-recording → popup, no new offscreen
const a4 = await runAssertion(
4,
'toolbar onClicked while recording → popup opens, no new offscreen target',
buffers,
async () => {
const offscreenCountBefore = countOffscreenTargets(browser, extensionId);
// Triggering the action during recording opens the popup
// (production code keeps popup set in REC mode); the click
// does NOT spawn a second offscreen.
await page.bringToFront();
await page.triggerExtensionAction(extension);
// Give the popup a moment to open + Chrome to settle.
await new Promise((r) => setTimeout(r, 500));
const offscreenCountAfter = countOffscreenTargets(browser, extensionId);
assertEqual(
offscreenCountAfter,
offscreenCountBefore,
'offscreen target count must not increase on second click during recording',
);
// popup must still be set (recording is ongoing).
const popupUrl = await getPopup(popupPage);
assertMatch(
popupUrl,
/src\/popup\/index\.html$/,
'popup must remain set while recording',
);
},
);
results.push(a4);
// ─── Wave 3 STUBBED assertions (Tasks 5-7 will wire these) ─────
const stubs: Array<{
index: number;
name: string;
taskNumber: number;
}> = [
{ index: 5, name: 'SAVE_ARCHIVE → download fires + zip appears', taskNumber: 5 },
{ index: 6, name: 'BUG B canonical: simulateUserStop → badge OFF + no recovery notif', taskNumber: 5 },
{ index: 7, name: 'RECORDING_ERROR codec-unsupported → badge ERR + recovery notif', taskNumber: 5 },
{ index: 8, name: 'BUG A canonical: onStartup → notification creates cleanly', taskNumber: 6 },
{ index: 9, name: 'icon file sizes meet floors', taskNumber: 6 },
{ index: 10, name: 'manifest has notifications + 3 icons', taskNumber: 6 },
{ index: 11, name: '35s recording → segments.length >= 3', taskNumber: 7 },
{ index: 12, name: 'ffprobe on extracted webm exits 0', taskNumber: 7 },
{ index: 13, name: 'archive shape — video + meta.json version match', taskNumber: 7 },
];
for (const s of stubs) {
const rec = await runAssertion(
s.index,
s.name,
buffers,
notYetImplemented(s.taskNumber),
);
results.push(rec);
}
// Suppress unused-warning placeholders for Tasks 5-7 helpers.
void expectedVersion;
void getIconSize;
void fireOnStartup;
void sendSyntheticRecordingError;
void getNotificationSnapshot;
void keepalivePing;
void simulateUserStop;
void getSegmentCount;
void assertArchiveShape;
void extractEntryToFile;
void assertTrue;
void assertGte;
void waitForDownloadedZip;
void mkdtempSync;
void existsSync;
void execSync;
void tmpdir;
void FFPROBE_BIN;
void ICON_SIZE_FLOORS;
return finalize(results);
} catch (setupErr) {
process.stderr.write(`\n*** Harness setup error: ${String(setupErr)}\n`);
return finalize(results);
} finally {
if (handles !== null) {
try {
await handles.browser.close();
} catch (closeErr) {
process.stderr.write(`(non-fatal: browser close threw: ${String(closeErr)})\n`);
}
}
}
}
/**
* Print the final summary line + return the exit code.
*
* @param results - All assertion records collected during the run.
* @returns 0 if all 14 passed, 1 otherwise.
*/
function finalize(results: ReadonlyArray<AssertionRecord>): number {
const passCount = results.filter((r) => r.passed).length;
const failCount = results.length - passCount;
process.stdout.write('\n' + '='.repeat(72) + '\n');
if (passCount === TOTAL_ASSERTIONS) {
process.stdout.write(`UAT harness: ${passCount}/${TOTAL_ASSERTIONS} assertions passed\n`);
return 0;
}
const firstFail = results.find((r) => !r.passed);
process.stdout.write(
`UAT harness: ${passCount}/${TOTAL_ASSERTIONS} assertions passed, ${failCount} failed`,
);
if (firstFail !== undefined) {
process.stdout.write(` (first failure: A${firstFail.index} ${firstFail.name})`);
}
process.stdout.write('\n');
return 1;
}
// Run + exit. Top-level await + explicit exit code so tsx returns
// the right status without leaving unhandled-promise spew on stderr.
const exitCode = await main();
process.exit(exitCode);