Mark
b3bfbf4a8d
5 plans across 5 waves (Wave 2 sequential per RESEARCH Pitfall 6 file overlap): - 03-01 Wave 1: rrweb DOM verification harness extension (A29; REQ-rrweb-dom-buffer; §10 #4) - 03-02 Wave 2: event-log verification harness extension (A30; REQ-user-event-log; §10 #5) - 03-03 Wave 3: §10 #8 password-filter PARTIAL verification (A31; D-P3-02 charter) - 03-04 Wave 4: §10 #9 RAM ceiling best-effort + Page.metrics scaffolding (A32; D-P3-04) - 03-05 Wave 5: §10 sweep VERIFICATION.md + REQUIREMENTS/ROADMAP/STATE marker flips (REQ-install-clean + REQ-rrweb-dom-buffer + REQ-user-event-log) Each plan has: - frontmatter (wave + depends_on + files_modified + autonomous + requirements + tags + must_haves) - tasks with mandatory <read_first> + <acceptance_criteria> + concrete <action> - <threat_model> block per security gate - Validation map row(s) added to 03-VALIDATION.md (10 tasks total) Expected UAT growth: 29/29 → 33/33 GREEN (A29-A32 + 03-05 docs). Expected vitest baseline preserved: 171/171. Expected Tier-1 FORBIDDEN_HOOK_STRINGS: 12 (A29+ ride production surfaces only). ROADMAP.md Phase 3 entry replaces "Plans: TBD" with full 5-plan list. VALIDATION.md status: planner_filled (nyquist_compliant: true; wave_0_complete: true). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
109 lines
8.7 KiB
Markdown
109 lines
8.7 KiB
Markdown
---
|
|
phase: 03
|
|
slug: spec-10-smoke-verification-dom-event-log-verification
|
|
status: planner_filled
|
|
nyquist_compliant: true
|
|
wave_0_complete: true
|
|
created: 2026-05-20
|
|
filled_by: gsd-planner
|
|
filled_at: 2026-05-20
|
|
---
|
|
|
|
# Phase 03 — Validation Strategy
|
|
|
|
> Per-phase validation contract for feedback sampling during execution.
|
|
|
|
**Phase 3 character:** This phase IS validation. The deliverables are
|
|
harness assertions (A29..A32) + a §10 sweep VERIFICATION.md, not new
|
|
production code. The per-task verification map below is filled in by
|
|
the planner during Plans 03-01..05 creation (each plan task's
|
|
`<automated>` block declares its verification command).
|
|
|
|
---
|
|
|
|
## Test Infrastructure
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| **Framework** | vitest 4.x (unit) + custom Puppeteer harness (UAT — `npm run test:uat`) |
|
|
| **Config file** | `vitest.config.ts` + `tests/uat/harness.test.ts` (orchestrator) |
|
|
| **Quick run command** | `npm test -- --run tests/<focused-file>.test.ts` |
|
|
| **Full suite command** | `npm test -- --run` (vitest) + `HEADLESS=1 SKIP_PROD_REBUILD=1 npm run test:uat` (UAT harness) |
|
|
| **Estimated runtime** | ~50s (vitest 171 tests) + ~110s (UAT harness 33 assertions) ≈ ~2.7 min full sweep |
|
|
|
|
---
|
|
|
|
## Sampling Rate
|
|
|
|
- **After every task commit:** Run focused test command (vitest single-file OR `npm run test:uat -- --grep A<NN>` for harness)
|
|
- **After every plan wave:** Run full vitest + full UAT harness — both MUST be GREEN
|
|
- **Before `/gsd-verify-work 3`:** Full suite GREEN + pre-checkpoint bundle gates 6/6 PASS (per saved memory `feedback-pre-checkpoint-bundle-gates.md`)
|
|
- **Max feedback latency:** ~2.7 min (full sweep); ~10s (focused vitest); ~25s (focused UAT assertion)
|
|
|
|
---
|
|
|
|
## Per-Task Verification Map
|
|
|
|
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|
|
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
|
|
| 03-01-01 | 03-01 | 1 | REQ-rrweb-dom-buffer | T-03-01-02 | Probe HTML appended without disturbing tokens.css link (preserves A18/A21) | bundle-gate (grep + build) | `grep -c 'id="probe-form"' tests/uat/extension-page-harness.html ; grep -c '<link rel="stylesheet" href="../../src/shared/tokens.css">' tests/uat/extension-page-harness.html ; ! grep -q '<textarea' tests/uat/extension-page-harness.html ; npm run build` | ✅ tests/uat/extension-page-harness.html exists | ⬜ pending |
|
|
| 03-01-02 | 03-01 | 1 | REQ-rrweb-dom-buffer | T-03-01-03 | A29 GREEN; FORBIDDEN_HOOK_STRINGS unchanged at 12 | harness | `npx tsc --noEmit && HEADLESS=1 SKIP_PROD_REBUILD=0 npm run test:uat` then verify stdout contains "UAT harness: 30/30" | ❌ Wave 1 (extension-page-harness.ts + harness-page-driver.ts + harness.test.ts to be appended) | ⬜ pending |
|
|
| 03-02-01 | 03-02 | 2 | REQ-user-event-log | T-03-02-01, T-03-02-02 | assertA30 page-side orchestrator dispatches 5 trigger types; tokens.css link still untouched | bundle-gate (tsc + grep) | `npx tsc --noEmit ; grep -c "assertA30" tests/uat/extension-page-harness.ts` | ❌ Wave 2 (assertA30 to be appended) | ⬜ pending |
|
|
| 03-02-02 | 03-02 | 2 | REQ-user-event-log | T-03-02-03 | A30 GREEN; 5 UserEvent.type literal values present in logs/events.json; Tier-1 inventory at 12 | harness | `HEADLESS=1 SKIP_PROD_REBUILD=0 npm run test:uat` then verify stdout contains "UAT harness: 31/31" | ❌ Wave 2 (driveA30 + drivers-array push to be appended) | ⬜ pending |
|
|
| 03-03-01 | 03-03 | 2 | N/A — §10 #8 PARTIAL per D-P3-02 | T-03-03-01 | assertA31 types SENTINEL into #probe-password and dispatches input event | bundle-gate (tsc + grep) | `npx tsc --noEmit ; grep -c 'A31_PASSWORD_SENTINEL' tests/uat/extension-page-harness.ts` | ❌ Wave 2 (assertA31 to be appended) | ⬜ pending |
|
|
| 03-03-02 | 03-03 | 2 | N/A — §10 #8 PARTIAL per D-P3-02 | T-03-03-04 | A31 GREEN; 2 negative-assertion checks pass (sentinel-absence + selector-target-absence) | harness | `HEADLESS=1 SKIP_PROD_REBUILD=0 npm run test:uat` then verify stdout contains "UAT harness: 32/32" AND A31 sentinel-containing count=0 | ❌ Wave 2 (driveA31 + drivers-array push to be appended) | ⬜ pending |
|
|
| 03-04-01 | 03-04 | 2 | N/A — §10 #9 best-effort per D-P3-04 | T-03-04-01, T-03-04-02 | A32 GREEN; page-realm-only diagnostic emitted; Tier-1 inventory at 12 | harness | `npx tsc --noEmit && HEADLESS=1 SKIP_PROD_REBUILD=0 npm run test:uat` then verify stdout contains "UAT harness: 33/33" AND "NOTE: page-realm only" diagnostic | ❌ Wave 2 (driveA32 + drivers-array push to be appended) | ⬜ pending |
|
|
| 03-05-01 | 03-05 | 3 | REQ-install-clean + REQ-rrweb-dom-buffer + REQ-user-event-log | T-03-05-01 | Pre-checkpoint bundle gates 6/6 PASS | bundle-gate (multi) | `npm run build ; npm test -- --run tests/background/no-test-hooks-in-prod-bundle.test.ts tests/background/sw-bundle-import.test.ts tests/i18n/ tests/build/` | ✅ All gate files already present from Phase 1+2 | ⬜ pending |
|
|
| 03-05-02 | 03-05 | 3 | REQ-rrweb-dom-buffer + REQ-user-event-log | T-03-05-01, T-03-05-02 | 03-VERIFICATION.md exists with 9/9 §10 scorecard + 3 overrides + 1 human_verification | docs (grep) | `test -f .planning/phases/03-spec-10-smoke-verification-dom-event-log-verification/03-VERIFICATION.md && wc -l < ... | awk '$1 >= 120' && grep -q 'overrides_applied: 3' ... && grep -q 'human_verification:' ...` | ❌ Wave 3 (file to be created) | ⬜ pending |
|
|
| 03-05-03 | 03-05 | 3 | REQ-install-clean + REQ-rrweb-dom-buffer + REQ-user-event-log | T-03-05-02 | REQUIREMENTS / ROADMAP / STATE markers flipped Complete | docs (grep) | `grep -c '^- \[x\] \*\*REQ-rrweb-dom-buffer' .planning/REQUIREMENTS.md && grep -c '^- \[x\] \*\*Phase 3' .planning/ROADMAP.md && grep -c 'completed_phases: 3' .planning/STATE.md` | ✅ Files exist; markers to be flipped | ⬜ pending |
|
|
|
|
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
|
|
|
**Total task count: 10 tasks across 5 plans (Plans 03-01..04 each ship 2
|
|
tasks; Plan 03-05 ships 3 tasks).**
|
|
|
|
---
|
|
|
|
## Wave 0 Requirements
|
|
|
|
Phase 3 has no Wave 0 (no RED test scaffolds needed — verification phase
|
|
extends existing harness; new assertions ride established Approach B
|
|
pattern). All test infrastructure is already in place from Plan 02-04:
|
|
|
|
- ✅ `tests/uat/extension-page-harness.ts` — page-side assertA* host
|
|
- ✅ `tests/uat/lib/harness-page-driver.ts` — host-side driveA* host
|
|
- ✅ `tests/uat/harness.test.ts` — orchestrator
|
|
- ✅ `tests/uat/lib/assertions.ts` — shared helpers
|
|
- ✅ `tests/uat/lib/zip.ts` — jszip-based archive parsing
|
|
- ✅ `tests/uat/lib/launch.ts` — Puppeteer Chrome launch + extension load
|
|
- ✅ `tests/background/no-test-hooks-in-prod-bundle.test.ts` — FORBIDDEN_HOOK_STRINGS lockstep
|
|
- ✅ vitest 4.x + Puppeteer already in package.json
|
|
|
|
*Phase 3 inherits all infrastructure from Phase 1 + 2.
|
|
`wave_0_complete: true` (no new infra needed).*
|
|
|
|
---
|
|
|
|
## Manual-Only Verifications
|
|
|
|
| Behavior | Requirement | Why Manual | Test Instructions |
|
|
|----------|-------------|------------|-------------------|
|
|
| Extension RAM ≤ 50 MB in background | CON-ram-ceiling (NFR; SPEC §10 #9) | Puppeteer Page.metrics is page-realm only; SW heap requires `performance.measureUserAgentSpecificMemory()` which throws SecurityError in MV3 (no COOP+COEP). chrome.devtools Memory API requires research budget out of Phase 3 charter. Per D-P3-04: operator/alpha-tester observation is the canonical path. | Load extension; idle 5 min; open `chrome://memory-internals` OR `chrome://extensions/` → "Service worker" link → DevTools → Memory tab; verify extension background RAM < 50 MB. Plan 03-04 includes the operator-facing instructions verbatim in VERIFICATION.md `human_verification` block. Plan 03-04 ships A32 informational scaffolding via `puppeteer.Page.metrics()` with explicit "page-realm only" diagnostic emitted on every run. |
|
|
|
|
*All other Phase 3 behaviors have automated verification via the UAT
|
|
harness (Plans 03-01..05 produce assertA29..A32 for SPEC §10 #4 rrweb
|
|
DOM, §10 #5 event log, §10 #8 password filter, §10 #9 RAM scaffolding).*
|
|
|
|
---
|
|
|
|
## Validation Sign-Off
|
|
|
|
- [x] All tasks have `<automated>` verify or Wave 0 dependencies — planner filled per-task map covering all 10 tasks across 5 plans
|
|
- [x] Sampling continuity: no 3 consecutive tasks without automated verify — verified; every task either runs `npm test`/`npm run test:uat` or has a grep-based bundle gate
|
|
- [x] Wave 0 covers all MISSING references — Phase 3 inherits infra from Plan 01-13 + 02-04; no Wave 0 needed
|
|
- [x] No watch-mode flags — focused commands use `--run` (vitest) or `npm run test:uat` (single-shot)
|
|
- [x] Feedback latency < ~3 min — confirmed by infrastructure inheritance (~2.7 min full sweep)
|
|
- [x] `nyquist_compliant: true` set in frontmatter — planner sign-off
|
|
|
|
**Approval:** planner-filled 2026-05-20; checker validates next pass.
|