docs(phase-03): add validation strategy — verification-only phase; infra inherited from Phase 1+2 .planning/phases/03-spec-10-smoke-verification-dom-event-log-verification/03-VALIDATION.md
This commit is contained in:
@@ -0,0 +1,91 @@
|
||||
---
|
||||
phase: 03
|
||||
slug: spec-10-smoke-verification-dom-event-log-verification
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
wave_0_complete: false
|
||||
created: 2026-05-20
|
||||
---
|
||||
|
||||
# Phase 03 — Validation Strategy
|
||||
|
||||
> Per-phase validation contract for feedback sampling during execution.
|
||||
|
||||
**Phase 3 character:** This phase IS validation. The deliverables are harness assertions (A29+) + a §10 sweep VERIFICATION.md, not new production code. The per-task verification map below is filled in by the planner during Plans 03-01..05 creation (each plan task's `<automated>` block declares its verification command).
|
||||
|
||||
---
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Framework** | vitest 4.x (unit) + custom Puppeteer harness (UAT — `npm run test:uat`) |
|
||||
| **Config file** | `vitest.config.ts` + `tests/uat/harness.test.ts` (orchestrator) |
|
||||
| **Quick run command** | `npm test -- --run tests/<focused-file>.test.ts` |
|
||||
| **Full suite command** | `npm test -- --run` (vitest) + `HEADLESS=1 SKIP_PROD_REBUILD=1 npm run test:uat` (UAT harness) |
|
||||
| **Estimated runtime** | ~50s (vitest 171 tests) + ~95s (UAT harness 29 → ~34 assertions) ≈ 2.5 min full sweep |
|
||||
|
||||
---
|
||||
|
||||
## Sampling Rate
|
||||
|
||||
- **After every task commit:** Run focused test command (vitest single-file OR `npm run test:uat -- --grep A<NN>` for harness)
|
||||
- **After every plan wave:** Run full vitest + full UAT harness — both MUST be GREEN
|
||||
- **Before `/gsd-verify-work 3`:** Full suite GREEN + pre-checkpoint bundle gates 6/6 PASS (per saved memory `feedback-pre-checkpoint-bundle-gates.md`)
|
||||
- **Max feedback latency:** ~2.5 min (full sweep); ~10s (focused vitest); ~20s (focused UAT assertion)
|
||||
|
||||
---
|
||||
|
||||
## Per-Task Verification Map
|
||||
|
||||
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|
||||
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
|
||||
| _Filled by planner during Plans 03-01..05 creation_ | _via planner_ | _via planner_ | _via planner_ | _via planner_ | _via planner_ | _via planner_ | _via planner_ | _via planner_ | ⬜ pending |
|
||||
|
||||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||||
|
||||
**Planner instructions:** When creating each task in Plans 03-01..05, populate one row in this table with:
|
||||
- `Task ID`: `03-XX-NN` format
|
||||
- `Requirement`: REQ-rrweb-dom-buffer | REQ-user-event-log | REQ-install-clean | (or "N/A — verification surface only")
|
||||
- `Test Type`: harness | unit | bundle-gate | manual-operator
|
||||
- `Automated Command`: exact CLI invocation that proves the task succeeded
|
||||
|
||||
---
|
||||
|
||||
## Wave 0 Requirements
|
||||
|
||||
Phase 3 has no Wave 0 (no RED test scaffolds needed — verification phase extends existing harness; new assertions ride established Approach B pattern). All test infrastructure is already in place from Plan 02-04:
|
||||
|
||||
- ✅ `tests/uat/extension-page-harness.ts` — page-side assertA* host
|
||||
- ✅ `tests/uat/lib/harness-page-driver.ts` — host-side driveA* host
|
||||
- ✅ `tests/uat/harness.test.ts` — orchestrator
|
||||
- ✅ `tests/uat/lib/assertions.ts` — shared helpers
|
||||
- ✅ `tests/uat/lib/zip.ts` — jszip-based archive parsing
|
||||
- ✅ `tests/uat/lib/launch.ts` — Puppeteer Chrome launch + extension load
|
||||
- ✅ `tests/background/no-test-hooks-in-prod-bundle.test.ts` — FORBIDDEN_HOOK_STRINGS lockstep
|
||||
- ✅ vitest 4.x + Puppeteer already in package.json
|
||||
|
||||
*Phase 3 inherits all infrastructure from Phase 1 + 2. wave_0_complete: true (no new infra needed).*
|
||||
|
||||
---
|
||||
|
||||
## Manual-Only Verifications
|
||||
|
||||
| Behavior | Requirement | Why Manual | Test Instructions |
|
||||
|----------|-------------|------------|-------------------|
|
||||
| Extension RAM ≤ 50 MB in background | CON-ram-ceiling (NFR; SPEC §10 #9) | Puppeteer Page.metrics is page-realm only; SW heap requires `performance.measureUserAgentSpecificMemory()` which throws SecurityError in MV3 (no COOP+COEP). chrome.devtools Memory API requires research budget out of Phase 3 charter. Per D-P3-04: operator/alpha-tester observation. | Load extension; idle 5 min; open `chrome://memory-internals` OR `chrome://extensions/` → "Service worker" link → DevTools → Memory tab; verify extension background RAM < 50 MB. Plan 03-04 includes the operator-facing instructions verbatim in VERIFICATION.md. Optional puppeteer.Page.metrics scaffolding ships in Plan 03-04 as a best-effort diagnostic with explicit "page-realm only" caveat. |
|
||||
|
||||
*All other Phase 3 behaviors have automated verification via the UAT harness (Plans 03-01..05 produce assertA29+ for SPEC §10 #4 rrweb DOM, §10 #5 event log, §10 #8 password filter).*
|
||||
|
||||
---
|
||||
|
||||
## Validation Sign-Off
|
||||
|
||||
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies — pending planner fill-in
|
||||
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify — verify after planner fills the table
|
||||
- [ ] Wave 0 covers all MISSING references — Phase 3 inherits infra; no Wave 0 needed
|
||||
- [ ] No watch-mode flags — verify in planner output (focused commands use `--run`)
|
||||
- [ ] Feedback latency < ~2.5 min — confirmed by infrastructure inheritance
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter — pending sign-off after planner completes
|
||||
|
||||
**Approval:** pending (planner fills per-task map; checker validates)
|
||||
Reference in New Issue
Block a user