109 lines
7.8 KiB
Markdown
109 lines
7.8 KiB
Markdown
# Phase 8: Nyquist Validation Track - Context
|
|
|
|
**Gathered:** 2026-04-13
|
|
**Status:** Ready for planning
|
|
|
|
<domain>
|
|
## Phase Boundary
|
|
|
|
Retro-fit a Nyquist-compliant validation record across all 7 v1.0 phases, aggregate pass/fail into a single milestone index, and triage any gaps into v1.1 defects or explicit deferrals. Pure audit of shipped code — no new product features, no refactors, no runtime validation (that's Phase 10).
|
|
|
|
</domain>
|
|
|
|
<decisions>
|
|
## Implementation Decisions
|
|
|
|
### File strategy & schema
|
|
- **Upgrade existing `{NN}-VALIDATION.md` in place** for each of the 7 v1.0 phases. Do NOT create a parallel file.
|
|
- Preserve the current execution-sampling sections (Sampling Rate, Per-Task Verification Map, Wave 0 Requirements, Manual-Only Verifications) — they are historical record.
|
|
- **Add a new top-level section `## Nyquist Record`** containing one row per phase success criterion from ROADMAP.md / milestones/v1.0-ROADMAP.md.
|
|
- Row shape: `| # | Success Criterion | Observable Check | Evidence | Status | Notes |`
|
|
- Exactly **one observable check per success criterion** (Nyquist rule — no hand-wavy "code looks right").
|
|
- Evidence cell is a markdown link or inline reference (see Evidence types below).
|
|
- **Frontmatter:** flip `nyquist_compliant: false → true` only when every row has Status = pass or explicit-deferred; add `nyquist_audited: YYYY-MM-DD` and `nyquist_auditor: <name>`.
|
|
|
|
### Evidence types (what counts)
|
|
Accepted, in priority order:
|
|
1. **Test evidence** — `pytest <path>::<testname>` plus the file path:line where it lives. Trust a recorded-green run from commit X; do NOT require live re-run.
|
|
2. **Commit evidence** — commit SHA + file path:line for code-only checks (e.g., "UAC self-elevation guard present at install.ps1:12-20").
|
|
3. **File-artifact evidence** — path to a generated artifact (e.g., a sample `.intunewin` byte-layout fixture).
|
|
4. **Manual-check evidence** — link to a dated note in the phase's `VERIFICATION.md` describing what was observed. Only allowed when no automated path exists.
|
|
- **Not accepted:** "code review says it works", "see comments in file", vague prose.
|
|
- Re-running tests live is NOT required. A green pytest run recorded in the commit log counts.
|
|
|
|
### Gap triage & index rollup
|
|
- **Gaps live as rows in the index**, not in a separate DEFECTS.md file. Keeping gap tracking co-located with the pass/fail rollup is the whole point of the index.
|
|
- Each gap row gets status ∈ {`fail-fix-v1.1`, `deferred-v1.2`, `wont-do`} and a mandatory `rationale` cell.
|
|
- `fail-fix-v1.1` rows MUST link to the fixing phase (9, 10, or 11) — if no phase owns the fix, the roadmap is wrong and must be amended before Phase 8 closes.
|
|
- `deferred-v1.2` / `wont-do` rows MUST have written rationale; no empty cells.
|
|
- Index file: **`.planning/milestones/v1.0-VALIDATION-INDEX.md`** (exact path per roadmap success criterion #2).
|
|
|
|
### Index structure
|
|
- Single flat table across all 7 phases: `| Phase | # | Criterion | Status | Evidence | Gap Link |`
|
|
- Header block above the table: audit date, auditor, overall tally (`X/Y pass, Z deferred, W fail-fix`).
|
|
- Per-phase subsections NOT used — flat table keeps the rollup scannable and greppable.
|
|
- **Sign-off:** dated checkbox line at the bottom: `- [x] Nyquist audit complete — <date> — <auditor>`. Same sign-off replicated in each phase's VALIDATION.md frontmatter.
|
|
|
|
### Audit execution approach
|
|
- **Sequential phase-by-phase (1 → 7).** Each audit informs the next (shared patterns, shared test infra), and ImpTune is small enough that parallelism would cost more coordination than it saves.
|
|
- **One plan per v1.0 phase audited** (7 plans: 08-01 through 08-07), plus **one final plan (08-08) for the index + triage sign-off**. Total: 8 plans.
|
|
- Each audit plan: read that phase's ROADMAP criteria → map each to an observable check → locate evidence → fill the Nyquist Record → commit.
|
|
- The final 08-08 plan: aggregate all 7 phase records into `v1.0-VALIDATION-INDEX.md`, confirm gap links against Phases 9/10/11, sign off.
|
|
|
|
### Claude's Discretion
|
|
- Exact wording / column widths in the Nyquist Record tables.
|
|
- Whether to also flip `status: draft → audited` in per-phase frontmatter (preferred but not mandatory).
|
|
- How to handle the Phase 1 "spike" row (`01-03-02`, upload to real Intune tenant) — likely flagged as `fail-fix-v1.1` linked to Phase 10 RTVAL-01, but the auditor confirms at audit time.
|
|
- Whether to cite commit SHAs as short (7-char) or full — auditor's call.
|
|
- How to phrase "observable check" sentences.
|
|
|
|
</decisions>
|
|
|
|
<code_context>
|
|
## Existing Code Insights
|
|
|
|
### Reusable Assets
|
|
- **Existing per-phase VALIDATION.md files** at `.planning/phases/{NN}-*/NN-VALIDATION.md` — already have frontmatter, execution sampling tables, and Wave 0 requirements. The audit upgrades these in place rather than creating new files.
|
|
- **`milestones/v1.0-ROADMAP.md`** — canonical source of success criteria per phase. The audit reads this, not the current `ROADMAP.md` (which is the v1.1 view).
|
|
- **`VERIFICATION.md` files** per phase — already record per-phase verification runs; manual-only evidence links point here.
|
|
- **Test suite under `tests/`** — pytest already green at v1.0 shipped commit; provides most test evidence for automated checks.
|
|
|
|
### Established Patterns
|
|
- **Frontmatter convention** — every phase doc uses YAML frontmatter; `nyquist_compliant: false` flag already exists in VALIDATION.md, ready to flip.
|
|
- **Per-task verification map** pattern uses `| Task ID | ... | Automated Command |` — the new Nyquist Record table mirrors this shape for consistency.
|
|
- **Evidence citation style** elsewhere in .planning uses commit SHAs and `file.py:line` references — reuse.
|
|
|
|
### Integration Points
|
|
- **Milestone-level index**: new file at `.planning/milestones/v1.0-VALIDATION-INDEX.md` — sits alongside existing `v1.0-ROADMAP.md`, same directory.
|
|
- **Roadmap back-links** — `fail-fix-v1.1` gap rows link into Phase 9 (UX), Phase 10 (RTVAL), Phase 11 (RWR) per the v1.1 phase mapping in REQUIREMENTS.md.
|
|
- **No source code touched** — this phase writes only to `.planning/` tree.
|
|
|
|
</code_context>
|
|
|
|
<specifics>
|
|
## Specific Ideas
|
|
|
|
- The existing `01-VALIDATION.md` Manual-Only Verifications row "Upload to real Intune tenant" is the clearest example of a gap: will flip to `fail-fix-v1.1` linking RTVAL-01 / Phase 10.
|
|
- "Nyquist-compliant" per the roadmap success criterion means: every success criterion maps to exactly one observable check with cited evidence. That's the guardrail — no auditor editorializing beyond that.
|
|
- User ran `/gsd:discuss-phase 8` and explicitly said "go ahead and do your best" — Claude has broad discretion on presentation details; lock only the decisions above.
|
|
|
|
</specifics>
|
|
|
|
<deferred>
|
|
## Deferred Ideas
|
|
|
|
### Pre-surfaced Nyquist gaps (feed into the audit, fixed elsewhere)
|
|
- **`POST /drivers/upload` returns HTTP 500** (reported 2026-04-13 during Phase 8 kickoff). Blocks UX-01 by definition — you cannot refresh the DriverDesc dropdown if the upload itself fails. The auditor MUST record this as a `fail-fix-v1.1` row in `v1.0-VALIDATION-INDEX.md` under Phase 2 (Driver Management, DRV-01..05) linking to **Phase 9 / UX-01** as the fixing phase. Root cause is unknown at audit-kickoff time; reproduction: attempt driver ZIP upload, check server console for the 500. Do NOT fix in Phase 8 (pure audit); the fix lands in Phase 9's UX-01 plan.
|
|
|
|
### Out of scope entirely
|
|
- **Live re-running of the v1.0 pytest suite at audit time** — not required for Nyquist. If a later phase wants a "green-at-audit-date" stamp, handle it in Phase 10 or later.
|
|
- **Automating index generation from per-phase files** — manual compilation is fine for 7 phases; revisit if v1.x grows past ~15 phases.
|
|
- **Converting this audit into a reusable template for future milestones** — capture as a v1.2+ chore, not in scope for v1.1.
|
|
|
|
</deferred>
|
|
|
|
---
|
|
|
|
*Phase: 08-nyquist-validation-track*
|
|
*Context gathered: 2026-04-13*
|