kawa
60654afc8e
- 5 rows: one per SCRPT-01..05 success criterion - All rows pass at template level (pytest-backed) - SYSTEM-context rows (SCRPT-01/02/03/04/05) cite Phase 10 RTVAL-02/03/04 with explicit attestation-only caveat per STATE.md 2026-04-13 - Frontmatter: nyquist_compliant true, nyquist_audited 2026-04-13
108 lines
13 KiB
Markdown
108 lines
13 KiB
Markdown
---
|
|
phase: 4
|
|
slug: script-generation
|
|
status: draft
|
|
nyquist_compliant: true
|
|
wave_0_complete: false
|
|
created: 2026-04-10
|
|
nyquist_audited: 2026-04-13
|
|
nyquist_auditor: Claude (gsd-executor, plan 08-04)
|
|
---
|
|
|
|
# Phase 4 — Validation Strategy
|
|
|
|
> Per-phase validation contract for feedback sampling during execution.
|
|
|
|
---
|
|
|
|
## Test Infrastructure
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| **Framework** | pytest >= 8.0 |
|
|
| **Config file** | none — discovered automatically |
|
|
| **Quick run command** | `python -m pytest tests/test_script_generator.py -x -q` |
|
|
| **Full suite command** | `python -m pytest tests/ -x -q` |
|
|
| **Estimated runtime** | ~10 seconds |
|
|
|
|
---
|
|
|
|
## Sampling Rate
|
|
|
|
- **After every task commit:** Run `python -m pytest tests/test_script_generator.py -x -q`
|
|
- **After every plan wave:** Run `python -m pytest tests/ -x -q`
|
|
- **Before `/gsd:verify-work`:** Full suite must be green
|
|
- **Max feedback latency:** 10 seconds
|
|
|
|
---
|
|
|
|
## Per-Task Verification Map
|
|
|
|
| Task ID | Plan | Wave | Requirement | Test Type | Automated Command | File Exists | Status |
|
|
|---------|------|------|-------------|-----------|-------------------|-------------|--------|
|
|
| 4-01-01 | 01 | 1 | SCRPT-01 | unit | `python -m pytest tests/test_script_generator.py::test_render_install_contains_pnputil -x` | ❌ W0 | ⬜ pending |
|
|
| 4-01-02 | 01 | 1 | SCRPT-01 | unit | `python -m pytest tests/test_script_generator.py::test_render_install_print_config -x` | ❌ W0 | ⬜ pending |
|
|
| 4-02-01 | 02 | 1 | SCRPT-02 | unit | `python -m pytest tests/test_script_generator.py::test_render_uninstall -x` | ❌ W0 | ⬜ pending |
|
|
| 4-02-02 | 02 | 1 | SCRPT-03 | unit | `python -m pytest tests/test_script_generator.py::test_render_detect -x` | ❌ W0 | ⬜ pending |
|
|
| 4-01-03 | 01 | 1 | SCRPT-04 | unit | `python -m pytest tests/test_script_generator.py::test_render_install_uac_guard -x` | ❌ W0 | ⬜ pending |
|
|
| 4-01-04 | 01 | 1 | SCRPT-05 | unit | `python -m pytest tests/test_script_generator.py::test_render_install_wow64_guard -x` | ❌ W0 | ⬜ pending |
|
|
| 4-03-01 | 03 | 2 | SCRPT-01 | integration | `python -m pytest tests/test_script_generator.py::test_install_endpoint -x` | ❌ W0 | ⬜ pending |
|
|
| 4-03-02 | 03 | 2 | SCRPT-02 | integration | `python -m pytest tests/test_script_generator.py::test_uninstall_endpoint -x` | ❌ W0 | ⬜ pending |
|
|
| 4-03-03 | 03 | 2 | SCRPT-03 | integration | `python -m pytest tests/test_script_generator.py::test_detect_endpoint -x` | ❌ W0 | ⬜ pending |
|
|
|
|
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
|
|
|
---
|
|
|
|
## Wave 0 Requirements
|
|
|
|
- [ ] `tests/test_script_generator.py` — stubs for SCRPT-01 through SCRPT-05 (unit + integration)
|
|
- [ ] `imptune/templates/scripts/install.ps1.j2` — Jinja2 template file
|
|
- [ ] `imptune/templates/scripts/uninstall.ps1.j2` — Jinja2 template file
|
|
- [ ] `imptune/templates/scripts/detect.ps1.j2` — Jinja2 template file
|
|
- [ ] `imptune/generators/script_generator.py` — render functions
|
|
- [ ] `imptune/api/scripts.py` — FastAPI router
|
|
|
|
---
|
|
|
|
## Manual-Only Verifications
|
|
|
|
| Behavior | Requirement | Why Manual | Test Instructions |
|
|
|----------|-------------|------------|-------------------|
|
|
| Install script runs on real Windows endpoint via Intune | SCRPT-01 | Requires real Intune + endpoint | Deploy .intunewin package to test device, verify printer appears |
|
|
| UAC elevation prompt appears for standard user | SCRPT-04 | Requires interactive desktop session | Run install.ps1 as standard user, verify UAC dialog |
|
|
| WOW64 relaunch works in 32-bit PS | SCRPT-05 | Requires 32-bit PowerShell host | Launch powershell.exe (x86), run install.ps1, verify relaunch |
|
|
|
|
---
|
|
|
|
## Nyquist Record
|
|
|
|
> Audited 2026-04-13 by Claude (gsd-executor, plan 08-04). One row per Phase 4 success criterion derived from `milestones/v1.0-ROADMAP.md` Phase 4 goal + plan outcomes (SCRPT-01..05), cross-checked against `04-VERIFICATION.md` (12/12 observable truths verified 2026-04-10) and `REQUIREMENTS.md` v1.0 SCRPT-0x block. Evidence cites committed tests, source lines, the dated VERIFICATION report, and — for rows whose proof requires real-device SYSTEM-context execution — the Phase 10 `RUNTIME-VALIDATION.md` report with explicit attestation-only caveats per STATE.md 2026-04-13.
|
|
>
|
|
> **Phase 4 goal (v1.0-ROADMAP.md):** *"System produces correct, production-ready PowerShell scripts handling all Intune and RMM execution contexts."*
|
|
>
|
|
> **Attestation-only caveat (STATE.md 2026-04-13):** Phase 10 RTVAL-02 (install on real endpoint), RTVAL-03 (detection script on real endpoint), and RTVAL-04 (uninstall on real endpoint) were accepted as **attestation-only PASSes** — the technician verbally confirmed success but did not produce IntuneManagementExtension.log excerpts, portal screenshots, or status captures. The user was warned twice about cumulative audit-trail damage and explicitly approved proceeding. Plan 10-03 closed the phase with this gap acknowledged in writing. Rows below that depend on SYSTEM-context runtime proof therefore record `pass` (Phase 10 signed off) but the Notes column states the weakened audit trail faithfully — this audit does not hide it.
|
|
|
|
| # | Success Criterion | Observable Check | Evidence | Status | Notes |
|
|
|---|-------------------|------------------|----------|--------|-------|
|
|
| 1 | **SCRPT-01** — Generate PowerShell install script (pnputil staging + Add-PrinterPort + Add-PrinterDriver + Add-Printer + Set-PrintConfiguration) | `pytest tests/test_script_generator.py::test_render_install_contains_pnputil` + `::test_render_install_print_config` + `::test_install_endpoint` — unit tests assert all 5 cmdlets appear in rendered template; integration test asserts `GET /printers/{id}/scripts/install` returns 200 PowerShell content with pnputil present | `tests/test_script_generator.py::test_render_install_contains_pnputil`, `::test_render_install_print_config`, `::test_install_endpoint`; `imptune/templates/scripts/install.ps1.j2` lines 40-67 (pnputil `/add-driver` + Add-PrinterPort + Add-PrinterDriver + Add-Printer + Set-PrintConfiguration); `imptune/generators/script_generator.py` `_duplex_map` + `render_install` (commits b4f2c64 RED, 8193e9d GREEN); `imptune/api/scripts.py` lines 38-59; 04-VERIFICATION.md truths 1 + 4 + 5 + 8; Phase 10 `RUNTIME-VALIDATION.md` RTVAL-02 (install succeeded on ARES-5CG5220YTM) | pass | **SYSTEM-context runtime proof is attestation-only per STATE.md 2026-04-13.** pnputil staging + $PSScriptRoot resolution under the real Intune SYSTEM context were confirmed verbally by the technician for RTVAL-02 but no IntuneManagementExtension.log excerpt or portal screenshot was captured. Phase 10 signed off the gap; rollout Phase 11 owns re-capture of full artifacts. Template-level correctness (cmdlet presence, positional ordering, duplex mapping) is fully automated via pytest. |
|
|
| 2 | **SCRPT-02** — Generate PowerShell uninstall script (Remove-Printer + Remove-PrinterDriver + Remove-PrinterPort in correct order) | `pytest tests/test_script_generator.py::test_render_uninstall` + `::test_uninstall_endpoint` — asserts all 3 Remove-* cmdlets appear in correct order (Printer → Driver → Port) with `-ErrorAction SilentlyContinue` on each; integration test asserts endpoint returns 200 | `tests/test_script_generator.py::test_render_uninstall`, `::test_uninstall_endpoint`; `imptune/templates/scripts/uninstall.ps1.j2` lines 2-4; `imptune/generators/script_generator.py::render_uninstall` line 70 (commit 6bff8f3); `imptune/api/scripts.py` lines 63-78 (commit b7b0d1b); 04-VERIFICATION.md truth 6 + truth 9; Phase 10 `RUNTIME-VALIDATION.md` RTVAL-04 (uninstall succeeded on real endpoint) | pass | **SYSTEM-context runtime proof is attestation-only per STATE.md 2026-04-13.** RTVAL-04 is the **third consecutive attestation-only** Phase 10 check — no `rtval-04-uninstall-log.txt` and no `rtval-04-uninstall-status.png` were captured. Template-level ordering and `-ErrorAction SilentlyContinue` safety are fully automated via pytest; real-device Remove-Printer behavior under SYSTEM rests on verbal technician confirmation only. |
|
|
| 3 | **SCRPT-03** — Generate Intune detection script (exit 0 when printer present, exit 1 when absent, with Write-Output on success) | `pytest tests/test_script_generator.py::test_render_detect` + `::test_detect_endpoint` — asserts `Get-Printer` check + `Write-Output` + `exit 0` on found branch + `exit 1` on absent branch; integration test asserts endpoint returns 200 | `tests/test_script_generator.py::test_render_detect`, `::test_detect_endpoint`; `imptune/templates/scripts/detect.ps1.j2` lines 2-8; `imptune/generators/script_generator.py::render_detect` line 92 (commit 6bff8f3); `imptune/api/scripts.py` lines 82-94; 04-VERIFICATION.md truth 7 + truth 10; `.planning/phases/04-script-generation/04-RESEARCH.md` State-of-the-Art table (Get-Printer cmdlet chosen over HKLM registry path as more reliable); Phase 10 `RUNTIME-VALIDATION.md` RTVAL-03 (Intune detection script evaluated as installed) | pass | **Documented deviation from REQUIREMENTS.md wording.** REQUIREMENTS.md says "registry check" but 04-RESEARCH.md supersedes with `Get-Printer` cmdlet — explicitly documented as more reliable before implementation. The functional Intune contract (Write-Output + exit 0 when present, exit 1 when absent) is correctly satisfied. **SYSTEM-context runtime proof is attestation-only per STATE.md 2026-04-13** — RTVAL-03 is the second consecutive attestation-only Phase 10 check; no `rtval-03-detection.png` or `rtval-03-detect-manual.txt` was captured. Real Intune evaluator behavior confirmed verbally only. |
|
|
| 4 | **SCRPT-04** — Install script detects SYSTEM vs user context and self-elevates via UAC when run by user | `pytest tests/test_script_generator.py::test_render_install_uac_guard` — asserts `WindowsIdentity::GetCurrent()`, `IsSystem` check, `IsInRole(Administrator)` check, and `Start-Process -Verb Runas` all present in rendered install template | `tests/test_script_generator.py::test_render_install_uac_guard`; `imptune/templates/scripts/install.ps1.j2` lines 22-33 (SYSTEM identity check + admin role check + self-elevation branch); 04-VERIFICATION.md truth 3; Phase 10 `RUNTIME-VALIDATION.md` RTVAL-02 (install succeeded under Intune SYSTEM context on ARES-5CG5220YTM — UAC guard correctly skipped elevation) | pass | **SYSTEM-context runtime proof is attestation-only per STATE.md 2026-04-13.** The `IsSystem` branch (skip elevation when run by Intune Management Extension as SYSTEM) was exercised in the attestation-only RTVAL-02 run. The user-interactive self-elevation branch (Start-Process -Verb Runas triggering a real UAC dialog for a standard user) is flagged as a `Manual-Only Verification` above and **was not exercised in Phase 10** (RTVAL only covered the Intune SYSTEM path, not standalone standard-user execution). Template-level correctness (both branches present, identity check first) is automated via pytest. |
|
|
| 5 | **SCRPT-05** — Install script includes 64-bit WOW64 relaunch guard for Intune's 32-bit execution context | `pytest tests/test_script_generator.py::test_render_install_wow64_guard` — positional assertion: `PROCESSOR_ARCHITECTURE` + `PROCESSOR_ARCHITEW6432` + `SysNative` relaunch block appears **before** the pnputil block in rendered install template (guard must be first executable block) | `tests/test_script_generator.py::test_render_install_wow64_guard`; `imptune/templates/scripts/install.ps1.j2` lines 12-16 (WOW64 guard) preceding lines 40+ (pnputil); 04-VERIFICATION.md truth 2; Phase 10 `RUNTIME-VALIDATION.md` RTVAL-02 (install succeeded end-to-end under Intune on 64-bit Windows) | pass | **SYSTEM-context runtime proof is attestation-only per STATE.md 2026-04-13.** The WOW64 relaunch path (Intune's 32-bit PS host → SysNative 64-bit relaunch → continue execution) is **not directly observable** from RTVAL-02's attestation-only confirmation — the technician only attested the printer installed, not that the WOW64 branch was taken. This check remains a `Manual-Only Verification` pending a real 32-bit PowerShell host trace. Template-level positional correctness (guard before pnputil) is fully automated via pytest. Row recorded as `pass` because Phase 10 signed off end-to-end install; full WOW64 trace is a Phase 11 rollout concern. |
|
|
|
|
**Audit outcome:** 5/5 rows `pass`. No `fail-fix-v1.1`, `deferred-v1.2`, or `wont-do` rows. Phase 4 is Nyquist-compliant *at the template level* — every SCRPT-0x success criterion has exactly one observable check with cited, committed evidence. **However**, SYSTEM-context runtime behavior (pnputil staging under SYSTEM, `$PSScriptRoot` resolution under SYSTEM, detect/uninstall under SYSTEM, WOW64 relaunch in real 32-bit Intune host) rests on attestation-only Phase 10 PASSes per STATE.md 2026-04-13. This audit records the weakened runtime audit trail faithfully in the Notes column rather than flipping rows to `fail-fix-v1.1` — Phase 10 signed off with explicit written acknowledgement of the attestation gap, and Phase 11 (Real-World Rollout) owns artifact re-capture before broad rollout. Zero gaps carry forward into 08-08 (rollup) beyond what STATE.md already tracks.
|
|
|
|
---
|
|
|
|
## Validation Sign-Off
|
|
|
|
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
|
|
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
|
|
- [ ] Wave 0 covers all MISSING references
|
|
- [ ] No watch-mode flags
|
|
- [ ] Feedback latency < 10s
|
|
- [ ] `nyquist_compliant: true` set in frontmatter
|
|
|
|
**Approval:** pending
|