docs(phase-10): create runtime validation phase plan
This commit is contained in:
+156
@@ -0,0 +1,156 @@
|
||||
---
|
||||
phase: 10-real-world-runtime-validation
|
||||
plan: 01
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md
|
||||
- .planning/phases/10-real-world-runtime-validation/evidence/.gitkeep
|
||||
autonomous: false
|
||||
requirements:
|
||||
- RTVAL-05
|
||||
must_haves:
|
||||
truths:
|
||||
- "A real .intunewin package generated by the current ImpTune build exists on disk and its SHA256 is recorded"
|
||||
- "A RUNTIME-VALIDATION.md scaffold exists with tenant, device, OS build, and driver vendor fields ready to fill"
|
||||
- "An evidence/ folder exists under the phase directory for screenshots and logs"
|
||||
artifacts:
|
||||
- path: ".planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md"
|
||||
provides: "Scaffolded evidence report with metadata table and empty result sections"
|
||||
contains: "## Tenant & Environment"
|
||||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/.gitkeep"
|
||||
provides: "Evidence folder placeholder so git tracks the directory"
|
||||
key_links:
|
||||
- from: "imptune package export UI"
|
||||
to: "RUNTIME-VALIDATION.md metadata"
|
||||
via: "technician records printer name, driver vendor, SHA256, build commit"
|
||||
pattern: "Package SHA256:.*[a-f0-9]{64}"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Produce the artifact under test and the evidence scaffold before any real-world testing begins.
|
||||
|
||||
Purpose: You cannot validate what you did not generate. Lock the exact commit, package, and metadata that will be sent to the tenant so the final report is reproducible.
|
||||
Output: A real .intunewin package generated by the running ImpTune instance and a RUNTIME-VALIDATION.md scaffold with all environment fields waiting to be filled in.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@C:/Users/SebastienQUEROL/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@C:/Users/SebastienQUEROL/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/REQUIREMENTS.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Scaffold RUNTIME-VALIDATION.md and evidence folder</name>
|
||||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/.gitkeep</files>
|
||||
<action>
|
||||
Create `RUNTIME-VALIDATION.md` under the phase directory with the following sections (empty values to be filled by technician in later tasks):
|
||||
|
||||
```
|
||||
# Runtime Validation Report — ImpTune v1.1
|
||||
|
||||
**Status:** DRAFT (in progress)
|
||||
**Started:** <YYYY-MM-DD>
|
||||
**Signed off by:** <name>
|
||||
**Signed off date:** <YYYY-MM-DD>
|
||||
|
||||
## Tenant & Environment
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Intune tenant (domain) | |
|
||||
| Test device hostname | |
|
||||
| Windows OS build | |
|
||||
| Device is Intune-managed | yes / no |
|
||||
| Driver vendor(s) tested | |
|
||||
| ImpTune commit SHA | |
|
||||
| Package file name | |
|
||||
| Package SHA256 | |
|
||||
| Generated at | |
|
||||
|
||||
## RTVAL-01 — Tenant ingestion
|
||||
Status: PENDING
|
||||
Evidence: (screenshot path)
|
||||
Notes:
|
||||
|
||||
## RTVAL-02 — Install under SYSTEM (pnputil + $PSScriptRoot)
|
||||
Status: PENDING
|
||||
Evidence: (IntuneManagementExtension.log excerpt path)
|
||||
Notes:
|
||||
|
||||
## RTVAL-03 — Detection rule reports Installed
|
||||
Status: PENDING
|
||||
Evidence: (screenshot path)
|
||||
Notes:
|
||||
|
||||
## RTVAL-04 — Uninstall under SYSTEM
|
||||
Status: PENDING
|
||||
Evidence: (log path + screenshot)
|
||||
Notes:
|
||||
|
||||
## Issues Found
|
||||
(list any defects; link to v1.1 tickets or defer rationale)
|
||||
|
||||
## Sign-off
|
||||
- [ ] All five RTVAL criteria PASS or have documented rationale
|
||||
- [ ] Evidence files committed under `evidence/`
|
||||
- [ ] Signed by: ___________________
|
||||
```
|
||||
|
||||
Also create `.planning/phases/10-real-world-runtime-validation/evidence/.gitkeep` as an empty file so git tracks the evidence directory.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>test -f .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md && test -f .planning/phases/10-real-world-runtime-validation/evidence/.gitkeep && grep -q "RTVAL-01" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md && grep -q "Sign-off" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md</automated>
|
||||
</verify>
|
||||
<done>RUNTIME-VALIDATION.md scaffold exists with all five RTVAL sections and a sign-off block; evidence/ folder exists in git.</done>
|
||||
</task>
|
||||
|
||||
<task type="checkpoint:human-action" gate="blocking">
|
||||
<name>Task 2: Technician generates a real .intunewin package from running ImpTune</name>
|
||||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md</files>
|
||||
<action>
|
||||
PAUSE for human technician. Claude cannot run Docker + click through the export UI + read a local file's hash on the user's box. The executor agent must present the following instructions and wait.
|
||||
|
||||
Technician MUST perform the following:
|
||||
|
||||
1. Start ImpTune from the current commit (`docker compose up` or the local dev command).
|
||||
2. Record the current commit SHA: `git rev-parse HEAD` and paste into RUNTIME-VALIDATION.md "ImpTune commit SHA".
|
||||
3. In the UI, pick or create a test printer using a driver vendor representative of the target MSP environment (e.g., HP Universal, Konica Minolta, Brother). Record the vendor in the report.
|
||||
4. Export the package via the package export button. Save the resulting `.intunewin` file locally.
|
||||
5. Compute its SHA256: `certutil -hashfile <file>.intunewin SHA256` (Windows) or `sha256sum <file>.intunewin` (Linux/WSL). Paste into report.
|
||||
6. Fill in tenant domain, test device hostname, Windows OS build (`winver` on device), and generation timestamp in the report metadata table.
|
||||
7. Copy the generated `.intunewin` file into `.planning/phases/10-real-world-runtime-validation/evidence/` (or note its archived location if too large for git).
|
||||
|
||||
Expected outcome: RUNTIME-VALIDATION.md metadata table is FULLY populated before proceeding to plan 10-02.
|
||||
|
||||
Resume signal: reply "metadata filled" (with the package SHA256) or describe blockers.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>! grep -E "^\| (Intune tenant|Test device hostname|Windows OS build|Driver vendor\(s\) tested|ImpTune commit SHA|Package file name|Package SHA256|Generated at) \| *\|" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md</automated>
|
||||
</verify>
|
||||
<done>Every row in the Tenant & Environment metadata table has a non-empty value; technician has replied "metadata filled".</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- RUNTIME-VALIDATION.md exists with scaffold
|
||||
- Metadata table populated (commit SHA, tenant, device, OS build, vendor, package SHA256)
|
||||
- A real .intunewin file is generated and located (path or archive recorded)
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
Technician signals "metadata filled"; report metadata table has no empty fields; package file exists and its SHA256 is recorded.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/10-real-world-runtime-validation/10-01-preflight-package-and-scaffold-SUMMARY.md`
|
||||
</output>
|
||||
+207
@@ -0,0 +1,207 @@
|
||||
---
|
||||
phase: 10-real-world-runtime-validation
|
||||
plan: 02
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on:
|
||||
- 10-01
|
||||
files_modified:
|
||||
- .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md
|
||||
- .planning/phases/10-real-world-runtime-validation/evidence/
|
||||
autonomous: false
|
||||
requirements:
|
||||
- RTVAL-01
|
||||
- RTVAL-02
|
||||
- RTVAL-03
|
||||
- RTVAL-04
|
||||
must_haves:
|
||||
truths:
|
||||
- "A real Intune tenant accepted the generated .intunewin without format errors (RTVAL-01)"
|
||||
- "A real Intune-managed Windows endpoint ran the install script under SYSTEM with pnputil staging drivers and $PSScriptRoot resolving (RTVAL-02)"
|
||||
- "Intune detection rule reports 'Installed' on the endpoint after install (RTVAL-03)"
|
||||
- "Intune uninstall removes the printer cleanly under SYSTEM context (RTVAL-04)"
|
||||
artifacts:
|
||||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/rtval-01-tenant-upload.png"
|
||||
provides: "Screenshot of Intune showing successful package upload"
|
||||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/rtval-02-install-log.txt"
|
||||
provides: "IntuneManagementExtension.log excerpt showing install success + pnputil + $PSScriptRoot"
|
||||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/rtval-03-detection.png"
|
||||
provides: "Screenshot of Intune device install status = Installed"
|
||||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/rtval-04-uninstall-log.txt"
|
||||
provides: "Log + printer-list verification showing printer removed under SYSTEM"
|
||||
key_links:
|
||||
- from: "Intune tenant"
|
||||
to: "generated .intunewin from plan 10-01"
|
||||
via: "Win32 app upload"
|
||||
pattern: "Package (uploaded|accepted)"
|
||||
- from: "install script ($PSScriptRoot)"
|
||||
to: "bundled driver folder inside .intunewin"
|
||||
via: "pnputil /add-driver staging under SYSTEM"
|
||||
pattern: "pnputil.*Published Name|Driver package added successfully"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Drive the four real-world RTVAL behavioral checks end-to-end on a live Intune tenant against a real Windows endpoint, and capture hard evidence (screenshots + device logs) for each.
|
||||
|
||||
Purpose: This is the phase's whole reason for existing. Every check here is a behavior no unit test can prove — it has to happen on real hardware with a real tenant.
|
||||
Output: evidence/ folder populated with RTVAL-01..04 artifacts and RUNTIME-VALIDATION.md status lines flipped from PENDING to PASS/FAIL.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@C:/Users/SebastienQUEROL/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@C:/Users/SebastienQUEROL/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md
|
||||
@.planning/REQUIREMENTS.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="checkpoint:human-action" gate="blocking">
|
||||
<name>Task 1: RTVAL-01 — Upload .intunewin to real Intune tenant</name>
|
||||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/rtval-01-tenant-upload.png, .planning/phases/10-real-world-runtime-validation/evidence/rtval-01-app-assigned.png</files>
|
||||
<action>
|
||||
PAUSE for human technician. Requires Intune tenant admin credentials — no complete free CLI path exists.
|
||||
|
||||
Technician MUST:
|
||||
|
||||
1. Sign in to https://intune.microsoft.com with tenant admin.
|
||||
2. Go to Apps → Windows → Add → App type: Windows app (Win32).
|
||||
3. Upload the `.intunewin` file from plan 10-01.
|
||||
4. OBSERVE: Intune parses the file and presents the app metadata form without format errors.
|
||||
5. Fill in minimal app info (name = "ImpTune RTVAL Test - <vendor>"), install command, uninstall command, detection rule (use the generated detect script: File exists / custom script as produced by ImpTune), requirement = Windows 10/11 x64, assignment = the single test device only.
|
||||
6. Save the app. OBSERVE: app appears in the apps list with "Waiting for install status" or similar.
|
||||
7. Capture screenshots:
|
||||
- `evidence/rtval-01-tenant-upload.png` — upload success page showing the parsed .intunewin metadata
|
||||
- `evidence/rtval-01-app-assigned.png` — app assignment screen showing the test device group
|
||||
8. Update RUNTIME-VALIDATION.md section "RTVAL-01 — Tenant ingestion":
|
||||
- Status: PASS if tenant accepted the package with NO format errors. FAIL otherwise.
|
||||
- Evidence: paths to the two screenshots.
|
||||
- Notes: any warnings Intune surfaced.
|
||||
|
||||
PASS: Intune accepted the upload and allowed assignment WITHOUT a format/extraction error.
|
||||
FAIL: tenant rejected the file, surfaced a parse error, or could not load detection metadata — record the exact error text verbatim in Notes.
|
||||
|
||||
Resume signal: reply "RTVAL-01 PASS" + screenshot paths, or "RTVAL-01 FAIL" with exact tenant error text.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>test -f .planning/phases/10-real-world-runtime-validation/evidence/rtval-01-tenant-upload.png && ! grep -A1 "RTVAL-01 — Tenant ingestion" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md | grep -q "Status: PENDING"</automated>
|
||||
</verify>
|
||||
<done>RTVAL-01 status is PASS or FAIL in the report; tenant upload screenshot committed; any FAIL has verbatim error text in Notes.</done>
|
||||
</task>
|
||||
|
||||
<task type="checkpoint:human-action" gate="blocking">
|
||||
<name>Task 2: RTVAL-02 — Install on device under SYSTEM; verify pnputil + $PSScriptRoot</name>
|
||||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/rtval-02-install-log.txt, .planning/phases/10-real-world-runtime-validation/evidence/rtval-02-install-status.png</files>
|
||||
<action>
|
||||
PAUSE for human technician. Requires physical/virtual access to the test Intune-managed Windows endpoint.
|
||||
|
||||
Technician MUST:
|
||||
|
||||
1. On the test device, force Intune sync: Settings → Accounts → Access work or school → Info → Sync (or restart `IntuneManagementExtension`).
|
||||
2. Wait for Intune to push the assignment. Typical latency 5–30 minutes.
|
||||
3. While install runs, open `C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\IntuneManagementExtension.log` and `AgentExecutor.log`.
|
||||
4. After install completes, extract the block covering this app's install run and save as `evidence/rtval-02-install-log.txt`.
|
||||
5. OBSERVE in the log — every item below MUST be visible:
|
||||
a. The install script ran under `NT AUTHORITY\SYSTEM` (script banner or process token).
|
||||
b. `$PSScriptRoot` resolved to the Intune app staging directory (not empty, not `C:\Windows\system32`).
|
||||
c. `pnputil /add-driver` (or equivalent) executed against the bundled driver folder and reported success (`Driver package added successfully` or `Published Name:`).
|
||||
d. Final script exit code is 0.
|
||||
6. Also capture Intune portal view: Apps → the test app → Device install status → test device → Install status = "Installed". Screenshot as `evidence/rtval-02-install-status.png`.
|
||||
7. Update RUNTIME-VALIDATION.md section "RTVAL-02":
|
||||
- Status: PASS only if ALL four observations hold. Otherwise FAIL with which observation failed.
|
||||
- Evidence: both file paths.
|
||||
|
||||
PASS: SYSTEM context + $PSScriptRoot resolved + pnputil staged drivers + exit 0.
|
||||
FAIL: ANY of the above missing. Record the exact log excerpt showing the failure in Notes; open a defect ticket in "Issues Found".
|
||||
|
||||
Resume signal: reply "RTVAL-02 PASS" + log path, or "RTVAL-02 FAIL" with failing observation and excerpt.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>test -f .planning/phases/10-real-world-runtime-validation/evidence/rtval-02-install-log.txt && ! grep -A1 "RTVAL-02 — Install under SYSTEM" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md | grep -q "Status: PENDING"</automated>
|
||||
</verify>
|
||||
<done>RTVAL-02 status is PASS or FAIL in the report; install log committed; for PASS the log shows SYSTEM, pnputil success, $PSScriptRoot resolved, and exit 0.</done>
|
||||
</task>
|
||||
|
||||
<task type="checkpoint:human-action" gate="blocking">
|
||||
<name>Task 3: RTVAL-03 — Confirm Intune detection rule reports Installed</name>
|
||||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/rtval-03-detection.png, .planning/phases/10-real-world-runtime-validation/evidence/rtval-03-detect-manual.txt</files>
|
||||
<action>
|
||||
PAUSE for human technician.
|
||||
|
||||
Technician MUST:
|
||||
|
||||
1. In Intune portal: Apps → the test app → Device install status → test device row → confirm "Installation status" = **Installed** (this is driven by the generated detect script).
|
||||
2. On the endpoint itself, confirm the printer is present:
|
||||
- PowerShell (user context OK for verification only): `Get-Printer | Where-Object Name -eq "<printer name>"` returns a row.
|
||||
- OR open Settings → Bluetooth & devices → Printers & scanners → confirm printer visible.
|
||||
3. Run the generated detect script manually once to cross-check:
|
||||
- Open admin PowerShell and execute the detect script; confirm exit code 0.
|
||||
4. Screenshot Intune "Installed" status as `evidence/rtval-03-detection.png`.
|
||||
5. Save detect script manual run transcript as `evidence/rtval-03-detect-manual.txt`.
|
||||
6. Update RUNTIME-VALIDATION.md section "RTVAL-03":
|
||||
- Status: PASS if Intune shows Installed AND manual detect script exits 0 AND printer is visible. Otherwise FAIL.
|
||||
|
||||
PASS: Intune "Installed" + printer visible + detect script exit 0.
|
||||
FAIL: Intune shows "Not detected", manual detect non-zero, or printer missing. Record in Notes.
|
||||
|
||||
Resume signal: reply "RTVAL-03 PASS" + evidence paths, or "RTVAL-03 FAIL" with observed status.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>test -f .planning/phases/10-real-world-runtime-validation/evidence/rtval-03-detection.png && ! grep -A1 "RTVAL-03 — Detection rule" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md | grep -q "Status: PENDING"</automated>
|
||||
</verify>
|
||||
<done>RTVAL-03 status is PASS or FAIL; detection screenshot and manual-run transcript committed.</done>
|
||||
</task>
|
||||
|
||||
<task type="checkpoint:human-action" gate="blocking">
|
||||
<name>Task 4: RTVAL-04 — Intune uninstall removes printer cleanly under SYSTEM</name>
|
||||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/rtval-04-uninstall-log.txt, .planning/phases/10-real-world-runtime-validation/evidence/rtval-04-uninstall-status.png</files>
|
||||
<action>
|
||||
PAUSE for human technician.
|
||||
|
||||
Technician MUST:
|
||||
|
||||
1. In Intune portal: change the app assignment for the test device from Required to Uninstall (or remove assignment and add as Uninstall).
|
||||
2. Force Intune sync on the device again. Wait for the uninstall to run.
|
||||
3. Extract the uninstall run from `IntuneManagementExtension.log` and save as `evidence/rtval-04-uninstall-log.txt`.
|
||||
4. OBSERVE in the log:
|
||||
a. Uninstall script ran under `NT AUTHORITY\SYSTEM`.
|
||||
b. Final script exit code 0.
|
||||
5. On the endpoint, confirm the printer is GONE:
|
||||
- `Get-Printer | Where-Object Name -eq "<printer name>"` returns nothing.
|
||||
- OR Settings → Printers & scanners → printer no longer listed.
|
||||
6. Confirm Intune portal shows "Not installed" for the test device.
|
||||
7. Screenshot as `evidence/rtval-04-uninstall-status.png`.
|
||||
8. Update RUNTIME-VALIDATION.md section "RTVAL-04":
|
||||
- Status: PASS if SYSTEM context + exit 0 + printer gone + Intune "Not installed". Otherwise FAIL with which check failed.
|
||||
|
||||
PASS: clean removal under SYSTEM, zero leftover printer artifacts.
|
||||
FAIL: printer still present, non-zero exit, or tenant shows install failure. Record in Notes.
|
||||
|
||||
Resume signal: reply "RTVAL-04 PASS" + evidence paths, or "RTVAL-04 FAIL" with failing check.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>test -f .planning/phases/10-real-world-runtime-validation/evidence/rtval-04-uninstall-log.txt && ! grep -A1 "RTVAL-04 — Uninstall under SYSTEM" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md | grep -q "Status: PENDING"</automated>
|
||||
</verify>
|
||||
<done>RTVAL-04 status is PASS or FAIL; uninstall log and status screenshot committed; printer confirmed removed on the device for PASS.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- evidence/rtval-01-tenant-upload.png exists
|
||||
- evidence/rtval-02-install-log.txt exists and shows SYSTEM + pnputil + $PSScriptRoot
|
||||
- evidence/rtval-03-detection.png exists and shows Installed
|
||||
- evidence/rtval-04-uninstall-log.txt exists and shows clean removal
|
||||
- RUNTIME-VALIDATION.md status lines for RTVAL-01..04 are PASS or FAIL (no PENDING)
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
All four RTVAL checkpoint tasks have signaled a final status (PASS or FAIL) with evidence files committed to evidence/. Any FAIL has a corresponding "Issues Found" entry in the report with defect ticket or deferral rationale.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/10-real-world-runtime-validation/10-02-live-intune-runtime-validation-SUMMARY.md`
|
||||
</output>
|
||||
@@ -0,0 +1,144 @@
|
||||
---
|
||||
phase: 10-real-world-runtime-validation
|
||||
plan: 03
|
||||
type: execute
|
||||
wave: 3
|
||||
depends_on:
|
||||
- 10-01
|
||||
- 10-02
|
||||
files_modified:
|
||||
- .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md
|
||||
- .planning/REQUIREMENTS.md
|
||||
- .planning/ROADMAP.md
|
||||
autonomous: false
|
||||
requirements:
|
||||
- RTVAL-05
|
||||
must_haves:
|
||||
truths:
|
||||
- "A reviewer can open RUNTIME-VALIDATION.md and read a complete signed-off report with tenant, device, OS build, driver vendor, screenshots/logs, and any issues"
|
||||
- "Every RTVAL-01..04 result is PASS/FAIL with cited evidence (no PENDING)"
|
||||
- "Any FAIL has either a v1.1 defect ticket or a written deferral rationale"
|
||||
- "REQUIREMENTS.md and ROADMAP.md mark Phase 10 complete"
|
||||
artifacts:
|
||||
- path: ".planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md"
|
||||
provides: "Final signed-off runtime validation report"
|
||||
contains: "Signed off by:"
|
||||
key_links:
|
||||
- from: "RUNTIME-VALIDATION.md Sign-off checklist"
|
||||
to: "evidence/*.png and evidence/*.txt files from plan 10-02"
|
||||
via: "markdown relative links"
|
||||
pattern: "\\]\\(evidence/.*\\)"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Convert the draft RUNTIME-VALIDATION.md into a signed-off, reviewable report and update tracking docs so Phase 10 can be marked complete.
|
||||
|
||||
Purpose: RTVAL-05 requires a reviewer-grade report, not just scattered evidence files. This plan closes the loop.
|
||||
Output: A final RUNTIME-VALIDATION.md committed to the repo plus REQUIREMENTS.md/ROADMAP.md ticks.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@C:/Users/SebastienQUEROL/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@C:/Users/SebastienQUEROL/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md
|
||||
@.planning/REQUIREMENTS.md
|
||||
@.planning/ROADMAP.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Finalize RUNTIME-VALIDATION.md report body</name>
|
||||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md</files>
|
||||
<action>
|
||||
Read the current RUNTIME-VALIDATION.md (populated by plans 10-01 and 10-02). Then:
|
||||
|
||||
1. For each RTVAL-01..04 section, ensure:
|
||||
- Status line is PASS or FAIL (not PENDING).
|
||||
- Evidence paths use markdown relative links like `[screenshot](evidence/rtval-01-tenant-upload.png)`.
|
||||
- Notes field has at least one sentence (either "no anomalies" or description of issues).
|
||||
2. For each FAIL, confirm there is a corresponding entry under "## Issues Found" formatted as:
|
||||
```
|
||||
- **[RTVAL-0X]** <short description> — <defect ticket ID or "Deferred to v1.2: <rationale>">
|
||||
```
|
||||
If a FAIL has no Issues Found entry yet, add one noting "TODO: triage — see Task 2 checkpoint".
|
||||
3. Update the "Status" field at the top from `DRAFT (in progress)` to `READY FOR SIGN-OFF`.
|
||||
|
||||
Do NOT fabricate results. If any section is still PENDING because plan 10-02 did not actually run, STOP and return an error to the executor — 10-03 must not be run before 10-02 is complete.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>! grep -q "Status: PENDING" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md && grep -q "READY FOR SIGN-OFF\|SIGNED OFF" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md</automated>
|
||||
</verify>
|
||||
<done>Report has zero PENDING status lines, all evidence links are markdown-valid, Issues Found is consistent with FAILs, and top-level status is READY FOR SIGN-OFF.</done>
|
||||
</task>
|
||||
|
||||
<task type="checkpoint:human-verify" gate="blocking">
|
||||
<name>Task 2: Human review and sign-off</name>
|
||||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md</files>
|
||||
<action>
|
||||
PAUSE for human reviewer. The reviewer (user acting as technician/owner) MUST:
|
||||
|
||||
1. Open `.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md`.
|
||||
2. Walk every RTVAL-01..04 section and click every evidence link. Confirm screenshots open and log excerpts are readable.
|
||||
3. For every FAIL, confirm the "Issues Found" entry is present AND has either:
|
||||
- a v1.1 defect ticket ID (open a ticket if needed), OR
|
||||
- an explicit deferral to v1.2 with written rationale.
|
||||
4. If satisfied, edit the report's Sign-off block:
|
||||
- Change `Signed off by:` to the reviewer's name.
|
||||
- Change `Signed off date:` to today's date.
|
||||
- Tick the three sign-off checkboxes.
|
||||
- Change the top-level Status from `READY FOR SIGN-OFF` to `SIGNED OFF`.
|
||||
5. If NOT satisfied, describe exactly which section is lacking and which additional evidence is required. The plan loops back to plan 10-02 (or a gap-closure plan).
|
||||
|
||||
Resume signal: reply "signed off" (after editing the file) or describe missing evidence.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>grep -q "^\*\*Status:\*\* SIGNED OFF" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md</automated>
|
||||
</verify>
|
||||
<done>Report top-level status = SIGNED OFF; Signed off by/date populated; all three sign-off checkboxes ticked.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 3: Tick RTVAL requirements and mark Phase 10 complete in tracking docs</name>
|
||||
<files>.planning/REQUIREMENTS.md, .planning/ROADMAP.md</files>
|
||||
<action>
|
||||
Only run after Task 2 signals "signed off". Verify by grepping the report for `Status: SIGNED OFF` — if absent, STOP.
|
||||
|
||||
Then:
|
||||
|
||||
1. In `.planning/REQUIREMENTS.md`:
|
||||
- Change `- [ ] **RTVAL-01**` through `- [ ] **RTVAL-05**` to `- [x]`.
|
||||
- In the Traceability table, change status for RTVAL-01..05 from `Pending` to `Complete`. If any RTVAL ended FAIL and was deferred, mark it `Deferred (v1.2)` instead and add a footnote link to the report's Issues Found entry.
|
||||
|
||||
2. In `.planning/ROADMAP.md`:
|
||||
- Change `- [ ] **Phase 10: Real-World Runtime Validation**` to `- [x]` and append `(completed <today>)`.
|
||||
- In the Progress table row for Phase 10, set `Plans Complete` to `3/3`, `Status` to `Complete`, and `Completed` to today's date.
|
||||
|
||||
Do not touch unrelated rows. Preserve existing formatting exactly.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>grep -q "\[x\] \*\*RTVAL-05\*\*" .planning/REQUIREMENTS.md && grep -q "\[x\] \*\*Phase 10" .planning/ROADMAP.md</automated>
|
||||
</verify>
|
||||
<done>REQUIREMENTS.md has RTVAL-01..05 ticked (or explicitly deferred); ROADMAP.md Phase 10 ticked with completion date and 3/3 plans.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- RUNTIME-VALIDATION.md top-level status = SIGNED OFF
|
||||
- Sign-off name + date populated
|
||||
- Zero PENDING status lines
|
||||
- REQUIREMENTS.md RTVAL-01..05 ticked
|
||||
- ROADMAP.md Phase 10 ticked
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
A fresh reviewer can clone the repo, open RUNTIME-VALIDATION.md, follow every evidence link, and reach the same PASS/FAIL verdicts. Tracking docs reflect phase completion.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/10-real-world-runtime-validation/10-03-report-signoff-SUMMARY.md`
|
||||
</output>
|
||||
Reference in New Issue
Block a user