208 lines
13 KiB
Markdown
208 lines
13 KiB
Markdown
---
|
||
phase: 10-real-world-runtime-validation
|
||
plan: 02
|
||
type: execute
|
||
wave: 2
|
||
depends_on:
|
||
- 10-01
|
||
files_modified:
|
||
- .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md
|
||
- .planning/phases/10-real-world-runtime-validation/evidence/
|
||
autonomous: false
|
||
requirements:
|
||
- RTVAL-01
|
||
- RTVAL-02
|
||
- RTVAL-03
|
||
- RTVAL-04
|
||
must_haves:
|
||
truths:
|
||
- "A real Intune tenant accepted the generated .intunewin without format errors (RTVAL-01)"
|
||
- "A real Intune-managed Windows endpoint ran the install script under SYSTEM with pnputil staging drivers and $PSScriptRoot resolving (RTVAL-02)"
|
||
- "Intune detection rule reports 'Installed' on the endpoint after install (RTVAL-03)"
|
||
- "Intune uninstall removes the printer cleanly under SYSTEM context (RTVAL-04)"
|
||
artifacts:
|
||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/rtval-01-tenant-upload.png"
|
||
provides: "Screenshot of Intune showing successful package upload"
|
||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/rtval-02-install-log.txt"
|
||
provides: "IntuneManagementExtension.log excerpt showing install success + pnputil + $PSScriptRoot"
|
||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/rtval-03-detection.png"
|
||
provides: "Screenshot of Intune device install status = Installed"
|
||
- path: ".planning/phases/10-real-world-runtime-validation/evidence/rtval-04-uninstall-log.txt"
|
||
provides: "Log + printer-list verification showing printer removed under SYSTEM"
|
||
key_links:
|
||
- from: "Intune tenant"
|
||
to: "generated .intunewin from plan 10-01"
|
||
via: "Win32 app upload"
|
||
pattern: "Package (uploaded|accepted)"
|
||
- from: "install script ($PSScriptRoot)"
|
||
to: "bundled driver folder inside .intunewin"
|
||
via: "pnputil /add-driver staging under SYSTEM"
|
||
pattern: "pnputil.*Published Name|Driver package added successfully"
|
||
---
|
||
|
||
<objective>
|
||
Drive the four real-world RTVAL behavioral checks end-to-end on a live Intune tenant against a real Windows endpoint, and capture hard evidence (screenshots + device logs) for each.
|
||
|
||
Purpose: This is the phase's whole reason for existing. Every check here is a behavior no unit test can prove — it has to happen on real hardware with a real tenant.
|
||
Output: evidence/ folder populated with RTVAL-01..04 artifacts and RUNTIME-VALIDATION.md status lines flipped from PENDING to PASS/FAIL.
|
||
</objective>
|
||
|
||
<execution_context>
|
||
@C:/Users/SebastienQUEROL/.claude/get-shit-done/workflows/execute-plan.md
|
||
@C:/Users/SebastienQUEROL/.claude/get-shit-done/templates/summary.md
|
||
</execution_context>
|
||
|
||
<context>
|
||
@.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md
|
||
@.planning/REQUIREMENTS.md
|
||
</context>
|
||
|
||
<tasks>
|
||
|
||
<task type="checkpoint:human-action" gate="blocking">
|
||
<name>Task 1: RTVAL-01 — Upload .intunewin to real Intune tenant</name>
|
||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/rtval-01-tenant-upload.png, .planning/phases/10-real-world-runtime-validation/evidence/rtval-01-app-assigned.png</files>
|
||
<action>
|
||
PAUSE for human technician. Requires Intune tenant admin credentials — no complete free CLI path exists.
|
||
|
||
Technician MUST:
|
||
|
||
1. Sign in to https://intune.microsoft.com with tenant admin.
|
||
2. Go to Apps → Windows → Add → App type: Windows app (Win32).
|
||
3. Upload the `.intunewin` file from plan 10-01.
|
||
4. OBSERVE: Intune parses the file and presents the app metadata form without format errors.
|
||
5. Fill in minimal app info (name = "ImpTune RTVAL Test - <vendor>"), install command, uninstall command, detection rule (use the generated detect script: File exists / custom script as produced by ImpTune), requirement = Windows 10/11 x64, assignment = the single test device only.
|
||
6. Save the app. OBSERVE: app appears in the apps list with "Waiting for install status" or similar.
|
||
7. Capture screenshots:
|
||
- `evidence/rtval-01-tenant-upload.png` — upload success page showing the parsed .intunewin metadata
|
||
- `evidence/rtval-01-app-assigned.png` — app assignment screen showing the test device group
|
||
8. Update RUNTIME-VALIDATION.md section "RTVAL-01 — Tenant ingestion":
|
||
- Status: PASS if tenant accepted the package with NO format errors. FAIL otherwise.
|
||
- Evidence: paths to the two screenshots.
|
||
- Notes: any warnings Intune surfaced.
|
||
|
||
PASS: Intune accepted the upload and allowed assignment WITHOUT a format/extraction error.
|
||
FAIL: tenant rejected the file, surfaced a parse error, or could not load detection metadata — record the exact error text verbatim in Notes.
|
||
|
||
Resume signal: reply "RTVAL-01 PASS" + screenshot paths, or "RTVAL-01 FAIL" with exact tenant error text.
|
||
</action>
|
||
<verify>
|
||
<automated>test -f .planning/phases/10-real-world-runtime-validation/evidence/rtval-01-tenant-upload.png && ! grep -A1 "RTVAL-01 — Tenant ingestion" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md | grep -q "Status: PENDING"</automated>
|
||
</verify>
|
||
<done>RTVAL-01 status is PASS or FAIL in the report; tenant upload screenshot committed; any FAIL has verbatim error text in Notes.</done>
|
||
</task>
|
||
|
||
<task type="checkpoint:human-action" gate="blocking">
|
||
<name>Task 2: RTVAL-02 — Install on device under SYSTEM; verify pnputil + $PSScriptRoot</name>
|
||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/rtval-02-install-log.txt, .planning/phases/10-real-world-runtime-validation/evidence/rtval-02-install-status.png</files>
|
||
<action>
|
||
PAUSE for human technician. Requires physical/virtual access to the test Intune-managed Windows endpoint.
|
||
|
||
Technician MUST:
|
||
|
||
1. On the test device, force Intune sync: Settings → Accounts → Access work or school → Info → Sync (or restart `IntuneManagementExtension`).
|
||
2. Wait for Intune to push the assignment. Typical latency 5–30 minutes.
|
||
3. While install runs, open `C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\IntuneManagementExtension.log` and `AgentExecutor.log`.
|
||
4. After install completes, extract the block covering this app's install run and save as `evidence/rtval-02-install-log.txt`.
|
||
5. OBSERVE in the log — every item below MUST be visible:
|
||
a. The install script ran under `NT AUTHORITY\SYSTEM` (script banner or process token).
|
||
b. `$PSScriptRoot` resolved to the Intune app staging directory (not empty, not `C:\Windows\system32`).
|
||
c. `pnputil /add-driver` (or equivalent) executed against the bundled driver folder and reported success (`Driver package added successfully` or `Published Name:`).
|
||
d. Final script exit code is 0.
|
||
6. Also capture Intune portal view: Apps → the test app → Device install status → test device → Install status = "Installed". Screenshot as `evidence/rtval-02-install-status.png`.
|
||
7. Update RUNTIME-VALIDATION.md section "RTVAL-02":
|
||
- Status: PASS only if ALL four observations hold. Otherwise FAIL with which observation failed.
|
||
- Evidence: both file paths.
|
||
|
||
PASS: SYSTEM context + $PSScriptRoot resolved + pnputil staged drivers + exit 0.
|
||
FAIL: ANY of the above missing. Record the exact log excerpt showing the failure in Notes; open a defect ticket in "Issues Found".
|
||
|
||
Resume signal: reply "RTVAL-02 PASS" + log path, or "RTVAL-02 FAIL" with failing observation and excerpt.
|
||
</action>
|
||
<verify>
|
||
<automated>test -f .planning/phases/10-real-world-runtime-validation/evidence/rtval-02-install-log.txt && ! grep -A1 "RTVAL-02 — Install under SYSTEM" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md | grep -q "Status: PENDING"</automated>
|
||
</verify>
|
||
<done>RTVAL-02 status is PASS or FAIL in the report; install log committed; for PASS the log shows SYSTEM, pnputil success, $PSScriptRoot resolved, and exit 0.</done>
|
||
</task>
|
||
|
||
<task type="checkpoint:human-action" gate="blocking">
|
||
<name>Task 3: RTVAL-03 — Confirm Intune detection rule reports Installed</name>
|
||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/rtval-03-detection.png, .planning/phases/10-real-world-runtime-validation/evidence/rtval-03-detect-manual.txt</files>
|
||
<action>
|
||
PAUSE for human technician.
|
||
|
||
Technician MUST:
|
||
|
||
1. In Intune portal: Apps → the test app → Device install status → test device row → confirm "Installation status" = **Installed** (this is driven by the generated detect script).
|
||
2. On the endpoint itself, confirm the printer is present:
|
||
- PowerShell (user context OK for verification only): `Get-Printer | Where-Object Name -eq "<printer name>"` returns a row.
|
||
- OR open Settings → Bluetooth & devices → Printers & scanners → confirm printer visible.
|
||
3. Run the generated detect script manually once to cross-check:
|
||
- Open admin PowerShell and execute the detect script; confirm exit code 0.
|
||
4. Screenshot Intune "Installed" status as `evidence/rtval-03-detection.png`.
|
||
5. Save detect script manual run transcript as `evidence/rtval-03-detect-manual.txt`.
|
||
6. Update RUNTIME-VALIDATION.md section "RTVAL-03":
|
||
- Status: PASS if Intune shows Installed AND manual detect script exits 0 AND printer is visible. Otherwise FAIL.
|
||
|
||
PASS: Intune "Installed" + printer visible + detect script exit 0.
|
||
FAIL: Intune shows "Not detected", manual detect non-zero, or printer missing. Record in Notes.
|
||
|
||
Resume signal: reply "RTVAL-03 PASS" + evidence paths, or "RTVAL-03 FAIL" with observed status.
|
||
</action>
|
||
<verify>
|
||
<automated>test -f .planning/phases/10-real-world-runtime-validation/evidence/rtval-03-detection.png && ! grep -A1 "RTVAL-03 — Detection rule" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md | grep -q "Status: PENDING"</automated>
|
||
</verify>
|
||
<done>RTVAL-03 status is PASS or FAIL; detection screenshot and manual-run transcript committed.</done>
|
||
</task>
|
||
|
||
<task type="checkpoint:human-action" gate="blocking">
|
||
<name>Task 4: RTVAL-04 — Intune uninstall removes printer cleanly under SYSTEM</name>
|
||
<files>.planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md, .planning/phases/10-real-world-runtime-validation/evidence/rtval-04-uninstall-log.txt, .planning/phases/10-real-world-runtime-validation/evidence/rtval-04-uninstall-status.png</files>
|
||
<action>
|
||
PAUSE for human technician.
|
||
|
||
Technician MUST:
|
||
|
||
1. In Intune portal: change the app assignment for the test device from Required to Uninstall (or remove assignment and add as Uninstall).
|
||
2. Force Intune sync on the device again. Wait for the uninstall to run.
|
||
3. Extract the uninstall run from `IntuneManagementExtension.log` and save as `evidence/rtval-04-uninstall-log.txt`.
|
||
4. OBSERVE in the log:
|
||
a. Uninstall script ran under `NT AUTHORITY\SYSTEM`.
|
||
b. Final script exit code 0.
|
||
5. On the endpoint, confirm the printer is GONE:
|
||
- `Get-Printer | Where-Object Name -eq "<printer name>"` returns nothing.
|
||
- OR Settings → Printers & scanners → printer no longer listed.
|
||
6. Confirm Intune portal shows "Not installed" for the test device.
|
||
7. Screenshot as `evidence/rtval-04-uninstall-status.png`.
|
||
8. Update RUNTIME-VALIDATION.md section "RTVAL-04":
|
||
- Status: PASS if SYSTEM context + exit 0 + printer gone + Intune "Not installed". Otherwise FAIL with which check failed.
|
||
|
||
PASS: clean removal under SYSTEM, zero leftover printer artifacts.
|
||
FAIL: printer still present, non-zero exit, or tenant shows install failure. Record in Notes.
|
||
|
||
Resume signal: reply "RTVAL-04 PASS" + evidence paths, or "RTVAL-04 FAIL" with failing check.
|
||
</action>
|
||
<verify>
|
||
<automated>test -f .planning/phases/10-real-world-runtime-validation/evidence/rtval-04-uninstall-log.txt && ! grep -A1 "RTVAL-04 — Uninstall under SYSTEM" .planning/phases/10-real-world-runtime-validation/RUNTIME-VALIDATION.md | grep -q "Status: PENDING"</automated>
|
||
</verify>
|
||
<done>RTVAL-04 status is PASS or FAIL; uninstall log and status screenshot committed; printer confirmed removed on the device for PASS.</done>
|
||
</task>
|
||
|
||
</tasks>
|
||
|
||
<verification>
|
||
- evidence/rtval-01-tenant-upload.png exists
|
||
- evidence/rtval-02-install-log.txt exists and shows SYSTEM + pnputil + $PSScriptRoot
|
||
- evidence/rtval-03-detection.png exists and shows Installed
|
||
- evidence/rtval-04-uninstall-log.txt exists and shows clean removal
|
||
- RUNTIME-VALIDATION.md status lines for RTVAL-01..04 are PASS or FAIL (no PENDING)
|
||
</verification>
|
||
|
||
<success_criteria>
|
||
All four RTVAL checkpoint tasks have signaled a final status (PASS or FAIL) with evidence files committed to evidence/. Any FAIL has a corresponding "Issues Found" entry in the report with defect ticket or deferral rationale.
|
||
</success_criteria>
|
||
|
||
<output>
|
||
After completion, create `.planning/phases/10-real-world-runtime-validation/10-02-live-intune-runtime-validation-SUMMARY.md`
|
||
</output>
|