From 63cf69f1142abbf7304c9d1bfdcfa0491ab0b229 Mon Sep 17 00:00:00 2001 From: Kawa Date: Thu, 2 Apr 2026 09:28:40 +0200 Subject: [PATCH] docs: map existing codebase Co-Authored-By: Claude Opus 4.6 (1M context) --- .planning/codebase/ARCHITECTURE.md | 302 +++++++++++++++++++++++++++++ .planning/codebase/CONCERNS.md | 221 +++++++++++++++++++++ .planning/codebase/CONVENTIONS.md | 210 ++++++++++++++++++++ .planning/codebase/INTEGRATIONS.md | 149 ++++++++++++++ .planning/codebase/STACK.md | 103 ++++++++++ .planning/codebase/STRUCTURE.md | 249 ++++++++++++++++++++++++ .planning/codebase/TESTING.md | 256 ++++++++++++++++++++++++ 7 files changed, 1490 insertions(+) create mode 100644 .planning/codebase/ARCHITECTURE.md create mode 100644 .planning/codebase/CONCERNS.md create mode 100644 .planning/codebase/CONVENTIONS.md create mode 100644 .planning/codebase/INTEGRATIONS.md create mode 100644 .planning/codebase/STACK.md create mode 100644 .planning/codebase/STRUCTURE.md create mode 100644 .planning/codebase/TESTING.md diff --git a/.planning/codebase/ARCHITECTURE.md b/.planning/codebase/ARCHITECTURE.md new file mode 100644 index 0000000..1c7ffda --- /dev/null +++ b/.planning/codebase/ARCHITECTURE.md @@ -0,0 +1,302 @@ +# Architecture + +**Analysis Date:** 2026-04-02 + +## Pattern Overview + +**Overall:** Monolithic PowerShell Application with WinForms UI and Async Runspace Pattern + +**Key Characteristics:** +- Single-file PowerShell script (6408 lines) serving as entry point +- Native WinForms GUI (no external UI framework dependencies) +- Asynchronous operations via dedicated PowerShell runspaces to prevent UI blocking +- Hashtable-based state management for inter-runspace communication +- PnP.PowerShell module for all SharePoint Online interactions +- Profile and template persistence via JSON files +- Region-based code organization for logical grouping + +## Layers + +**Presentation Layer (GUI):** +- Purpose: User interface and interaction handling +- Location: `Sharepoint_ToolBox.ps1` lines 2990-3844 (GUI setup) + event handlers +- Contains: WinForms controls, dialogs, input validation, visual updates +- Depends on: Shared helpers, Settings layer +- Used by: Event handlers, runspace callbacks via synchronized hashtable + +**Application Layer (Business Logic):** +- Purpose: Core operations for each feature (permissions, storage, templates, search, duplicates) +- Location: `Sharepoint_ToolBox.ps1` multiple regions: + - Permissions: lines 1784-2001 + - Storage: lines 2002-2110 + - File Search: lines 2112-2233 + - Duplicates: lines 2235-2408 + - Templates: lines 475-1360 + - Transfer/Bulk: lines 2410-3000 +- Contains: PnP API calls, data aggregation, report generation +- Depends on: PnP.PowerShell module, Presentation feedback +- Used by: Event handlers via runspaces, HTML/CSV export functions + +**Data Access Layer:** +- Purpose: File I/O, persistence, caching +- Location: `Sharepoint_ToolBox.ps1` dedicated regions: + - Profile Management: lines 48-127 + - Settings: lines 129-154 + - Template Management: lines 475-533 +- Contains: JSON serialization/deserialization, profile CRUD, settings management +- Depends on: File system access +- Used by: Application layer, GUI initialization + +**Export & Reporting Layer:** +- Purpose: Transform data to CSV and interactive HTML +- Location: `Sharepoint_ToolBox.ps1`: + - Permissions HTML: lines 1361-1617 + - Storage HTML: lines 1619-1784 + - Search HTML: lines 2112-2233 + - Duplicates HTML: lines 2235-2408 + - Transfer HTML: lines 2412-2547 +- Contains: HTML template generation, JavaScript for interactivity, CSV formatting +- Depends on: Application layer data, System.Drawing for styling +- Used by: Feature implementations for export operations + +**Integration Layer:** +- Purpose: External service communication (SharePoint, PnP.PowerShell) +- Location: `Sharepoint_ToolBox.ps1` PnP function regions +- Contains: Connect-PnPOnline, Get-PnP* cmdlets, authentication handling +- Depends on: PnP.PowerShell module, credentials from user input +- Used by: Application layer operations + +**Utilities & Helpers:** +- Purpose: Cross-cutting formatting, UI helpers, internationalization +- Location: `Sharepoint_ToolBox.ps1`: + - Shared Helpers: lines 4-46 + - Internationalization: lines 2732-2989 + - UI Control Factories: lines 3119-3146 +- Contains: Write-Log, Format-Bytes, EscHtml, T() translator, control builders +- Depends on: System.Windows.Forms, language JSON file +- Used by: All other layers + +## Data Flow + +**Permissions Report Generation:** + +1. User selects site(s) and report options in GUI (Permissions tab) +2. Click "Générer le rapport" triggers event handler at line 4068+ +3. Validation via `Validate-Inputs` (line 30) +4. GUI triggers runspace via `Start-Job` with user parameters +5. Runspace calls `Generate-PnPSitePermissionRpt` (line 1852) +6. `Generate-PnPSitePermissionRpt` connects to SharePoint via `Connect-PnPOnline` (line 1864) +7. Recursive permission scanning: + - `Get-PnPWebPermission` (line 1944) for site/webs + - `Get-PnPListPermission` (line 1912) for lists and libraries + - `Get-PnPFolderPermission` (line 1882) for folders (if enabled) + - `Get-PnPPermissions` (line 1786) extracts individual role assignments +8. Results accumulated in `$script:AllPermissions` array +9. Export based on format choice: + - CSV: `Merge-PermissionRows` (line 1363) then `Export-Csv` + - HTML: `Export-PermissionsToHTML` (line 1389) generates interactive report +10. Output file path returned to UI via synchronized hashtable +11. User can open report via `btnPermOpen` click handler + +**Storage Metrics Scan:** + +1. User selects storage options and sites +2. Click "Générer les métriques" triggers runspace job +3. Job calls `Get-SiteStorageMetrics` (line 2004) +4. Per-site or per-library scanning: + - Connect to web via `Connect-PnPOnline` + - `Get-PnPList` retrieves document libraries (if per-library mode) + - `Get-PnPFolderStorageMetric` for library/root metrics + - `Collect-FolderStorage` (recursive nested function) walks folder tree to configured depth +5. Results accumulate in `$script:storageResults` with hierarchy intact +6. HTML or CSV export writes report file +7. File path communicated back to UI + +**Site Picker (Browse Sites):** + +1. User clicks "Voir les sites" button +2. `Show-SitePicker` dialog opens (line 212) +3. User clicks "Charger les sites" button +4. Dialog initializes `$script:_pkl` state hashtable (line 315) +5. Runspace spawned in `btnLoad.Add_Click` (line 395) +6. Runspace connects to admin site and retrieves all sites via `Get-PnPTenantSite` +7. Results queued back to UI via synchronized `$script:_pkl.Sync` hashtable +8. Timer polls `$script:_pkl.Sync` and updates ListView asynchronously +9. User filters by text, sorts columns, checks/unchecks sites +10. Returns selected site URLs in `$script:SelectedSites` array + +**File Search:** + +1. User enters search criteria (extensions, regex, date ranges, etc.) +2. Click "Lancer la recherche" triggers runspace +3. Runspace uses PnP Search API (KQL) with filters: + - File extension filters via `fileExtension:ext1` OR syntax + - Date range filters via `Created >= date` + - Regex applied client-side after retrieval +4. Results paginated and accumulated +5. Exported to CSV or HTML with interactive filtering/sorting + +**Duplicate Detection:** + +1. User chooses file or folder mode and comparison criteria +2. Click "Lancer le scan" triggers runspace +3. File duplicates: Search API with filename-based grouping +4. Folder duplicates: Enumerate all folders, compare attributes (size, dates, subfolder/file counts) +5. Results grouped by match criteria +6. HTML export shows grouped duplicates with visual indicators (green/orange for matching/differing fields) + +**Template Capture & Apply:** + +1. Capture mode: `Show-TemplateManager` dialog (line 542) + - User selects "Capture from site" + - Runspace scans site structure via `Get-PnPList`, `Get-PnPFolderItem`, `Get-PnPWebPermission` + - Captures libraries, folders, permission groups, site logo, title + - Persisted to `Sharepoint_Templates.json` +2. Apply mode: User selects template and target site + - Runspace creates lists/libraries via `New-PnPList` + - Replicates folder structure via `New-PnPFolder` + - Applies permission groups if selected + - Logs creation results + +**State Management:** + +- `$script:` variables hold state across runspace calls (profiles, sites, results, settings) +- Synchronized hashtables (`$script:_pkl`, `$script:_sync`) enable runspace-to-UI communication +- Timer at line 3850-3870 polls synchronized hashtable and updates GUI with progress/results +- Event handlers trigger jobs but don't block waiting for completion (asynchronous pattern) + +## Key Abstractions + +**Runspace Encapsulation:** + +- Purpose: Execute long-running SharePoint operations without freezing GUI +- Pattern: `$job = Start-Job -ScriptBlock { ... } -RunspacePool $rsPool` +- Example: `Start-NextStorageScan` (line 4536) manages storage scan runspace jobs +- Trade-off: Requires careful state management via shared hashtables; no direct closures + +**Hashtable-Based State:** + +- Purpose: Share mutable state between main runspace and job runspaces +- Pattern: `$sync = @{ Data = @(); Status = "Running" }` passed to job +- Example: `$script:_pkl` (line 315) manages site picker state across checkbox events +- Benefit: Avoids closure complexity; timer can poll changes safely + +**Dialog Modal Isolation:** + +- Purpose: Site picker and template manager as isolated UI contexts +- Pattern: `Show-SitePicker` and `Show-TemplateManager` create self-contained `Form` objects +- State stored in `$script:_pkl` and `$script:_tpl` respectively +- Returns result arrays (selected sites, template data) to main form + +**Language Translation System:** + +- Purpose: Internationalization without external dependencies +- Pattern: `T("key")` function (line 2908) looks up keys in `$script:LangDict` hashtable +- Source: `lang/fr.json` contains French translations; English is hardcoded +- Used throughout: All UI labels, buttons, messages use `T()` for localization + +**HTML Export Templates:** + +- Purpose: Dynamically generate interactive HTML reports with embedded JavaScript +- Pattern: String templates with `@"` heredoc syntax containing HTML/CSS/JS +- Examples: + - `Export-PermissionsToHTML` (line 1389): Responsive table, collapsible groups, copy-to-clipboard + - `Export-StorageToHTML` (line 1621): Tree visualization, sorting, filtering + - `Export-DuplicatesToHTML` (line 2235): Grouped duplicates with visual indicators +- Benefit: No external libraries; reports are self-contained single-file HTML + +## Entry Points + +**Main GUI Form:** +- Location: `Sharepoint_ToolBox.ps1` line 2992 +- Triggers: Script execution via `.ps1` file or PowerShell IDE +- Responsibilities: + - Initialize WinForms components (form, controls, menus) + - Load and populate profiles/settings from JSON + - Register event handlers for all buttons and controls + - Run main event loop `[void]$form.ShowDialog()` + +**Feature Event Handlers:** +- Location: Various in lines 4068+ (Event Handlers region) +- Examples: + - `btnPermRun.Add_Click` → Permissions report generation + - `btnStorRun.Add_Click` → Storage metrics scan + - `btnSearchRun.Add_Click` → File search + - `btnDupRun.Add_Click` → Duplicate detection +- Pattern: Validate inputs, start runspace job, launch progress animation, register cleanup callback + +**Background Runspaces:** +- Entry: `Start-Job -ScriptBlock { Generate-PnPSitePermissionRpt ... }` +- Execution: PnP cmdlets execute within runspace's isolated context +- Completion: Job completion callback writes results to synchronized hashtable; timer detects and updates UI + +**Language Switch:** +- Location: Menu → Language submenu (line 3011+) +- Handler: `Switch-AppLanguage` (line 4167) +- Updates: All UI labels via `Update-UILanguage` (line 2951) + +## Error Handling + +**Strategy:** Try/Catch with graceful degradation; errors logged to UI RichTextBox + +**Patterns:** + +1. **Runspace Error Handling:** + ```powershell + try { $result = Get-PnPList } + catch { Write-Log "Error: $($_.Exception.Message)" "Red" } + ``` + +2. **Connection Validation:** + - `Validate-Inputs` (line 30) checks required fields before operation + - `Connect-PnPOnline` fails if credentials invalid; caught and logged + +3. **File I/O Protection:** + ```powershell + if (Test-Path $path) { + try { $data = Get-Content $path -Raw | ConvertFrom-Json } + catch {} # Silently ignore JSON parse errors + } + ``` + +4. **UI Update Safety:** + - `Write-Log` checks `if ($script:LogBox -and !$script:LogBox.IsDisposed)` before updating + - Prevents access to disposed UI objects after form close + +5. **Missing Configuration Handling:** + - Settings default to English + current directory if file missing + - Profiles default to empty array if file missing + - Templates default to empty if file corrupted + +## Cross-Cutting Concerns + +**Logging:** +- Framework: `Write-Log` function (line 6) +- Pattern: Writes colored messages to RichTextBox + host console +- Usage: All operations log status (connecting, scanning, exporting) +- Timestamps: `Get-Date -Format 'HH:mm:ss'` prefixes each message + +**Validation:** +- Entry point: `Validate-Inputs` (line 30) checks ClientID and Site URL +- Pattern: Early return if missing; user sees MessageBox with missing field hint +- Localization: Error messages use `T()` function for i18n + +**Authentication:** +- Method: Interactive browser login via `Connect-PnPOnline -Interactive` +- Pattern: PnP module opens browser for Azure AD consent; token cached within session +- Credential scope: Per site connection; multiple connections supported (for multi-site operations) +- Token management: Automatic via PnP.PowerShell; no manual handling + +**Asynchronous Progress:** +- Animation: `Start-ProgressAnim` (line 3845) flashes "Running..." in status label +- Polling: Timer at line 3850-3870 checks `$job.State` and synchronized hashtable every 300ms +- Cleanup: `Stop-ProgressAnim` (line 3850) stops animation when job completes + +**UI Responsiveness:** +- Pattern: `[System.Windows.Forms.Application]::DoEvents()` called during long operations +- Benefit: Allows UI events (button clicks, close) to process while waiting +- Cost: Runspace jobs recommended for truly long operations (>5 second operations) + +--- + +*Architecture analysis: 2026-04-02* diff --git a/.planning/codebase/CONCERNS.md b/.planning/codebase/CONCERNS.md new file mode 100644 index 0000000..e9e3111 --- /dev/null +++ b/.planning/codebase/CONCERNS.md @@ -0,0 +1,221 @@ +# Codebase Concerns + +**Analysis Date:** 2026-04-02 + +## Tech Debt + +**Silent Error Handling (Widespread):** +- Issue: 38 empty `catch` blocks that suppress errors without logging +- Files: `Sharepoint_ToolBox.ps1` (lines 1018, 1020, 1067, 1068, 1144, 2028, 2030, etc.) +- Impact: Failures go unnoticed, making debugging difficult. Users don't know why operations fail. Error conditions are hidden from logs. +- Fix approach: Add logging to all `catch` blocks. Use `BgLog` for background tasks, `Write-Log` for UI threads. Example: `catch { BgLog "Folder enumeration failed: $_" "DarkGray" }` instead of `catch {}` + +**Resource Cleanup Issues:** +- Issue: Runspace and PowerShell objects created in background jobs may not be properly disposed if exceptions occur +- Files: `Sharepoint_ToolBox.ps1` lines 1040-1052, 4564-4577, 5556-5577 +- Impact: Memory leaks possible if UI interactions are interrupted. Zombie runspaces could accumulate over multiple operations. +- Fix approach: Wrap all runspace/PS object creation in try-finally blocks. Ensure `$rs.Dispose()` and `$ps.Dispose()` are called in finally block, not just in success path + +**Overly Broad Error Suppression:** +- Issue: 27 instances of `-ErrorAction SilentlyContinue` spread throughout code +- Files: `Sharepoint_ToolBox.ps1` (e.g., lines 1142, 1188, 2070, 4436, etc.) +- Impact: Real failures indistinguishable from expected failures (e.g., list doesn't exist vs. connection failed). Masks bugs. +- Fix approach: Use selective error suppression. Only suppress when you've explicitly checked for the condition (e.g., "if list doesn't exist, create it"). Otherwise use `-ErrorAction Stop` with explicit try-catch. + +**Inconsistent JSON Error Handling:** +- Issue: JSON parsing in `Load-Profiles`, `Load-Settings`, `Load-Templates` uses empty catch blocks +- Files: `Sharepoint_ToolBox.ps1` lines 61, 140, 568-570 +- Impact: Corrupted JSON files silently fail and return empty defaults, losing user data silently +- Fix approach: Log actual error message. Implement validation schema. Create backup of corrupted files. + +## Known Bugs + +**Blank Client ID Warning Not Actionable:** +- Symptoms: "WARNING: No Client ID returned" appears but doesn't prevent further operations or clear user input +- Files: `Sharepoint_ToolBox.ps1` line 4237 +- Trigger: Azure AD app registration completes but returns null ClientId (can happen with certain tenant configurations) +- Workaround: Manually register app via Azure Portal and paste Client ID +- Fix approach: Check for null ClientId before continuing, clear the warning state properly + +**Group Member Addition Silent Failures:** +- Symptoms: Members appear not to be added to sites, but no error shown in UI +- Files: `Sharepoint_ToolBox.ps1` lines 1222-1225, 5914, 5922 (try-catch with SilentlyContinue) +- Trigger: User exists but cannot be added to group (permissions, licensing, or source-specific SP group issues) +- Workaround: Manual group membership assignment +- Fix approach: Replace SilentlyContinue with explicit logging of why Add-PnPGroupMember failed + +**Folder Metadata Loss in Template Application:** +- Symptoms: Folder permissions captured correctly but not reapplied when permissions=true but structure already exists +- Files: `Sharepoint_ToolBox.ps1` lines 1229-1292 (folder-level permission application depends on library structure map being built first) +- Trigger: Target library created by Apply-FolderTree but permissions application logic expects library to already exist in template structure +- Workaround: Delete and recreate target library, or manually apply permissions via SharePoint UI +- Fix approach: Build library map before applying folder tree, or add validation that all referenced libraries exist + +**CSV Import for Bulk Operations Not Validated:** +- Symptoms: Invalid CSV format silently fails, users see no clear error, form appears unresponsive +- Files: `Sharepoint_ToolBox.ps1` lines 5765-5800 (CSV parsing with inadequate error context) +- Trigger: CSV with missing headers, wrong delimiter, or invalid format +- Workaround: Edit CSV manually to match expected format, restart tool +- Fix approach: Add CSV schema validation before processing, show specific validation errors + +## Security Considerations + +**Client ID and Tenant URL Hardcoded in Temp Files:** +- Risk: Temp registration script contains unencrypted Client ID and Tenant ID in plaintext +- Files: `Sharepoint_ToolBox.ps1` lines 4210-4245 (temp file creation) +- Current mitigation: Temp file cleanup attempted but not guaranteed if process crashes +- Recommendations: Use SecureString to pass credentials, delete temp file with -Force in finally block, or use named pipes instead of files + +**No Validation of Tenant URL Format:** +- Risk: Arbitrary URLs accepted, could be typos leading to authentication against wrong tenant +- Files: `Sharepoint_ToolBox.ps1` lines 4306-4317, 30-43 (Validate-Inputs) +- Current mitigation: URL used as-is, relies on PnP authentication failure to catch issues +- Recommendations: Add regex validation for SharePoint tenant URLs, warn on suspicious patterns + +**Profile File Contains Credentials in Plaintext:** +- Risk: `Sharepoint_Export_profiles.json` contains Client ID and Tenant URL in plaintext on disk +- Files: `Sharepoint_ToolBox.ps1` lines 50-72 (profile persistence) +- Current mitigation: File located in user home directory (Windows ACL protection), but still plaintext +- Recommendations: Consider encrypting profile file with DPAPI, or move to Windows Credential Manager + +**PnP PowerShell Module Trust Not Validated:** +- Risk: Module imported without version pinning, could load compromised version +- Files: `Sharepoint_ToolBox.ps1` lines 151, 1151, 4218, 4521, 5833 (Import-Module PnP.PowerShell) +- Current mitigation: None +- Recommendations: Pin module version in manifest, use `-MinimumVersion` parameter, check module signature + +## Performance Bottlenecks + +**Synchronous UI Freezes During Large Operations:** +- Problem: File search with 50,000 result limit processes all results at once, building HTML string in memory +- Files: `Sharepoint_ToolBox.ps1` lines 2112-2133 (Export-SearchResultsToHTML builds entire table in string) +- Cause: All results concatenated into single `$rows` string before sending to UI +- Improvement path: Implement pagination in HTML reports, stream results rather than buffering all in memory. For large datasets, chunk exports into multiple files. + +**Folder Storage Recursion Not Depth-Limited by Default:** +- Problem: `Collect-FolderStorage` recurses unlimited depth unless explicitly capped, can take hours on deep folder structures +- Files: `Sharepoint_ToolBox.ps1` lines 2009-2032, 4432-4455 +- Cause: CurrentDepth compared against FolderDepth limit, but FolderDepth defaults to 999 if not set +- Improvement path: Default to depth 3-4, show estimated scan time based on depth, implement cancellation token + +**No Parallel Processing for Multiple Sites:** +- Problem: Sites processed sequentially in Permissions/Storage reports, one site blocks all others +- Files: `Sharepoint_ToolBox.ps1` lines 4379-4401 (foreach loop in permissions scan) +- Cause: Single-threaded approach with `Connect-PnPOnline` context switches +- Improvement path: Queue-based processing for multiple sites (partially done for storage scans), implement async context management + +**HTML Report Generation for Large Duplicates List:** +- Problem: Export-DuplicatesToHTML builds entire HTML in memory, slow for 10,000+ duplicates +- Files: `Sharepoint_ToolBox.ps1` lines 2235-2400 (HTML string concatenation in loop) +- Cause: All groups converted to HTML before writing to file +- Improvement path: Stream HTML generation, write to file incrementally, implement lazy-loading tables in browser + +## Fragile Areas + +**Language System (T() function):** +- Files: `Sharepoint_ToolBox.ps1` (translation lookups throughout, ~15 hardcoded English fallbacks like "Veuillez renseigner") +- Why fragile: Language loading can fail silently, UI control updates hardcoded at multiple locations, no fallback chain for missing translations +- Safe modification: Add validation that all UI strings have corresponding translation keys before form creation. Create helper function that returns English default if key missing. +- Test coverage: No tests for translation system. Gaps: Missing translations for error messages, hardcoded "Veuillez renseigner" strings that bypass T() function + +**Profile Management:** +- Files: `Sharepoint_ToolBox.ps1` lines 50-127 +- Why fragile: Profile list is in-memory array that syncs with JSON file. If Save-Profiles fails, changes are lost. No transaction semantics. +- Safe modification: Implement write-lock pattern. Create backup before write. Validate JSON before replacing file. +- Test coverage: No validation that profile save actually persists. Race condition if opened in multiple instances. + +**Runspace State Machine:** +- Files: `Sharepoint_ToolBox.ps1` lines 1034-1095, 4580-4650 (runspace creation, async timer polling) +- Why fragile: UI state (`btnGenPerms.Enabled = false`) set before runspace begins, but no explicit state reset if runspace crashes or hangs +- Safe modification: Implement state enum (Idle, Running, Done, Error). Always reset state in finally block. Set timeout on runspace execution. +- Test coverage: No timeout tests. Gaps: What happens if runspace hangs indefinitely? Button remains disabled forever. + +**Site Picker List View:** +- Files: `Sharepoint_ToolBox.ps1` lines 157-250 (_Pkl-* functions) +- Why fragile: AllSites list updates while UI may be reading it (SuppressCheck flag used but incomplete synchronization) +- Safe modification: Use proper locking or rebuild entire list atomically. Current approach relies on flag which may miss updates. +- Test coverage: No concurrent access tests. Gaps: What if site is checked while sort is happening? + +## Scaling Limits + +**Permissions Report HTML with Large Item Counts:** +- Current capacity: Tested up to ~5,000 items, performance degrades significantly +- Limit: HTML table becomes unusable in browser above 10,000 rows (sorting, filtering slow) +- Scaling path: Implement client-side virtual scrolling in HTML template, paginate into multiple reports, add server-side filtering before export + +**File Search Result Limit:** +- Current capacity: 50,000 result maximum hardcoded +- Limit: Beyond 50,000 files, results truncated without warning +- Scaling path: Implement pagination in SharePoint Search API, show "more results available" warning, allow user to refine search + +**Runspace Queue Processing:** +- Current capacity: Single queue per scan, sequential processing +- Limit: If background job produces messages faster than timer dequeues, queue could grow unbounded +- Scaling path: Implement back-pressure (slow producer if queue > 1000 items), implement priority queue + +**Profile JSON File Size:** +- Current capacity: Profiles loaded entirely into memory, no limit on file size +- Limit: If user creates 1,000+ profiles, JSON file becomes slow to load/save +- Scaling path: Implement profile paging, index file by profile name, lazy-load profile details + +## Dependencies at Risk + +**PnP.PowerShell Module Version Mismatch:** +- Risk: Module API changes between major versions, cmdlet parameter changes +- Impact: Features relying on specific cmdlet parameters break silently +- Migration plan: Pin to stable version range in script header. Create version compatibility matrix. Test against 2-3 stable versions. + +**System.Windows.Forms Dependency:** +- Risk: WinForms support in PowerShell 7 is deprecated, future versions may not ship it +- Impact: GUI completely broken on future PowerShell versions +- Migration plan: Consider migrating to WPF or cross-platform GUI framework (Avalonia). Current WinForms code is tied to Assembly loading. + +## Missing Critical Features + +**No Operation Cancellation:** +- Problem: Running operations (permissions scan, storage metrics, file search) cannot be stopped mid-execution +- Blocks: User stuck waiting for slow operations to complete, no way to abort except kill process + +**No Audit Log:** +- Problem: No record of who ran what operation, what results were exported, when last backup occurred +- Blocks: Compliance, troubleshooting + +**No Dry-Run for Most Operations:** +- Problem: Only version cleanup has dry-run. Permission changes, site creation applied immediately without preview +- Blocks: Prevents risk assessment before making changes + +## Test Coverage Gaps + +**PnP Connection Failures:** +- What's not tested: Connection timeouts, intermittent network issues, authentication failures mid-operation +- Files: `Sharepoint_ToolBox.ps1` lines 36, 157, 170, 2036, 4458, etc. (Connect-PnPOnline calls) +- Risk: Tool may hang indefinitely if connection drops. No retry logic. +- Priority: High + +**Malformed JSON Resilience:** +- What's not tested: Templates.json, Profiles.json, Settings.json with invalid JSON, missing fields, type mismatches +- Files: `Sharepoint_ToolBox.ps1` lines 61, 140, 568 (ConvertFrom-Json) +- Risk: Tool fails to start or loses user data +- Priority: High + +**Large-Scale Operations:** +- What's not tested: Permissions scan on site with 50,000+ items, storage metrics on 10,000+ folders, file search returning 40,000+ results +- Files: Bulk scanning functions throughout +- Risk: Memory exhaustion, timeout, UI freeze +- Priority: Medium + +**Runspace Cleanup on Error:** +- What's not tested: Runspace exception handling, cleanup if UI window closes during background operation +- Files: `Sharepoint_ToolBox.ps1` lines 1040-1052, 4564-4577, 5556-5577 +- Risk: Zombie processes, resource leaks +- Priority: Medium + +**CSV Format Validation:** +- What's not tested: Invalid column headers, wrong delimiter, missing required columns in bulk operations +- Files: `Sharepoint_ToolBox.ps1` lines 5765-5800 +- Risk: Silent failures, partial data import +- Priority: Medium + +--- + +*Concerns audit: 2026-04-02* diff --git a/.planning/codebase/CONVENTIONS.md b/.planning/codebase/CONVENTIONS.md new file mode 100644 index 0000000..595955c --- /dev/null +++ b/.planning/codebase/CONVENTIONS.md @@ -0,0 +1,210 @@ +# Coding Conventions + +**Analysis Date:** 2026-04-02 + +## Naming Patterns + +**Functions:** +- PascalCase for public functions: `Write-Log`, `Get-ProfilesFilePath`, `Load-Profiles`, `Save-Profiles`, `Show-InputDialog` +- Verb-Noun format using standard PowerShell verbs: Get-, Load-, Save-, Show-, Export-, Apply-, Validate-, Merge-, Refresh-, Switch- +- Private/internal functions prefixed with underscore: `_Pkl-FormatMB`, `_Pkl-Sort`, `_Pkl-Repopulate`, `_Tpl-Repopulate`, `_Tpl-Log` +- Descriptive names reflecting operation scope: `Get-SiteStorageMetrics`, `Collect-FolderStorage`, `Collect-WebStorage`, `Export-PermissionsToHTML` + +**Variables:** +- camelCase for local variables: `$message`, `$color`, `$data`, `$index`, `$siteSrl` +- PascalCase for control variables and form elements: `$form`, `$LogBox`, `$ClientId`, `$SiteURL` +- Prefixed script-scope variables with `$script:` for shared state: `$script:LogBox`, `$script:Profiles`, `$script:SelectedSites`, `$script:_pkl`, `$script:_tpl` +- Abbreviated but meaningful names in tight loops: `$s` (site), `$e` (event), `$i` (index), `$m` (message), `$c` (color) +- Hashtable keys use camelCase: `@{ name = "...", clientId = "...", tenantUrl = "..." }` + +**Parameters:** +- Type hints included in function signatures: `[string]$Message`, `[array]$Data`, `[switch]$IncludeSubsites`, `[int]$CurrentDepth` +- Optional parameters use `= $null` or `= $false` defaults: `[string]$Color = "LightGreen"`, `[System.Windows.Forms.Form]$Owner = $null` +- Single-letter abbreviated parameters in nested functions: `param($s, $e)` for event handlers + +**File/Directory Names:** +- Single main script file: `Sharepoint_ToolBox.ps1` +- Settings/profile files: `Sharepoint_Settings.json`, `Sharepoint_Export_profiles.json`, `Sharepoint_Templates.json` +- Generated report files use pattern: `{ReportType}_{site/date}_{timestamp}.{csv|html}` + +## Code Style + +**Formatting:** +- No explicit formatter configured +- Indentation: 4 spaces (PowerShell default) +- Line length: practical limit around 120 characters (some HTML generation lines exceed this) +- Braces on same line for blocks: `function Name { ... }`, `if ($condition) { ... }` +- Region markers used for file organization: `#region ===== Section Name =====` and `#endregion` + +**Regions Organization (in `Sharepoint_ToolBox.ps1`):** +- Shared Helpers (utility functions) +- Profile Management (profile CRUD, loading/saving) +- Settings (configuration handling) +- Site Picker (dialog and list management) +- Template Management (capture, apply, storage) +- HTML Export: Permissions and Storage (report generation) +- PnP: Permissions and Storage Metrics (SharePoint API operations) +- File Search (advanced file search functionality) +- Transfer (file/folder transfer operations) +- Bulk Site Creation (site creation from templates) +- Internationalization (multi-language support) +- GUI (main form and controls definition) +- Event Handlers (button clicks, selections, menu actions) +- Structure (folder tree CSV parsing) + +**Comments:** +- Inline comments explain non-obvious logic: `# Groups rows that share the same Users + Permissions` +- Block comments precede major sections: `# -- Top bar --`, `# -- Site list (ListView with columns) --` +- Section separators use dashes: `# ── Profile Management ─────────────────────────────────` +- Descriptive comments in complex functions explain algorithm: `# Recursively collects subfolders up to $MaxDepth levels deep` +- No JSDoc/TSDoc style - pure text comments + +## Import Organization + +**Module Imports:** +- `Add-Type -AssemblyName` for .NET assemblies at script start: + - `System.Windows.Forms` for UI controls + - `System.Drawing` for colors and fonts +- `Import-Module PnP.PowerShell` dynamically when needed in background runspace blocks +- No explicit order beyond UI assemblies first + +## Error Handling + +**Patterns:** +- Broad `try-catch` blocks with minimal logging: `try { ... } catch {}` +- Silent error suppression common: empty catch blocks swallow exceptions +- Explicit error capture in key operations: `catch { $Sync.Error = $_.Exception.Message }` +- Error logging via `Write-Log` with color coding: + - Red for critical failures: `Write-Log "Erreur: $message" "Red"` + - Yellow for informational messages: `Write-Log "Processing..." "Yellow"` + - DarkGray for skipped items: `Write-Log "Skipped: $reason" "DarkGray"` +- Exception messages extracted and logged: `$_.Exception.Message` +- Validation checks return boolean: `if ([string]::IsNullOrWhiteSpace(...)) { return $false }` + +## Logging + +**Framework:** Native `Write-Log` function + UI RichTextBox display + +**Patterns:** +```powershell +function Write-Log { + param([string]$Message, [string]$Color = "LightGreen") + if ($script:LogBox -and !$script:LogBox.IsDisposed) { + # Append to UI with timestamp and color + $script:LogBox.AppendText("$(Get-Date -Format 'HH:mm:ss') - $Message`n") + } + Write-Host $Message # Also output to console +} +``` + +**Logging locations:** +- Long-running operations log to RichTextBox in real-time via background runspace queue +- Background functions use custom `BgLog` helper that queues messages: `function BgLog([string]$m, [string]$c="LightGreen")` +- Colors indicate message type: LightGreen (success), Yellow (info), Cyan (detail), DarkGray (skip), Red (error) +- Timestamps added automatically: `HH:mm:ss` format + +## Validation + +**Input Validation:** +- Null/whitespace checks: `[string]::IsNullOrWhiteSpace($variable)` +- Array/collection size checks: `$array.Count -gt 0`, `$items -and $items.Count -gt 0` +- Index bounds validation: `if ($idx -lt 0 -or $idx -ge $array.Count) { return }` +- UI MessageBox dialogs for user-facing errors: `[System.Windows.Forms.MessageBox]::Show(...)` +- Function-level validation via `Validate-Inputs` pattern + +## String Handling + +**HTML Escaping:** +- Custom `EscHtml` function escapes special characters for HTML generation: + ```powershell + function EscHtml([string]$s) { + return $s -replace '&','&' -replace '<','<' -replace '>','>' -replace '"','"' + } + ``` +- Applied to all user-supplied data rendered in HTML reports + +**String Interpolation:** +- Double-quoted strings with `$variable` expansion: `"URL: $url"` +- Format operator for complex strings: `"Template '{0}' saved" -f $name` +- Localization helper function `T` for i18n strings: `T "profile"`, `T "btn.save"` + +## Function Design + +**Size:** Functions range from 5-50 lines for utilities, 100+ lines for complex operations + +**Parameters:** +- Explicit type declarations required +- Optional parameters use default values +- Switch parameters for boolean flags: `[switch]$IncludeSubsites` +- Complex objects passed as reference (arrays, hashtables) + +**Return Values:** +- Functions return results or arrays: `return @()`, `return $data` +- Boolean results for validation: `return $true` / `return $false` +- PSCustomObject for structured data: `[PSCustomObject]@{ Name = ...; Value = ... }` +- Void operations often silent or logged + +## Object/Struct Patterns + +**PSCustomObject for Data:** +```powershell +[PSCustomObject]@{ + name = "value" + clientId = "value" + capturedAt = (Get-Date -Format 'dd/MM/yyyy HH:mm') + options = @{ structure = $true; permissions = $true } +} +``` + +**Hashtable for Mutable State:** +```powershell +$script:_pkl = @{ + AllSites = @() + CheckedUrls = [System.Collections.Generic.HashSet[string]]::new() + SortCol = 0 + SortAsc = $true +} +``` + +**Synchronized Hashtable for Thread-Safe State:** +```powershell +$sync = [hashtable]::Synchronized(@{ + Done = $false + Error = $null + Queue = [System.Collections.Generic.Queue[object]]::new() +}) +``` + +## Class/Type Usage + +- Minimal custom classes; primarily uses `[PSCustomObject]` +- Extensive use of .NET types: + - `[System.Windows.Forms.*]` for UI controls + - `[System.Drawing.Color]` for colors + - `[System.Drawing.Font]` for typography + - `[System.Drawing.Point]`, `[System.Drawing.Size]` for positioning + - `[System.Collections.Generic.HashSet[string]]` for efficient lookups + - `[System.Collections.Generic.Queue[object]]` for message queues + - `[System.Management.Automation.Runspaces.*]` for background execution + +## Date/Time Formatting + +**Consistent format:** `dd/MM/yyyy HH:mm` (European format) +- Used for timestamps in reports and logs +- Timestamps in logs: `HH:mm:ss` +- Storage/file metrics: `dd/MM/yyyy HH:mm` + +## Performance Patterns + +**Batch Operations:** +- Form updates wrapped in `BeginUpdate()` / `EndUpdate()` to prevent flickering +- ListView population optimized: clear, populate, sort in batch + +**Threading:** +- Long-running PnP operations execute in separate runspace +- Main UI thread communicates via synchronized hashtable + timer polling +- Async UI updates via `Timer` with `DoEvents()` for responsiveness + +--- + +*Convention analysis: 2026-04-02* diff --git a/.planning/codebase/INTEGRATIONS.md b/.planning/codebase/INTEGRATIONS.md new file mode 100644 index 0000000..234eb9a --- /dev/null +++ b/.planning/codebase/INTEGRATIONS.md @@ -0,0 +1,149 @@ +# External Integrations + +**Analysis Date:** 2026-04-02 + +## APIs & External Services + +**SharePoint Online:** +- Service: Microsoft SharePoint Online (via Microsoft 365) +- What it's used for: Site management, permission auditing, file search, storage metrics, templating, bulk operations + - SDK/Client: PnP.PowerShell module + - Auth: Azure AD interactive login (ClientId required) + - Connection method: `Connect-PnPOnline -Url -Interactive -ClientId ` + - Search: SharePoint Search API using KQL (keyword query language) via `Submit-PnPSearchQuery` + +**Azure AD:** +- Service: Microsoft Entra ID (formerly Azure Active Directory) +- What it's used for: User authentication and app registration + - SDK/Client: PnP.PowerShell (handles auth flow) + - Auth: Interactive browser-based login + - App Registration: Required with delegated permissions configured + - No service principal or client secret used (interactive auth only) + +## Data Storage + +**Databases:** +- None detected - Application uses file-based storage only + +**File Storage:** +- Service: Local filesystem only +- Connection: Configured data folder for JSON files +- Client: PowerShell native file I/O +- Configuration: `Sharepoint_Settings.json` stores dataFolder path + +**Caching:** +- Service: None detected +- In-memory collections used during session (synchronized hashtables for runspace communication) + +## Authentication & Identity + +**Auth Provider:** +- Azure AD (Microsoft Entra ID) + - Implementation: Interactive browser-based OAuth 2.0 flow + - No client secrets or certificates + - User must have access to target SharePoint tenant + - App registration required with delegated permissions + +**Registration Process:** +- User creates Azure AD App Registration +- Client ID stored in profile for reuse +- Helper script available: `Register-PnPEntraIDAppForInteractiveLogin` (via PnP.PowerShell) +- Result file: Temporary JSON stored in system temp folder, user copies Client ID manually + +## Monitoring & Observability + +**Error Tracking:** +- None detected - Errors written to UI log box via `Write-Log` function +- Location: UI RichTextBox control in application + +**Logs:** +- Approach: In-app console logging + - Function: `Write-Log $Message [Color]` writes timestamped messages to UI log box + - Colors: LightGreen (default), Red (errors), Yellow (KQL queries), DarkOrange (dry-run operations) + - File location: `C:\Users\SebastienQUEROL\Documents\projets\Sharepoint\Sharepoint_ToolBox.ps1` (lines 6-17) + +## CI/CD & Deployment + +**Hosting:** +- Not applicable - Desktop application (local execution) + +**CI Pipeline:** +- None detected + +**Execution Model:** +- Direct script execution: `.\Sharepoint_Toolbox.ps1` +- No installation/setup required beyond PowerShell and PnP.PowerShell module + +## Environment Configuration + +**Required env vars:** +- None required - All configuration stored in JSON files +- User inputs via GUI: Client ID, Tenant URL, Site URL + +**Secrets location:** +- Not applicable - Interactive auth uses no stored secrets +- User manages Client ID (non-sensitive app identifier) +- Session credentials handled by Azure AD auth flow (in-memory only) + +**Configuration files:** +- `Sharepoint_Settings.json` - Data folder, language preference +- `Sharepoint_Export_profiles.json` - Saved connection profiles (Tenant URL, Client ID) +- `Sharepoint_Templates.json` - Captured site templates + +## Webhooks & Callbacks + +**Incoming:** +- None detected + +**Outgoing:** +- None detected + +## Search & Query Integration + +**SharePoint Search API:** +- Usage: File search across libraries using KQL +- Location: `Sharepoint_ToolBox.ps1` lines 4744-4773 (search query building) +- Function: `Submit-PnPSearchQuery -Query $kql` +- Pagination: Automatic via PnP.PowerShell +- Client-side filtering: Regex filters applied after results fetched +- Query example: Supports file extension, name/path patterns, creation/modification date ranges, author filters, max result limits + +## Export & Report Formats + +**Output Formats:** +- CSV: PowerShell `Export-Csv` cmdlet (UTF-8 encoding, no type info) +- HTML: Custom HTML generation with: + - Interactive tables (sorting, filtering by column) + - Collapsible sections (durable state via CSS/JS) + - Charts and metrics visualization + - Inline styling (no external CSS file) + +**Export Functions:** +- `Export-PermissionsToHTML` (line 1389) +- `Export-StorageToHTML` (line 1621) +- `Export-SearchResultsToHTML` (line 2112) +- `Export-DuplicatesToHTML` (line 2235) +- `Export-TransferVerifyToHTML` (line 2412) + +## Bulk Import Formats + +**CSV Input:** +- Bulk member add: Expects columns for site, group, user email +- Bulk site creation: Site name, alias, owner email, description +- Bulk file transfer: Source site/path, destination site/path +- Folder structure: Library name, folder path, permissions + +**Parsing:** +- PowerShell `Import-Csv` - Standard CSV parsing +- Headers used as property names + +## API Rate Limiting + +**SharePoint Online:** +- No explicit rate limiting handling detected +- Assumes PnP.PowerShell handles throttling internally +- Pagination used for large result sets (PageSize 2000 for list items) + +--- + +*Integration audit: 2026-04-02* diff --git a/.planning/codebase/STACK.md b/.planning/codebase/STACK.md new file mode 100644 index 0000000..06be8ee --- /dev/null +++ b/.planning/codebase/STACK.md @@ -0,0 +1,103 @@ +# Technology Stack + +**Analysis Date:** 2026-04-02 + +## Languages + +**Primary:** +- PowerShell 5.1+ - All application logic, UI, and SharePoint integration + +## Runtime + +**Environment:** +- Windows PowerShell 5.1+ or PowerShell Core 7.0+ +- .NET Framework (built-in with PowerShell) + +**Execution Model:** +- Desktop application (WinForms GUI) +- Synchronous runspace threading for async operations without blocking UI + +## Frameworks + +**UI:** +- System.Windows.Forms (native .NET, bundled with PowerShell) +- System.Drawing (native .NET for graphics and colors) + +**SharePoint/Cloud Integration:** +- PnP.PowerShell (latest version) - All SharePoint Online API interactions +- Azure AD App Registration for authentication (required) + +**Testing:** +- No dedicated test framework detected (manual testing assumed) + +**Build/Dev:** +- No build system (single .ps1 script file executed directly) + +## Key Dependencies + +**Critical:** +- PnP.PowerShell - Required for all SharePoint Online operations + - Location: Installed via `Install-Module PnP.PowerShell` + - Used for: Site enumeration, permissions scanning, storage metrics, file search, templating, bulk operations + - Connection: Interactive authentication via Azure AD App (ClientId required) + +**Infrastructure:** +- System.Windows.Forms - Desktop UI framework +- System.Drawing - UI graphics and rendering +- Microsoft.SharePoint.Client - Underlying SharePoint CSOM (via PnP.PowerShell) + +## Configuration + +**Environment:** +- `Sharepoint_Settings.json` - User preferences (data folder location, language) +- `Sharepoint_Export_profiles.json` - Saved connection profiles (Tenant URL, Client ID) +- `Sharepoint_Templates.json` - Site structure templates (captured and reapplied) + +**Build:** +- Single executable: `Sharepoint_ToolBox.ps1` +- Launched directly: `.\Sharepoint_Toolbox.ps1` + +**Localization:** +- File: `lang/fr.json` - French translations +- Default: English (en) +- Loaded dynamically at runtime + +## Platform Requirements + +**Development:** +- Windows Operating System (WinForms is Windows-only) +- PowerShell 5.1+ +- Internet connection for Azure AD authentication +- Access to SharePoint Online tenant + +**Production:** +- Windows 10/11 (or Windows Server 2016+) +- PowerShell 5.1 minimum +- Azure AD tenant with properly configured app registration +- Network access to target SharePoint Online sites + +## Data Persistence + +**Local Storage:** +- JSON files in configurable data folder (default: `Sharepoint_Export_profiles.json`, `Sharepoint_Templates.json`, `Sharepoint_Settings.json`) +- CSV exports of reports and bulk operation results +- HTML reports with interactive UI + +**No external databases** - All storage is file-based and local + +## Authentication + +**Method:** +- Azure AD Interactive Login (user-initiated browser-based auth) +- Client ID (App Registration ID) required +- No client secrets or certificates (interactive auth flow only) +- PnP.PowerShell handles Azure AD token acquisition + +## Known Versions + +- PowerShell: 5.1 (minimum requirement stated in README) +- PnP.PowerShell: Not pinned (latest version recommended) + +--- + +*Stack analysis: 2026-04-02* diff --git a/.planning/codebase/STRUCTURE.md b/.planning/codebase/STRUCTURE.md new file mode 100644 index 0000000..45de744 --- /dev/null +++ b/.planning/codebase/STRUCTURE.md @@ -0,0 +1,249 @@ +# Codebase Structure + +**Analysis Date:** 2026-04-02 + +## Directory Layout + +``` +Sharepoint-Toolbox/ +├── .planning/ # GSD planning documentation +├── .git/ # Git repository +├── .gitea/ # Gitea configuration +├── .claude/ # Claude IDE configuration +├── examples/ # CSV example files for bulk operations +│ ├── bulk_add_members.csv # Template: add users to groups +│ ├── bulk_create_sites.csv # Template: create multiple sites +│ ├── bulk_transfer.csv # Template: transfer site ownership +│ └── folder_structure.csv # Template: create folder hierarchies +├── lang/ # Language/localization files +│ └── fr.json # French language translations +├── Sharepoint_ToolBox.ps1 # Main application (6408 lines) +├── Sharepoint_Settings.json # User settings (data folder, language preference) +├── Sharepoint_Export_profiles.json # Saved connection profiles (generated at runtime) +├── Sharepoint_Templates.json # Saved site templates (generated at runtime) +├── README.md # Documentation and feature overview +├── TODO.md # Future feature roadmap +└── SPToolbox-logo.png # Application logo +``` + +## Directory Purposes + +**`examples/`:** +- Purpose: Reference CSV templates for bulk operations +- Contains: CSV example files demonstrating column structure for bulk tasks +- Key files: + - `bulk_add_members.csv`: User email and group name mappings + - `bulk_create_sites.csv`: Site title, URL, type, language + - `bulk_transfer.csv`: Source site, target owner email + - `folder_structure.csv`: Folder paths to create under libraries +- Non-versioned: May contain user data after operations +- Access: Referenced by bulk operation dialogs; templates shown in UI + +**`lang/`:** +- Purpose: Store language packs for UI localization +- Contains: JSON files with key-value pairs for UI text +- Key files: + - `fr.json`: Complete French translations for all UI elements, buttons, messages +- Naming: `.json` (e.g., `en.json`, `fr.json`) +- Loading: `Load-Language` function (line 2933) reads and caches translation dict +- Integration: `T("key")` function (line 2908) looks up translations at runtime + +**`.planning/`:** +- Purpose: GSD (GitHub Sync & Deploy) planning and analysis documents +- Contains: Generated documentation for architecture, structure, conventions, concerns +- Generated: By GSD mapping tools; not manually edited +- Committed: Yes, tracked in version control + +## Key File Locations + +**Entry Points:** + +- `Sharepoint_ToolBox.ps1` (lines 1-6408): Single monolithic PowerShell script + - Execution: `.ps1` file run directly or sourced from PowerShell ISE/terminal + - Initialization: Lines 6386-6408 load settings, language, profiles, then show main form + - Exit: Triggered by form close or exception; automatic cleanup of runspaces + +- Main GUI Form instantiation: `Sharepoint_ToolBox.ps1` line 2992 + - Creates WinForms.Form object + - Registers all event handlers + - Shows dialog via `[void]$form.ShowDialog()` at line 6405 + +**Configuration:** + +- `Sharepoint_Settings.json`: User preferences + - Structure: `{ "dataFolder": "...", "lang": "en" }` + - Loaded by: `Load-Settings` (line 136) + - Saved by: `Save-Settings` (line 147) + - Auto-created: If missing, defaults to English + script root directory + +- `Sharepoint_Export_profiles.json`: Connection profiles (auto-created) + - Structure: `{ "profiles": [ { "name": "Prod", "clientId": "...", "tenantUrl": "..." }, ... ] }` + - Loaded by: `Load-Profiles` (line 57) + - Saved by: `Save-Profiles` (line 68) + - Location: Determined by `Get-ProfilesFilePath` (line 50) - same as settings + +- `Sharepoint_Templates.json`: Captured site templates (auto-created) + - Structure: `{ "templates": [ { "name": "...", "libraries": [...], "groups": [...], ... }, ... ] }` + - Loaded by: `Load-Templates` (line 484) + - Saved by: `Save-Templates` (line 495) + - Location: Same folder as profiles/settings + +**Core Logic:** + +- Permissions Report: `Sharepoint_ToolBox.ps1` lines 1784-2001 + - `Generate-PnPSitePermissionRpt` (line 1852): Main permission scanning function + - `Get-PnPWebPermission` (line 1944): Recursive site/subsite scanning + - `Get-PnPListPermission` (line 1912): Library and list enumeration + - `Get-PnPFolderPermission` (line 1882): Folder-level permission scanning + - `Get-PnPPermissions` (line 1786): Individual role assignment extraction + +- Storage Metrics: `Sharepoint_ToolBox.ps1` lines 2002-2110 + - `Get-SiteStorageMetrics` (line 2004): Main storage scan function + - Nested `Collect-FolderStorage` (line 2010): Recursive folder traversal + - Nested `Collect-WebStorage` (line 2034): Per-web storage collection + +- File Search: `Sharepoint_ToolBox.ps1` lines 2112-2233 + - Search API integration via PnP.PowerShell + - KQL (Keyword Query Language) filter construction + - Client-side regex filtering after API retrieval + +- Template Management: `Sharepoint_ToolBox.ps1` lines 475-1360 + - `Show-TemplateManager` (line 542): Template dialog + - Capture state machine in dialog event handlers + - Template persistence via JSON serialization + +- Duplicate Detection: `Sharepoint_ToolBox.ps1` lines 2235-2408 + - File mode: Search API with grouping by filename + - Folder mode: Direct library enumeration + comparison + - HTML export with grouped UI + +**Testing:** + +- No automated test framework present +- Manual testing via GUI interaction +- Examples folder (`examples/`) provides test data templates + +**Localization:** + +- `lang/fr.json`: French translations + - Format: JSON object with `"_name"` and `"_code"` metadata + translation keys + - Loading: `Load-Language` (line 2933) parses JSON into `$script:LangDict` + - Usage: `T("key")` replaces hardcoded English strings with translations + - UI Update: `Update-UILanguage` (line 2951) updates all registered controls + +## Naming Conventions + +**Files:** + +- Main application: `Sharepoint_ToolBox.ps1` (PascalCase with underscore separator) +- Settings/data: `Sharepoint_.json` (e.g., `Sharepoint_Settings.json`) +- Generated exports: `__.` or `__.` + - Permissions: `Permissions__.csv/html` + - Storage: `Storage__.csv/html` + - Search: `FileSearch_.csv/html` + - Duplicates: `Duplicates__.csv/html` +- Language files: `.json` (lowercase, e.g., `fr.json`) +- Example files: `_.csv` (lowercase with underscore, e.g., `bulk_add_members.csv`) + +**PowerShell Functions:** + +- Public functions: `Verb-Noun` naming (e.g., `Generate-PnPSitePermissionRpt`, `Get-SiteStorageMetrics`) +- Private/internal functions: Prefixed with `_` or grouped in regions (e.g., `_Pkl-Sort`, `_Pkl-Repopulate`) +- Event handlers: Declared inline in `.Add_Click` or `.Add_TextChanged` blocks; not named separately +- Nested functions (inside others): CamelCase with parent context (e.g., `Collect-FolderStorage` inside `Get-SiteStorageMetrics`) + +**Variables:** + +- Script-scope state: `$script:` (e.g., `$script:AllPermissions`, `$script:DataFolder`) +- Local function scope: camelCase (e.g., `$result`, `$dlg`, `$lists`) +- Control references: descriptive with type suffix (e.g., `$txtClientId`, `$btnPermRun`, `$cboProfile`) +- Control variables stored in script scope: Prefixed `$script:` for access across event handlers +- Temporary arrays: `$` (e.g., `$sites`, `$folders`, `$results`) + +**Types:** + +- Region markers: `#region ===== =====` (e.g., `#region ===== GUI =====`) +- Comments: Double hash for section comments (e.g., `# ── Label helper ──`) + +**Exports:** + +- HTML: Class names like `permission-table`, `storage-tree`, `duplicate-group` +- CSV: Column headers match object property names (e.g., `Title`, `URL`, `Permissions`) + +## Where to Add New Code + +**New Feature:** + +1. Create a new region section in `Sharepoint_ToolBox.ps1`: + ```powershell + #region ===== [Feature Name] ===== + function [Verb]-[Feature](...) { ... } + #endregion + ``` + +2. Primary code locations: + - Core logic: After line 2408 (after duplicates region, before Transfer region) + - PnP interaction: Own `#region` mirroring storage/permissions pattern + - HTML export helper: Create function like `Export-[Feature]ToHTML` in dedicated region + +3. Add UI tab or button: + - Create new TabPage in `$tabs` (around line 3113+) + - Register event handler for execution button in Event Handlers region (line 4068+) + - Add label in French translation file (`lang/fr.json`) + +4. Add menu item if needed: + - Modify MenuStrip construction around line 3001-3027 + - Register handler in Event Handlers region + +5. Persist settings: + - Add properties to `Sharepoint_Settings.json` structure + - Update `Load-Settings` (line 136) to include new fields + - Update `Save-Settings` (line 147) to serialize new fields + +**New Component/Module:** + +- Keep as internal functions (no separate files) +- If complexity exceeds 500 lines, consider refactoring into regions +- Pattern: All code stays in single `Sharepoint_ToolBox.ps1` file +- Dependencies: Use script-scope variables for shared state + +**Utilities:** + +- Shared helpers: `Shared Helpers` region (line 4-46) + - Add new helper function here if used by multiple features + - Examples: `Write-Log`, `Format-Bytes`, `EscHtml`, `Validate-Inputs` + +- UI control factories: Lines 3119-3146 + - Add new `New-` helper for frequently used UI patterns + - Examples: `New-Group`, `New-Check`, `New-Radio`, `New-ActionBtn` + +- Internationalization: `Internationalization` region (line 2732-2989) + - Add new translation keys to `lang/fr.json` + - Update `T()` function if new resolution logic needed + +## Special Directories + +**`examples/`:** +- Purpose: CSV templates for user reference +- Generated: No; committed as examples +- Committed: Yes, tracked in version control +- Accessed by: Bulk operation dialogs (not directly imported by code; users download manually) +- Content: Non-executable; user-facing documentation + +**`.planning/`:** +- Purpose: GSD-generated codebase analysis and planning documents +- Generated: Yes; created by GSD mapping tools during `/gsd:map-codebase` +- Committed: Yes; documents are version-controlled +- Accessed by: `/gsd:plan-phase` and `/gsd:execute-phase` commands for context +- Content: Markdown documents describing architecture, structure, conventions, concerns, stack, integrations + +**Generated Files (Runtime):** + +Files created at runtime, not part of initial repository: +- `Sharepoint_Export_profiles.json`: User-created connection profiles +- `Sharepoint_Templates.json`: User-captured site templates +- Export reports: `*_.(csv|html)` files in output folder (default: script root or user-selected) + +--- + +*Structure analysis: 2026-04-02* diff --git a/.planning/codebase/TESTING.md b/.planning/codebase/TESTING.md new file mode 100644 index 0000000..19ccf1f --- /dev/null +++ b/.planning/codebase/TESTING.md @@ -0,0 +1,256 @@ +# Testing Patterns + +**Analysis Date:** 2026-04-02 + +## Test Framework + +**Status:** No automated testing framework detected + +**Infrastructure:** Not applicable +- No test runner (Jest, Vitest, Pester) +- No test configuration files +- No test suite in codebase +- No CI/CD pipeline test stage configured + +## Testing Approach + +**Current Testing Model:** Manual testing via GUI + +**Test Methods:** +- **GUI Testing:** All functionality tested through WinForms UI + - Manual interaction with controls and dialogs + - Visual verification of results in generated reports + - Log output observation in RichTextBox +- **Report Validation:** HTML and CSV exports manually reviewed for correctness +- **API Integration:** Manual testing of PnP.PowerShell operations against live SharePoint tenant +- **Regression Testing:** Ad-hoc manual verification of features after changes + +## Code Organization for Testing + +**Testability Patterns:** Limited + +The monolithic single-file architecture (`Sharepoint_ToolBox.ps1` at 6408 lines) makes isolated unit testing challenging. Key observations: + +**Tight Coupling to UI:** +- Core business logic embedded in event handlers +- Heavy reliance on global `$script:` scope for state +- Example: `Load-Profiles` reads from `Get-ProfilesFilePath`, which is file-system dependent +- Site picker functionality (`Show-SitePicker`) spawns background runspace but depends on form being instantiated + +**Hard Dependencies:** +- PnP.PowerShell module imported dynamically in background runspace blocks +- File system access (profiles, templates, settings) not abstracted +- SharePoint connection state implicit in PnP connection context + +**Areas with Better Isolation:** +- Pure utility functions like `Format-Bytes`, `EscHtml` could be unit tested +- Data transformation functions like `Merge-PermissionRows` accept input arrays and return structured output +- HTML generation in `Export-PermissionsToHTML` and `Export-StorageToHTML` could be tested against expected markup + +## Background Runspace Pattern + +**Async Execution Model:** +Most long-running operations execute in separate PowerShell runspace to prevent UI blocking: + +```powershell +# 1. Create synchronized hashtable for communication +$sync = [hashtable]::Synchronized(@{ + Done = $false + Error = $null + Result = $null + Queue = [System.Collections.Generic.Queue[object]]::new() +}) + +# 2. Define background script block (has access to passed parameters only) +$bgScript = { + param($Url, $ClientId, $Sync) + try { + Import-Module PnP.PowerShell -ErrorAction Stop + Connect-PnPOnline -Url $Url -Interactive -ClientId $ClientId + # Perform work + $Sync.Result = $data + } catch { + $Sync.Error = $_.Exception.Message + } finally { + $Sync.Done = $true + } +} + +# 3. Launch in runspace +$rs = [System.Management.Automation.Runspaces.RunspaceFactory]::CreateRunspace() +$rs.ApartmentState = "STA"; $rs.ThreadOptions = "ReuseThread"; $rs.Open() +$ps = [System.Management.Automation.PowerShell]::Create() +$ps.Runspace = $rs +[void]$ps.AddScript($bgScript) +[void]$ps.AddArgument($url) +$hnd = $ps.BeginInvoke() + +# 4. Poll completion with timer +$tmr = New-Object System.Windows.Forms.Timer +$tmr.Interval = 300 +$tmr.Add_Tick({ + if ($sync.Done) { + [void]$ps.EndInvoke($hnd) + $rs.Close(); $rs.Dispose() + # Update UI with $sync.Result + } +}) +$tmr.Start() +``` + +**Used for:** +- Site picker loading: `Show-SitePicker` (lines 212-471) +- Template capture: Background job in template manager (lines 900+) +- Site creation: Background job in bulk creation (lines 1134+) +- Permission/storage export: Operations triggered from event handlers +- File search: Background search execution + +## Message Queue Pattern + +**For logging from background runspaces:** + +```powershell +# Background function enqueues messages +function BgLog([string]$m, [string]$c="LightGreen") { + $Sync.Queue.Enqueue([PSCustomObject]@{ Text=$m; Color=$c }) +} + +# Main thread timer dequeues and displays +$tmr.Add_Tick({ + while ($sync.Queue.Count -gt 0) { + $msg = $sync.Queue.Dequeue() + _Tpl-Log -Box $textBox -Msg $msg.Text -Color $msg.Color + } +}) +``` + +**Rationale:** Avoids cross-thread UI access violations by queueing messages from worker thread. + +## Common Testing Patterns in Code + +**Null/Existence Checks:** +```powershell +# Before using objects +if ($script:LogBox -and !$script:LogBox.IsDisposed) { ... } +if ($data -and $data.Count -gt 0) { ... } +if ([string]::IsNullOrWhiteSpace($value)) { return $false } +``` + +**Error Logging in Loops:** +```powershell +# Catch errors in data processing, log, continue +foreach ($item in $items) { + try { + # Process item + } catch { + BgLog " Skipped: $($_.Exception.Message)" "DarkGray" + } +} +``` + +**Validation Before Operations:** +```powershell +function Validate-Inputs { + if ([string]::IsNullOrWhiteSpace($script:txtClientId.Text)) { + [System.Windows.Forms.MessageBox]::Show(...) + return $false + } + return $true +} + +# Called before starting long operations +if (-not (Validate-Inputs)) { return } +``` + +## Data Flow Testing Approach + +**For Feature Development:** + +1. **Manual Test Cases** (observed pattern, not formalized): + - Permissions Export: + - Select site with multiple libraries + - Choose CSV format + - Verify CSV contains all libraries and permissions + - Test HTML format in browser for interactivity + + - Storage Metrics: + - Run with `PerLibrary` flag + - Verify folder hierarchy is captured + - Test recursive subsite inclusion + - Validate byte calculations + + - Template Capture/Apply: + - Capture from source site + - Verify JSON structure + - Create new site from template + - Verify structure, permissions, settings applied + + - File Search: + - Test regex patterns + - Verify date filtering + - Test large result sets (pagination) + - Check CSV/HTML output + +2. **Visual Verification:** + - Log output reviewed in RichTextBox for progress + - Generated HTML reports tested in multiple browsers + - CSV files opened in Excel for format verification + +## Fragility Points & Testing Considerations + +**PnP Connection Management:** +- No connection pooling; each operation may create new connection +- Interactive auth prompt appears per runspace +- **Risk:** Auth failures not consistently handled +- **Testing Need:** Mock PnP module or use test tenant + +**HTML Generation:** +- String concatenation for large HTML documents (lines 1475+) +- Inline CSS for styling +- JavaScript for interactivity +- **Risk:** Complex HTML fragments prone to markup errors +- **Testing Need:** Validate HTML structure and JavaScript functionality + +**JSON Persistence:** +- Profiles, templates, settings stored in JSON +- ConvertTo-Json/-From-Json without depth specification can truncate +- **Risk:** Nested objects may not round-trip correctly +- **Testing Need:** Validate all object types persist/restore + +**Background Runspace Cleanup:** +- Runspace and PowerShell objects must be disposed +- Timer must be stopped and disposed +- **Risk:** Resource leaks if exception occurs before cleanup +- **Testing Need:** Verify cleanup in error paths + +## Suggested Testing Improvements + +**Unit Testing:** +1. Extract pure functions (no UI/file system dependencies) + - `Format-Bytes`, `EscHtml`, `Merge-PermissionRows` + - HTML generation functions + +2. Use Pester framework for PowerShell unit tests: + ```powershell + Describe "Format-Bytes" { + It "Formats bytes to GB" { + Format-Bytes (1GB * 2) | Should -Be "2 GB" + } + } + ``` + +3. Mock file system and PnP operations for integration tests + +**Integration Testing:** +1. Use test SharePoint tenant for functional testing +2. Automate report generation and validation +3. Script common user workflows + +**Regression Testing:** +1. Maintain test suite of sites with known structures +2. Generate reports, compare outputs +3. Run before major releases + +--- + +*Testing analysis: 2026-04-02*