Sharepoint-Toolbox/.planning/phases/09-storage-visualization/09-02-PLAN.md

---
phase: 09-storage-visualization
plan: 02
type: execute
wave: 2
depends_on:
  - "09-01"
files_modified:
  - SharepointToolbox/Services/StorageService.cs
autonomous: true
requirements:
  - VIZZ-02
must_haves:
  truths:
    - "CollectFileTypeMetricsAsync enumerates files from all non-hidden document libraries"
    - "Files are grouped by extension with summed size and count"
    - "Results are sorted by TotalSizeBytes descending"
    - "Existing CollectStorageAsync method is not modified"
  artifacts:
    - path: "SharepointToolbox/Services/StorageService.cs"
      provides: "Implementation of CollectFileTypeMetricsAsync"
      contains: "CollectFileTypeMetricsAsync"
  key_links:
    - from: "SharepointToolbox/Services/StorageService.cs"
      to: "SharepointToolbox/Core/Models/FileTypeMetric.cs"
      via: "Groups CSOM file data into FileTypeMetric records"
      pattern: "new FileTypeMetric"
    - from: "SharepointToolbox/Services/StorageService.cs"
      to: "SharepointToolbox/Core/Helpers/ExecuteQueryRetryHelper.cs"
      via: "Throttle-safe query execution"
      pattern: "ExecuteQueryRetryHelper\\.ExecuteQueryRetryAsync"
---

<objective>
Implement CollectFileTypeMetricsAsync in StorageService -- enumerate files across all non-hidden document libraries using CSOM CamlQuery, aggregate by file extension, and return sorted FileTypeMetric list.

Purpose: Provides the data layer for chart visualization (VIZZ-02). The ViewModel will call this after the main storage scan completes.
Output: Updated StorageService.cs with CollectFileTypeMetricsAsync implementation
</objective>

<execution_context>
@C:/Users/dev/.claude/get-shit-done/workflows/execute-plan.md
@C:/Users/dev/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/09-storage-visualization/09-01-SUMMARY.md

<interfaces>
<!-- From Plan 09-01 -->
From SharepointToolbox/Core/Models/FileTypeMetric.cs:
```csharp
public record FileTypeMetric(string Extension, long TotalSizeBytes, int FileCount)
{
    public string DisplayLabel => string.IsNullOrEmpty(Extension)
        ? "No Extension"
        : Extension.TrimStart('.').ToUpperInvariant();
}
```

From SharepointToolbox/Services/IStorageService.cs:
```csharp
public interface IStorageService
{
    Task<IReadOnlyList<StorageNode>> CollectStorageAsync(
        ClientContext ctx, StorageScanOptions options,
        IProgress<OperationProgress> progress, CancellationToken ct);

    Task<IReadOnlyList<FileTypeMetric>> CollectFileTypeMetricsAsync(
        ClientContext ctx,
        IProgress<OperationProgress> progress,
        CancellationToken ct);
}
```

From SharepointToolbox/Core/Helpers/ExecuteQueryRetryHelper.cs:
```csharp
public static class ExecuteQueryRetryHelper
{
    public static async Task ExecuteQueryRetryAsync(
        ClientContext ctx,
        IProgress<OperationProgress>? progress = null,
        CancellationToken ct = default);
}
```

<!-- Existing StorageService structure (DO NOT modify existing methods) -->
From SharepointToolbox/Services/StorageService.cs:
```csharp
public class StorageService : IStorageService
{
    public async Task<IReadOnlyList<StorageNode>> CollectStorageAsync(...) { ... }
    private static async Task<StorageNode> LoadFolderNodeAsync(...) { ... }
    private static async Task CollectSubfoldersAsync(...) { ... }
}
```
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Implement CollectFileTypeMetricsAsync in StorageService</name>
  <files>SharepointToolbox/Services/StorageService.cs</files>
  <action>
    Add the `CollectFileTypeMetricsAsync` method to the existing `StorageService` class. Do NOT modify any existing methods (CollectStorageAsync, LoadFolderNodeAsync, CollectSubfoldersAsync). Add the new method after the existing `CollectStorageAsync` method.

    Add this method to the `StorageService` class:

    ```csharp
    public async Task<IReadOnlyList<FileTypeMetric>> CollectFileTypeMetricsAsync(
        ClientContext ctx,
        IProgress<OperationProgress> progress,
        CancellationToken ct)
    {
        ct.ThrowIfCancellationRequested();

        // Load all non-hidden document libraries
        ctx.Load(ctx.Web,
            w => w.Lists.Include(
                l => l.Title,
                l => l.Hidden,
                l => l.BaseType,
                l => l.ItemCount));
        await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);

        var libs = ctx.Web.Lists
            .Where(l => !l.Hidden && l.BaseType == BaseType.DocumentLibrary)
            .ToList();

        // Accumulate file sizes by extension across all libraries
        var extensionMap = new Dictionary<string, (long totalSize, int count)>(StringComparer.OrdinalIgnoreCase);

        int libIdx = 0;
        foreach (var lib in libs)
        {
            ct.ThrowIfCancellationRequested();
            libIdx++;
            progress.Report(new OperationProgress(libIdx, libs.Count,
                $"Scanning files by type: {lib.Title} ({libIdx}/{libs.Count})"));

            // Use CamlQuery to enumerate all files in the library
            // Paginate with 500 items per batch to avoid list view threshold issues
            var query = new CamlQuery
            {
                ViewXml = @"<View Scope='RecursiveAll'>
                    <Query>
                        <Where>
                            <Eq>
                                <FieldRef Name='FSObjType' />
                                <Value Type='Integer'>0</Value>
                            </Eq>
                        </Where>
                    </Query>
                    <ViewFields>
                        <FieldRef Name='FileLeafRef' />
                        <FieldRef Name='File_x0020_Size' />
                    </ViewFields>
                    <RowLimit Paged='TRUE'>500</RowLimit>
                </View>"
            };

            ListItemCollection items;
            do
            {
                ct.ThrowIfCancellationRequested();
                items = lib.GetItems(query);
                ctx.Load(items, ic => ic.ListItemCollectionPosition,
                    ic => ic.Include(
                        i => i["FileLeafRef"],
                        i => i["File_x0020_Size"]));
                await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);

                foreach (var item in items)
                {
                    string fileName = item["FileLeafRef"]?.ToString() ?? string.Empty;
                    string sizeStr = item["File_x0020_Size"]?.ToString() ?? "0";

                    if (!long.TryParse(sizeStr, out long fileSize))
                        fileSize = 0;

                    string ext = Path.GetExtension(fileName).ToLowerInvariant();
                    // ext is "" for extensionless files, ".docx" etc. for others

                    if (extensionMap.TryGetValue(ext, out var existing))
                        extensionMap[ext] = (existing.totalSize + fileSize, existing.count + 1);
                    else
                        extensionMap[ext] = (fileSize, 1);
                }

                // Move to next page
                query.ListItemCollectionPosition = items.ListItemCollectionPosition;
            }
            while (items.ListItemCollectionPosition != null);
        }

        // Convert to FileTypeMetric list, sorted by size descending
        return extensionMap
            .Select(kvp => new FileTypeMetric(kvp.Key, kvp.Value.totalSize, kvp.Value.count))
            .OrderByDescending(m => m.TotalSizeBytes)
            .ToList();
    }
    ```

    Make sure to add `using System.IO;` at the top of the file if not already present (for `Path.GetExtension`).

    Design notes:
    - Uses `Scope='RecursiveAll'` in CamlQuery to get files from all subfolders without explicit recursion
    - `FSObjType=0` filter ensures only files (not folders) are returned
    - Paged query with 500-item batches avoids list view threshold (5000 default) issues
    - File_x0020_Size is the internal name for file size in SharePoint
    - Extensions normalized to lowercase for consistent grouping (".DOCX" and ".docx" merge)
    - Dictionary uses OrdinalIgnoreCase comparer as extra safety
    - Existing methods (CollectStorageAsync, LoadFolderNodeAsync, CollectSubfoldersAsync) are NOT touched
  </action>
  <verify>
    <automated>cd "C:\Users\dev\Documents\projets\Sharepoint" && dotnet build SharepointToolbox/SharepointToolbox.csproj --no-incremental 2>&1 | tail -5</automated>
  </verify>
  <done>StorageService.cs implements CollectFileTypeMetricsAsync. Method enumerates files via CamlQuery with paging, groups by extension, returns IReadOnlyList&lt;FileTypeMetric&gt; sorted by TotalSizeBytes descending. Existing CollectStorageAsync is unchanged. Project compiles with 0 errors.</done>
</task>

</tasks>

<verification>
- `dotnet build SharepointToolbox/SharepointToolbox.csproj` succeeds with 0 errors
- StorageService now implements both IStorageService methods
- CollectFileTypeMetricsAsync uses paginated CamlQuery (RowLimit 500, Paged=TRUE)
- Extensions normalized to lowercase
- Results sorted by TotalSizeBytes descending
- No modifications to CollectStorageAsync, LoadFolderNodeAsync, or CollectSubfoldersAsync
</verification>

<success_criteria>
StorageService fully implements IStorageService. CollectFileTypeMetricsAsync can enumerate files by extension from any SharePoint site. The project compiles cleanly and existing storage scan behavior is unaffected.
</success_criteria>

<output>
After completion, create `.planning/phases/09-storage-visualization/09-02-SUMMARY.md`
</output>