Sharepoint-Toolbox/.planning/phases/09-storage-visualization/09-02-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
09-storage-visualization	02	execute	2	09-01	SharepointToolbox/Services/StorageService.cs	true	VIZZ-02	truths artifacts key_links CollectFileTypeMetricsAsync enumerates files from all non-hidden document libraries Files are grouped by extension with summed size and count Results are sorted by TotalSizeBytes descending Existing CollectStorageAsync method is not modified path provides contains SharepointToolbox/Services/StorageService.cs Implementation of CollectFileTypeMetricsAsync CollectFileTypeMetricsAsync from to via pattern SharepointToolbox/Services/StorageService.cs SharepointToolbox/Core/Models/FileTypeMetric.cs Groups CSOM file data into FileTypeMetric records new FileTypeMetric from to via pattern SharepointToolbox/Services/StorageService.cs SharepointToolbox/Core/Helpers/ExecuteQueryRetryHelper.cs Throttle-safe query execution ExecuteQueryRetryHelper.ExecuteQueryRetryAsync

Implement CollectFileTypeMetricsAsync in StorageService -- enumerate files across all non-hidden document libraries using CSOM CamlQuery, aggregate by file extension, and return sorted FileTypeMetric list.

Purpose: Provides the data layer for chart visualization (VIZZ-02). The ViewModel will call this after the main storage scan completes. Output: Updated StorageService.cs with CollectFileTypeMetricsAsync implementation

<execution_context> @C:/Users/dev/.claude/get-shit-done/workflows/execute-plan.md @C:/Users/dev/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/09-storage-visualization/09-01-SUMMARY.md From SharepointToolbox/Core/Models/FileTypeMetric.cs: ```csharp public record FileTypeMetric(string Extension, long TotalSizeBytes, int FileCount) { public string DisplayLabel => string.IsNullOrEmpty(Extension) ? "No Extension" : Extension.TrimStart('.').ToUpperInvariant(); } ```

From SharepointToolbox/Services/IStorageService.cs:

public interface IStorageService
{
    Task<IReadOnlyList<StorageNode>> CollectStorageAsync(
        ClientContext ctx, StorageScanOptions options,
        IProgress<OperationProgress> progress, CancellationToken ct);

    Task<IReadOnlyList<FileTypeMetric>> CollectFileTypeMetricsAsync(
        ClientContext ctx,
        IProgress<OperationProgress> progress,
        CancellationToken ct);
}

From SharepointToolbox/Core/Helpers/ExecuteQueryRetryHelper.cs:

public static class ExecuteQueryRetryHelper
{
    public static async Task ExecuteQueryRetryAsync(
        ClientContext ctx,
        IProgress<OperationProgress>? progress = null,
        CancellationToken ct = default);
}

From SharepointToolbox/Services/StorageService.cs:

public class StorageService : IStorageService
{
    public async Task<IReadOnlyList<StorageNode>> CollectStorageAsync(...) { ... }
    private static async Task<StorageNode> LoadFolderNodeAsync(...) { ... }
    private static async Task CollectSubfoldersAsync(...) { ... }
}

Task 1: Implement CollectFileTypeMetricsAsync in StorageService SharepointToolbox/Services/StorageService.cs Add the `CollectFileTypeMetricsAsync` method to the existing `StorageService` class. Do NOT modify any existing methods (CollectStorageAsync, LoadFolderNodeAsync, CollectSubfoldersAsync). Add the new method after the existing `CollectStorageAsync` method.

Add this method to the `StorageService` class:

```csharp
public async Task<IReadOnlyList<FileTypeMetric>> CollectFileTypeMetricsAsync(
    ClientContext ctx,
    IProgress<OperationProgress> progress,
    CancellationToken ct)
{
    ct.ThrowIfCancellationRequested();

    // Load all non-hidden document libraries
    ctx.Load(ctx.Web,
        w => w.Lists.Include(
            l => l.Title,
            l => l.Hidden,
            l => l.BaseType,
            l => l.ItemCount));
    await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);

    var libs = ctx.Web.Lists
        .Where(l => !l.Hidden && l.BaseType == BaseType.DocumentLibrary)
        .ToList();

    // Accumulate file sizes by extension across all libraries
    var extensionMap = new Dictionary<string, (long totalSize, int count)>(StringComparer.OrdinalIgnoreCase);

    int libIdx = 0;
    foreach (var lib in libs)
    {
        ct.ThrowIfCancellationRequested();
        libIdx++;
        progress.Report(new OperationProgress(libIdx, libs.Count,
            $"Scanning files by type: {lib.Title} ({libIdx}/{libs.Count})"));

        // Use CamlQuery to enumerate all files in the library
        // Paginate with 500 items per batch to avoid list view threshold issues
        var query = new CamlQuery
        {
            ViewXml = @"<View Scope='RecursiveAll'>
                <Query>
                    <Where>
                        <Eq>
                            <FieldRef Name='FSObjType' />
                            <Value Type='Integer'>0</Value>
                        </Eq>
                    </Where>
                </Query>
                <ViewFields>
                    <FieldRef Name='FileLeafRef' />
                    <FieldRef Name='File_x0020_Size' />
                </ViewFields>
                <RowLimit Paged='TRUE'>500</RowLimit>
            </View>"
        };

        ListItemCollection items;
        do
        {
            ct.ThrowIfCancellationRequested();
            items = lib.GetItems(query);
            ctx.Load(items, ic => ic.ListItemCollectionPosition,
                ic => ic.Include(
                    i => i["FileLeafRef"],
                    i => i["File_x0020_Size"]));
            await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);

            foreach (var item in items)
            {
                string fileName = item["FileLeafRef"]?.ToString() ?? string.Empty;
                string sizeStr = item["File_x0020_Size"]?.ToString() ?? "0";

                if (!long.TryParse(sizeStr, out long fileSize))
                    fileSize = 0;

                string ext = Path.GetExtension(fileName).ToLowerInvariant();
                // ext is "" for extensionless files, ".docx" etc. for others

                if (extensionMap.TryGetValue(ext, out var existing))
                    extensionMap[ext] = (existing.totalSize + fileSize, existing.count + 1);
                else
                    extensionMap[ext] = (fileSize, 1);
            }

            // Move to next page
            query.ListItemCollectionPosition = items.ListItemCollectionPosition;
        }
        while (items.ListItemCollectionPosition != null);
    }

    // Convert to FileTypeMetric list, sorted by size descending
    return extensionMap
        .Select(kvp => new FileTypeMetric(kvp.Key, kvp.Value.totalSize, kvp.Value.count))
        .OrderByDescending(m => m.TotalSizeBytes)
        .ToList();
}
```

Make sure to add `using System.IO;` at the top of the file if not already present (for `Path.GetExtension`).

Design notes:
- Uses `Scope='RecursiveAll'` in CamlQuery to get files from all subfolders without explicit recursion
- `FSObjType=0` filter ensures only files (not folders) are returned
- Paged query with 500-item batches avoids list view threshold (5000 default) issues
- File_x0020_Size is the internal name for file size in SharePoint
- Extensions normalized to lowercase for consistent grouping (".DOCX" and ".docx" merge)
- Dictionary uses OrdinalIgnoreCase comparer as extra safety
- Existing methods (CollectStorageAsync, LoadFolderNodeAsync, CollectSubfoldersAsync) are NOT touched

cd "C:\Users\dev\Documents\projets\Sharepoint" && dotnet build SharepointToolbox/SharepointToolbox.csproj --no-incremental 2>&1 | tail -5 StorageService.cs implements CollectFileTypeMetricsAsync. Method enumerates files via CamlQuery with paging, groups by extension, returns IReadOnlyList<FileTypeMetric> sorted by TotalSizeBytes descending. Existing CollectStorageAsync is unchanged. Project compiles with 0 errors. - `dotnet build SharepointToolbox/SharepointToolbox.csproj` succeeds with 0 errors - StorageService now implements both IStorageService methods - CollectFileTypeMetricsAsync uses paginated CamlQuery (RowLimit 500, Paged=TRUE) - Extensions normalized to lowercase - Results sorted by TotalSizeBytes descending - No modifications to CollectStorageAsync, LoadFolderNodeAsync, or CollectSubfoldersAsync

<success_criteria> StorageService fully implements IStorageService. CollectFileTypeMetricsAsync can enumerate files by extension from any SharePoint site. The project compiles cleanly and existing storage scan behavior is unaffected. </success_criteria>

After completion, create `.planning/phases/09-storage-visualization/09-02-SUMMARY.md`