Sharepoint-Toolbox/.planning/milestones/v1.0-phases/04-bulk-operations-and-provisioning/04-RESEARCH.md

Phase 4: Bulk Operations and Provisioning - Research

Researched: 2026-04-03 Domain: SharePoint CSOM bulk operations, PnP Framework provisioning, Microsoft Graph group management, CSV parsing Confidence: HIGH

Summary

Phase 4 introduces five new tabs (Transfer, Bulk Members, Bulk Sites, Folder Structure, Templates) that perform write operations against SharePoint Online and Microsoft 365 Groups. The core challenge is implementing reliable per-item error handling with continue-on-error semantics, CSV import/validation/preview, and cancellation support -- all following the established FeatureViewModelBase pattern.

The file transfer feature uses CSOM's MoveCopyUtil for cross-site file copy/move operations. Bulk member addition requires Microsoft Graph SDK (new dependency) for M365 Group operations, with CSOM fallback for classic SharePoint groups. Site creation uses PnP Framework's SiteCollection.CreateAsync. Template capture reads site structure via CSOM properties, and template application manually recreates structure rather than using the heavy PnP Provisioning Engine. CSV parsing uses CsvHelper (new dependency). Folder structure creation uses CSOM's Folder.Folders.Add.

Primary recommendation: Use a shared BulkOperationRunner<T> pattern across all bulk operations to enforce continue-on-error, per-item reporting, cancellation, and retry-failed semantics consistently. Keep template capture/apply as manual CSOM operations (not PnP Provisioning Engine) to match the lightweight scope defined in CONTEXT.md.

<user_constraints>

User Constraints (from CONTEXT.md)

Locked Decisions

• Continue all items on error, report summary at end -- never stop on first failure
• Errors logged live in log panel (red) AND user can export CSV of failed items after completion
• Failed-items CSV includes: original row data, error message, timestamp -- user can fix and re-import
• "Retry Failed" button appears after completion with partial failures, re-runs only failed items
• User can also manually re-import the corrected failed-items CSV
• Always show confirmation summary dialog before any bulk write operation starts
• File transfer: user chooses Copy or Move; conflict policy per operation: Skip/Overwrite/Rename
• Move deletes source files only after successful transfer to destination
• Source and destination selection: site picker + library/folder tree browser on both sides
• Metadata preservation: best effort via CSOM; if rejected, log warning and continue
• Template capture: user selects what to capture via checkboxes
• Available capture options: Libraries, Folders, Permission groups, Site logo, Site settings
• No custom columns, content types, or navigation links in v1
• Template remembers source site type and creates same type on apply
• Templates persisted as JSON locally
• Dedicated Templates tab with list, capture, apply, rename/delete
• Bundled example CSV templates shipped with app for each bulk operation
• After CSV import: full validation pass + DataGrid preview with valid/invalid indicators
• CSV encoding: UTF-8 with BOM detection
• Each operation gets its own top-level tab: Transfer, Bulk Members, Bulk Sites, Folder Structure, Templates
• Each tab follows FeatureViewModelBase pattern (AsyncRelayCommand + IProgress + CancellationToken)

Claude's Discretion

• Exact CSV column names and schemas for each bulk operation type
• Tree browser component implementation details for file transfer source/destination
• Template JSON schema structure
• PnP Provisioning Engine usage details for template apply
• Confirmation dialog layout and wording
• DataGrid preview grid column configuration
• Retry mechanism implementation details

Deferred Ideas (OUT OF SCOPE)

None -- discussion stayed within phase scope </user_constraints>

<phase_requirements>

Phase Requirements

ID	Description	Research Support
BULK-01	Transfer files/folders between sites with progress tracking	MoveCopyUtil CSOM API for cross-site copy/move; download-then-upload fallback pattern from existing PowerShell
BULK-02	Add members to groups in bulk from CSV	Microsoft Graph SDK batch API (up to 20 members per PATCH); CSOM fallback for classic SP groups
BULK-03	Create multiple sites in bulk from CSV	PnP Framework SiteCollection.CreateAsync with TeamSiteCollectionCreationInformation / CommunicationSiteCollectionCreationInformation
BULK-04	All bulk operations support cancellation	CancellationToken threaded through BulkOperationRunner; check between items
BULK-05	Bulk errors reported per-item	BulkOperationResult model with per-item status; failed-items CSV export
TMPL-01	Capture site structure as template	CSOM Web/List/Folder/Group property reads; manual recursive folder enumeration
TMPL-02	Apply template to create new site	SiteCollection.CreateAsync + manual CSOM library/folder/group/settings creation
TMPL-03	Templates persist locally as JSON	System.Text.Json serialization following SettingsRepository pattern
TMPL-04	Manage templates (create, rename, delete)	TemplateRepository with same atomic write pattern as SettingsRepository
FOLD-01	Create folder structures from CSV	CSOM Folder.Folders.Add with parent-first ordering
FOLD-02	Example CSV templates provided	Existing /examples/ CSV files already present; bundle as embedded resources
</phase_requirements>

Standard Stack

Core (already in project)

Library	Version	Purpose	Why Standard
PnP.Framework	1.18.0	SharePoint CSOM extensions, site creation, folder/file operations	Already used; provides SiteCollection.CreateAsync, MoveCopyUtil wrappers
CommunityToolkit.Mvvm	8.4.2	MVVM pattern, ObservableProperty, AsyncRelayCommand	Already used for all ViewModels
Microsoft.Identity.Client	4.83.3	MSAL authentication	Already used; Graph SDK will share token cache
Serilog	4.3.1	Structured logging	Already used for all operations
System.Text.Json	(built-in)	JSON serialization for templates	Already used in SettingsRepository/ProfileRepository

New Dependencies

Library	Version	Purpose	Why Needed
CsvHelper	33.1.0	CSV parsing with type mapping, validation, BOM handling	De facto standard for .NET CSV; handles encoding, quoting, type conversion automatically. Avoids hand-rolling parser
Microsoft.Graph	5.x (latest stable)	M365 Group member management, batch API	Required for BULK-02 (adding members to M365 Groups); CSOM cannot manage M365 Group membership directly
Microsoft.Graph.Core	3.x (transitive)	Graph SDK core/auth	Transitive dependency of Microsoft.Graph

Alternatives Considered

Instead of	Could Use	Tradeoff
CsvHelper	Manual parsing (Split + StreamReader)	CsvHelper handles RFC 4180 edge cases, BOM detection, type mapping. Manual parsing risks bugs on quoted fields, encoding
Microsoft.Graph SDK	Raw HTTP to Graph API	SDK handles auth token injection, batch splitting, retry, serialization. Not worth hand-rolling
PnP Provisioning Engine for templates	Manual CSOM property reads + writes	Provisioning Engine is heavyweight, captures far more than needed (content types, navigation, custom actions). Manual approach matches PS app behavior and v1 scope

Installation:

dotnet add SharepointToolbox/SharepointToolbox.csproj package CsvHelper --version 33.1.0
dotnet add SharepointToolbox/SharepointToolbox.csproj package Microsoft.Graph --version 5.74.0

Architecture Patterns

Recommended Project Structure (new files)

SharepointToolbox/
  Core/
    Models/
      BulkOperationResult.cs       # Per-item result: Success/Failed/Skipped + error message
      BulkMemberRow.cs             # CSV row model for member addition
      BulkSiteRow.cs               # CSV row model for site creation
      TransferJob.cs               # Source/dest site+library+conflict policy
      FolderStructureRow.cs        # CSV row model for folder structure
      SiteTemplate.cs              # Template JSON model
      SiteTemplateOptions.cs       # Capture options (booleans for each section)
      TemplateLibraryInfo.cs       # Library captured in template
      TemplateFolderInfo.cs        # Folder tree captured in template
      TemplatePermissionGroup.cs   # Permission group captured in template
  Infrastructure/
    Persistence/
      TemplateRepository.cs        # JSON persistence for templates (like SettingsRepository)
  Services/
    IFileTransferService.cs        # Interface for file copy/move operations
    FileTransferService.cs         # CSOM MoveCopyUtil + download/upload fallback
    IBulkMemberService.cs          # Interface for bulk member addition
    BulkMemberService.cs           # Graph SDK batch API + CSOM fallback
    IBulkSiteService.cs            # Interface for bulk site creation
    BulkSiteService.cs             # PnP Framework SiteCollection.CreateAsync
    ITemplateService.cs            # Interface for template capture/apply
    TemplateService.cs             # CSOM property reads for capture, CSOM writes for apply
    IFolderStructureService.cs     # Interface for folder creation from CSV
    FolderStructureService.cs      # CSOM folder creation
    ICsvValidationService.cs       # Interface for CSV validation + preview data
    CsvValidationService.cs        # CsvHelper-based parsing, schema validation, row validation
    Export/
      BulkResultCsvExportService.cs # Failed-items CSV export (re-importable format)
  ViewModels/
    Tabs/
      TransferViewModel.cs
      BulkMembersViewModel.cs
      BulkSitesViewModel.cs
      FolderStructureViewModel.cs
      TemplatesViewModel.cs
  Views/
    Tabs/
      TransferView.xaml(.cs)
      BulkMembersView.xaml(.cs)
      BulkSitesView.xaml(.cs)
      FolderStructureView.xaml(.cs)
      TemplatesView.xaml(.cs)
    Dialogs/
      ConfirmBulkOperationDialog.xaml(.cs)  # Pre-write confirmation
      FolderBrowserDialog.xaml(.cs)         # Tree browser for library/folder selection

Pattern 1: BulkOperationRunner (continue-on-error with per-item reporting)

What: A generic helper that iterates items, catches per-item exceptions, tracks results, and checks cancellation between items. When to use: Every bulk operation (transfer, members, sites, folders). Example:

// Core pattern shared by all bulk operations
public static class BulkOperationRunner
{
    public static async Task<BulkOperationSummary<TItem>> RunAsync<TItem>(
        IReadOnlyList<TItem> items,
        Func<TItem, int, CancellationToken, Task> processItem,
        IProgress<OperationProgress> progress,
        CancellationToken ct)
    {
        var results = new List<BulkItemResult<TItem>>();
        for (int i = 0; i < items.Count; i++)
        {
            ct.ThrowIfCancellationRequested();
            progress.Report(new OperationProgress(i + 1, items.Count, $"Processing {i + 1}/{items.Count}..."));
            try
            {
                await processItem(items[i], i, ct);
                results.Add(BulkItemResult<TItem>.Success(items[i]));
            }
            catch (OperationCanceledException) { throw; } // cancellation propagates
            catch (Exception ex)
            {
                results.Add(BulkItemResult<TItem>.Failed(items[i], ex.Message));
            }
        }
        return new BulkOperationSummary<TItem>(results);
    }
}

Pattern 2: CSV Import Pipeline (validate -> preview -> execute)

What: Three-step flow: parse CSV with CsvHelper, validate all rows, present DataGrid preview with valid/invalid indicators, then execute on user confirmation. When to use: Bulk Members, Bulk Sites, Folder Structure tabs. Example:

// CsvHelper usage with BOM-tolerant configuration
public List<CsvValidationRow<T>> ParseAndValidate<T>(Stream csvStream)
{
    using var reader = new StreamReader(csvStream, Encoding.UTF8, detectEncodingFromByteOrderMarks: true);
    using var csv = new CsvReader(reader, new CsvConfiguration(CultureInfo.InvariantCulture)
    {
        HasHeaderRecord = true,
        MissingFieldFound = null,        // handle missing fields gracefully
        HeaderValidated = null,          // custom validation instead
        DetectDelimiter = true,          // auto-detect ; vs ,
        TrimOptions = TrimOptions.Trim,
    });
    
    var rows = new List<CsvValidationRow<T>>();
    csv.Read();
    csv.ReadHeader();
    while (csv.Read())
    {
        try
        {
            var record = csv.GetRecord<T>();
            var errors = Validate(record);
            rows.Add(new CsvValidationRow<T>(record, errors));
        }
        catch (Exception ex)
        {
            rows.Add(CsvValidationRow<T>.ParseError(csv.Context.Parser.RawRecord, ex.Message));
        }
    }
    return rows;
}

Pattern 3: Failed-Items CSV Export (re-importable)

What: Export failed items as CSV identical to input format but with appended Error and Timestamp columns. User can fix and re-import. When to use: After any bulk operation completes with partial failures. Example:

// Failed-items CSV: same columns as input + Error + Timestamp
public async Task ExportFailedItemsAsync<T>(
    IReadOnlyList<BulkItemResult<T>> failedItems,
    string filePath, CancellationToken ct)
{
    await using var writer = new StreamWriter(filePath, false, new UTF8Encoding(true));
    await using var csv = new CsvWriter(writer, CultureInfo.InvariantCulture);
    
    // Write original columns + error column
    csv.WriteHeader<T>();
    csv.WriteField("Error");
    csv.WriteField("Timestamp");
    await csv.NextRecordAsync();
    
    foreach (var item in failedItems.Where(r => !r.IsSuccess))
    {
        csv.WriteRecord(item.Item);
        csv.WriteField(item.ErrorMessage);
        csv.WriteField(DateTime.UtcNow.ToString("o"));
        await csv.NextRecordAsync();
    }
}

Pattern 4: File Transfer via CSOM (download-then-upload approach)

What: The existing PowerShell app downloads files to a temp folder, then uploads to the destination. This is the most reliable cross-site approach when using CSOM directly. MoveCopyUtil is an alternative for same-tenant operations. When to use: BULK-01 file transfer. Example:

// Approach A: MoveCopyUtil (preferred for same-tenant, simpler)
var srcPath = ResourcePath.FromDecodedUrl(sourceFileServerRelativeUrl);
var dstPath = ResourcePath.FromDecodedUrl(destFileServerRelativeUrl);
MoveCopyUtil.CopyFileByPath(ctx, srcPath, dstPath, overwrite, new MoveCopyOptions
{
    KeepBoth = false, // or true for "Rename" conflict policy
    ResetAuthorAndCreatedOnCopy = false,
});
await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);

// Approach B: Download-then-upload (reliable cross-site fallback)
// 1. Download: ctx.Web.GetFileByServerRelativeUrl(url).OpenBinaryStream()
// 2. Upload: destFolder.Files.Add(new FileCreationInformation { ContentStream, Overwrite, Url })

Anti-Patterns to Avoid

• Stopping on first error in bulk operations: All bulk ops MUST continue-on-error per CONTEXT.md. Never throw from the item loop.
• Using PnP Provisioning Engine for template capture/apply: It captures far more than needed (content types, custom actions, navigation, page layouts). The v1 scope only captures libraries, folders, permission groups, logo, settings. Manual CSOM reads are simpler, lighter, and match the PS app behavior.
• Parsing CSV manually with string.Split: CSV has too many edge cases (quoted fields containing delimiters, embedded newlines, BOM). Use CsvHelper.
• Creating M365 Groups via CSOM for member addition: CSOM cannot manage M365 Group membership. Must use Microsoft Graph API.
• Blocking UI thread during file download/upload: All service methods are async, run via FeatureViewModelBase.RunOperationAsync.

Don't Hand-Roll

Problem	Don't Build	Use Instead	Why
CSV parsing	Custom StreamReader + Split	CsvHelper 33.1.0	RFC 4180 quoting, BOM detection, delimiter detection, type mapping, error handling
M365 Group member management	Raw HTTP calls to Graph	Microsoft.Graph SDK 5.x	Token management, batch splitting (20-per-request auto), retry, deserialization
Bulk operation error handling	Copy-paste try/catch in each ViewModel	Shared BulkOperationRunner	Ensures consistent continue-on-error, per-item tracking, cancellation, and retry-failed across all 5 bulk operations
CSV field escaping on export	Manual quote doubling	CsvHelper CsvWriter	RFC 4180 compliance, handles all edge cases
Template JSON serialization	Manual JSON string building	System.Text.Json with typed models	Already used in project; handles nulls, escaping, indentation

Key insight: The bulk operation infrastructure (runner, result model, failed-items export, retry) is shared across all five features. Building it as a reusable component in Wave 0 prevents duplicated error handling logic and ensures consistent behavior.

Common Pitfalls

Pitfall 1: MoveCopyUtil fails on cross-site-collection operations with special characters

What goes wrong: MoveCopyUtil.MoveFileByPath / CopyFileByPath can fail silently or throw when file/folder names contain special characters (#, %, accented characters) in cross-site-collection scenarios. Why it happens: SharePoint URL encoding differences between site collections. How to avoid: Use ResourcePath.FromDecodedUrl() (not string URLs) for all MoveCopyUtil calls. For files with problematic names, fall back to download-then-upload approach. Warning signs: Sporadic failures on files with non-ASCII names.

Pitfall 2: M365 Group member addition requires Graph permissions, not SharePoint permissions

What goes wrong: App registration has SharePoint permissions but not Graph permissions. Member addition silently fails or returns 403. Why it happens: M365 Groups are Azure AD objects, not SharePoint objects. Adding members requires GroupMember.ReadWrite.All or Group.ReadWrite.All Graph permission. How to avoid: Document required Graph permissions. Detect permission errors and surface clear message: "App registration needs Group.ReadWrite.All permission." Warning signs: 403 Forbidden on Graph batch requests.

Pitfall 3: Site creation is asynchronous -- polling required

What goes wrong: SiteCollection.CreateAsync returns a URL but the site may not be immediately accessible. Subsequent operations (add libraries, folders) fail with 404. Why it happens: SharePoint site provisioning is asynchronous. The site creation API returns before all components are ready. How to avoid: After site creation, poll the site URL with retry/backoff until it responds (up to 2-3 minutes for Teams sites). The existing ExecuteQueryRetryHelper pattern can be adapted. Warning signs: 404 errors when connecting to newly created sites.

Pitfall 4: CancellationToken must be checked between items, not within CSOM calls

What goes wrong: Cancellation appears unresponsive because CSOM ExecuteQueryAsync doesn't accept CancellationToken natively. Why it happens: CSOM's ExecuteQueryAsync() has no CancellationToken overload. How to avoid: Check ct.ThrowIfCancellationRequested() before each item in the bulk loop. For long-running single-item operations (large file transfer), check between download and upload phases. Warning signs: Cancel button pressed but operation continues for a long time.

Pitfall 5: CSV delimiter detection -- semicolon vs comma

What goes wrong: European Excel exports use semicolons; North American exports use commas. Wrong delimiter means all data lands in one column. Why it happens: Excel's CSV export uses the system's list separator, which varies by locale. How to avoid: CsvHelper's DetectDelimiter = true handles this automatically. The existing PS app already does manual detection (checking for ;). CsvHelper is more robust. Warning signs: All columns merged into first column during import.

Pitfall 6: TeamSite creation requires at least one owner

What goes wrong: SiteCollection.CreateAsync with TeamSiteCollectionCreationInformation fails if no owner is specified. Why it happens: M365 Groups require at least one owner. How to avoid: Pre-flight CSV validation must flag rows with Type=Team and empty Owners as invalid. The PS app already checks this (line 5862). Warning signs: Cryptic error from Graph API during site creation.

Pitfall 7: Template capture -- system lists must be excluded

What goes wrong: Capturing all lists includes system libraries (Style Library, Site Pages, Form Templates, etc.) that fail or create duplicates on apply. Why it happens: CSOM Web.Lists returns all lists, including hidden system ones. How to avoid: Filter out hidden lists (list.Hidden == true) and system templates (BaseTemplate check). The PS app filters with !$_.Hidden (line 936). Also exclude well-known system lists by name: "Style Library", "Form Templates", "Site Assets" (unless user-created). Warning signs: Error "list already exists" when applying template.

Code Examples

File Copy/Move with MoveCopyUtil (CSOM)

// Source: https://learn.microsoft.com/en-us/dotnet/api/microsoft.sharepoint.client.movecopyutil
public async Task CopyFileAsync(
    ClientContext ctx,
    string sourceServerRelativeUrl,
    string destServerRelativeUrl,
    ConflictPolicy conflictPolicy,
    IProgress<OperationProgress> progress,
    CancellationToken ct)
{
    var srcPath = ResourcePath.FromDecodedUrl(sourceServerRelativeUrl);
    var dstPath = ResourcePath.FromDecodedUrl(destServerRelativeUrl);
    
    var options = new MoveCopyOptions
    {
        KeepBoth = conflictPolicy == ConflictPolicy.Rename,
        ResetAuthorAndCreatedOnCopy = false, // preserve metadata best-effort
    };
    
    bool overwrite = conflictPolicy == ConflictPolicy.Overwrite;
    MoveCopyUtil.CopyFileByPath(ctx, srcPath, dstPath, overwrite, options);
    await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);
}

Download-then-Upload Fallback (cross-site)

// Source: existing PowerShell app pattern (Sharepoint_ToolBox.ps1 lines 5362-5427)
public async Task TransferFileViaStreamAsync(
    ClientContext srcCtx, string srcServerRelUrl,
    ClientContext dstCtx, string dstFolderServerRelUrl, string fileName,
    bool overwrite,
    IProgress<OperationProgress> progress, CancellationToken ct)
{
    // Download from source
    var fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(srcCtx, srcServerRelUrl);
    using var memStream = new MemoryStream();
    await fileInfo.Stream.CopyToAsync(memStream, ct);
    memStream.Position = 0;
    
    // Upload to destination
    var destFolder = srcCtx.Web.GetFolderByServerRelativeUrl(dstFolderServerRelUrl);
    var fileCreation = new FileCreationInformation
    {
        ContentStream = memStream,
        Url = fileName,
        Overwrite = overwrite,
    };
    destFolder.Files.Add(fileCreation);
    await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(dstCtx, progress, ct);
}

Site Creation with PnP Framework

// Source: https://pnp.github.io/pnpframework/api/PnP.Framework.Sites.SiteCollection.html
public async Task<string> CreateTeamSiteAsync(
    ClientContext adminCtx, string title, string alias,
    string? description, CancellationToken ct)
{
    var creationInfo = new TeamSiteCollectionCreationInformation
    {
        DisplayName = title,
        Alias = alias,
        Description = description ?? string.Empty,
        IsPublic = false,
    };
    
    using var siteCtx = await adminCtx.CreateSiteAsync(creationInfo);
    siteCtx.Load(siteCtx.Web, w => w.Url);
    await siteCtx.ExecuteQueryAsync();
    return siteCtx.Web.Url;
}

public async Task<string> CreateCommunicationSiteAsync(
    ClientContext adminCtx, string title, string siteUrl,
    string? description, CancellationToken ct)
{
    var creationInfo = new CommunicationSiteCollectionCreationInformation
    {
        Title = title,
        Url = siteUrl,
        Description = description ?? string.Empty,
    };
    
    using var siteCtx = await adminCtx.CreateSiteAsync(creationInfo);
    siteCtx.Load(siteCtx.Web, w => w.Url);
    await siteCtx.ExecuteQueryAsync();
    return siteCtx.Web.Url;
}

Graph SDK Batch Member Addition

// Source: https://learn.microsoft.com/en-us/graph/api/group-post-members
// Source: https://martin-machacek.com/blogPost/0b71abb2-87c9-4c88-9157-eb1ae4d6603b
public async Task AddMembersToGroupAsync(
    GraphServiceClient graphClient, string groupId,
    IReadOnlyList<string> userPrincipalNames,
    CancellationToken ct)
{
    // Graph PATCH can add up to 20 members at once via members@odata.bind
    foreach (var batch in userPrincipalNames.Chunk(20))
    {
        ct.ThrowIfCancellationRequested();
        var memberRefs = batch.Select(upn =>
            $"https://graph.microsoft.com/v1.0/users/{upn}").ToList();
        
        var requestBody = new Group
        {
            AdditionalData = new Dictionary<string, object>
            {
                { "members@odata.bind", memberRefs }
            }
        };
        
        await graphClient.Groups[groupId].PatchAsync(requestBody, cancellationToken: ct);
    }
}

Template Capture via CSOM

// Source: existing PowerShell pattern (Sharepoint_ToolBox.ps1 lines 909-1024)
public async Task<SiteTemplate> CaptureTemplateAsync(
    ClientContext ctx, SiteTemplateOptions options,
    IProgress<OperationProgress> progress, CancellationToken ct)
{
    var web = ctx.Web;
    ctx.Load(web, w => w.Title, w => w.Description, w => w.Language,
             w => w.SiteLogoUrl, w => w.WebTemplate, w => w.ServerRelativeUrl);
    await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);
    
    var template = new SiteTemplate
    {
        Name = string.Empty, // set by caller
        SourceUrl = ctx.Url,
        CapturedAt = DateTime.UtcNow,
        SiteType = web.WebTemplate == "GROUP" ? "Team" : "Communication",
        Options = options,
    };
    
    if (options.CaptureSettings)
    {
        template.Settings = new TemplateSettings
        {
            Title = web.Title,
            Description = web.Description,
            Language = (int)web.Language,
        };
    }
    
    if (options.CaptureLogo)
    {
        template.Logo = new TemplateLogo { LogoUrl = web.SiteLogoUrl };
    }
    
    if (options.CaptureLibraries)
    {
        var lists = ctx.LoadQuery(web.Lists.Where(l => !l.Hidden));
        await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);
        
        foreach (var list in lists)
        {
            ct.ThrowIfCancellationRequested();
            // Load root folder + enumerate folders recursively
            ctx.Load(list, l => l.Title, l => l.BaseType, l => l.BaseTemplate, l => l.RootFolder);
            await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);
            
            var folders = await EnumerateFoldersRecursiveAsync(ctx, list.RootFolder, progress, ct);
            template.Libraries.Add(new TemplateLibraryInfo
            {
                Name = list.Title,
                BaseType = list.BaseType.ToString(),
                BaseTemplate = (int)list.BaseTemplate,
                Folders = folders,
            });
        }
    }
    
    return template;
}

Folder Structure Creation from CSV

// Source: existing PowerShell pattern (Sharepoint_ToolBox.ps1 lines 6162-6193)
public async Task CreateFoldersAsync(
    ClientContext ctx, string libraryTitle,
    IReadOnlyList<string> folderPaths, // sorted parent-first
    IProgress<OperationProgress> progress, CancellationToken ct)
{
    var list = ctx.Web.Lists.GetByTitle(libraryTitle);
    ctx.Load(list, l => l.RootFolder.ServerRelativeUrl);
    await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);
    
    var baseUrl = list.RootFolder.ServerRelativeUrl.TrimEnd('/');
    
    for (int i = 0; i < folderPaths.Count; i++)
    {
        ct.ThrowIfCancellationRequested();
        progress.Report(new OperationProgress(i + 1, folderPaths.Count,
            $"Creating folder {i + 1}/{folderPaths.Count}: {folderPaths[i]}"));
        
        // Resolve creates all intermediate folders if they don't exist
        var folder = ctx.Web.Folders.Add($"{baseUrl}/{folderPaths[i]}");
        await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);
    }
}

Recommended CSV Schemas

Bulk Members (bulk_add_members.csv):

GroupName,GroupUrl,Email,Role
Marketing Team,https://contoso.sharepoint.com/sites/Marketing,user1@contoso.com,Member
Marketing Team,https://contoso.sharepoint.com/sites/Marketing,manager@contoso.com,Owner

Note: The existing example only has Email. Extending with GroupName/GroupUrl/Role enables bulk addition to multiple groups.

Bulk Sites (bulk_create_sites.csv) -- matches existing:

Name;Alias;Type;Template;Owners;Members

Keep semicolon delimiter for Excel compatibility. CsvHelper's DetectDelimiter handles both.

Folder Structure (folder_structure.csv) -- matches existing:

Level1;Level2;Level3;Level4

Hierarchical columns. Non-empty cells build the path. Already present in /examples/.

State of the Art

Old Approach	Current Approach	When Changed	Impact
PnP Sites Core (OfficeDevPnP.Core)	PnP.Framework 1.18.0	2021 migration	Same API surface, new namespace. Project already uses PnP.Framework
SP.MoveCopyUtil (JavaScript/REST)	CSOM MoveCopyUtil (C#)	Always available	Same underlying SharePoint API, different entry point
Microsoft Graph SDK v4	Microsoft Graph SDK v5	2023	New fluent API, BatchRequestContentCollection, breaking namespace changes
Manual Graph HTTP calls	Graph SDK batch with auto-split	v5+	SDK handles 20-per-batch splitting automatically

Deprecated/outdated:

• PnP Sites Core (OfficeDevPnP.Core): Replaced by PnP.Framework. Do not reference the old package.
• Graph SDK v4 BatchRequestContent: Replaced by v5 BatchRequestContentCollection which auto-splits beyond 20 requests.

Open Questions

1. Graph SDK authentication integration with MSAL

   • What we know: The project uses MsalClientFactory to create PublicClientApplication instances. Graph SDK needs a TokenCredentialProvider.
   • What's unclear: Exact wiring to share the same MSAL token cache between PnP Framework and Graph SDK.
   • Recommendation: Create a GraphClientFactory that obtains tokens from the same MsalClientFactory. Use DelegateAuthenticationProvider or BaseBearerTokenAuthenticationProvider wrapping the existing PCA. Research this during implementation -- the token scopes differ (Graph needs https://graph.microsoft.com/.default, PnP uses https://{tenant}.sharepoint.com/.default).

2. MoveCopyUtil vs download-then-upload for cross-site-collection

   • What we know: MoveCopyUtil works within the same tenant. The PS app uses download-then-upload.
   • What's unclear: Whether MoveCopyUtil handles all cross-site-collection scenarios reliably (special characters, large files).
   • Recommendation: Implement MoveCopyUtil as primary approach (simpler, server-side). Fall back to download-then-upload if MoveCopyUtil fails. The fallback is proven in the PS app.

3. Bulk member CSV schema -- Group identification

   • What we know: The current example CSV only has Email column. For bulk addition to multiple groups, we need group identification.
   • What's unclear: Whether users want to add to one selected group (UI picker) or multiple groups (CSV column).
   • Recommendation: Support both -- UI group picker for single-group scenario, CSV with GroupUrl column for multi-group. The CSV schema is Claude's discretion per CONTEXT.md.

Validation Architecture

Test Framework

Property	Value
Framework	xunit 2.9.3 + Moq 4.20.72
Config file	SharepointToolbox.Tests/SharepointToolbox.Tests.csproj
Quick run command	dotnet test SharepointToolbox.Tests --filter "Category!=Integration" --no-build -q
Full suite command	dotnet test SharepointToolbox.Tests --no-build

Phase Requirements to Test Map

Req ID	Behavior	Test Type	Automated Command	File Exists?
BULK-01	File transfer service handles copy/move/conflict policies	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~FileTransferService -x	Wave 0
BULK-02	Bulk member service processes CSV rows, calls Graph API	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~BulkMemberService -x	Wave 0
BULK-03	Bulk site service creates team/communication sites from CSV	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~BulkSiteService -x	Wave 0
BULK-04	BulkOperationRunner stops on cancellation	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~BulkOperationRunner -x	Wave 0
BULK-05	BulkOperationRunner collects per-item errors	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~BulkOperationRunner -x	Wave 0
TMPL-01	Template service captures site structure correctly	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~TemplateService -x	Wave 0
TMPL-02	Template service applies template to new site	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~TemplateService -x	Wave 0
TMPL-03	TemplateRepository persists/loads JSON correctly	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~TemplateRepository -x	Wave 0
TMPL-04	TemplateRepository supports CRUD operations	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~TemplateRepository -x	Wave 0
FOLD-01	Folder structure service creates folders from parsed paths	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~FolderStructureService -x	Wave 0
FOLD-02	Example CSVs parse correctly with CsvValidationService	unit	dotnet test SharepointToolbox.Tests --filter FullyQualifiedName~CsvValidation -x	Wave 0

Sampling Rate

• Per task commit: dotnet test SharepointToolbox.Tests --filter "Category!=Integration" --no-build -q
• Per wave merge: dotnet test SharepointToolbox.Tests --no-build
• Phase gate: Full suite green before /gsd:verify-work

Wave 0 Gaps

• SharepointToolbox.Tests/Services/BulkOperationRunnerTests.cs -- covers BULK-04, BULK-05
• SharepointToolbox.Tests/Services/CsvValidationServiceTests.cs -- covers FOLD-02, CSV parsing
• SharepointToolbox.Tests/Services/TemplateRepositoryTests.cs -- covers TMPL-03, TMPL-04
• SharepointToolbox.Tests/Services/BulkResultCsvExportServiceTests.cs -- covers failed-items export
• CsvHelper package added to test project if needed for test CSV generation

Sources

Primary (HIGH confidence)

• PnP Framework API - SiteCollection - Site creation methods
• PnP Framework API - CommunicationSiteCollectionCreationInformation - Communication site creation
• CSOM MoveCopyUtil.CopyFileByPath - File copy API
• CSOM MoveCopyUtil.MoveFileByPath - File move API
• Microsoft Graph - Add members to group - Graph member addition
• CsvHelper official site - CSV parsing library
• CsvHelper NuGet - Version 33.1.0

Secondary (MEDIUM confidence)

• Provisioning modern team sites programmatically - Site creation patterns
• PnP Core SDK - Copy/Move content - CreateCopyJobs pattern documentation
• Graph SDK batch member addition example - Batch API usage
• Existing PowerShell app (Sharepoint_ToolBox.ps1) - Proven patterns for all operations

Tertiary (LOW confidence)

• Graph SDK authentication integration with existing MSAL setup - needs validation during implementation

Metadata

Confidence breakdown:

• Standard stack: HIGH - PnP.Framework already in use; CsvHelper and Graph SDK are de facto standards with official documentation
• Architecture: HIGH - Patterns directly port from working PowerShell app; BulkOperationRunner is a well-understood generic pattern
• Pitfalls: HIGH - Documented from real-world experience in the PS app and Microsoft issue trackers
• Template capture/apply: MEDIUM - Manual CSOM approach is straightforward but may hit edge cases with specific site types or regional settings
• Graph SDK auth wiring: LOW - Needs validation; sharing MSAL tokens between PnP and Graph SDK has nuances

Research date: 2026-04-03 Valid until: 2026-05-03 (stable APIs, PnP Framework release cycle is quarterly)