Sharepoint-Toolbox/.planning/milestones/v1.0-phases/04-bulk-operations-and-provisioning/04-03-PLAN.md

---
phase: 04
plan: 03
title: FileTransferService Implementation
status: pending
wave: 1
depends_on:
  - 04-01
files_modified:
  - SharepointToolbox/Services/FileTransferService.cs
  - SharepointToolbox.Tests/Services/FileTransferServiceTests.cs
autonomous: true
requirements:
  - BULK-01
  - BULK-04
  - BULK-05

must_haves:
  truths:
    - "FileTransferService copies files using CSOM MoveCopyUtil.CopyFileByPath with ResourcePath.FromDecodedUrl"
    - "FileTransferService moves files using MoveCopyUtil.MoveFileByPath then deletes source only after success"
    - "Conflict policy maps to MoveCopyOptions: Skip=catch-and-skip, Overwrite=overwrite:true, Rename=KeepBoth:true"
    - "Recursive folder enumeration collects all files before transferring"
    - "BulkOperationRunner handles per-file error reporting and cancellation"
    - "Metadata preservation is best-effort (ResetAuthorAndCreatedOnCopy=false)"
  artifacts:
    - path: "SharepointToolbox/Services/FileTransferService.cs"
      provides: "CSOM file transfer with copy/move/conflict support"
      exports: ["FileTransferService"]
  key_links:
    - from: "FileTransferService.cs"
      to: "BulkOperationRunner.cs"
      via: "per-file processing delegation"
      pattern: "BulkOperationRunner.RunAsync"
    - from: "FileTransferService.cs"
      to: "MoveCopyUtil"
      via: "CSOM file operations"
      pattern: "MoveCopyUtil.CopyFileByPath|MoveFileByPath"
---

# Plan 04-03: FileTransferService Implementation

## Goal

Implement `FileTransferService` for copying and moving files/folders between SharePoint sites using CSOM `MoveCopyUtil`. Supports Copy/Move modes, Skip/Overwrite/Rename conflict policies, recursive folder transfer, best-effort metadata preservation, and per-file error reporting via `BulkOperationRunner`.

## Context

`IFileTransferService`, `TransferJob`, `ConflictPolicy`, `TransferMode`, and `BulkOperationRunner` are defined in Plan 04-01. The service follows the established pattern: receives `ClientContext` as parameter, uses `ExecuteQueryRetryHelper` for all CSOM calls, never stores contexts.

Key CSOM APIs:
- `MoveCopyUtil.CopyFileByPath(ctx, srcPath, dstPath, overwrite, options)` for copy
- `MoveCopyUtil.MoveFileByPath(ctx, srcPath, dstPath, overwrite, options)` for move
- `ResourcePath.FromDecodedUrl()` required for special characters (Pitfall 1)
- `MoveCopyOptions.KeepBoth = true` for Rename conflict policy
- `MoveCopyOptions.ResetAuthorAndCreatedOnCopy = false` for metadata preservation

## Tasks

### Task 1: Implement FileTransferService

**Files:**
- `SharepointToolbox/Services/FileTransferService.cs`

**Action:**

Create `FileTransferService.cs`:
```csharp
using System.IO;
using Microsoft.SharePoint.Client;
using Serilog;
using SharepointToolbox.Core.Models;
using SharepointToolbox.Infrastructure.Auth;

namespace SharepointToolbox.Services;

public class FileTransferService : IFileTransferService
{
    public async Task<BulkOperationSummary<string>> TransferAsync(
        ClientContext sourceCtx,
        ClientContext destCtx,
        TransferJob job,
        IProgress<OperationProgress> progress,
        CancellationToken ct)
    {
        // 1. Enumerate files from source
        progress.Report(new OperationProgress(0, 0, "Enumerating source files..."));
        var files = await EnumerateFilesAsync(sourceCtx, job, progress, ct);

        if (files.Count == 0)
        {
            progress.Report(new OperationProgress(0, 0, "No files found to transfer."));
            return new BulkOperationSummary<string>(new List<BulkItemResult<string>>());
        }

        // 2. Build source and destination base paths
        var srcBasePath = BuildServerRelativePath(sourceCtx, job.SourceLibrary, job.SourceFolderPath);
        var dstBasePath = BuildServerRelativePath(destCtx, job.DestinationLibrary, job.DestinationFolderPath);

        // 3. Transfer each file using BulkOperationRunner
        return await BulkOperationRunner.RunAsync(
            files,
            async (fileRelUrl, idx, token) =>
            {
                // Compute destination path by replacing source base with dest base
                var relativePart = fileRelUrl;
                if (fileRelUrl.StartsWith(srcBasePath, StringComparison.OrdinalIgnoreCase))
                    relativePart = fileRelUrl.Substring(srcBasePath.Length).TrimStart('/');

                // Ensure destination folder exists
                var destFolderRelative = dstBasePath;
                var fileFolder = Path.GetDirectoryName(relativePart)?.Replace('\\', '/');
                if (!string.IsNullOrEmpty(fileFolder))
                {
                    destFolderRelative = $"{dstBasePath}/{fileFolder}";
                    await EnsureFolderAsync(destCtx, destFolderRelative, progress, token);
                }

                var fileName = Path.GetFileName(relativePart);
                var destFileUrl = $"{destFolderRelative}/{fileName}";

                await TransferSingleFileAsync(sourceCtx, destCtx, fileRelUrl, destFileUrl, job, progress, token);

                Log.Information("Transferred: {Source} -> {Dest}", fileRelUrl, destFileUrl);
            },
            progress,
            ct);
    }

    private async Task TransferSingleFileAsync(
        ClientContext sourceCtx,
        ClientContext destCtx,
        string srcFileUrl,
        string dstFileUrl,
        TransferJob job,
        IProgress<OperationProgress> progress,
        CancellationToken ct)
    {
        var srcPath = ResourcePath.FromDecodedUrl(srcFileUrl);
        var dstPath = ResourcePath.FromDecodedUrl(dstFileUrl);

        bool overwrite = job.ConflictPolicy == ConflictPolicy.Overwrite;
        var options = new MoveCopyOptions
        {
            KeepBoth = job.ConflictPolicy == ConflictPolicy.Rename,
            ResetAuthorAndCreatedOnCopy = false, // best-effort metadata preservation
        };

        try
        {
            if (job.Mode == TransferMode.Copy)
            {
                MoveCopyUtil.CopyFileByPath(sourceCtx, srcPath, dstPath, overwrite, options);
                await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(sourceCtx, progress, ct);
            }
            else // Move
            {
                MoveCopyUtil.MoveFileByPath(sourceCtx, srcPath, dstPath, overwrite, options);
                await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(sourceCtx, progress, ct);
            }
        }
        catch (ServerException ex) when (job.ConflictPolicy == ConflictPolicy.Skip &&
                                          ex.Message.Contains("already exists", StringComparison.OrdinalIgnoreCase))
        {
            Log.Warning("Skipped (already exists): {File}", srcFileUrl);
        }
    }

    private async Task<IReadOnlyList<string>> EnumerateFilesAsync(
        ClientContext ctx,
        TransferJob job,
        IProgress<OperationProgress> progress,
        CancellationToken ct)
    {
        var list = ctx.Web.Lists.GetByTitle(job.SourceLibrary);
        var rootFolder = list.RootFolder;
        ctx.Load(rootFolder, f => f.ServerRelativeUrl);
        await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);

        var baseFolderUrl = rootFolder.ServerRelativeUrl.TrimEnd('/');
        if (!string.IsNullOrEmpty(job.SourceFolderPath))
            baseFolderUrl = $"{baseFolderUrl}/{job.SourceFolderPath.TrimStart('/')}";

        var folder = ctx.Web.GetFolderByServerRelativeUrl(baseFolderUrl);
        var files = new List<string>();
        await CollectFilesRecursiveAsync(ctx, folder, files, progress, ct);
        return files;
    }

    private async Task CollectFilesRecursiveAsync(
        ClientContext ctx,
        Folder folder,
        List<string> files,
        IProgress<OperationProgress> progress,
        CancellationToken ct)
    {
        ct.ThrowIfCancellationRequested();

        ctx.Load(folder, f => f.Files.Include(fi => fi.ServerRelativeUrl),
                         f => f.Folders);
        await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);

        foreach (var file in folder.Files)
        {
            files.Add(file.ServerRelativeUrl);
        }

        foreach (var subFolder in folder.Folders)
        {
            // Skip system folders
            if (subFolder.Name.StartsWith("_") || subFolder.Name == "Forms")
                continue;
            await CollectFilesRecursiveAsync(ctx, subFolder, files, progress, ct);
        }
    }

    private async Task EnsureFolderAsync(
        ClientContext ctx,
        string folderServerRelativeUrl,
        IProgress<OperationProgress> progress,
        CancellationToken ct)
    {
        try
        {
            var folder = ctx.Web.GetFolderByServerRelativeUrl(folderServerRelativeUrl);
            ctx.Load(folder, f => f.Exists);
            await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);
            if (folder.Exists) return;
        }
        catch { /* folder doesn't exist, create it */ }

        // Create folder using Folders.Add which creates intermediate folders
        ctx.Web.Folders.Add(folderServerRelativeUrl);
        await ExecuteQueryRetryHelper.ExecuteQueryRetryAsync(ctx, progress, ct);
    }

    private static string BuildServerRelativePath(ClientContext ctx, string library, string folderPath)
    {
        // Extract site-relative URL from context URL
        var uri = new Uri(ctx.Url);
        var siteRelative = uri.AbsolutePath.TrimEnd('/');
        var basePath = $"{siteRelative}/{library}";
        if (!string.IsNullOrEmpty(folderPath))
            basePath = $"{basePath}/{folderPath.TrimStart('/')}";
        return basePath;
    }
}
```

**Verify:**
```bash
dotnet build SharepointToolbox/SharepointToolbox.csproj --no-restore -q
```

**Done:** FileTransferService compiles. Implements copy/move via MoveCopyUtil with all three conflict policies, recursive folder enumeration, folder auto-creation at destination, best-effort metadata preservation, per-file error handling via BulkOperationRunner, and cancellation checking between files.

### Task 2: Create FileTransferService unit tests

**Files:**
- `SharepointToolbox.Tests/Services/FileTransferServiceTests.cs`

**Action:**

Create `FileTransferServiceTests.cs`. Since CSOM classes (ClientContext, MoveCopyUtil) cannot be mocked directly, these tests verify the service compiles and its helper logic. Integration testing requires a real SharePoint tenant. Mark integration-dependent tests with Skip.

```csharp
using SharepointToolbox.Core.Models;
using SharepointToolbox.Services;

namespace SharepointToolbox.Tests.Services;

public class FileTransferServiceTests
{
    [Fact]
    public void FileTransferService_Implements_IFileTransferService()
    {
        var service = new FileTransferService();
        Assert.IsAssignableFrom<IFileTransferService>(service);
    }

    [Fact]
    public void TransferJob_DefaultValues_AreCorrect()
    {
        var job = new TransferJob();
        Assert.Equal(TransferMode.Copy, job.Mode);
        Assert.Equal(ConflictPolicy.Skip, job.ConflictPolicy);
    }

    [Fact]
    public void ConflictPolicy_HasAllValues()
    {
        Assert.Equal(3, Enum.GetValues<ConflictPolicy>().Length);
        Assert.Contains(ConflictPolicy.Skip, Enum.GetValues<ConflictPolicy>());
        Assert.Contains(ConflictPolicy.Overwrite, Enum.GetValues<ConflictPolicy>());
        Assert.Contains(ConflictPolicy.Rename, Enum.GetValues<ConflictPolicy>());
    }

    [Fact]
    public void TransferMode_HasAllValues()
    {
        Assert.Equal(2, Enum.GetValues<TransferMode>().Length);
        Assert.Contains(TransferMode.Copy, Enum.GetValues<TransferMode>());
        Assert.Contains(TransferMode.Move, Enum.GetValues<TransferMode>());
    }

    [Fact(Skip = "Requires live SharePoint tenant")]
    public async Task TransferAsync_CopyMode_CopiesFiles()
    {
        // Integration test — needs real ClientContext
    }

    [Fact(Skip = "Requires live SharePoint tenant")]
    public async Task TransferAsync_MoveMode_DeletesSourceAfterCopy()
    {
        // Integration test — needs real ClientContext
    }

    [Fact(Skip = "Requires live SharePoint tenant")]
    public async Task TransferAsync_SkipConflict_DoesNotOverwrite()
    {
        // Integration test — needs real ClientContext
    }
}
```

**Verify:**
```bash
dotnet build SharepointToolbox.slnx --no-restore -q && dotnet test SharepointToolbox.Tests --no-build --filter "FullyQualifiedName~FileTransferService" -q
```

**Done:** FileTransferService tests pass (4 pass, 3 skip). Service is fully implemented and compiles.

**Commit:** `feat(04-03): implement FileTransferService with MoveCopyUtil and conflict policies`