diff --git a/PACKAGING.md b/PACKAGING.md index cb0113a..18f2324 100644 --- a/PACKAGING.md +++ b/PACKAGING.md @@ -4,7 +4,7 @@ Three distribution targets share one source tree (`src/`, a plain static app): | Target | Output | Build host | | --- | --- | --- | -| Desktop | `Motionity Setup 1.0.0.exe`, `Motionity-1.0.0-x64.AppImage`, `.flatpak` | Windows for `.exe`, Linux (or WSL2) for AppImage/Flatpak | +| Desktop | `Motionity Setup 1.0.0.exe`, `Motionity-1.0.0-x64.AppImage`, `.flatpak` | Linux or WSL2 builds all of them; Windows builds only the `.exe` targets | | Container | `motionity:latest`, ~86 MB on disk / ~57 MB pulled | any Docker host | | Bare metal | `src/` behind Node, nginx or Caddy | any | @@ -63,6 +63,10 @@ npm run dist:win # NSIS installer + portable .exe -> dist/ npm run dist:appimage # .AppImage -> dist/ npm run dist:flatpak # .flatpak -> dist/ npm run dist:linux # both Linux targets + +npm run dist:linux:wsl # from Windows: Linux bundles, built in WSL +npm run dist:win:wsl # from Windows: .exe targets, built and left in WSL +npm run dist:win:wsl:portable # same, portable only (needs no Wine in the distro) ``` Each `dist:*` script re-runs `vendor` and regenerates `build/icon.png` @@ -107,9 +111,17 @@ WARNING: Cannot open 2 files The ACL is intact (the owner still has FullControl), so this is a filter driver, not a permissions problem. Fixes, in order of preference: -1. Exclude the repository's `dist/` directory in the endpoint protection agent. -2. Build on a machine or CI runner without that agent. -3. Ship `dist/win-unpacked/` — `electron-builder --win dir` is unaffected. +1. **Build the Windows targets in WSL**, where the agent cannot see the files at + all — `npm run dist:win:wsl`, covered in [its own section](#can-the-exe-be-built-in-wsl-too) + below. Needs no admin rights on the Windows side. +2. Exclude the repository's `dist/` directory in the endpoint protection agent. +3. Build on a machine or CI runner without that agent. +4. Ship `dist/win-unpacked/` — `electron-builder --win dir` is unaffected. + +The same agent can also quarantine the *finished* unsigned `.exe` on write, not +just lock it during the build. That is what `-KeepInWsl` is for: the artifact +stays in the distro and is uploaded to the release from there, so it never +crosses onto NTFS. ### Running `npm run dev` from a VS Code terminal @@ -124,7 +136,8 @@ $env:ELECTRON_RUN_AS_NODE=$null; npm run dev # PowerShell ### Is WSL enough for AppImage and Flatpak? -Yes, and `build-release.ps1 -UseWsl` does it for you from Windows: +Yes, and `build-release.ps1 -UseWsl` does it for you from Windows (`-WinInWsl` +moves the `.exe` targets there too — see [below](#can-the-exe-be-built-in-wsl-too)): ```powershell ./scripts/build-release.ps1 -UseWsl # .exe on Windows, Linux bundles in WSL @@ -138,8 +151,9 @@ Both bundles have been built this way on this machine (`Ubuntu`, WSL2 kernel - the `npm ci`, `vendor` and icon steps run **once on the Windows side**, and the WSL build reads them back through `/mnt/c` — one worktree, no second checkout; -- only `linux-appimage` and `linux-flatpak` are delegated. `-UseWsl` on a Linux - host, or with no Linux target, warns and changes nothing; +- `-UseWsl` delegates only `linux-appimage` and `linux-flatpak`; add `-WinInWsl` + to send `win` / `win-nsis` / `win-portable` as well. On a Linux host, or with no + target selected for WSL, both warn and change nothing; - the distro is the first installed one that is not `docker-desktop`, override with `-WslDistro`. `docker-desktop` is skipped deliberately: it is Docker's own LinuxKit VM, with no apt and no home to install the flatpak runtimes into, and @@ -172,8 +186,9 @@ inside WSL you need `libfuse2` (or WSL2 kernel has the user namespaces and `/dev/fuse` that bubblewrap wants, but there is no `xdg-desktop-portal` to fall back on. -**`.exe`: no.** Build it on the Windows side. Cross-building NSIS from Linux -needs Wine and rules out signing. +**`.exe`: yes** — see the next section. It is opt-in (`-WinInWsl`) because Windows +builds those targets natively too; the reason to move them is the endpoint agent +described above, not portability. One-time distro setup, from PowerShell: @@ -187,6 +202,8 @@ Then inside Ubuntu: sudo apt update sudo apt install -y nodejs npm libfuse2 # AppImage (libfuse2 only to run it) sudo apt install -y flatpak flatpak-builder elfutils # Flatpak +sudo dpkg --add-architecture i386 && sudo apt update # only for the NSIS .exe (see below) +sudo apt install -y wine # " " flatpak remote-add --if-not-exists --user flathub \ https://dl.flathub.org/repo/flathub.flatpakrepo flatpak install --user -y flathub \ @@ -195,6 +212,103 @@ flatpak install --user -y flathub \ org.electronjs.Electron2.BaseApp//23.08 ``` +### Can the `.exe` be built in WSL too? + +Yes, and it is the way out of the "Access denied" failure above, since nothing +unsigned is written to a Windows filesystem. + +```powershell +npm run dist:win:wsl # NSIS + portable, built and left in WSL — build only +npm run dist:win:wsl:portable # portable only — no Wine, no sudo needed +npm run release:binaries:wsl # same .exe targets, then uploaded to the Gitea release +npm run release:wsl # everything incl. Linux + the container image push + +./scripts/build-release.ps1 -WinInWsl # copy the .exe back into dist/ +./scripts/build-release.ps1 -WinInWsl -KeepInWsl # leave it in the distro +``` + +Verified on this machine (`Ubuntu`, WSL2 kernel 6.18): a 138 MB portable +`motionity-v2.0.2-win-x64-portable.exe`, `PE32 executable for MS Windows (GUI), +Nullsoft Installer self-extracting archive`, with the icon and version resources +applied, byte-identical whether read in the distro or after the copy back into +`dist/`. The NSIS installer needs the wine setup below and has not been built this +way yet. + +**What each Windows target needs on the Linux side.** + +| Target | Wine? | Why | +| --- | --- | --- | +| `-Targets win-portable` | no | electron-builder's NSIS bundle ships a native Linux `makensis`, and the exe's icon and version strings are written by the `resedit` JS package, not by `rcedit.exe`. | +| `-Targets win-nsis` | **yes, 32-bit capable** | NSIS builds its uninstaller by *executing* the installer stub it has just linked, so a Windows PE has to run. | +| `-Targets win` | yes | Both of the above in one packaging pass. | + +**Why the NSIS target cannot avoid Wine.** electron-builder links the installer +once with `BUILD_UNINSTALLER` defined, runs it to produce `uninstaller.exe`, then +links the real installer, which *embeds that file*: +`templates/nsis/include/installer.nsh` does +`File "/oname=${UNINSTALL_FILENAME}" "${UNINSTALLER_OUT_FILE}"`. There is no +option to skip the first pass. The stub it executes is **PE32/i386** even for an +x64 app, so a 64-bit-only Wine is not enough either: + +```bash +sudo dpkg --add-architecture i386 +sudo apt update +sudo apt install -y wine +``` + +`build-release.ps1` probes for a usable wine before packaging (missing or broken +wine is an error; a wine with no `i386-windows` directory is a warning), because +the failure otherwise arrives ~200 MB into the build naming ntdll rather than the +missing package. + +**Do not use `toolsets.wine=1.0.1` for this.** electron-builder can download its +own Wine 11 bundle instead of using the distro's, which looks like it would avoid +the apt install, and it does download and verify cleanly. Its Linux build is +unusable: `lib/wine/x86_64-unix/` only, with no `*-windows` PE builtin directory +and no `syswow64`, so it fails after the app is already packaged with + +``` +wine: failed to load .../wine-11.0-linux-x86_64-*/lib/wine/x86_64-unix/ntdll.dll error c0000135 +0024:err:environ:run_wineboot failed to start wineboot 1 +``` + +`c0000135` is `STATUS_DLL_NOT_FOUND`. Leaving `toolsets.wine` unset is what makes +electron-builder use the distro's `wine` on Linux, which is the working path. If +that bundle was already downloaded, `rm -rf ~/.cache/electron-builder/wine@1.0.1` +reclaims it. + +If you cannot install anything in the distro, `-Targets win-portable` is a +complete answer: a single self-contained `.exe`, no installer, no Wine, no root. + +**Why the build also passes `win.signExecutable=false`.** With no certificate +configured, electron-builder still walks the signing path, and on Linux that +path shells out to `signtool.exe` under Wine *before* discovering there is +nothing to sign — `spawn wine ENOENT`, build over. `signExecutable: false` skips +signing while still applying the icon and version metadata. +(`signAndEditExecutable: false` would drop those too, which is not wanted.) Both +overrides are passed on the command line for the WSL build only, so a native +Windows build behaves exactly as before. These releases are unsigned either way. + +**Uploading straight from the distro.** With `-KeepInWsl`, `build-release.ps1` +writes `dist/wsl-artifacts.json` naming the distro, the staging directory and the +files it deliberately did not copy back. `publish.ps1` reads it and runs the +`curl` upload *inside* the distro for those files. The Gitea token reaches WSL +through `WSLENV` and is written to a `mktemp` config file by bash — it is in +neither `wsl.exe`'s arguments nor the distro's process table. `SHA256SUMS.txt` +still covers every artifact, hashes for the staged ones coming from `sha256sum` +in the distro; it is text, so nothing objects to it landing in `dist/`. + +Two things to know when writing more of this plumbing: + +- These `.ps1` files are stored with **CRLF**, so a multi-line here-string handed + to `bash -lc` arrives with a `\r` on every line (`set: - : invalid option`, + `cd: $'/path\r': No such file or directory`). `ConvertTo-BashScript` strips it. +- Never combine `set -e` with an explicit `exit 0` under `bash -lc`. A login + shell sources `~/.bash_logout`, Ubuntu's ends in a `clear_console` test that + fails with no tty, and errexit promotes that to the shell's exit status: + `wsl -e bash -lc 'set -e; exit 0'` returns **1**. `-l` has to stay, because + node from nvm or fnm is only on the login `PATH`. + Those three refs must match `build.flatpak.runtimeVersion` / `baseVersion` in `package.json`; `build-release.ps1` reads them from there when it checks. diff --git a/package.json b/package.json index e8aa50f..6dbacc1 100644 --- a/package.json +++ b/package.json @@ -18,11 +18,15 @@ "dist:flatpak": "npm run vendor && npm run icons && electron-builder --linux flatpak", "dist:linux": "npm run vendor && npm run icons && electron-builder --linux AppImage flatpak", "dist:linux:wsl": "pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/build-release.ps1 -Targets linux -UseWsl", + "dist:win:wsl": "pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/build-release.ps1 -Targets win -WinInWsl -KeepInWsl", + "dist:win:wsl:portable": "pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/build-release.ps1 -Targets win-portable -WinInWsl -KeepInWsl", "docker:build": "docker build -t motionity:latest .", "docker:run": "docker run --rm -p 8080:8080 motionity:latest", "release:build": "pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/build-release.ps1", "release:binaries": "pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/publish.ps1 -BinariesOnly", + "release:binaries:wsl": "pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/publish.ps1 -BinariesOnly win -WinInWsl -KeepInWsl", "release": "pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/publish.ps1 -PublishRelease", + "release:wsl": "pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/publish.ps1 -PublishRelease -UseWsl -WinInWsl -KeepInWsl", "test": "node --test \"test/**/*.test.js\"" }, "dependencies": { diff --git a/scripts/build-release.ps1 b/scripts/build-release.ps1 index 9410781..91a5b08 100644 --- a/scripts/build-release.ps1 +++ b/scripts/build-release.ps1 @@ -26,7 +26,7 @@ 404s. Windows builds the .exe targets; AppImage and Flatpak need a Linux host or - WSL (see PACKAGING.md). Nothing here cross-builds. + WSL (see PACKAGING.md). -UseWsl makes that split automatic: the .exe targets run on Windows, and the Linux ones are handed to a WSL distro over /mnt/c, against this same worktree. @@ -34,6 +34,47 @@ side and the WSL build reads them through the mount, so the artifacts still land in dist/ and the checksum step below sees every one of them. + -WinInWsl sends the .exe targets to that distro as well. The reason to want + that is not portability: a locked-down Windows machine's endpoint agent takes an + exclusive lock on the freshly written unsigned Motionity.exe and the 7-Zip step + that packs the installer payload then cannot read it (PACKAGING.md has the + error). Building in the distro's ext4 filesystem is not visible to that agent. + Signing is the one thing lost, and these builds are unsigned either way — the + WSL build passes win.signExecutable=false, because the signtool.exe path runs + under Wine and there is no certificate for it to use. + + What each Windows target needs on the Linux side: + + win-portable nothing. electron-builder's NSIS bundle ships a native Linux + makensis, and rcedit was replaced by the resedit JS package, so + the icon and version strings are written without Wine. + win-nsis Wine, with 32-bit support, unavoidably. NSIS builds its + uninstaller by *executing* the installer stub it has just linked + (NsisTarget's computeScriptAndSignUninstaller) because the + installer then embeds that uninstaller as a file + (templates/nsis/include/installer.nsh: `File "/oname=..." + "${UNINSTALLER_OUT_FILE}"`). There is no flag to skip it, and the + stub is PE32/i386, so a 64-bit-only Wine cannot run it either. + Assert-WslWine checks for a usable one before building. + + Do NOT reach for toolsets.wine=1.0.1 here. That bundle exists and + downloads cleanly, but the Linux build of it is unusable: it ships + lib/wine/x86_64-unix only, with no *-windows PE builtin directory + and no syswow64, so wine dies with + wine: failed to load .../x86_64-unix/ntdll.dll error c0000135 + after the app has already been packaged. The distro's own wine is + the working path. + + "win" is both, in one packaging pass. -Targets win-portable is the way to get a + usable .exe out of a machine where you cannot apt-install anything. + + -KeepInWsl goes further and never copies the WSL-built artifacts back: they + stay in the staging directory, and dist/ gets only SHA256SUMS.txt plus + wsl-artifacts.json naming them. Use it when the agent quarantines the finished + unsigned .exe on write, not just during the build — publish.ps1 reads that + manifest and uploads those files to Gitea from inside the distro, so the .exe + never lands on NTFS at all. + .EXAMPLE ./scripts/build-release.ps1 Build every target at v. @@ -54,15 +95,32 @@ ./scripts/build-release.ps1 -Targets linux -UseWsl -WslDistro Ubuntu-24.04 Both Linux bundles, in a named distro. +.EXAMPLE + ./scripts/build-release.ps1 -WinInWsl + Every target built in WSL, artifacts copied back into dist/. Nothing unsigned + is written to NTFS while the build runs. + +.EXAMPLE + ./scripts/build-release.ps1 -WinInWsl -KeepInWsl + Same, but the artifacts stay in the distro. dist/ gets SHA256SUMS.txt and + wsl-artifacts.json; publish.ps1 uploads from there. + +.EXAMPLE + ./scripts/build-release.ps1 -Targets win-portable -WinInWsl -KeepInWsl + The portable .exe only, built and left in WSL. Needs no Wine and no root in the + distro, and writes nothing to NTFS. + .EXAMPLE ./scripts/build-release.ps1 -SkipVendor -SkipDeps Reuse src/vendor/ and node_modules as they are — the fast rebuild. #> [CmdletBinding()] param( - # Platforms to package. "win" is NSIS + portable; "linux-appimage" and - # "linux-flatpak" are the two Linux bundles, and "linux" is shorthand for both. - [ValidateSet("win", "linux", "linux-appimage", "linux-flatpak")] + # Platforms to package. "win" is NSIS + portable in one pass, and "win-nsis" / + # "win-portable" are those two separately — worth having because only the NSIS + # one needs Wine when built in WSL. "linux-appimage" and "linux-flatpak" are the + # two Linux bundles, and "linux" is shorthand for both. + [ValidateSet("win", "win-nsis", "win-portable", "linux", "linux-appimage", "linux-flatpak")] [string[]]$Targets = @("win", "linux"), # Version used in the artifact names. Defaults to v, @@ -84,6 +142,14 @@ param( # docker-desktop. [string]$WslDistro, + # Send the "win" target to WSL too. Implies -UseWsl. Nothing unsigned is then + # written to NTFS during the build, which is what the endpoint agent reacts to. + [switch]$WinInWsl, + + # Leave the WSL-built artifacts in the distro instead of copying them into + # dist/. dist/ still gets SHA256SUMS.txt and wsl-artifacts.json. + [switch]$KeepInWsl, + # Remove dist/ before building. [switch]$Clean ) @@ -115,12 +181,43 @@ function Get-ArtifactName { # --- WSL plumbing ------------------------------------------------------------- +function ConvertTo-BashScript { + <# + Strip CR. This file is stored with CRLF line endings, so a multi-line + here-string handed to bash arrives with a \r on every line and bash treats it + as part of the last token: + + set: - : invalid option + cd: $'/home/kawa/.cache/motionity-build\r': No such file or directory + + Single-line commands are unaffected, which is exactly why this is easy to + miss until a script gains a second line. + #> + param([Parameter(Mandatory)][AllowEmptyString()][string]$Script) + return $Script -replace "`r", "" +} + +<# + A second trap in the same area, worth stating once: never combine `set -e` with an + explicit `exit 0` under `bash -lc`. + + wsl -d Ubuntu -e bash -lc 'set -e; exit 0' -> 1 + wsl -d Ubuntu -e bash -lc 'exit 0' -> 0 + + A login shell sources ~/.bash_logout on the way out, and Ubuntu's default one ends + in `[ -x /usr/bin/clear_console ] && /usr/bin/clear_console -q`, which fails with + no tty attached. With errexit set, that failure becomes the shell's exit status, + and a script that did its job reports failure. -l is not negotiable (node from nvm + or fnm is only on the login PATH), so the scripts below use `set -u` and check the + commands that matter by hand. +#> + function Invoke-Wsl { param([Parameter(Mandatory)][string]$Distro, [Parameter(Mandatory)][string]$Command) Write-Host " > wsl -d $Distro -- $Command" -ForegroundColor DarkGray # bash -lc, so PATH matches an interactive shell: node installed through nvm # or fnm is not on the default non-login PATH. - & wsl.exe -d $Distro -e bash -lc $Command + & wsl.exe -d $Distro -e bash -lc (ConvertTo-BashScript $Command) if ($LASTEXITCODE -ne 0) { throw "the WSL build in '$Distro' failed with exit code $LASTEXITCODE." } @@ -132,6 +229,30 @@ function Test-WslCommand { return ($LASTEXITCODE -eq 0) } +function Invoke-WslCapture { + <# + Like Invoke-Wsl, but returns the distro's stdout instead of echoing the + command. For the small queries (a path, a checksum listing) whose output is + the point and whose command line is noise. + + Positional arguments go to bash as $1..$n; $0 is a label. Passing them this + way rather than interpolating into $Command keeps quoting out of it. + #> + param( + [Parameter(Mandatory)][string]$Distro, + [Parameter(Mandatory)][string]$Command, + [string[]]$ScriptArgs = @() + ) + # 2>&1: stderr is merged in so a failure can be reported with what the distro + # actually said. Callers filter the lines they want, so the noise is harmless. + $out = @(& wsl.exe -d $Distro -e bash -lc (ConvertTo-BashScript $Command) "motionity-build" @ScriptArgs 2>&1 | + ForEach-Object { $_.ToString().Replace("`0", "") }) + if ($LASTEXITCODE -ne 0) { + throw "a query in WSL distro '$Distro' failed with exit code ${LASTEXITCODE}: $($out -join ' | ')" + } + return $out +} + function ConvertTo-BashArg { <# Single quotes, always. The artifactName arguments carry a literal ${ext} @@ -214,6 +335,47 @@ function Get-WslStageDir { return $path } +function Assert-WslWine { + <# + The NSIS target needs a wine in the distro that can run a 32-bit PE, because + the installer stub it has to execute is PE32/i386 even for an x64 app. Checked + up front: without it the build dies after packaging ~200 MB, with an error + that names ntdll rather than the missing package. + + The i386 check is a warning, not an error. The directory list below covers the + usual layouts but cannot cover every distro or a hand-built wine, and a false + negative must not block a build that would have worked. + #> + param([Parameter(Mandatory)][string]$Distro) + + $probe = @' +set -u +command -v wine >/dev/null 2>&1 || exit 10 +wine --version >/dev/null 2>&1 || exit 11 +for d in /usr/lib/wine /usr/lib64/wine /usr/lib/x86_64-linux-gnu/wine /usr/local/lib/wine /opt/wine*/lib/wine; do + [ -d "$d/i386-windows" ] && exit 0 +done +exit 12 +'@ + & wsl.exe -d $Distro -e bash -lc (ConvertTo-BashScript $probe) "motionity-build" *> $null + $code = $LASTEXITCODE + + $installHint = @" +Inside the distro (the i386 architecture is what provides the 32-bit loader): + sudo dpkg --add-architecture i386 + sudo apt update + sudo apt install -y wine +Or skip NSIS entirely — the portable .exe runs no PE, so it needs neither Wine nor root: + ./scripts/build-release.ps1 -Targets win-portable -WinInWsl -KeepInWsl +"@ + + switch ($code) { + 10 { throw "no wine in WSL distro '$Distro', and the NSIS uninstaller cannot be built without one.`n$installHint" } + 11 { throw "wine is installed in '$Distro' but will not run ('wine --version' failed). A broken or partial install cannot build the NSIS uninstaller.`n$installHint" } + 12 { Write-Warning "wine in '$Distro' looks 64-bit only (no i386-windows directory found). The NSIS installer stub is PE32/i386, so the build will likely fail at 'building target=nsis'. If it does:`n$installHint" } + } +} + function Assert-WslBuildEnv { <# Fail before the build rather than during it. electron-builder's own error @@ -230,9 +392,16 @@ function Assert-WslBuildEnv { throw "no node in WSL distro '$Distro'. Inside it: sudo apt update && sudo apt install -y nodejs npm" } - # AppImage needs nothing else: electron-builder downloads its own appimage - # tooling and writes the squashfs itself. libfuse2 is only needed to *run* the - # result, which is not this script's job. + # NSIS links the installer and then *runs* it with BUILD_UNINSTALLER defined to + # get the uninstaller out, so building it on Linux means executing a Windows PE. + if (@($WslTargets | Where-Object { $_ -eq "win" -or $_ -eq "win-nsis" }).Count) { + Assert-WslWine -Distro $Distro + } + + # AppImage and the Windows targets need nothing else: electron-builder downloads + # its own appimage tooling and writes the squashfs itself, and it downloads the + # NSIS bundle whose makensis is a native Linux binary. libfuse2 is only needed to + # *run* an AppImage, which is not this script's job. if ($WslTargets -notcontains "linux-flatpak") { return } if (-not (Test-WslCommand $Distro 'command -v flatpak-builder')) { @@ -279,8 +448,9 @@ try { Write-Host " output : $distDir" Write-Host "" - # Expand "linux" to its two concrete bundles and drop duplicates, so the rest - # of the script only ever deals with win / linux-appimage / linux-flatpak. + # Expand "linux" to its two concrete bundles and drop duplicates. "win" is left + # alone rather than expanded the same way: electron-builder builds nsis and + # portable from one packaging pass, and splitting it would unpack Electron twice. $resolvedTargets = @() foreach ($t in $Targets) { if ($t -eq "linux") { $resolvedTargets += "linux-appimage", "linux-flatpak" } @@ -293,18 +463,27 @@ try { # targets to a WSL distro; without it they stay a warning rather than an error, # because the other right answer is running this whole script under pwsh on a # Linux box, where they build natively. + # + # -WinInWsl adds "win" to that set. Not because Windows cannot build it, but + # because a locked-down Windows machine's endpoint agent interferes with the + # unsigned .exe while the installer is being packed (see the header). $onWindows = ($env:OS -eq "Windows_NT") - $wslTargets = @($resolvedTargets | Where-Object { $_ -like "linux-*" }) - $useWslHere = $onWindows -and $UseWsl -and [bool]$wslTargets.Count + $wantWsl = $UseWsl -or $WinInWsl + $wslTargets = @($resolvedTargets | Where-Object { $_ -like "linux-*" -or ($WinInWsl -and $_ -like "win*") }) + $useWslHere = $onWindows -and $wantWsl -and [bool]$wslTargets.Count - if ($onWindows -and $wslTargets.Count -and -not $UseWsl) { + $linuxTargets = @($resolvedTargets | Where-Object { $_ -like "linux-*" }) + if ($onWindows -and $linuxTargets.Count -and -not $wantWsl) { Write-Warning "the Linux targets need a Linux host or WSL — electron-builder cannot produce AppImage or Flatpak on Windows. Add -UseWsl to build them in a WSL distro (PACKAGING.md has the setup)." } - if ($UseWsl -and -not $onWindows) { - Write-Warning "-UseWsl ignored: this is already a Linux host, so the Linux targets build natively." + if ($wantWsl -and -not $onWindows) { + Write-Warning "-UseWsl/-WinInWsl ignored: this is already a Linux host, so every target builds natively." } - if ($UseWsl -and $onWindows -and -not $wslTargets.Count) { - Write-Warning "-UseWsl ignored: no Linux target was requested (-Targets $($Targets -join ', '))." + if ($wantWsl -and $onWindows -and -not $wslTargets.Count) { + Write-Warning "-UseWsl ignored: no target was selected for WSL (-Targets $($Targets -join ', ')). Add -WinInWsl to send the Windows targets there too." + } + if ($KeepInWsl -and -not $useWslHere) { + Write-Warning "-KeepInWsl ignored: nothing is being built in WSL." } $wslRepo = $null @@ -313,10 +492,10 @@ try { $WslDistro = Get-WslDistro -Requested $WslDistro $wslRepo = Get-WslPath -Distro $WslDistro -WindowsPath $repoRoot $wslStage = Get-WslStageDir -Distro $WslDistro - Write-Host "Linux targets go to WSL" -ForegroundColor Cyan + Write-Host "Targets going to WSL: $($wslTargets -join ', ')" -ForegroundColor Cyan Write-Host " distro : $WslDistro" Write-Host " worktree: $wslRepo" - Write-Host " staging : $wslStage (copied back into dist/)" + Write-Host " staging : $wslStage$(if ($KeepInWsl) { ' (left there — -KeepInWsl)' } else { ' (copied back into dist/)' })" Assert-WslBuildEnv -Distro $WslDistro -WslTargets $wslTargets -Pkg $pkg Write-Host "" } @@ -374,6 +553,15 @@ try { throw 'electron-builder not found in node_modules — run "npm ci" (or drop -SkipDeps).' } + # Clear this tag's previous bundles from the staging directory once, before the + # loop — not per target. Per target, the glob would delete the artifacts an + # earlier target had just staged, which only went unnoticed while every target + # copied its output back immediately. + if ($useWslHere) { + Invoke-Wsl -Distro $WslDistro -Command ( + "mkdir -p $(ConvertTo-BashArg $wslStage) && rm -f $(ConvertTo-BashArg $wslStage)/$prefix-*") + } + foreach ($target in $resolvedTargets) { Write-Host "Packaging $target..." -ForegroundColor Cyan @@ -388,6 +576,18 @@ try { "-c.portable.artifactName=$(Get-ArtifactName "$prefix-win-x64-portable")" ) } + "win-nsis" { + $builderArgs = @( + "--win", "nsis", "--publish", "never", + "-c.nsis.artifactName=$(Get-ArtifactName "$prefix-win-x64-setup")" + ) + } + "win-portable" { + $builderArgs = @( + "--win", "portable", "--publish", "never", + "-c.portable.artifactName=$(Get-ArtifactName "$prefix-win-x64-portable")" + ) + } "linux-appimage" { $builderArgs = @( "--linux", "AppImage", "--publish", "never", @@ -402,7 +602,26 @@ try { } } - if ($useWslHere -and $target -like "linux-*") { + # Two overrides that only apply to a Windows target built on Linux: + # + # signExecutable=false electron-builder still walks the signing path with no + # certificate configured, and on Linux that path shells + # out to signtool.exe under Wine before it discovers + # there is nothing to sign — `spawn wine ENOENT`, build + # over. false skips signing while still applying the + # icon and version strings (signAndEditExecutable=false + # would drop those too, which is not what is wanted). + # toolsets.wine is deliberately NOT set: leaving it unset is what makes + # electron-builder use the distro's own `wine` on Linux, and the 1.0.1 bundle + # it would otherwise download is unusable there (see the header). + # + # Passed here rather than put in package.json so a native Windows build keeps + # behaving exactly as it did. + if ($useWslHere -and $wslTargets -contains $target -and $target -like "win*") { + $builderArgs += "-c.win.signExecutable=false" + } + + if ($useWslHere -and $wslTargets -contains $target) { # `node cli.js` rather than node_modules/.bin/electron-builder: the # extensionless shim npm writes on Windows is a sh script, and whether # it is executable across the mount depends on the drvfs options. @@ -411,16 +630,16 @@ try { # the bundles are copied back afterwards. `cp -f`, never `cp -p`: # preserving modes means chmod, which is the EPERM this avoids. # - # The rm clears this tag's previous bundles from the staging directory, - # so the copy back cannot pick up an artifact from an earlier build that - # electron-builder did not overwrite this time round. + # With -KeepInWsl there is no copy back at all: the point is that no + # unsigned .exe is ever written to NTFS, and a `cp` into dist/ is exactly + # the write the endpoint agent would quarantine. $quotedArgs = @($builderArgs | ForEach-Object { ConvertTo-BashArg $_ }) -join " " $stage = ConvertTo-BashArg $wslStage $wslCommand = "cd $(ConvertTo-BashArg $wslRepo)" + - " && mkdir -p $stage" + - " && rm -f $stage/$prefix-*" + - " && USE_HARD_LINKS=false node node_modules/electron-builder/cli.js $quotedArgs $(ConvertTo-BashArg "-c.directories.output=$wslStage")" + - " && cp -f $stage/$prefix-* $(ConvertTo-BashArg "$wslRepo/dist")/" + " && USE_HARD_LINKS=false node node_modules/electron-builder/cli.js $quotedArgs $(ConvertTo-BashArg "-c.directories.output=$wslStage")" + if (-not $KeepInWsl) { + $wslCommand += " && cp -f $stage/$prefix-* $(ConvertTo-BashArg "$wslRepo/dist")/" + } Invoke-Wsl -Distro $WslDistro -Command $wslCommand } else { @@ -438,16 +657,80 @@ try { } | Remove-Item -Force $built = @(Get-ChildItem $distDir -Filter "$prefix-*" -File | Sort-Object Name) - if (-not $built.Count) { - throw "electron-builder reported success but no $prefix-* artifact landed in $distDir." + + # --- Artifacts left in the distro ------------------------------------------ + # With -KeepInWsl the bundles never crossed the mount, so their names and hashes + # have to come from the distro. The same intermediates are cleared there first: + # `.exe.blockmap` matches the $prefix-* glob too, and would otherwise + # be listed as a release artifact. + $manifestPath = Join-Path $distDir "wsl-artifacts.json" + $stagedLines = @() + if ($useWslHere -and $KeepInWsl) { + $listScript = @' +set -u +cd "$1" || { echo "staging directory $1 is gone" >&2; exit 1; } +rm -f ./*.blockmap ./*.nsis.7z ./latest*.yml ./builder-debug.yml +shopt -s nullglob +files=("$2"-*) +if [ ${#files[@]} -gt 0 ]; then sha256sum "${files[@]}"; fi +'@ + $stagedLines = @(Invoke-WslCapture -Distro $WslDistro -Command $listScript ` + -ScriptArgs @($wslStage, $prefix) | Where-Object { $_ -match '^[0-9a-f]{64}\s\s\S' }) + } + $stagedNames = @($stagedLines | ForEach-Object { ($_ -split '\s\s', 2)[1] } | Sort-Object) + + # A name built in WSL this run wins over a same-named file sitting in dist/ from + # an earlier native build. That leftover is stale by definition, and it is also + # the likeliest file on the machine to be locked or quarantined — which is the + # whole reason for building in the distro. + if ($stagedNames.Count) { + $shadowed = @($built | Where-Object { $stagedNames -contains $_.Name }) + if ($shadowed.Count) { + Write-Warning "ignoring $($shadowed.Count) stale file(s) in dist/ superseded by this run's WSL build: $(($shadowed | ForEach-Object Name) -join ', '). Delete them (or pass -Clean) to keep dist/ honest." + $built = @($built | Where-Object { $stagedNames -notcontains $_.Name }) + } + } + + if (-not $built.Count -and -not $stagedNames.Count) { + throw "electron-builder reported success but no $prefix-* artifact landed in $(if ($KeepInWsl) { "$distDir or $wslStage" } else { $distDir })." + } + + # publish.ps1 globs dist/ for what to upload, so the files that stayed behind + # need an explicit hand-off. A stale manifest from a previous -KeepInWsl run + # would make it upload artifacts this build did not produce, so it is removed + # whenever this run left nothing in the distro. + if ($stagedNames.Count) { + $manifest = [ordered]@{ + tag = $Tag + distro = $WslDistro + stageDir = $wslStage + files = @($stagedNames) + } + [System.IO.File]::WriteAllText($manifestPath, + ($manifest | ConvertTo-Json -Depth 3), [System.Text.UTF8Encoding]::new($false)) + } + elseif (Test-Path $manifestPath) { + Remove-Item -Force $manifestPath } # --- Checksums ------------------------------------------------------------ Write-Host "Writing checksums..." -ForegroundColor Cyan $sumsPath = Join-Path $distDir "SHA256SUMS.txt" - $lines = foreach ($f in $built) { - "$((Get-FileHash -Algorithm SHA256 $f.FullName).Hash.ToLower()) $($f.Name)" - } + # sha256sum's own output format is already ` `, so the lines from the + # distro go in verbatim and the two sets sort together by file name. + $lines = @( + @(foreach ($f in $built) { + try { + "$((Get-FileHash -Algorithm SHA256 $f.FullName).Hash.ToLower()) $($f.Name)" + } + catch { + # An artifact in dist/ that cannot even be read is the endpoint agent + # again, and hashing is not the step to paper over it: publish.ps1 + # would try to upload the same unreadable file next. + throw "cannot read $($f.FullName) to hash it ($($_.Exception.Message.Trim())). On a machine whose security agent locks unsigned executables, build with -WinInWsl -KeepInWsl and delete the leftovers in dist/ (or pass -Clean)." + } + }) + $stagedLines + ) | Sort-Object { ($_ -split '\s\s', 2)[1] } # ASCII with LF: a BOM or CRLF makes `sha256sum -c` reject the first line. [System.IO.File]::WriteAllText($sumsPath, ($lines -join "`n") + "`n", [System.Text.ASCIIEncoding]::new()) Write-Host "" @@ -456,6 +739,14 @@ try { foreach ($f in $built) { Write-Host " $($f.FullName) ($([math]::Round($f.Length / 1MB, 1)) MB)" -ForegroundColor Green } + if ($stagedNames.Count) { + Write-Host " in WSL ($WslDistro), not copied to NTFS:" -ForegroundColor Green + foreach ($n in $stagedNames) { + Write-Host " $wslStage/$n" -ForegroundColor Green + } + Write-Host " reachable from Windows as \\wsl.localhost\$WslDistro$(($wslStage -replace '/', '\'))\" -ForegroundColor DarkGray + Write-Host " publish.ps1 uploads these from inside the distro (dist/wsl-artifacts.json)" -ForegroundColor DarkGray + } Write-Host " $sumsPath" -ForegroundColor Green } finally { diff --git a/scripts/publish.ps1 b/scripts/publish.ps1 index ffff219..30dda9d 100644 --- a/scripts/publish.ps1 +++ b/scripts/publish.ps1 @@ -13,12 +13,25 @@ registry, so -PublishRelease attaches them to the Gitea release for that tag instead (creating the release if it does not exist). + An existing release is added to, not recreated: artifacts it does not have yet + are appended, and ones it already carries under the same name are replaced by the + freshly built file. That makes re-running a release after a rebuild safe, and + keeps the attachments in agreement with the SHA256SUMS.txt uploaded beside them. + -NoReplace turns a name collision back into an error. + -BinariesOnly ships just the installers: no docker build, no docker login, no image push, and the release upload is implied. It takes the target list to build (win, linux-appimage, linux-flatpak — comma-separated) and that list overrides -Targets. The Linux targets need a Linux host or WSL: on Windows, add -UseWsl and build-release.ps1 hands them to a WSL distro. + -WinInWsl and -KeepInWsl are forwarded to build-release.ps1 and together keep + the unsigned .exe off NTFS entirely: it is built in the distro and, because + -KeepInWsl skips the copy back, it is still there at upload time. This script + then reads dist/wsl-artifacts.json and runs the curl upload *inside* the distro + for those files, so the endpoint agent never sees a write it can quarantine. + The token reaches the distro through WSLENV, not through the command line. + Credentials are read, in order of precedence: 1. -Username / -Password parameters 2. $env:GITEA_USER / $env:GITEA_TOKEN @@ -50,6 +63,11 @@ Full release from Windows: image push, .exe installers built natively, AppImage and Flatpak built in WSL, everything attached to release v1.1.0. +.EXAMPLE + ./scripts/publish.ps1 -Tag v1.1.0 -PublishRelease -WinInWsl -KeepInWsl + Full release with every installer built in WSL and uploaded from there — no + unsigned binary is ever written to a Windows filesystem. + .EXAMPLE ./scripts/publish.ps1 -BinariesOnly win -NoBinaryBuild -Tag v1.1.0 Retry a failed upload: attach the installers already in dist/ without rebuilding @@ -105,8 +123,10 @@ param( # Skip building the desktop installers. [switch]$NoBinaries, - # Forwarded to build-release.ps1. "linux" is shorthand for both Linux bundles. - [ValidateSet("win", "linux", "linux-appimage", "linux-flatpak")] + # Forwarded to build-release.ps1. "linux" is shorthand for both Linux bundles; + # "win" is NSIS + portable, and win-nsis / win-portable are those two on their + # own (only the NSIS one needs Wine when built in WSL). + [ValidateSet("win", "win-nsis", "win-portable", "linux", "linux-appimage", "linux-flatpak")] [string[]]$Targets = @("win", "linux"), [switch]$SkipVendor, @@ -115,6 +135,12 @@ param( [switch]$UseWsl, [string]$WslDistro, + # Forwarded to build-release.ps1: build the Windows targets in WSL too, and + # leave what WSL built inside the distro. With -KeepInWsl the upload below runs + # in the distro instead of on Windows, so the .exe never reaches NTFS. + [switch]$WinInWsl, + [switch]$KeepInWsl, + # Reuse the installers already in dist/ instead of re-running the build. For # retrying a failed upload without paying for the build again. [switch]$NoBinaryBuild, @@ -123,7 +149,7 @@ param( # -PublishRelease, since building alone is what build-release.ps1 already does. # Takes the target list to build (comma-separated), which overrides -Targets: # -BinariesOnly win,linux-appimage - [ValidateSet("win", "linux-appimage", "linux-flatpak")] + [ValidateSet("win", "win-nsis", "win-portable", "linux-appimage", "linux-flatpak")] [string[]]$BinariesOnly, # Attach the installers to the Gitea release for $Tag, creating the release if @@ -136,7 +162,14 @@ param( # Gitea base URL for the API. Defaults to https://. [string]$ApiBase, - # Replace release attachments that already exist under the same name. + # Fail instead of replacing an attachment that already exists under the same + # name. The default is to replace, because re-running a release for the same tag + # after a rebuild is the normal case and the new file is the one that matches + # SHA256SUMS.txt. + [switch]$NoReplace, + + # Deprecated: replacing is now the default, so this does nothing. Kept so + # existing commands and scripts do not start failing on an unknown parameter. [switch]$Force ) @@ -249,20 +282,203 @@ function Send-ReleaseAsset { } } +function ConvertTo-BashScript { + <# + Strip CR. This file is stored with CRLF line endings, so a multi-line + here-string handed to bash arrives with a \r on every line and bash reads it + as part of the last token — `set: - : invalid option`, and paths that end in + a literal \r. Single-line commands never show it. + #> + param([Parameter(Mandatory)][AllowEmptyString()][string]$Script) + return $Script -replace "`r", "" +} + +<# + And the companion trap: `set -e` plus an explicit `exit 0` under `bash -lc` yields + 1, because a login shell sources ~/.bash_logout on exit and Ubuntu's ends in a + `[ -x /usr/bin/clear_console ] && ...` that fails with no tty, which errexit then + promotes to the shell's status. -l has to stay (node from nvm/fnm lives on the + login PATH), so the scripts below set only `-u` and check what matters explicitly. +#> + +function Get-WslArtifactManifest { + <# + build-release.ps1 -KeepInWsl writes dist/wsl-artifacts.json for the artifacts + it deliberately did not copy onto NTFS. It removes the file whenever a build + leaves nothing behind, so its presence means "these files are in the distro"; + the tag is still checked, because a -NoBinaryBuild run for a different tag + would otherwise upload the previous release's binaries under the new one. + #> + param([Parameter(Mandatory)][string]$DistDir, [Parameter(Mandatory)][string]$Tag) + + $path = Join-Path $DistDir "wsl-artifacts.json" + if (-not (Test-Path $path)) { return $null } + + $manifest = Get-Content $path -Raw | ConvertFrom-Json + if (-not $manifest.files -or -not @($manifest.files).Count) { return $null } + if ($manifest.tag -ne $Tag) { + throw "$path was written for tag '$($manifest.tag)', not '$Tag' — those artifacts belong to another release. Rebuild, or delete the file if it is stale." + } + if (-not $manifest.distro -or -not $manifest.stageDir) { + throw "$path is missing the distro or stageDir field — delete it and rebuild." + } + return $manifest +} + +function Invoke-WithWslEnv { + <# + Run a script block with $Name exported into WSL through WSLENV. + + WSLENV is the only way to hand a value to a WSL process without putting it in + an argument list, and an argument list is exactly where a token must not be: + wsl.exe's own command line is readable from the Windows process table. The + previous WSLENV is restored rather than overwritten, because a distro may + rely on entries somebody else put there (PATH translation flags in + particular are positional and easy to break). + #> + param( + [Parameter(Mandatory)][string]$Name, + [Parameter(Mandatory)][string]$Value, + [Parameter(Mandatory)][scriptblock]$Body + ) + + $previousValue = [Environment]::GetEnvironmentVariable($Name, "Process") + $previousWslEnv = $env:WSLENV + [Environment]::SetEnvironmentVariable($Name, $Value, "Process") + $env:WSLENV = if ($previousWslEnv) { "$previousWslEnv`:$Name" } else { $Name } + try { + & $Body + } + finally { + [Environment]::SetEnvironmentVariable($Name, $previousValue, "Process") + if ($null -eq $previousWslEnv) { + Remove-Item Env:\WSLENV -ErrorAction SilentlyContinue + } + else { + $env:WSLENV = $previousWslEnv + } + } +} + +function Test-WslArtifacts { + <# + Every file the manifest names must still be in the staging directory. Without + this the first missing one surfaces as a curl error about an unreadable + upload part, halfway through a release. + #> + param( + [Parameter(Mandatory)][string]$Distro, + [Parameter(Mandatory)][string]$StageDir, + [Parameter(Mandatory)][string[]]$Names + ) + + $script = @' +set -u +cd "$1" || { echo "staging directory $1 is gone" >&2; exit 1; } +shift +missing=0 +for f in "$@"; do + [ -f "$f" ] || { echo "$f" >&2; missing=1; } +done +exit $missing +'@ + & wsl.exe -d $Distro -e bash -lc (ConvertTo-BashScript $script) "motionity-publish" $StageDir @Names 2>&1 | + ForEach-Object { Write-Host " $_" -ForegroundColor DarkGray } + if ($LASTEXITCODE -ne 0) { + throw "artifacts named in dist/wsl-artifacts.json are missing from ${StageDir} in '$Distro' (listed above) — rebuild with -WinInWsl -KeepInWsl, or drop -NoBinaryBuild." + } +} + +function Get-WslArtifactMtime { + <# + Oldest mtime among the staged artifacts, as a local DateTime, so the + -NoBinaryBuild staleness check works on WSL-resident files too. + #> + param( + [Parameter(Mandatory)][string]$Distro, + [Parameter(Mandatory)][string]$StageDir, + [Parameter(Mandatory)][string[]]$Names + ) + + $script = @' +set -u +cd "$1" || exit 1 +shift +stat -c %Y -- "$@" | sort -n | head -n 1 +'@ + $out = (& wsl.exe -d $Distro -e bash -lc (ConvertTo-BashScript $script) "motionity-publish" $StageDir @Names) + $epoch = (@($out) -join "").Replace("`0", "").Trim() + if ($LASTEXITCODE -ne 0 -or $epoch -notmatch '^\d+$') { return $null } + return [System.DateTimeOffset]::FromUnixTimeSeconds([int64]$epoch).LocalDateTime +} + +function Send-ReleaseAssetFromWsl { + <# + Upload one staged file as a release attachment, with curl running inside the + distro. Same Gitea endpoint and the same --config indirection for the token as + Send-ReleaseAsset; the only reason for a second implementation is that the + file must not be copied to NTFS to be read. + + The config file is written by bash from $GITEA_UPLOAD_TOKEN (arriving via + WSLENV) rather than interpolated into the command string, so the token is in + neither wsl.exe's arguments nor the distro's process table. mktemp creates it + 0600, and the trap removes it even if curl dies. + #> + param( + [Parameter(Mandatory)][string]$Distro, + [Parameter(Mandatory)][string]$StageDir, + [Parameter(Mandatory)][string]$Name, + [Parameter(Mandatory)][string]$Uri + ) + + # fail-with-body needs curl 7.76+ (Ubuntu 22.04 ships 7.81); the Windows path + # above already assumes it, so the two behave the same on an HTTP error. + $script = @' +set -u +command -v curl >/dev/null 2>&1 || { echo "curl is not installed in this WSL distro: sudo apt install -y curl" >&2; exit 127; } +[ -n "${GITEA_UPLOAD_TOKEN:-}" ] || { echo "GITEA_UPLOAD_TOKEN did not reach the distro — is WSLENV being overwritten?" >&2; exit 2; } +cfg=$(mktemp) || { echo "could not create a temp file for the curl config" >&2; exit 1; } +trap 'rm -f "$cfg"' EXIT +printf 'header = "Authorization: token %s"\nsilent\nshow-error\nfail-with-body\n' "$GITEA_UPLOAD_TOKEN" > "$cfg" || exit 1 +cd "$1" || exit 1 +# Last command on purpose: curl's status is the script's status. +curl --config "$cfg" --write-out ' http %{http_code}, %{size_upload} bytes uploaded\n' -F "attachment=@$2" "$3" +'@ + Write-Host " > [$Distro] curl --config -F attachment=@$Name `"$Uri`"" -ForegroundColor DarkGray + & wsl.exe -d $Distro -e bash -lc (ConvertTo-BashScript $script) "motionity-publish" $StageDir $Name $Uri + if ($LASTEXITCODE -ne 0) { + throw "upload of '$StageDir/$Name' from '$Distro' failed (exit $LASTEXITCODE)." + } +} + function Publish-BinaryRelease { <# Attach the installers to the release for $Tag, creating that release if it - does not exist yet. Re-uploading the same file name is a delete + upload, - which needs -Force: overwriting an asset someone may already have linked is - not something to do silently. + does not exist yet. An existing release is added to, never recreated. + + A file name the release already carries is replaced: Gitea does not treat + attachment names as unique, so uploading over one without removing it first + leaves two assets with the same name and no way for anyone to tell which is + which. Replacing is the default because the alternative is a release whose + binaries disagree with its own SHA256SUMS.txt after a rebuild; -NoReplace + restores the strict behaviour. + + Delete-then-upload, in that order, for the same reason — which does mean a + failed upload leaves the old asset gone. Recover with -NoBinaryBuild, which + re-attaches from dist/ (or from the distro) without rebuilding. #> param( [Parameter(Mandatory)][string]$ApiRoot, [Parameter(Mandatory)][string]$RepoPath, [Parameter(Mandatory)][string]$Tag, [Parameter(Mandatory)][string]$Token, - [Parameter(Mandatory)][string[]]$Artifacts, - [switch]$Force + # Windows-side files, uploaded by curl.exe. + [string[]]$Artifacts = @(), + # Files still in a WSL staging directory, uploaded by curl inside the distro. + [string[]]$WslArtifacts = @(), + [string]$WslDistro, + [string]$WslStageDir, + [switch]$NoReplace ) $releasesUri = "$ApiRoot/repos/$RepoPath/releases" @@ -282,20 +498,39 @@ function Publish-BinaryRelease { Write-Host " reusing release $Tag (id $($release.id))" -ForegroundColor DarkGray } - foreach ($path in $Artifacts) { - $name = Split-Path -Leaf $path - $existing = $release.assets | Where-Object { $_.name -eq $name } - if ($existing) { - if (-not $Force) { - throw "release $Tag already has an attachment named '$name' — pass -Force to replace it." + # One list so the asset-already-exists handling is written once: only the final + # transfer differs between a file on NTFS and one left in the distro. + $uploads = @() + foreach ($path in $Artifacts) { $uploads += @{ Name = (Split-Path -Leaf $path); Path = $path; InWsl = $false } } + foreach ($name in $WslArtifacts) { $uploads += @{ Name = $name; Path = $null; InWsl = $true } } + + foreach ($upload in $uploads) { + $name = $upload.Name + # @() because a release can already hold several assets under one name — an + # earlier run that uploaded without deleting, or a partial retry. Unwrapped, + # $existing.id would be an array and the DELETE would go to a malformed URL. + $existing = @($release.assets | Where-Object { $_.name -eq $name }) + if ($existing.Count) { + if ($NoReplace) { + throw "release $Tag already has an attachment named '$name', and -NoReplace was passed. Drop it to replace the file, or upload under a different tag." + } + foreach ($asset in $existing) { + Write-Host " replacing attachment '$name' (asset $($asset.id))..." -ForegroundColor DarkGray + Invoke-GiteaApi -Method DELETE -Token $Token ` + -Uri "$releasesUri/$($release.id)/assets/$($asset.id)" | Out-Null } - Write-Host " replacing existing attachment '$name'..." -ForegroundColor DarkGray - Invoke-GiteaApi -Method DELETE -Token $Token ` - -Uri "$releasesUri/$($release.id)/assets/$($existing.id)" | Out-Null } - $encoded = [System.Uri]::EscapeDataString($name) - Send-ReleaseAsset -Token $Token -Path $path ` - -Uri "$releasesUri/$($release.id)/assets?name=$encoded" + else { + Write-Host " adding attachment '$name'..." -ForegroundColor DarkGray + } + $encoded = [System.Uri]::EscapeDataString($name) + $assetUri = "$releasesUri/$($release.id)/assets?name=$encoded" + if ($upload.InWsl) { + Send-ReleaseAssetFromWsl -Distro $WslDistro -StageDir $WslStageDir -Name $name -Uri $assetUri + } + else { + Send-ReleaseAsset -Token $Token -Path $upload.Path -Uri $assetUri + } } return "$ApiRoot/repos/$RepoPath/releases/tags/$Tag" @@ -315,6 +550,12 @@ try { if ($NoBinaryBuild -and $NoBinaries) { throw "-NoBinaryBuild reuses the build that -NoBinaries skips entirely — pick one." } + if ($Force) { + Write-Warning "-Force is deprecated and ignored: replacing an attachment that already exists is now the default. -NoReplace is the opt-out." + } + if ($Force -and $NoReplace) { + throw "-Force and -NoReplace ask for opposite things — drop -Force, it is already the default." + } if ($binariesOnlyMode) { # Nothing to build, log into or push on the container side, and uploading # is the whole point (build-release.ps1 alone covers "just build them"). @@ -360,7 +601,7 @@ try { else { Write-Host " image : skipped (-BinariesOnly)" } - Write-Host " binaries : $(if ($NoBinaries) { 'skipped' } elseif ($NoBinaryBuild) { 'dist/ (reused, not rebuilt)' } else { $Targets -join ', ' })" + Write-Host " binaries : $(if ($NoBinaries) { 'skipped' } elseif ($NoBinaryBuild) { 'dist/ (reused, not rebuilt)' } else { $Targets -join ', ' })$(if ($WinInWsl) { ' (all in WSL)' } elseif ($UseWsl) { ' (Linux ones in WSL)' })$(if ($KeepInWsl) { ', uploaded from the distro' })" Write-Host " release : $(if ($PublishRelease) { "$ReleaseRepo @ $Tag" } else { 'not uploaded' })" Write-Host "" @@ -386,7 +627,10 @@ try { # --- Release artifacts ---------------------------------------------------- # Built before the push so a failing build doesn't leave a pushed image with # no matching installers for the same tag. - $artifacts = @() + $artifacts = @() + $wslArtifacts = @() + $wslUploadDistro = $null + $wslStageDir = $null if (-not $NoBinaries) { $distDir = Join-Path $repoRoot "dist" @@ -397,12 +641,28 @@ try { } # Uploading an installer older than the code it claims to be is the one - # way this flag can quietly go wrong, so say so rather than assume. + # way this flag can quietly go wrong, so say so rather than assume. The + # artifacts a -KeepInWsl build left in the distro count as present here: + # dist/ can legitimately hold nothing but SHA256SUMS.txt. + $manifest = Get-WslArtifactManifest -DistDir $distDir -Tag $Tag $oldest = (Get-ChildItem $distDir -Filter "motionity-$Tag-*" -File | Sort-Object LastWriteTime | Select-Object -First 1) - if (-not $oldest) { - throw "no installers matching motionity-$Tag-* in $distDir — what is on disk was built under a different tag. Drop -NoBinaryBuild." + $oldestName = if ($oldest) { $oldest.Name } else { $null } + $oldestTime = if ($oldest) { $oldest.LastWriteTime } else { $null } + + if ($manifest) { + Write-Host " $(@($manifest.files).Count) artifact(s) staged in '$($manifest.distro)':$($manifest.stageDir)" -ForegroundColor DarkGray + Test-WslArtifacts -Distro $manifest.distro -StageDir $manifest.stageDir -Names @($manifest.files) + $wslTime = Get-WslArtifactMtime -Distro $manifest.distro -StageDir $manifest.stageDir -Names @($manifest.files) + if ($wslTime -and (-not $oldestTime -or $wslTime -lt $oldestTime)) { + $oldestTime = $wslTime + $oldestName = @($manifest.files)[0] + } } + if (-not $oldestTime) { + throw "no installers matching motionity-$Tag-* in $distDir and no dist/wsl-artifacts.json for $Tag — what is on disk was built under a different tag. Drop -NoBinaryBuild." + } + # src/ is the app: every extension the packaged tree actually serves, # plus the packaging scripts themselves. $newer = Get-ChildItem $repoRoot -Recurse -Include *.js, *.cjs, *.mjs, *.html, *.css, *.json -File | @@ -410,10 +670,10 @@ try { $_.FullName -notlike "$distDir*" -and $_.FullName -notlike "*\node_modules\*" -and $_.FullName -notlike "*/node_modules/*" -and - $_.LastWriteTime -gt $oldest.LastWriteTime + $_.LastWriteTime -gt $oldestTime } if ($newer) { - Write-Warning "$($oldest.Name) predates $($newer.Count) source file(s) — the installers may not contain your latest changes (newest: $(($newer | Sort-Object LastWriteTime -Descending)[0].Name))." + Write-Warning "$oldestName predates $($newer.Count) source file(s) — the installers may not contain your latest changes (newest: $(($newer | Sort-Object LastWriteTime -Descending)[0].Name))." } } else { @@ -431,13 +691,34 @@ try { Targets = $Targets SkipVendor = $SkipVendor UseWsl = $UseWsl + WinInWsl = $WinInWsl + KeepInWsl = $KeepInWsl } if ($WslDistro) { $buildParams["WslDistro"] = $WslDistro } & (Join-Path $PSScriptRoot "build-release.ps1") @buildParams + + $manifest = Get-WslArtifactManifest -DistDir $distDir -Tag $Tag + if ($manifest) { + Test-WslArtifacts -Distro $manifest.distro -StageDir $manifest.stageDir -Names @($manifest.files) + } } - $artifacts = @(Get-ChildItem $distDir -Filter "motionity-$Tag-*" -File | ForEach-Object FullName) - if (-not $artifacts.Count) { throw "no installers for $Tag found in $distDir." } + if ($manifest) { + $wslArtifacts = @($manifest.files) + $wslUploadDistro = $manifest.distro + $wslStageDir = $manifest.stageDir + } + + # A name that exists both in dist/ and in the distro is the WSL build's, and + # the dist/ copy is a leftover from an earlier native build — uploading both + # would collide on the release's asset names anyway. + $artifacts = @(Get-ChildItem $distDir -Filter "motionity-$Tag-*" -File | + Where-Object { $wslArtifacts -notcontains $_.Name } | ForEach-Object FullName) + if (-not $artifacts.Count -and -not $wslArtifacts.Count) { + throw "no installers for $Tag found in $distDir." + } + # SHA256SUMS.txt covers both sets and is written on the Windows side either + # way — it is text, so nothing objects to it landing in dist/. $sums = Join-Path $distDir "SHA256SUMS.txt" if (Test-Path $sums) { $artifacts += $sums } Write-Host "" @@ -467,13 +748,34 @@ try { # --- Release attachments -------------------------------------------------- $releaseUrl = $null if ($PublishRelease) { - if (-not $artifacts.Count) { + if (-not $artifacts.Count -and -not $wslArtifacts.Count) { throw "-PublishRelease has nothing to upload (was -NoBinaries set?)." } Write-Host "Uploading artifacts to release $Tag..." -ForegroundColor Cyan - $Password = Resolve-Token -Provided $Password -Purpose "release upload to $ReleaseRepo" - $releaseUrl = Publish-BinaryRelease -ApiRoot $apiRoot -RepoPath $ReleaseRepo -Tag $Tag ` - -Token $Password -Artifacts $artifacts -Force:$Force + $Password = Resolve-Token -Provided $Password -Purpose "release upload to $ReleaseRepo" + + $publishArgs = @{ + ApiRoot = $apiRoot + RepoPath = $ReleaseRepo + Tag = $Tag + Token = $Password + Artifacts = $artifacts + NoReplace = $NoReplace + } + if ($wslArtifacts.Count) { + $publishArgs["WslArtifacts"] = $wslArtifacts + $publishArgs["WslDistro"] = $wslUploadDistro + $publishArgs["WslStageDir"] = $wslStageDir + # The token is exported for the whole upload rather than per file: WSLENV + # is process-wide state, and setting and restoring it around every + # attachment is more windows in which a concurrent wsl.exe sees it. + $releaseUrl = Invoke-WithWslEnv -Name "GITEA_UPLOAD_TOKEN" -Value $Password -Body { + Publish-BinaryRelease @publishArgs + } + } + else { + $releaseUrl = Publish-BinaryRelease @publishArgs + } Write-Host "" } @@ -482,10 +784,13 @@ try { Write-Host "Pushed:" -ForegroundColor Green foreach ($t in $tags) { Write-Host " $t" -ForegroundColor Green } } - if ($artifacts.Count) { + if ($artifacts.Count -or $wslArtifacts.Count) { $where = if ($PublishRelease) { "attached to release $Tag" } else { "built locally — attach to a release manually" } Write-Host "Artifacts ($where):" -ForegroundColor Green foreach ($a in $artifacts) { Write-Host " $a" -ForegroundColor Green } + foreach ($a in $wslArtifacts) { + Write-Host " [$wslUploadDistro] $wslStageDir/$a" -ForegroundColor Green + } if ($releaseUrl) { Write-Host " $ApiBase/$ReleaseRepo/releases/tag/$Tag" -ForegroundColor Green } } }