diff --git a/README.md b/README.md index fd4d58f..e06cf52 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@

DockMV

-Move Docker containers — and their data — from one host to another, from a web UI, in a few clicks. +Move Docker containers — **and their data** — from one host to another, from a web UI, in a few clicks. It handles the whole container, not just the image: named volumes, anonymous volumes, bind mounts, user-defined networks, published ports, environment, capabilities, restart policy, healthchecks and @@ -10,148 +10,173 @@ Two ways to move things: | Mode | What happens | When to use it | | --- | --- | --- | -| **Host to host over SSH** | The source connects to the target over SSH and streams everything straight into the target's `docker cp` / `docker load`. Nothing touches disk in between. | The two hosts can reach each other. | +| **Host to host over SSH** | The source streams everything straight into the target's `docker cp` / `docker load`. Nothing touches disk in between. | The two hosts can reach each other. | | **Migration package** | Builds a self-contained folder or `.tar` holding the data, the images, and a plain-bash `install.sh`. Carry it on a disk, run the script on the target. | Air-gapped targets, or when you want the move reviewed and replayed later. | -The target needs **nothing installed**: no agent, no Python, no Go. Just `sshd`, a working `docker` -CLI, `bash` and `gzip`. +The target needs **nothing installed**: no agent, no Python, no Go — just `sshd`, `docker`, `bash` and `gzip`. --- -## Quick start +## Install -### Run it in a container (recommended) +Run DockMV on the **source** host (the one holding the containers to move). -On the **source** host: +### Docker Compose — recommended ```bash git clone https://git.azuze.fr/kawa/DockMV.git dockmv && cd dockmv -docker compose up -d --build +docker compose up -d docker compose logs dockmv # prints the URL, including the access token ``` -Then open the printed URL. It binds to `127.0.0.1` only; reach it from your laptop with a tunnel: +Open the printed URL. It binds to `127.0.0.1` only; reach it from your laptop with a tunnel: ```bash ssh -L 8080:127.0.0.1:8080 you@source-host ``` -### Run it bare-metal +Uses the published image `git.azuze.fr/kawa/dockmv:latest`. Pin a version with `VERSION=v1.2.0 docker compose up -d`, +and set a fixed token with `DOCKMV_TOKEN` in the compose file to keep the same URL across restarts. -The binary is fully static and embeds the web UI, so there is nothing to install alongside it. +### Prebuilt binary + +Grab the archive for your platform from the [releases](https://git.azuze.fr/kawa/DockMV/releases) +(`dockmv--.tar.gz`, `.zip` on Windows). The binary is static and embeds the web UI — +nothing to install alongside it. ```bash -make build # needs Go 1.25+ and Node 20+ ... or just `go build .` if you skip the UI rebuild +tar xzf dockmv-linux-amd64.tar.gz +./dockmv-linux-amd64 serve +``` + +### From source + +```bash +git clone https://git.azuze.fr/kawa/DockMV.git dockmv && cd dockmv +make build # rebuilds the UI, then the binary ./dockmv serve ``` -Prebuilt for several platforms: +`go build .` alone also works — the built UI is committed, so the Node toolchain is optional. -```bash -make release # dist/dockmv-linux-amd64, -linux-arm64, -darwin-arm64, -windows-amd64 -``` +> DockMV needs access to the Docker socket on the source host: run it as a user in the `docker` +> group, or as root. That is equivalent to root on that host, so keep the UI on loopback. -`dockmv` needs access to the Docker socket on the source host, so run it as a user in the -`docker` group (or as root). +--- + +## Dependencies + +### Source host — where DockMV runs + +| Requirement | Notes | +| --- | --- | +| Docker daemon + access to `/var/run/docker.sock` | reads containers and streams their data | +| Docker Compose | only for the compose install | + +Nothing else. The binary is static: no libc, no runtime, no Python. + +### Target host — where containers land + +| Requirement | Why | +| --- | --- | +| `sshd`, reachable from the source | transport | +| `docker` CLI + a working daemon | everything | +| the login user can use docker | either in the `docker` group, or tick *run docker through `sudo -n`* | +| `gzip` | compressed transfers; without it the tool falls back to uncompressed | +| `bash` | only for the migration package installer | + +The **connect** button runs a preflight and tells you which of these are missing, plus the target's +free disk space and architecture. + +### Building from source + +| Tool | Version | +| --- | --- | +| Go | 1.25+ | +| Node | 20+ (22 in CI) — only to rebuild the UI | +| `make` | optional, wraps the two above | +| PowerShell 7+ | only for `make publish` / `make release` | + +Libraries: [`docker/docker`](https://github.com/docker/docker) v28.3.3 and `golang.org/x/crypto` +on the Go side; React 19, Vite 7 and TypeScript 5.9 on the UI side. That is the whole list. --- ## Using it -1. **Containers tab** — everything on the source host, grouped by compose project. - Tick the ones to move. Use *select all*, *select running*, or the compose-project checkbox for - batch selection. -2. **Expand a row** (`▸`) to choose per-container details: the name on the target, whether the image - is pulled or transferred, whether networks and ports come along, and — per mount — whether to - **copy the data**, **create it empty**, or **not mount it at all**. Bind mounts can be relocated to - a different path on the target; named volumes can be renamed. -3. **Apply to selected** in the toolbar does the same thing to every selected container at once - (*copy all data*, *skip binds*, *image: pull on target*, …). -4. **Right panel** — add the target host, hit *connect*, review the options, then either - **migrate over SSH** or **build a package**. -5. **Preview the commands** shows the exact `docker` invocations that will run on the target. Nothing - is hidden. -6. **Jobs tab** — live progress per container and per mount, with the full command log. +1. **Containers tab** — everything on the source host, grouped by compose project. Tick what to move. +2. **Expand a row** (`▸`) for per-container details: target name, image pulled or transferred, + networks and ports, and — per mount — **copy the data**, **create it empty**, or **do not mount it**. + Bind mounts can be relocated; named volumes renamed. +3. **Apply to selected** does the same thing to every selected container at once. +4. **Right panel** — add the target host, *connect*, then **migrate over SSH** or **build a package**. +5. **Jobs tab** — live progress per container and per mount, with the full command log. -Start with **dry run** ticked. It performs every check and prints every command without changing -anything on the target. +Start with **dry run** ticked: it runs every check and prints every command without touching the target. +**Preview the commands** shows the exact `docker` invocations that will run. Nothing is hidden. ---- +
+How the data is actually moved -## How the data is actually moved - -The interesting part is that there is exactly **one** mechanism for every kind of data location: +One mechanism for every kind of data location: ``` source daemon ──CopyFromContainer(/mount/path)──▶ tar stream ──gzip──▶ ssh ──▶ docker cp -a - ctr:/parent ``` -The source container's own mount path is read through the Docker archive API — the same thing -`docker cp` uses. That means: +The mount is read through the Docker archive API — the same thing `docker cp` uses. So: -- named volumes, anonymous volumes and bind mounts are all handled identically; -- no helper image is pulled, and the container's image does not need `tar` inside it; +- named volumes, anonymous volumes and bind mounts are handled identically; +- no helper image is pulled, and the image does not need `tar` inside it; - it works whether the container is running or stopped; -- file ownership, permissions, symlinks and hardlinks are preserved (`docker cp -a`). +- ownership, permissions, symlinks and hardlinks are preserved (`docker cp -a`). -On the target the container is **created first, started last**. Creating it is what makes Docker -materialise the named volumes and bind directories; the data is then copied into the stopped -container, and only then is it started. +On the target the container is **created first, started last** — creating it is what makes Docker +materialise the volumes and bind directories; data is copied into the stopped container, then it +starts. Mounts declared **read-only** get a throwaway container (never started) with the volume +attached writable, and it is removed straight after. -A mount the container declares **read-only** cannot be written through the container itself. For -those, a throwaway container is created (never started) with the same volume attached writable, the -data is copied into it, and it is removed straight after. +**Faithfully reproduced:** image (pull or layer transfer), command, entrypoint, environment, labels, +working directory, user, hostname, published and exposed ports, all mount types, user-defined +networks with subnets and aliases, DNS, extra hosts, capabilities, devices, sysctls, ulimits, +security options, restart policy, stop signal and timeout, healthcheck, log driver, memory/CPU/pids +limits, privileged, read-only rootfs, init, and the PID/IPC/UTS/userns modes. -### What is faithfully reproduced +Settings that come from the **image** are deliberately not re-emitted, so the recreated container +carries only genuine run-time overrides and keeps working when the image is updated. -Image (by pull or by layer transfer), command, entrypoint, environment, labels, working directory, -user, hostname, published and exposed ports, all mount types, user-defined networks with their -subnets and the container's aliases, DNS settings, extra hosts, capabilities, devices, sysctls, -ulimits, security options, restart policy, stop signal and timeout, healthcheck, log driver and -options, memory/CPU/pids limits, privileged, read-only rootfs, init, and the PID/IPC/UTS/userns -modes. +**What it will not do:** -Settings that come from the **image** are deliberately not re-emitted — the recreated container -carries only genuine run-time overrides, so it stays readable and keeps working when the image is -later updated. +- `--rm` is never reapplied — a container that deletes itself cannot be inspected. +- `--volumes-from` and `--network container:other` are not reproduced; you are warned. +- Swarm services are out of scope. Plain containers only. +- **Live databases**: copying a running database's files is crash-consistent at best. The default + stops the source container while copying — leave it on, or migrate a dump instead. +- **Cross-architecture**: an `amd64` image will not run on `arm64`. The preflight shows the target's arch. -### What it will not do for you +
-- **`--rm` is never reapplied.** A migrated container that deletes itself cannot be inspected. -- **`--volumes-from` is not reproduced.** You are warned; migrate the other container and mount - explicitly. -- **`--network container:other`** requires the other container to be migrated too. You are warned. -- **Swarm services** are out of scope. This tool moves plain containers. -- **Live databases**: copying a running database's files gives you a crash-consistent snapshot at - best. The default is to stop the source container while copying — leave it on. For anything you - really care about, take a dump instead and migrate that. -- **Cross-architecture moves**: an `amd64` image will not run on an `arm64` target. The preflight - shows the target's architecture; check it. - ---- - -## Safety +
+Safety The tool can stop containers and read every volume on the host, so it is treated as a privileged admin tool: -- It binds to **`127.0.0.1` by default**. Binding anywhere else automatically generates an access - token and prints it. -- **SSH host keys are verified** exactly like OpenSSH. An unknown key is refused until you approve - the fingerprint in the UI; a *changed* key is refused outright until you explicitly replace it. - Trusted keys go to `/known_hosts`. -- **Credentials are not persisted unless you ask.** By default the password or key lives in memory - for the session. Ticking *remember* writes it to `/connections.json`, mode `0600`. -- **Nothing on the target is overwritten by default.** If a container name already exists the item - fails; you choose *skip*, *rename* or *replace* explicitly. An existing **volume** is reused and - merged into, never silently deleted, unless you pick *replace*. +- Binds to **`127.0.0.1` by default**. Binding elsewhere auto-generates an access token and prints it. +- **SSH host keys are verified** like OpenSSH. An unknown key is refused until you approve the + fingerprint in the UI; a *changed* key is refused outright. Trusted keys go to `/known_hosts`. +- **Credentials are not persisted unless you ask.** *Remember* writes them to + `/connections.json`, mode `0600`. +- **Nothing on the target is overwritten by default.** An existing container name fails the item; + you pick *skip*, *rename* or *replace*. An existing volume is reused and merged into, never + silently deleted, unless you pick *replace*. - Every command that runs on the target is echoed into the job log. -- Bind mounts of `/var/run/docker.sock`, `/proc`, `/sys`, `/dev` and `/` are flagged, and a mount at - `/` is refused outright. +- Bind mounts of `/var/run/docker.sock`, `/proc`, `/sys`, `/dev` and `/` are flagged; a mount at `/` is refused. ---- +
-## The migration package +
+The migration package `build package` produces: @@ -171,8 +196,8 @@ On the target: ./install.sh # restore ``` -The installer never parses the manifest — every command is written out literally, so it can be read -and audited before running. It checksums each payload before feeding it to Docker, and supports: +The installer never parses the manifest — every command is written out literally, so it can be +audited before running. It checksums each payload, and supports: ``` --dry-run print every command without changing anything @@ -186,24 +211,10 @@ and audited before running. It checksums each payload before feeding it to Docke --sudo prefix docker with sudo -n ``` ---- +
-## Target host requirements - -| Requirement | Why | -| --- | --- | -| `sshd`, reachable from the source | transport | -| `docker` CLI + a working daemon | everything | -| the login user can use docker | either in the `docker` group, or tick *run docker through `sudo -n`* | -| `gzip` | compressed transfers; without it the tool falls back to uncompressed | -| `bash` | only for the migration package installer | - -The **connect** button runs a preflight and tells you which of these are missing, plus the target's -free disk space and architecture. - ---- - -## Command line +
+Command line and HTTP API ``` dockmv [serve] [flags] start the web interface (default) @@ -228,10 +239,6 @@ dockmv version dockmv inspect --sizes | jq '.containers[] | {name, image, mounts}' ``` ---- - -## HTTP API - Everything the UI does is available over HTTP. Pass the token as `X-Auth-Token` when one is set. ``` @@ -254,9 +261,10 @@ POST /api/jobs/{id}/cancel GET /api/packages, /api/packages/{name}/download ``` ---- +
-## Development +
+Development ``` web/ React + TypeScript UI (vite) @@ -276,30 +284,20 @@ make ui # rebuild the embedded UI cd web && npm run dev # UI dev server on :5173, proxying /api to :8080 ``` -### Verifying it works +`internal/webui/dist` is committed so that a plain `go build .` produces a working binary without a +Node toolchain. Rerun `make ui` after changing anything under `web/`. -On a Linux host with Docker (a VM is fine): +End-to-end tests, on a Linux host with Docker (a VM is fine): ```bash go test ./... # unit tests: command rendering, plan resolution, # and the generated installer, checked with bash - -go test -tags e2e ./test/... -v # end to end, against the real daemon: - # creates a container with a named volume, a - # read-only bind mount and an anonymous volume, - # writes files into all three, builds a package, - # runs the generated install.sh, then reads the - # files back out of the restored container +go test -tags e2e ./test/... -v # end to end, against the real daemon ``` -The e2e tests restore onto the same daemon under a suffixed name and clean up after -themselves, so a single machine is enough: - - `TestPackageRoundTrip` — builds a package and runs the generated `install.sh` for real. -- `TestSSHMigration` — drives the host-to-host engine over a genuine SSH connection to - `127.0.0.1`, so the whole transport (ssh, gzip streaming, the target's docker CLI, the - staging container for read-only mounts, the verify step) is exercised. It needs - `DM_SSH_HOST`, `DM_SSH_USER` and `DM_SSH_KEY`, and skips without them: +- `TestSSHMigration` — drives the host-to-host engine over a genuine SSH connection to `127.0.0.1`, + exercising the whole transport. Needs `DM_SSH_HOST`, `DM_SSH_USER` and `DM_SSH_KEY`, skips without them: ```bash ssh-keygen -t ed25519 -N '' -f ~/.ssh/dm_loop @@ -308,8 +306,7 @@ themselves, so a single machine is enough: go test -tags e2e ./test/... -run TestSSHMigration -v ``` -Both were run against Debian 13 with Docker 29.7.2, alongside a browser pass over the web -UI covering the trust prompt, a batch migration and a package build. +They restore onto the same daemon under a suffixed name and clean up, so one machine is enough. +Both were run against Debian 13 with Docker 29.7.2, alongside a browser pass over the web UI. -`internal/webui/dist` is committed so that a plain `go build .` produces a working binary without a -Node toolchain. Rerun `make ui` after changing anything under `web/`. +