pivot to Python: replace Kotlin/JVM with stdlib zipapp
Reasons stacked up:
- AV: unsigned JARs that auto-download binaries + upload files trigger
Windows Defender false-positives more often than Python scripts
invoked by code-signed python.exe.
- Qt UI option: PySide6 opens a path to a real Qt UI (matching Prism's
look) if needed later. JVM Qt bindings are abandoned.
- frazclient already needs Python; inlining as 'import cloud_sync' is
zero overhead vs the launcher always shelling out to java.
Implementation:
- cloud_sync package: cli.py (argparse), creds.py, scope.py,
restic.py (binary discovery + auto-download + sha256 verify),
sync.py (pull/push subprocess restic).
- pyproject.toml with hatchling backend; pip-installable.
- Makefile builds cloud-sync.pyz via python -m zipapp (~53 KB).
- 33 pytest tests, stdlib only on runtime.
- CI workflow runs pytest matrix (3.10/3.11/3.12) + builds pyz.
- DESIGN.md + README.md updated to reflect Python.
E2E verified against local restic-rest-server:
pull empty → push initial → rm -rf local → pull restores → modify+push
creates second snapshot → client forget --prune blocked by --append-only.
Throws away ~565 LOC of Kotlin (and 18 jar tests) committed earlier in
this same session. Net result is ~250 LOC Python + 33 tests = smaller
and more aligned with the rest of the stack.
This commit is contained in:
@@ -1,12 +1,12 @@
|
||||
# cloud-sync — design
|
||||
|
||||
Per-Discord-user state sync for Minecraft. Pulls on launch, pushes on exit. Single JAR drops into Prism / MMC / ATLauncher / frazclient as a pre-launch + post-exit hook.
|
||||
Per-Discord-user state sync for Minecraft. Pulls on launch, pushes on exit. Single Python zipapp drops into Prism / MMC / ATLauncher / frazclient as a pre-launch + post-exit hook.
|
||||
|
||||
**Data plane:** `restic-rest-server` with `--private-repos --append-only`. Clients hit this directly with their per-user password.
|
||||
|
||||
**Control plane:** `cloud-svc` Go service with two listeners — a provisioning port reachable from automc-net (called by discord-bot) and a loopback admin port (called by automc-setup wizard). Players never touch cloud-svc.
|
||||
|
||||
**Client:** `cloud-sync.jar` subprocesses restic. ~200 LOC.
|
||||
**Client:** `cloud-sync.pyz` (Python 3.10+, stdlib only) subprocesses restic. ~300 LOC. Distributed as a zipapp (single-file). Python over Java for two reasons: (a) launcher's PostExit hook can call any subprocess so language doesn't matter, (b) custom unsigned JARs that download binaries + upload files are textbook Windows Defender false-positive triggers, while Python invoked by signed `python.exe` mostly sidesteps that.
|
||||
|
||||
## Why this shape
|
||||
|
||||
@@ -29,8 +29,8 @@ flowchart LR
|
||||
pl["player PC"]:::external
|
||||
op["operator
|
||||
(via SSH)"]:::external
|
||||
jar["cloud-sync.jar
|
||||
(in launcher's
|
||||
jar["cloud-sync.pyz
|
||||
(Python; in launcher's
|
||||
pre/post hooks)"]:::deploy
|
||||
restic["restic binary
|
||||
(auto-downloaded
|
||||
@@ -115,7 +115,7 @@ Revocation = operator runs `automc-setup cloud revoke <discord_id>` which hits t
|
||||
|
||||
## On-disk layout (client)
|
||||
|
||||
cloud-sync.jar stores its state under `<pack-folder>/.cloud-sync/` — per-instance, hidden by leading dot. Auto-excluded from cloud sync so a player can't accidentally upload their own credentials.
|
||||
cloud-sync.pyz stores its state under `<pack-folder>/.cloud-sync/` — per-instance, hidden by leading dot. Auto-excluded from cloud sync so a player can't accidentally upload their own credentials.
|
||||
|
||||
```
|
||||
<pack-folder>/
|
||||
@@ -145,21 +145,21 @@ Probed in order:
|
||||
|
||||
### Jar placement
|
||||
|
||||
Stateless. Lives wherever the operator put it. Prism / MMC config references absolute path. One jar can serve N instances; each gets its own `.cloud-sync/` underneath its own `--pack-folder`.
|
||||
Stateless. Lives wherever the operator put it. Prism / MMC config references absolute path. One pyz can serve N instances; each gets its own `.cloud-sync/` underneath its own `--pack-folder`.
|
||||
|
||||
## Client flow
|
||||
|
||||
### `cloud-sync.jar pull`
|
||||
### `cloud-sync.pyz pull`
|
||||
|
||||
```
|
||||
1. Load creds from <pack-folder>/.cloud-token (format: discord_id:password on one line)
|
||||
2. Locate or auto-download restic binary into <jar dir>/restic-<version>/
|
||||
2. Locate or auto-download restic binary into <pack-folder>/.cloud-sync/restic-<version>
|
||||
3. restic -r rest:https://<url>/<discord_id>/ snapshots --latest 1 --json
|
||||
4. If no snapshots → exit 0 (first run on this machine, nothing to restore)
|
||||
5. restic restore latest --target <pack-folder> --include-from cloud-scope.txt
|
||||
```
|
||||
|
||||
### `cloud-sync.jar push`
|
||||
### `cloud-sync.pyz push`
|
||||
|
||||
```
|
||||
1. Same creds + restic locator as pull
|
||||
@@ -201,7 +201,7 @@ Recommendation: **option 1** (multi-key per repo). On `/register`, the bot calls
|
||||
|
||||
- restic-rest-server with `--private-repos --append-only --htpasswd-file`
|
||||
- discord-bot `/register` extension: mint password, htpasswd add, `restic init` repo, `restic key add` player key
|
||||
- cloud-sync.jar that subprocesses restic for pull/push
|
||||
- cloud-sync.pyz that subprocesses restic for pull/push
|
||||
- Auto-download restic binary on first run from upstream GitHub release
|
||||
- Server-side nightly prune cron with operator-side master password key
|
||||
|
||||
@@ -236,7 +236,7 @@ New code:
|
||||
|
||||
Estimate: ~300 LOC kept, ~600 LOC new. Net smaller than current cloud-svc.
|
||||
|
||||
Also delete `cloud_pull` / `cloud_push` from `frazclient/client.py` (these get obsoleted by `cloud-sync.jar` calls).
|
||||
Also delete `cloud_pull` / `cloud_push` from `frazclient/client.py` (these get obsoleted by `import cloud_sync` calls; frazclient depends on the same package).
|
||||
|
||||
## Topology consequences for `automc/docs/network-exposure.md`
|
||||
|
||||
@@ -260,7 +260,7 @@ Operator endpoints are loopback-only and require SSH access to john to reach. No
|
||||
|
||||
| Repo | Purpose |
|
||||
|---|---|
|
||||
| `Timemachine/cloud-sync` (this) | Kotlin/Gradle JAR that subprocesses restic |
|
||||
| `Timemachine/cloud-sync` (this) | Python 3.10+ package + zipapp that subprocesses restic |
|
||||
| `Timemachine/cloud-svc` | **Reshaped** — control plane only. Two-port Go service for provisioning + operator ops. NOT archived. |
|
||||
| `Timemachine/discord-bot` | Extended `/register` flow calls cloud-svc to provision; DMs returned password |
|
||||
| `Timemachine/automc` | `setup` wizard adds `automc-setup cloud {list,prune,revoke,quota}` subcommands hitting cloud-svc's loopback admin port. Quadlet templates for both restic-ao (new flags) and cloud-svc (two listeners). `database/schema.sql` unchanged. |
|
||||
@@ -272,6 +272,6 @@ All locked 2026-06-02:
|
||||
- [x] cloud-svc reshapes to control plane, not archived
|
||||
- [x] Two-port split — automc-net for provisioning, loopback for operator
|
||||
- [x] Server-side prune via operator master password key on each repo. On `provision`, cloud-svc runs `restic init` then `restic key add` with the operator-master password as a SECOND key. The nightly pruner uses the operator key to open any repo.
|
||||
- [x] cloud-sync.jar auto-downloads restic binary. Matches `packwiz-installer-bootstrap` pattern. First run hits `https://github.com/restic/restic/releases` for the matching platform binary, caches under `<jar dir>/restic-<version>/`. `--no-download` flag for air-gapped operators.
|
||||
- [x] cloud-sync.pyz auto-downloads restic binary. Matches `packwiz-installer-bootstrap` pattern. First run hits `https://github.com/restic/restic/releases` for the matching platform binary, caches under `<pack-folder>/.cloud-sync/restic-<version>`. SHA256 verified against the release's `SHA256SUMS` file. `--no-download` flag for air-gapped operators.
|
||||
- [x] Nightly prune at 04:00 UTC. `time.Ticker` inside cloud-svc; no external cron. `--prune-time HH:MM` flag in case operators want a different window.
|
||||
- [x] Per-caller tokens, NOT shared. cloud-svc reads `CLOUD_PROVISIONING_TOKENS_BOT`, `CLOUD_PROVISIONING_TOKENS_<OTHER>` env vars — one per known caller. Logs include the matched caller name so audit trails show which service made each call. Adding a future caller (e.g., a portal) means a new env var, not a token rotation.
|
||||
|
||||
Reference in New Issue
Block a user