Skip to content

[Feature]: Git-backed OVPack backup relay for cross-machine handoff #4166

Description

@hachimin-wh

Issue Origin

Observed or reproduced in a real environment

Problem Statement

Individual users run OpenViking on several personal machines one at a time. Switching machines means continuing on machine B where machine A left off — a backup handoff, not two-way sync: no concurrent writers, no deletion propagation, explicit and rejectable over implicit. Today this is done by hand (ad-hoc copies): no version record, and copying the workspace drags along runtime state (indexes, locks, queues).

ov backup already produces the right artifact — OVPack v3 whole-account archive, manifest + content_sha256, accounts/API keys/runtime state excluded — but it can only write a local path. Missing pieces:

Primitive Status Gap
ov backup ✅ OVPack v3 local paths only, no remote destination
ov restore ✅ manifest/checksum validation, merge-overwrite (#4003) no pull+verify workflow; return ≠ searchable
ov wait ✅ exists not wired into backup flow — session extraction kept running 2m50s after commit in our test, so immediate backup misses data
Git relay anti-clobber/retention policy only via hand-written scripts

The user's own private Git repository is the natural relay: already in their workflow, versioned, access-controlled.

Proposed Solution

Target model (single writer, non-simultaneous machines):

Machine A (in use): ov wait → ov backup → fetch & compare remote OID → commit
                    → push with explicit OID lease            (manual first)
Machine B (new):    fetch artifact → sha256/manifest check → ov restore
                    → ov wait → verify searchable → record applied OID

v1 scope (six items):

  1. Remote destination config for backup — one Git remote per account/server; all Git operations in the ov CLI client, credentials via local credential helper / SSH agent; the server stores no Git credentials and never touches the remote.
  2. Anti-clobber push (commit-OID CAS) — record the applied remote OID in the OV workspace after each successful push/restore; before push, fetch & compare — remote advanced but not applied locally → actionable "restore first" error; pushes use an explicit expected-OID lease (--force-with-lease=<branch>:<expected_oid>; zero OID for the very first push); a fresh machine with no record cannot first-push over an existing remote backup. All verified (see evidence).
  3. Completion barrierov wait before backup (enforced or documented as required). Evidence: extraction drain took 2m50s; skipping it loses in-flight extractions.
  4. Retention: latest-only — one visible artifact (orphan branch, lease-forced replacement); branch-protection rejection → actionable error. keep-N deferred (needs history rewriting).
  5. Pull+verify+restore workflow — fetch → metadata/sha256 check → existing ov restoreov wait → verify searchable. Fetching never touches VikingFS before an explicit restore.
  6. First-push confirmation with content disclosure — show what will be uploaded (contents + size), require explicit confirmation. OVPack includes privacy/ in plaintext (verified) — a private repo is access control, not end-to-end encryption; exclude/encryption deferred.

Questions for maintainers:

  1. Preferred route: (a) official docs + maintained example scripts (the full loop was demonstrated end-to-end with plain git commands), (b) built-in ov client subcommands for push/pull, or (c) a generic destination abstraction with Git as one implementation? We can deliver any; (a) is days, (b) keeps the server untouched.
  2. Is latest-only retention acceptable for v1, with keep-N deferred?

Alternatives Considered

  • Status quo (USB / ad-hoc workspace copy) — no versioning, runtime-state pollution, no integrity checks
  • Cloud-drive sync of the workspace — same pollution; no manifest/integrity; conflicts opaque
  • Mirror restore (propagate deletions) — contradicts the intended merge-overwrite semantics (fix(pack): 安全恢复账号级 OVPack 备份 #4003); deferred

Feature Area

Core (Client/Engine)

Use Case

Personal multi-machine users are one of OpenViking's real usage shapes. The complete relay loop is already achievable with plain git commands (proven below); what is missing is official sanction and productization: discoverability, correct defaults (completion barrier, anti-clobber), and documented semantics.

Example API (Optional)

Illustrative CLI shape (client-side only; exact naming open):


# machine A — end of session
ov wait
ov backup --remote git@github.com:me/ov-relay.git --branch relay --confirm

# machine B — start of session
ov restore --from-remote git@github.com:me/ov-relay.git --branch relay --verify
ov wait

Additional Context

Verified end-to-end (isolated instance, api_key mode, local bare repo as relay — nothing pushed to real GitHub):

Stage Result
ov backup (ADMIN key) 48,284 B / 112 entries / ~0.16s; zip container; manifest format_version: 3 with content_sha256
Pack inspection privacy/ included in plaintext (confirms disclosure requirement); .watch_tasks.json included (see #4139); accounts/API keys correctly excluded
Fresh machine simulation workspace wiped → old user/admin keys all Invalid API Key (key store not in pack) → re-issued via root
ov restore (clean pack) returns in 0.12s (logical writes only); ov wait then drains reindex (Embedding×65), 0 errors
ov find restored markers score 0.729 / 0.505; experiences & privacy configs intact
CAS push protection zero-OID first push ✅; no-record fresh machine: client ABORT + git stale info on bypass ✅; lease push advances gen1 3d814d7… → gen2 9fd8ffb… ✅; stale machine rejected at both layers ✅

Blocking prerequisite bug (#4139): with watch tasks present, backup packs viking://resources/.watch_tasks.json while restore rejects it → [INVALID_ARGUMENT] cannot import watch task control file; the round-trip is broken (fix PRs in flight: #4149, #4150). Only workaround today: cancel watch tasks before backup. Acceptance criterion 7 depends on it.

Acceptance criteria (v1):

  1. A Git remote can be configured per account/server; Git credentials never leave the client machine
  2. Push refuses to clobber a newer remote backup the local instance has not applied (client record + explicit OID lease)
  3. Backup is preceded by ov wait (enforced or documented); skipping it produces a documented warning
  4. The relay branch shows a single visible artifact; branch-protection rejection yields an actionable error
  5. Pull workflow verifies sha256/manifest before restore; docs state restore-return ≠ searchable and show the ov wait step
  6. First push displays pack contents/size and requires explicit confirmation
  7. Instances with watch tasks can complete backup → restore (depends on [Bug]: Backup packs .watch_tasks.json but restore rejects it — backup/restore round-trip broken whenever a watch task exists #4139)
  8. Docs cover: handoff ≠ sync (restore is merge-overwrite, deletions are not propagated); backup is online and non-atomic; a fresh machine requires re-issuing all client keys; backup/restore needs an ADMIN key in api_key mode (ROOT is blocked by the path allowlist, USER keys are for daily ops); ov wait double timeout (drain budget vs. CLI HTTP timeout)

Related: #4139 (blocking prerequisite bug; fixes proposed in #4149 / #4150); #4003 (merge-overwrite is intended restore semantics); #3909 / #3704 (scheduled sync exists only for inbound resources); #3516 (peer/identity continuity — different problem, listed to prevent a false duplicate).

Contribution

  • I am willing to contribute to implementing this feature

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions