GO-7348 spacev2: greenfield space orchestration rewrite - #3190
Draft
requilence wants to merge 25 commits into
Draft
GO-7348 spacev2: greenfield space orchestration rewrite#3190requilence wants to merge 25 commits into
requilence wants to merge 25 commits into
Conversation
Replace single GetMessages call that loads all messages into memory with batched iteration (100 messages per batch). Add OrderAsc field to GetMessagesRequest for forward pagination. Extract shared chatMessageToSearchDoc helper to eliminate duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unnecessary OrderAsc field from GetMessagesRequest. Paginate using existing BeforeOrderId with batch[0].OrderId as cursor, since queryMessages always returns results sorted ascending. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
docs/SpaceController.md is the clean-room reimplementation spec; the spacev2 package is the inert skeleton (not registered in bootstrap).
…te path) - addStatic no longer double-closes the ready channel of a failed entry (found by writing the steady-state test) - new tests: ensure-vs-closeAll shutdown race; apply drives Update on an existing controller; SpaceView detail change flows end-to-end through the subscription UpdateKeys path; convertSpaceError contract
Port of v1 mode.StateMachine with two sanctioned changes: waiters receive the start-failure cause (v1 TODO [MR] — waiters got nil), and CanTransition is dropped (§11.3 — gating lives in the controller mode mapping; offloading stays non-terminal for CancelLeave). Same-mode requests during a divergent in-flight transition now get ErrTransitionInProcess instead of a handle to the process being torn down.
Remove the M1-M3 orchestration implementation (service, controller, state machine, registry, watcher, factory, tech provider, marketplace) and its tests, leaving only HANDOFF.md. Per the greenfield mandate the next implementer starts fresh from the docs, not the existing shape. Full code preserved on branch spacev2-agent-wip (recover: git checkout spacev2-agent-wip -- space/spacev2).
Greenfield restart per HANDOFF.md: replace the mode-state-machine + five-controller-types shape with a per-space reconciler. DESIGN.md is the normative architecture doc. - state.go: State/Target enums, pure decide(status, wanted) -> target, Fatal() error marker for non-retryable backend failures. - controller.go: one reconcile goroutine per space converges current state to the target through a narrow Backend seam (AccountStatus/Load/Unload/ Offload/Join); controller owns retry with exponential backoff; input/ decision sequence numbers make WaitLoaded's terminal answers race-free against concurrent input changes and prevent stale-poke re-attempts. Pause/unload is a first-class Loaded->Idle transition. - registry.go: idempotent getOrCreate keyed by spaceId, no waiting map, no error caching; concurrent close-all. - Unit tests (fake backend with step-concurrency detector) cover load/ pause/delete/restore/join-accept/join-reject/offload-without-load, transient retry, fatal park+surface+recover, mid-load status flips, shutdown during load, waiter semantics; 10x -race clean. Claude-Session: https://claude.ai/code/session_01AHPvaSRimgrQmY2BWpomsU
Backend implementation for the reconciler (backends.go, presetloader.go): - Load: guest-key decode, optimistic-Ok LocalStatus publication, clientspace.BuildSpace via spacecore, access-type write, mandatory-objects + ACL-head gates, then the reused post-load domain components (aclobjectmanager, participantwatcher, aclnotifications, migration, personalmigration for the personal space) hosted in a child app; the presetLoader shim satisfies their spaceloader.SpaceLoader barrier with the already-built space, so OnSpaceLoad/OnSpaceUnload keep firing from aclobjectmanager exactly as in v1. Terminal build failures map to the same LocalStatus/RemoteStatus writes as v1's spaceloader and are Fatal; transient ones keep Loading and are retried by the controller. - Unload (pause): close pipeline then space; ocache eviction stops sync; keeps on-disk data and LocalStatus. - Offload: AddSpaceToDelete always; skip when already Missing; storage (10s lock timeout) -> files -> indexes -> LocalStatusMissing; missing storage tolerated (idempotent). - Join: participant-status + LocalStatusUnknown writes, then hosts the any-sync aclwaiter until accept (Active+head) / reject (Deleted+head + decline notification), blocking as the reconciler's join step. Also fixes a controller determinism hole the new tests exposed: the demand flag is now snapshotted together with inputSeq, so a SetWanted racing the status read can no longer produce a fatal outcome stamped stale (which was silently re-attempted). Backend unit tests use embed-nil fakes for the SpaceView/storage/indexer/offloader seams; 30x -race suite runs clean. Claude-Session: https://claude.ai/code/session_01AHPvaSRimgrQmY2BWpomsU
- bootstrap.go: tech-space resolution as an explicit, pure decision tree (new-account create / load / offline-old-account fallbacks / nodes-report- missing), isolated from v1's nested initAccount; real create/load impls over spacecore + clientspace.NewTechSpace; ensurePersonalSpace heals a missing personal SpaceView on every start (subsumes v1's SkipCheckSpaceView path); marketplace becomes a static virtual-space entry with reindex-once (a failed reindex now keeps reporting its error instead of v1's silent success on the second call). - watcher.go: SpaceView objectsubscription -> onEvent; no dedupqueue — the controller's buffered poke coalesces bursts; initial snapshot replayed via Iterate (OnAdded only fires for later additions). - service.go: component skeleton — Init (deps, deterministic ids, account metadata), Run (marketplace -> resolve tech space -> register on child app -> techSpaceReady -> personal ensure -> watcher -> new-account synchronous personal load -> lazy drain -> StartSync -> persist network id), Close (watcher -> controllers -> tech space). Lazy mode is just wanted=false on discovery: no deferredStatuses cache; drain is a SetWanted sweep. - controller.go: close the second stale-decision hole the stress runs caught — SetWanted now bumps inputSeq in the same critical section as the wanted write (previously a separate Poke could let the loop snapshot new demand with an old seq stamp and re-attempt a fatal load). 75x -race suite runs clean. Claude-Session: https://claude.ai/code/session_01AHPvaSRimgrQmY2BWpomsU
- api.go: full verb surface on the reconciler primitives — Get (controller must exist; promotes paused/deferred spaces), Wait (creates the controller directly once the SpaceView exists; no 500ms polling), Create/CreateOneToOne (single build path: core-space create -> MarkSpaceCreated -> SpaceViewCreate -> WaitLoaded; one-to-one revives a locally-removed view), Join/InviteJoin (unidirectional: write Joining/Active + acl head, reconciler does the rest), CancelLeave/Delete (status writes), AddStreamable (idempotent), accessors, PreloadRemainingSpaces, UpdateRemoteStatus/UpdateSharedLimits/AllSpaceIds (deletioncontroller.SpaceManager, compile-time asserted), AllLoadedSpaceIds + SyncAllSpaceHeads (GO-7302 foreground kick), participant-remove notification, stream auto-join with backoff, documented error set. - service.go: watcher policy — remote-deleted reaction (notify + flip AccountStatusDeleted, no controller build, mirroring v1's early return), lazy-mode preferred-broken fallback, computeLazyMode with view validation. - bootstrap fix: v1 createAccount never creates the derived personal space — a new account gets the tech space plus one *created* first space (firstCreatedSpaceId). resolveTechSpace now reports the old-account path, which is the only place the personal SpaceView is healed eagerly. - Tests: API fixture over scripted techspace/notification fakes covering all verbs, discovery policies, remote-deleted reaction, preload drain, preferred-broken release; suite -race clean. Claude-Session: https://claude.ai/code/session_01AHPvaSRimgrQmY2BWpomsU
Package space becomes a thin facade: the v1 Service interface verbatim (all space/mock_space users keep working without regeneration), the error set aliased to spacev2's values (errors.Is works across both import paths, incl. the historical ErrSpaceStorageMissig spelling), and New() returning the v2 service. spacev2 takes over CName client.space; the SpaceView watcher subscription id therefore matches v1's. Removed v1 orchestration: service/load/create/join/init/metadata/waiter/ streamable/spacesub/spacewatcher + tests, dedupqueue, spacefactory, the five controller types (personal/shareable/streamable/marketplace + generic), spaceprocess (mode state machine, loader/offloader/joiner/initial) and the v1-only components spaceoffloader + syncstopper (dead code). The reused pipeline components stay under space/internal/ (spacestatus, spaceloader + builder as interface seams for them, aclobjectmanager, participantwatcher, aclnotifications, migration, personalmigration, dependencies, objectprovider). core/anytype/bootstrap.go: drop the spacefactory registration; space.New() now boots v2. .mockery.yaml: drop entries for deleted packages. Verified: go build ./... green; go vet over space+core (tests compile); test suites pass for ./space/..., ./core/anytype/..., ./core/block/..., ./core/acl/..., ./core/indexer/..., ./core/subscription/... Runtime verification against real accounts is the remaining step (HANDOFF updated). Claude-Session: https://claude.ai/code/session_01AHPvaSRimgrQmY2BWpomsU
Review finding (confirmed independently by 4 lenses + 8 adversarial verifiers): v1's lazy mode collapsed to eager after the first backlog release (the releasing flag in decideAndApplySpaceStatus), but v2's wantedOnDiscovery kept returning false for non-preferred spaces forever. A SpaceView first discovered after the drain — a space shared to this account or created on another device mid-session — was created with wanted=false and never background-loaded or synced until manually opened. Fix: a lazyReleased atomic flag set by drainDeferredLater BEFORE the SetWanted sweep (an event racing the sweep either sees the flag or its controller is already registered and swept — no gap), checked by wantedOnDiscovery. Also covers CancelLeave of a non-preferred space after the drain. Regression test pins the late-discovery eager behavior. Claude-Session: https://claude.ai/code/session_01AHPvaSRimgrQmY2BWpomsU
Coverage provided by https://github.com/seriousben/go-patch-cover-action |
Base automatically changed from
go-6758-optimize-full-chat-messages-reindex
to
develop
July 3, 2026 20:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
space/spacev2/(design rationale inspace/spacev2/DESIGN.md, build log inspace/spacev2/HANDOFF.md)clientspace,techspace,spacecore,spaceinfo, storage/migration — only orchestration (lifecycle, state machine, watcher, registry, lazy-load/pause/unload) is rewrittenspaceis now a thin façade preserving the v1Serviceinterface (so existingspace/mock_spaceconsumers need no changes);spacev2ownsclient.spaceBase branch note
Stacked on #3081 (GO-6758) since this branch was cut before that PR merged to
develop. Retarget todeveloponce #3081 lands.Status
Draft — all unit suites are green (
go build ./...,go vet ./...,go test ./space/..., plus consumer suites), including-racestress runs on the reconciler's seq-freshness logic. Not yet done: real end-to-end runtime verification (existing accounts of all vintages loading/syncing, cross-device profile + push, lazy mode against a real client) — flipping out of draft once that's verified.Test plan
go build ./...go vet ./...go test ./space/...(incl. new spacev2 unit + race-stress tests)