docs(platform): document the cache and coordination engine (Redis / Valkey)

[daric93]

Why / What / How

Why. The self-hosting docs never described the Redis-compatible engine the platform depends on for caching, distributed locking, rate limiting, spend counters and the SSE streams that carry agent output to the browser. Two consequences were invisible to self-hosters:

• The backend always connects with a cluster client. backend/data/redis_client.py sets RedisClient = RedisCluster unconditionally, so a single standalone node cannot work. Nothing in the docs said so, and the failure mode is an opaque connection error.
• The two distributions already run different engines. The Compose stack ships redis:7; the single-container image runs Valkey. That was only discoverable by reading docker-compose.platform.yml and single-container/entrypoint.sh.

A self-hoster pointing the platform at ElastiCache, Memorystore or any single-node instance currently has to read the client, the Compose file and the entrypoint to discover all of this.

What. Adds a Cache and coordination engine section to Advanced Setup, between Database selection and AutoGPT Agent Server Advanced set up.

How. The section is organised around what a reader actually has to decide: which engine, what topology it must provide, how to select it locally, and what a managed deployment has to support. Everything it describes works on dev as it stands today — no accompanying code change is required.

Changes 🏗️

docs/platform/advanced_setup.md — new Cache and coordination engine section covering:

• Redis as the default and Valkey as a tested alternative, with a table of where each engine is used today (Compose stack, single-container image, CI coverage, connection settings) and the operational — not functional — grounds on which to choose
• Cluster mode is required — the topology both engines must provide, and which distribution forms it how
• Switching the Compose stack to Valkey with a docker-compose.override.yml, including the uid 999 requirement below
• Using a managed or external deployment — cluster mode, sharded pub/sub, announced addresses + REDIS_USE_ANNOUNCED_ADDRESS, and which file the backend actually reads REDIS_HOST/REDIS_PORT from
• Engine version floor implied by the commands the backend actually issues

The page does not claim Valkey is better, and does not assert specific licence terms — both projects' licensing has changed, so it tells readers to check the licence of the tag they pin. redis:7 remains the Compose default and the engine every backend CI leg covers.

docs/platform/getting-started.md — the make start-core row claimed it starts "Postgres, Redis, RabbitMQ". It actually starts PostgreSQL, the three-shard cache cluster, RabbitMQ, FalkorDB and ClamAV, and runs migrations.

No code or Compose changes here; this PR is docs only.

The one sharp edge, and why it's a warning block

Switching the shard image alone silently runs the cluster as root. Both images gate their privilege drop on being invoked as their own server binary — redis-server for Redis, valkey-server for Valkey — and the Compose command lines call redis-server, which is a symlink under Valkey. Stock redis:7 drops to uid 999 by itself, so this only bites after switching engines, and nothing surfaces it. The override snippet therefore carries user: "999:999" on all four services, with a warning explaining why. uid 999 is redis in redis:7 and valkey in valkey/valkey:8.1 and owns the /data workdir where nodes.conf is written, so the single numeric value is correct for either engine.

Checklist 📋

For code changes:

Docs-only PR — no code, Compose or configuration changes. Every load-bearing claim was still verified against a running stack rather than read off the source.

• I have clearly listed my changes in the PR description
• I have made a test plan
• I have tested my changes according to the test plan:
  • The documented override really points all three shards and the init sidecar at Valkey, and the cluster forms — docker compose up -d redis-0 redis-1 redis-2 redis-init reaches cluster_state:ok on Valkey, uid=999, nodes.conf written and correctly owned, seed healthcheck healthy
  • The override YAML parses, and user: "999:999" must stay quoted — unquoted it is a YAML 1.1 sexagesimal integer, not the string Compose needs
  • Without user:, the shards run as root on Valkey; stock redis:7 drops to uid 999 unaided. Verified getent passwd 999 → redis in redis:7, valkey in valkey/valkey:8.1, and ran both under --user 999:999 with the real cluster command line
  • The Valkey image ships redis-server/redis-cli as symlinks, so the Compose command lines and health checks work unaltered
  • REDIS_HOST/REDIS_PORT must be changed in the x-backend-env block — they are hardcoded there, so Compose's shell-environment precedence does not apply. All nine services that read them take them from that block and none sets them separately
  • REDIS_PASSWORD does work from backend/.env — it is absent from x-backend-env and commented out in .env.default
  • docker compose up -d db rabbitmq clamav falkordb migrate avoids the bundled shards — merged depends_on for those five references no redis-*, and none carries a profiles: key
  • Valkey's stated CI coverage is real — platform-single-container-docker.yml builds and smoke-tests the single-container image on every push and PR touching autogpt_platform/**
  • Sharded pub/sub, EXPIRE … NX, LPOP count, GETEX set the version floor; no Redis modules are used — command inventory across the backend, plus a 46-command compatibility run on a Valkey 8.1 three-shard cluster against a redis:7.4.10 control showing zero Valkey-only failures
  • One near-miss checked and cleared: Valkey renames redis_mode → server_mode in INFO server. Nothing in this repo or in redis-py 5.3.1 reads either field
  • GitBook syntax: {% hint %}/{% endhint %} balanced (3 open, 3 close, valid styles), all three tables well-formed, and the #engine-version-floor cross-reference resolves
  • Rebased onto current gitbook after docs(platform): sync dev back into GitBook #14019; no new page added, so no SUMMARY.md change is needed

For configuration changes:

Not applicable — no configuration changes in this PR.

Related

• Part of Document the cache/coordination engine requirements, and make the Compose engine selectable (REDIS_IMAGE) #14012.
• feat(platform): make the cache engine selectable via REDIS_IMAGE, with a Valkey CI leg #14047 (→ dev) proposes a REDIS_IMAGE variable that would reduce the override in Switching the Compose stack to Valkey to a single environment variable, and adds a Valkey leg to backend CI. This PR does not depend on it — it documents the override that works today. If feat(platform): make the cache engine selectable via REDIS_IMAGE, with a Valkey CI leg #14047 lands, that subsection wants a short follow-up to lead with the one-liner, and the CI row in the comparison table can then credit the backend leg too. The two PRs can be reviewed and merged in either order.
• Overlaps with docs(platform): document single-container operations #13760, which documents the single-container distribution's Valkey topology — worth coordinating. Happy to rework against whatever lands there first.

Base branch

Targets gitbook, per docs/content/contribute/index.md ("create a pull request targeting the gitbook branch"). Retargeting to dev isn't an option for this diff — docs/platform/ differs between the two branches, so the change is written against the gitbook layout.

Follow-up, deliberately out of scope

The same stale start-core description lives in code, in autogpt_platform/Makefile (the header comment and the help text). Not touched here because this branch is docs-only and targets gitbook.

[CLAassistant]

All committers have signed the CLA.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• dev

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 12f6fc83-b21e-4e46-9420-8ee529e4e4ca

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}