Dream Mode Overview

Dream Mode Overview

Self-evolving agent fleet. Every slave's agent turns become training signal; a nightly pass consolidates fleet-wide wisdom; every slave benefits on the next turn.

---

The thesis

AI agent platforms today are memoryless across devices. A pattern your backend engineer discovers on Monday — "use this specific retry strategy when calling the payments API" — is locked inside their local reasoning bank. A second device on Tuesday rediscovers the same lesson from scratch.

Dream Mode closes that loop. The entire fleet becomes a single learning system: every device's experiences contribute, every device benefits.

---

The 6-step loop

1. Capture

Every agent turn already captures a trajectory — task description, tool sequence, result snippet, success score, workspace ID, failure flag. This happens on every slave regardless of Dream Mode.

Key detail: both successes and failures are captured. Failures are flagged (failure_pattern=1) so downstream distillation can extract avoidance rules ("don't do X, it fails on Y") alongside preferred paths.

2. Push (every 2h)

With Dream Mode enabled for a device:

• Every 2 hours, the slave builds an export envelope from unexported trajectories + agent memory summaries + consumption feedback (which consolidated entries did the slave actually use)
• Deep-scrubs every string through the redaction pipeline (API keys, tokens, home paths, emails)
• Runs a secondary high-entropy audit to catch provider-specific prefixes (sk-, ghp_, xoxb-) even if they slipped past pattern matching
• POSTs to master's /api/fleet/learnings over the same JWT channel as telemetry
• Exponential backoff (1×→8× cadence) on consecutive push failures

3. Ingest

Master writes each envelope to the fleet_learnings table with:

• device_id — which slave contributed
• workspace_id — scope for tenant-aware consolidation
• learning_type — trajectory | memory_summary | consumption_feedback
• received_at — timestamp
• consolidated_version — set later by the dream pass

Ingestion-time dedup within a single envelope prevents one slave's repeated pattern from inflating master's usage counts.

4. Dream (nightly at 03:00 local + threshold-triggered)

The Dream Scheduler runs a three-pass consolidation:

Pass 1 — Deduplicate Group incoming trajectories across devices by (trajectoryHash, workspaceId). Winning trajectory per cluster = highest success_score. Count unique contributing devices.

Pass 2 — Distill (LLM) For each high-frequency cluster (≥3 devices, ≥5 total usage), send the top trajectories to the master's configured default LLM. Failure clusters get the avoidance-rule prompt; success clusters get the preferred-path prompt.

The LLM returns structured JSON insight:

{
  "taskShape": "Call the payments API with retries",
  "preferredPath": "Exponential backoff, max 3 attempts",
  "avoidancePath": "Don't retry immediately — triggers rate limit",
  "triggerCondition": "HTTP 429 response"
}

A cost cap (default 500¢/pass) caps how many clusters get distilled per run. Lowest-rank clusters fall through without LLM processing.

Pass 3 — Rank Score each cluster by maxScore × (ingestionUsage + adoptionUsage) × adoptionSuccessRatio. Keep top N (default 500). The adoptionUsage signal boosts clusters that slaves have actually used and benefited from, not just ones that looked good at ingestion.

5. Broadcast

Consolidated output writes to consolidated_learnings with an incremented version. The next config bundle (which every slave pulls every 30s) carries:

• consolidatedLearnings.entries[] — trajectories with insight JSON, workspace scope
• consolidatedLearnings.memorySummaries[] — consolidated agent memory per slot hash
• consolidatedLearnings.templatePatches[] — persona addenda for specific agent templates that have high-signal fleet clusters

6. Apply

When a slave pulls the new bundle, it:

1. Upserts each trajectory into reasoning_bank with source_tag='fleet_consolidated' and preserved workspace_id
2. Writes consolidated memory summaries into local agent_memory with the same source tag
3. Appends template patches to agent_gallery.fleet_instructions_md

On the next agent turn, retrieval via findSimilarTrajectories() now prefers fleet-consolidated rows over locally-minted ones (explicit ORDER BY clause). Template spawn merges instructionsMd + fleetInstructionsMd so agents get both base persona + fleet wisdom.

The loop closes. The slave is measurably smarter on the next task.

---

Workspace scope

Every trajectory and consolidated entry carries a workspace_id. This means:

• A pattern captured in workspace project-a on Device A consolidates with the same pattern in workspace project-a on Device B
• It does not bleed into workspace project-b on any device
• Fleet-wide learnings (un-scoped, workspace_id IS NULL) are available to every workspace — these are the ones that came from cross-workspace patterns at ingestion time

This is the primary tenant-isolation mechanism for Dream Mode.

---

What's protected

Redaction

Before any envelope leaves a slave:

1. Pattern-based scrubbing — known API-key patterns (sk-*, ghp_*, xoxb-*, AKIA*, etc.), JWT tokens, email addresses, home directory paths → replaced with [REDACTED]
2. Secondary entropy audit — any string > 20 chars with Shannon entropy > 4.5 bits/char is dropped (catches base64-encoded secrets, novel provider formats)
3. Trajectories with ANY redacted field are dropped entirely — we'd rather lose learning signal than leak

Rate limits

• 500 trajectories/device/24h default quota (checkTrajectoryQuota)
• 500 KB envelope size cap
• 100 trajectories per envelope
• 50 memory summaries per envelope

Cost control

• $5 default cap per dream pass (TITANX_DREAM_MAX_COST_CENTS=500)
• Pass drops lowest-rank clusters first if estimated cost exceeds cap
• Fully observable — every pass emits a fleet.learning.dream_pass activity log entry with token counts + cost

Kill switches

• Global: fleet.learning.globalDisabled in secrets vault — disables across the entire master
• Per-device: fleet.learning.enabled=false in managed config keys — disables for one specific slave

---

What's observable

In Governance → Fleet Learning (master only):

• Learnings received per device, rolling 30d
• Last dream run timestamp + version + elapsed time
• Consolidated entries — browse all, with per-device contribution breakdown
• Template patches — which templates got persona additions, cluster provenance
• Per-stage failure counters from the last pass (feedback, trajectoryLoad, distillation, memorySummaries, rank, write)
• "Run Dream Now" button — admin-reauth-gated, useful for testing or catch-up after outage

---

What's opt-in for a reason

Dream Mode is off by default for every device. To enable:

1. On master: flip fleet.learning.enabled=true in managed config → pushes to every slave on next 30s bundle pull
2. Per-device override: master admin can keep Dream Mode on fleet-wide but exclude specific devices via managed config per-key

See Enabling Dream Mode for the full operator procedure.

Why opt-in:

• Envelopes contain summarized task output — some orgs have policy constraints
• Redaction is best-effort, not certified for regulated data (HIPAA, FedRAMP, etc.)
• Pilot deployments should validate the redaction pipeline against their own secret patterns first

---

Where to go next

Want to...	Read
Turn it on safely	Enabling Dream Mode
Understand the distillation prompt + JSON schema	Dream Pass Internals
Browse consolidated patterns	Consolidated Learnings Dashboard
Audit the redaction pipeline	Privacy and Redaction
See the per-stage retry logic	Source: dreamScheduler.ts