Release v1.28.1: improvements and bug fixes since v1.28.0

.github/agents/lb-message-pipeline.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "MessagePipeline specialist covering the React context, zustand store, subscription management, and render state building. Use for data flow from Player to panels."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on the Lichtblick MessagePipeline — the central data bus that connects Players to Panels.
 

.github/agents/lb-orchestrator.agent.md

@@ -44,10 +44,14 @@ You are the top-level routing agent for the Lichtblick monorepo. Your job is to
 Parallelism happens at two distinct layers — do not conflate them:
 
 1. **Within a single task** — governed by the Write-Ownership Principle above. Multiple agents may read the same files; exactly one writer edits a given path. This is logical parallelism inside one checkout; no git worktree is involved.
-2. **Across independent tasks** — handled at the session level using git worktrees (`setup-worktree.prompt.md`), set up **once per task by a human before** orchestration begins, ideally with a separate session per worktree.
+2. **Across independent tasks** — handled at the session level using git worktrees (`lb-setup-worktree.prompt.md`), set up **once per task by a human before** orchestration begins, ideally with a separate session per worktree.
 
 This orchestrator does **not** create worktrees: it has no `execute` tool, and prompt files are human-invoked entry points, not agent-callable. Do not route a task expecting a sub-agent to provision its own worktree — sub-agent invocations share this workspace's filesystem and working directory.
 
+## Release Requests
+
+Release-cutting requests (creating a release branch, triggering a release, or verifying that a release/RC published successfully) are **not** routed to a sub-agent. Point the user to `.github/prompts/lb-release.prompt.md`, which is a human-invoked runbook. Like `lb-setup-worktree.prompt.md`, it is a prompt file rather than an agent tool target, and this orchestrator has no `execute` tool, so release execution stays a human-invoked entry point rather than an agent delegation.
+
 ## Routing Rules
 
 ### Tier 1: Cross-cutting (action-capable)

.github/agents/lb-panel-image.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "Image panel specialist covering camera image visualization within the 3D rendering context (ImageMode). Use for image display, camera models, still image and H.264 video decoding, and annotation overlays."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on the Lichtblick Image panel — which is actually a specialized mode of the 3D panel (ThreeDeeRender).
 

.github/agents/lb-panel-log.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "Log panel specialist covering virtualized log display, react-window VariableSizeList, dynamic row heights, autoscroll behavior, and log level filtering. Use for log visualization and filtering issues."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on the Lichtblick Log panel — a high-performance log viewer with virtualized rendering and filtering.
 

.github/agents/lb-panel-map.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "Map panel specialist covering Leaflet integration, GeoJSON rendering, NavSatFix message handling, and the FilteredPointLayer pixel-deduplication system. Use for geographic visualization and GPS data display."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on the Lichtblick Map panel — a geographic visualization tool built on Leaflet for displaying GPS/NavSatFix data.
 

.github/agents/lb-panel-plot.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "Plot panel specialist covering PlotCoordinator, TimestampDatasetsBuilder, Chart.js Worker rendering, OffscreenCanvas, and time-series data extraction. Use for plot visualization, chart performance, and dataset management."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on the Lichtblick Plot panel — a high-performance time-series chart built on Chart.js with Worker-based rendering.
 

.github/agents/lb-panel-state-transitions.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "StateTransitions panel specialist covering discrete state visualization using TimeBasedChart, message-path extraction, and preloaded data range subscriptions. Use for state timeline display and discrete event visualization."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on the Lichtblick StateTransitions panel — a timeline visualization for discrete state changes extracted from messages.
 

.github/agents/lb-panel-user-scripts.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "UserScripts panel specialist covering the Monaco editor integration, TypeScript compilation, script execution in SharedWorkers, diagnostics, and the user script API. Use for script editor, script runtime, and script diagnostics."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on the Lichtblick UserScripts panel — the in-app TypeScript editor that lets users write custom message transformations.
 

.github/agents/lb-panels-general.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "General panel infrastructure specialist covering PanelExtensionAdapter, renderState building, panel lifecycle, pauseFrame, and the extension API contract. Use for panel framework patterns and creating new panels."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on the Lichtblick panel infrastructure — the framework that connects panels to the MessagePipeline and manages their lifecycle.
 

.github/agents/lb-player.agent.md

@@ -189,6 +189,9 @@ const cancel = subscribeMessageRange({
 | `WorkerIterableSource` | Runs an `IIterableSource` in a Web Worker via Comlink |
 | `WorkerSerializedIterableSource` | Serialized variant for Worker-based sources |
 
+`IIterableSource` also exposes optional lifecycle hooks: `terminate?()` for cleanup on close, and
+`prewarm?()` to warm a source before it becomes active (used by pooled/multi-file sources; failures are non-fatal).
+
 ### IteratorResult Types
 ```typescript
 type IteratorResult =
@@ -288,6 +291,8 @@ type IteratorResult =
 - `packages/suite-base/src/players/IterablePlayer/DeserializingIterableSource.ts` — lazy deserialization + sampling
 - `packages/suite-base/src/players/IterablePlayer/BlockLoader.ts` — preloading for full subscriptions
 - `packages/suite-base/src/players/IterablePlayer/WorkerIterableSource.ts` — Worker-based source (Comlink)
+- `packages/suite-base/src/players/IterablePlayer/shared/MultiIterableSource.ts` — multi-file/multi-URL orchestration; deep dive in `@lb-remote-connection`'s knowledge base
+- `packages/suite-base/src/players/IterablePlayer/shared/HydratedSourcePool.ts` — bounded LRU pool for resident heavyweight per-file readers; deep dive in `@lb-remote-connection`'s knowledge base
 - `packages/suite-base/src/players/IterablePlayer/CachingIterableSource.ts` — block-level cache
 - `packages/suite-base/src/players/FoxgloveWebSocketPlayer/index.ts` — WebSocket player
 - `packages/suite-base/src/players/FoxgloveWebSocketPlayer/WorkerSocketAdapter.ts` — Worker WebSocket
@@ -301,6 +306,7 @@ type IteratorResult =
 - Modifying `#allTopics` identity without triggering `reset-playback-iterator` (misses new topics)
 - Calling `seekPlayback` before initialization completes (must store in `#seekTarget` for later)
 - Not clamping times to `[start, end]` range (causes source errors)
+- When `initialize()` and `terminate()` can race on shared mutable fields, `terminate()` must capture fields into locals before any `await`, clear instance fields synchronously, and run dispose/cleanup in a `finally` block so rejections do not skip cleanup or touch newer state; this pattern was established in `WorkerSerializedIterableSource.terminate()` this session, and similar sibling lifecycles (for example `WorkerIterableSource`) have not yet been audited
 - UserScriptPlayer: emitting upstream messages twice when scripts change (use empty messages array)
 
 ## Skills Reference

.github/agents/lb-remote-connection.agent.md

@@ -54,13 +54,14 @@ IterablePlayer (state machine, tick loop)
 URLs: [url1.mcap, url2.mcap, url3.mcap]
     │
     ▼
-MultiIterableSource (500MB total cache ÷ N sources)
+MultiIterableSource (500MB total cache ÷ N, floored by MIN_CACHE_PER_SOURCE_BYTES)
     │
-    ├── McapIterableSource(url1, ~166MB cache)
-    ├── McapIterableSource(url2, ~166MB cache)
-    └── McapIterableSource(url3, ~166MB cache)
+    ├── shared HydratedSourcePool (bounds resident McapIndexedReaders by count + bytes)
+    │      ├── McapIterableSource(url1, ~166MB cache)
+    │      ├── McapIterableSource(url2, ~166MB cache)
+    │      └── McapIterableSource(url3, ~166MB cache)
     │
-    ▼ (each initialized in parallel)
+    ▼ (each initialized under bounded concurrency)
 mergeSequentialIterators (min-heap, lazy source activation)
     │
     ▼
@@ -83,6 +84,9 @@ WorkerSerializedIterableSource → BufferedIterableSource → Player
 | `IterablePlayer/Mcap/BatchingReadable.ts` | Coalesces nearby `IReadable.read()` calls into fewer, larger reads |
 | `IterablePlayer/Mcap/McapIterableSource.ts` | Factory: indexed vs streaming decision |
 | `IterablePlayer/Mcap/McapIndexedIterableSource.ts` | Random access via chunk indexes |
+| `IterablePlayer/Mcap/readerWeight.ts` | Heuristic byte-weight estimate for a `McapIndexedReader`, used as the pool's `weigh()` hook |
+| `IterablePlayer/shared/HydratedSourcePool.ts` | Bounded LRU pool of resident heavyweight per-file readers (byte + count budget) |
+| `IterablePlayer/shared/multiFileHydrationOptions.ts` | Shared hydration-override merging (`maxHydratedSources`/`maxHydratedBytes`/`initConcurrency`), used by both data source factories and the MCAP worker |
 | `IterablePlayer/shared/MultiIterableSource.ts` | Multi-file orchestration |
 | `IterablePlayer/shared/utils/mergeSequentialIterators.ts` | Heap-based lazy iterator merge |
 | `IterablePlayer/shared/utils/sourceTimeOverlap.ts` | Time range filtering |
@@ -124,8 +128,40 @@ WorkerSerializedIterableSource → BufferedIterableSource → Player
 
 ### Error Recovery
 - **`keepReconnectingCallback` set** (browser mode): unlimited retries, UI notified of reconnection state
-- **No callback**: two errors within 100ms → fatal, rejects all pending reads
 - **Single error**: destroys connection, retries via `#updateState()`
+- **No callback**: two errors within 100ms → fatal, routes through shared `#closeWithError(error)` cleanup
+- **`close()` + fatal path**: both destroy the active stream, reject pending `#readRequests`, cancel `#activeUncachedReads`, and reset `#virtualBuffer`
+
+---
+
+## HydratedSourcePool (Bounded Reader Pool)
+
+`HydratedSourcePool` bounds **resident heavyweight per-file `McapIndexedReader` instances** (reader/deserializer state sized from chunk-index and channel counts). It does **not** bound decoded message data or each source's byte cache; those still live in `BufferedIterableSource`, `CachedFilelike`, and downstream playback buffers.
+
+```typescript
+const pool = new HydratedSourcePool({
+  maxCount: dataSource.maxHydratedSources ?? 4,
+  maxBytes: dataSource.maxHydratedBytes ?? 512 * 1024 * 1024,
+});
+```
+(`minResident` is not passed at the call site — it relies on `HydratedSourcePool`'s own internal default of `1`.)
+
+- **Hybrid budget**: `maxCount` caps resident readers, `maxBytes` caps total estimated resident weight, and `minResident` keeps a small floor hot (clamped so it never exceeds the count cap). `MultiIterableSource` constructs one shared pool and passes it to **both** the local-files and remote-urls branches.
+- **API contract**:
+  - `acquire(token, hydrator)`: open or reuse a resident reader, increment its pin count, refresh LRU recency, and keep it non-evictable until a matching `release(token)`. After `terminate()`, `acquire()` throws instead of rehydrating.
+  - `release(token)`: decrement the pin count (floor 0) and opportunistically evict
+  - `admit(token, hydrator, value)`: seed an already-open reader (for example from `initialize()` / prewarm) without calling `open()`. If the token is already resident, or the pool is already terminated, the redundant value is closed instead of replacing the existing entry.
+  - `terminate()`: mark the pool terminated first, then close every resident entry in parallel so late `acquire()` / `admit()` calls cannot repopulate it
+- **Eviction semantics**: entries live in a `Map<object, Entry>` whose insertion order is the LRU order. Re-inserting on access moves an entry to the most-recent end. Eviction scans from oldest to newest and closes the first **unpinned** entries until the pool is back under budget. If everything remaining is pinned, the pool may temporarily stay over budget by design.
+- **Weight heuristic** (`readerWeight.ts`): `weigh()` defaults to `1`, but MCAP readers use an approximate byte estimate so larger readers churn first:
+  ```typescript
+  READER_BASE_BYTES +
+    reader.chunkIndexes.length * BYTES_PER_CHUNK_INDEX_BASE +
+    messageIndexEntries * BYTES_PER_MESSAGE_INDEX_ENTRY +
+    reader.channelsById.size * BYTES_PER_CHANNEL
+  ```
+  where `messageIndexEntries` sums `chunkIndex.messageIndexOffsets.size` across all chunk indexes. There is no `cacheBytes` term — the weight reflects only reader/index/channel structure size, not each source's byte-cache allocation. The absolute numbers are heuristic; relative weighting is what matters.
+- **Indexed URL reuse vs unindexed bypass**: for `type: "url"` sources, `McapIterableSource` keeps one session-long `#persistentReadable` (`RemoteFileReadable` + `CachedFilelike` cache) and reuses it across pool eviction/re-`acquire()`; the pool only evicts the heavyweight `McapIndexedReader` layer, not that transport/cache. If a source proves unindexed (or the URL path falls back to raw `fetch()` streaming), it skips `HydratedSourcePool` entirely and lives in `#eagerInner` for the whole session, so unindexed files are outside the pool's count/byte budget. `#persistentReadable` is also closed unconditionally when the whole `McapIterableSource` is `terminate()`d at normal session end — not only on failed init or unindexed fallback.
 
 ---
 
@@ -180,18 +216,22 @@ getBackfillMessages({ topics, time }) {
 ### Cache Budget Distribution
 ```typescript
 const totalCache = dataSource.totalCacheSizeInBytes ?? 500 * 1024 * 1024;  // 500MB total
-const perSourceCache = Math.floor(totalCache / urls.length);
-// Each source gets: totalCache / N
+const minPerSource = dataSource.minCachePerSourceBytes ?? 10 * 1024 * 1024;  // 10MB floor
+const perSourceCache = Math.max(minPerSource, Math.floor(totalCache / urls.length));
 ```
 - 2 files → 250MB each
 - 5 files → 100MB each
-- More files = more network re-fetches per file
+- Hundreds of files → floor at 10MiB per source by default (override `minCachePerSourceBytes`)
+- Prevents `totalCache / N` from dropping below a viable per-file metadata-read budget; if `perSourceCache * N > totalCache`, a `log.warn` notes the aggregate may exceed the nominal budget
 
 ### Initialization
-1. Creates N `McapIterableSource` instances (one per URL) with divided cache budget
-2. Initializes all sources **in parallel** (`Promise.all`)
-3. Merges results: topics, datatypes, metadata, topicStats, alerts, publishersByTopic
-4. Sorts sources by start time (`source.getStart()`)
+1. Constructs one shared `HydratedSourcePool` before branching on local files vs remote URLs (`maxHydratedSources ?? 4`, `maxHydratedBytes ?? 512 MiB`) and passes `pool` to every source constructor.
+2. Creates N `McapIterableSource` instances (one per URL) with the divided cache budget; `readAheadEnabled` still defaults to `true` for both single- and multi-file sessions. What changes for multi-file (`numSources > 1`) is `readAheadBufferBytes`, which defaults to `min(2 MiB, perSourceCache / 4)` instead of the legacy 50 MiB default, bounding read-ahead so it doesn't outrun the smaller per-source cache slice.
+3. Initializes sources under a `Semaphore` (`initConcurrency ?? 4`, normalized to a positive integer) instead of unconstrained `Promise.all`; the returned `Initialization[]` still preserves input order.
+4. Merges results: topics, datatypes, metadata, topicStats, alerts, publishersByTopic.
+5. Sorts sources by start time (`source.getStart()`), then `#prewarmEarliestSources()` prewarms the earliest 3 sources in **reverse** order so the t=0 source ends up most-recently-used in the pool. Prewarm failures are logged at debug and treated as non-fatal.
+
+- `terminate()`: runs `Promise.allSettled` over `source.terminate?.()`, logs each rejection, re-throws the first rejection after cleanup, and always tears down `pool.terminate()` in a `finally` block.
 
 ### Message Iteration — Lazy Sequential Merge
 ```typescript
@@ -308,15 +348,16 @@ WorkerSerializedIterableSource       McapIterableSourceWorker.worker.ts
 6. **Lazy sequential merge** avoids simultaneous HTTP streams to all remote files
 7. **Indexed MCAP** required for acceptable remote performance (random access via chunk indexes)
 8. **WASM decompression preload** prevents initialization failures under slow network
-9. **Cache budget division** for multi-file: consider fewer large files over many small ones
+9. **Cache budget division** for multi-file uses a 10MiB per-source floor; many small files still increase refetch pressure and reader churn
 10. **BufferedIterableSource** (10s, 300MB) smooths network latency for playback
 11. **Read coalescing** (`BatchingReadable`): merges `read()` calls arriving in the same microtask tick (gap <64KiB, ≤4MiB span) before they reach `CachedFilelike`
 
 ## Common Pitfalls
 - Server missing `Accept-Ranges: bytes` header → fails at open
 - CORS not exposing `Accept-Ranges` header → browser can't detect range support
 - Unindexed MCAP over remote → falls back to full streaming (slow, limited to 1GB)
-- Too many remote files → cache per file drops below useful threshold
+- Too many remote files → per-source cache hits the 10MiB floor, aggregate cache may exceed the nominal total, and readers churn more often
+- Worker teardown races → follow the capture-before-`await` plus `disposeRemote()`-in-`finally` pattern from `WorkerSerializedIterableSource.terminate()` or an older terminate can leak/dispose the wrong worker
 - S3 presigned URLs with non-GET method → `BrowserHttpReader.open()` requires GET
 
 ## Skills Reference
@@ -325,3 +366,5 @@ WorkerSerializedIterableSource       McapIterableSourceWorker.worker.ts
 - Worker/Comlink patterns: `read_file(".github/skills/web-workers/SKILL.md")`
 - BufferedIterableSource or BlockLoader details: `read_file(".github/skills/caching-internals/SKILL.md")`
 - Network/read latency profiling and buffering performance: `read_file(".github/skills/performance/SKILL.md")`
+- Bounded reader pool implementation: `read_file("packages/suite-base/src/players/IterablePlayer/shared/HydratedSourcePool.ts")`
+- Reader weight heuristic: `read_file("packages/suite-base/src/players/IterablePlayer/Mcap/readerWeight.ts")`

.github/agents/lb-websocket-connection.agent.md

@@ -1,6 +1,7 @@
 ---
 description: "WebSocket connection specialist covering FoxgloveWebSocketPlayer, WorkerSocketAdapter, and the Foxglove WebSocket protocol. Use for live data streaming, connection lifecycle, and real-time message handling."
 tools: ["read", "edit", "search", "execute"]
+---
 
 You are an expert on live WebSocket connections in Lichtblick — the real-time data path from robot/simulation to visualization.