[typescript] McapReader: add option to prefetch message indexes in parallel - #1641
[typescript] McapReader: add option to prefetch message indexes in parallel#1641snosenzo wants to merge 5 commits into
McapReader: add option to prefetch message indexes in parallel#1641Conversation
260831c to
0d605a1
Compare
Made-with: Cursor
Made-with: Cursor
4b7d0a6 to
c60ca97
Compare
793ef8b to
bb71f02
Compare
There was a problem hiding this comment.
All ten prior threads are resolved — the latest iteration addresses every concern from previous rounds. Traced the prefetch → CachedReadable byte cache → ChunkCursor.loadMessageIndexes() lifecycle: offsets and sizes align across #startMessageIndexPrefetch, #isMessageIndexCached, and the readFullMessageIndexRange path in ChunkCursor, so cache hits land correctly. The concurrent-read dedup via the pending map, error-once-then-fallback semantics, cumulative-bytes budget cap, and per-chunk cache check before awaiting all trace correctly.
Test coverage is thorough: prefetch-before-read, gated in-flight await, error surfacing with fallback, no-concurrent-reads throw, cache-budget-respected, and the new CachedReadable concurrent-read / double-count / replaced-entry tests all exercise the important paths.
No new blockers.
|
@snosenzo do we still want this? |
Changelog
-[typescript] Add optional
IReadablefieldsupportsConcurrentReadsto advertise capability toMcapReadersprefetchMessageIndexesargument toMcapIndexedReaderwhich will enable caching and prefetch message indexes if the underlying Readable supports concurrent reads.Docs
None. Documented in JSDoc docs
Description
This allows message indexes to be loaded in parallel starting when the Reader is initialized which significantly speeds up message index reading. Currently the concurrency is locked to up to 6 concurrent requests, based around browser HTTP:1 limits. This could be specified in the future, but I didn't want to add more api surface area.
If prefetch is enabled and no cache size is specified it will automatically size the cache to the size of the message indexes in the mcap, with a safe maximum cache size of 256MB.
If one of the prefetches fails then it will bail on prefetching and return the error in the first
readMessagescall so that the error can be surfaced to the owner of the Reader. Subsequent reads will act normally.The losses are expected for the no/lowest-latency, single read cases, since this benchmark uses an in memory mcap buffer and the overhead of the requests wins out. Enabling this option is not advised in those cases. This option is meant to be used primarily for remote file data sources which have a latencies of >20ms. In that range the savings are very apparent.