Add cross-language MCAP performance benchmarks.

[clalancette]

Changelog

None.

Docs

None.

Description

I've recently been looking at/thinking about MCAP performance. However, it turns out that we don't currently have a good way to talk about performance since we can't measure it.

Fix that by adding a benchmarking/ directory with read and write benchmarks for C++, Rust, Go, Python, and TypeScript. Each language has its own *_bench/ subdirectory. TypeScript supports unchunked/chunked/zstd (LZ4 compression is unavailable in wasm-lz4).

Benchmark modes:

• Fixed-payload: 1M messages x 100 bytes, single channel
• Mixed-payload: 10-second simulated robot recording with 5 channels (/imu 96B@200Hz, /odom 296B@50Hz, /tf 80-1600B@100Hz, /lidar 230KB@10Hz, /camera 512KB@15Hz) — 3750 messages, ~102 MB
• Filtered reads: topic filter (/imu), time range (seconds 3-5), and combined topic+time (/lidar seconds 4-6) using mixed files

All modes run across 4 compression types (unchunked, chunked, zstd, lz4) with configurable payload fill: uniform (0x42, best-case compression) or varied (deterministic pattern for realistic ratios).

make bench runs the full matrix: both fill patterns, all compression modes, fixed + mixed payloads, and filtered reads. Includes peak memory tracking and summary tables via run_bench.sh.

[clalancette]

And I'm noticing now that there is cpp/bench, rust/benches, and typescript/benchmarks . However, I believe that the benchmarks I've done here are more useful and comprehensive, and allow us to do comparisons across languages. If reviewers agree, I'll remove the ones in the per-language directories in favor of these top-level ones.

[claude]

All prior threads are now resolved. The fixes for TS /tf cycling, Go streaming reads, TS peak RSS, Go per-channel sequence counters, shell deduplication, and .gitignore all look good.

Two minor suggestions inline (duplicate TS import, temp file cleanup). Nothing blocking.

One open question: have you run the full make bench end-to-end on a clean checkout? The README says ~20-25 minutes — curious if that holds and whether the summary tables render cleanly across all 5 languages.

[claude]

suggestion: make clean removes build artifacts but not the benchmark output files in $BENCH_DIR (defaults to /tmp). A full run writes ~40 MCAP files (5 langs × 4 modes × {fixed, mixed}) — several GB total with the mixed-payload scenario. Consider adding a clean-bench target or a note in the README about manual cleanup, so users don't silently accumulate multi-GB temp files.

clean-bench:
	rm -f $(BENCH_DIR)/bench_*.mcap $(BENCH_DIR)/bench_*_results_*.tsv

(Or just document it — either way.)

[claude]

Spot-checked the prior round of fixes: per-channel chanSeq is in place across TS/Go/C++/Rust/Python, Go read now uses os.Open+mcap.NewReader, TS RSS now goes through process.resourceUsage().maxRSS, set_lang_cmds extracted, .gitignore added, and TSV filenames already include _${FILL} so the two make bench runs don't clobber each other. 👍

A few new things to look at, in rough order of importance:

• rust_bench/Cargo.toml's path = "../../rust" resolves to the workspace virtual manifest rather than the mcap package — works in some Cargo versions, fragile in others. Worth pointing at ../../rust/mcap explicitly.
• ru_maxrss units differ between Linux (KB) and macOS (bytes); the memory table converts unconditionally, so Mac numbers will be silently ~1024× off.
• One existing thread is still unaddressed: make clean doesn't touch the ~GB of bench_*.mcap files left in $BENCH_DIR after a full run. A clean-bench target or a README note would close it out.

Smaller items (default case in set_lang_cmds, Python timing including file-close, README "@mcap/core" wording) inline.

Open question: have you actually run make bench end-to-end on a fresh clone with the current rust_bench/Cargo.toml? If yes, that resolves the workspace-path concern.

[claude]

nit: the TypeScript bench imports ../../typescript/core/src/index.ts (and the nodejs/support packages) directly from source, not from the published @mcap/core. That's fine — it's actually what you want for an in-repo benchmark — but (@mcap/core) here implies the published package. A small clarification like TypeScript benchmarks (in-repo @mcap/core source) would prevent confusion.

[claude]

The read-completeness gap from my last pass is closed: all five read benches now emit msg_count (col 10), and run_bench.sh enforces it — verify_read_counts (exact count for fixed + mixed) and check_filter_counts (non-zero + cross-language agreement). Traced the fixed and mixed paths across C++/Go/Rust/Python/TS; a reader that silently under-reads would now abort the run instead of posting a fast time. Also confirmed the unrelated round_trip.rs/filter.rs changes are gone from the diff.

No new blockers. Two optional nit threads remain open (clean-bench target for the ~GB of bench_*.mcap files, README @mcap/core wording) — neither blocking.