feat: skip O_DIRECT when the page cache is warm (sample-read gate) - #33
feat: skip O_DIRECT when the page cache is warm (sample-read gate)#33AlperenKonukbay wants to merge 2 commits into
Conversation
…ng O_DIRECT mincore can report 0.0 residency under cgroup-managed runners even when the page cache is demonstrably hot (buffered repeat measured at 21.6 GB/s right after a 0.0 reading), which silently forces every warm reload onto the ~2x-slower direct path. Gate O_DIRECT behind a 4-thread 16 MiB spread sample read: page-hot files scale with reader threads while cache-cold reads saturate ~2-4 GB/s aggregate, so a 5 GB/s threshold separates the tiers with margin. mincore remains the fast positive signal; the probe only runs when mincore says cold. FLASHPACK_SAMPLE_PROBE=0 restores the old behavior; FLASHPACK_SAMPLE_PROBE_MIN_GBPS tunes the threshold.
AlperenKonukbay
changed the title
|
@fal-9000 can you review and report on the difference between O_DIRECT and buffered reads for warm caches |
|
Review by fal-9000 (GPT-5.6 Sol, xhigh) — reviewed at head No correctness issue found. On the O_DIRECT vs buffered question (@GencSaracaydin): Checks were green at review time. |
3 participants
The parallel reader always uses O_DIRECT, even when the node's page cache is already hot for the pack, leaving memory-speed reads unused. This adds a cheap gate: sample a few spread 16 MiB windows before committing; if mincore residency is high or the sampled bandwidth clears 5 GB/s, switch to buffered reads. Knobs:
FLASHPACK_SAMPLE_PROBE,FLASHPACK_SAMPLE_PROBE_MIN_GBPS.Measured on id-lora (38 GB per-request transformer re-reads, A/B apps differing by exactly this commit):
Same-seed outputs byte-identical across arms. The observed bandwidths match the independently measured tier ceilings, confirming the mechanism. Two honest notes: the warm figures come from one node pair per arm (replicated once on a fresh allocation), so treat the percentages as a band, not a point estimate; and on filesystem-warm but page-cold nodes the probe can clear the 5 GB/s threshold and flip to buffered early — that flip measured favorably where we saw it, but it means the gate is bandwidth-driven, not strictly "stay O_DIRECT until page-warm".
Summary claim: up to +40–50% on warm-cache nodes; when cold, no measurable regression beyond node variance (n=3/arm). Complementary to #30 — prefetch warms the cache, this gate is what lets reads use the warmth.