Direct-IO fast paths + page-cache correctness and robustness fixes - #138
Open
OBrezhniev wants to merge 42 commits into
Open
Direct-IO fast paths + page-cache correctness and robustness fixes#138OBrezhniev wants to merge 42 commits into
OBrezhniev wants to merge 42 commits into
Conversation
readToBuffer loaded every page into a cache buffer and then copied page->destination, so a large sequential read (e.g. a 100-350MB zkey/ptau section) cost a full extra copy and held the whole section in page buffers (maxPagesLoaded is bumped to fit the read). For reads >= directReadThreshold (1MB) with no overlapping dirty pages, read straight from the fd into the destination buffer instead. Small reads (section headers, ULE32 navigation) keep using the cache. _rangeHasDirtyPages() guards correctness on read-write files; positioned fd.read (pread) keeps concurrent section reads safe. On a sha256 groth16 prove this cut peak memory ~4.7GB -> ~3.9GB by dropping the duplicate page buffers, and removed the page->destination memcpy that was ~13% of main-thread self-time in the flame profile. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ache Mirrors the direct-read path. The cached write() loads pages, copies buff->page, marks them dirty, and flushes page->disk later, so a large write (e.g. a zkey/ptau section) costs an extra copy and holds the whole region in page buffers. For writes >= directWriteThreshold (1MB) to a region with no cached pages, write straight to disk via positioned fd.write instead. The guard is stricter than the read path's: it falls back to the cache when ANY page in range is cached (even clean), since a clean cached page would go stale after a direct write. Small writes (headers, ULE32) keep using the cache. The direct write awaits completion, so it leaves no deferred flush. Verified byte-exact on real files (large+small mixed writes, mid-file overwrite, read-back) and end to end via the snarkjs Full process suite (49/49). Symmetric with the read path's memory win for large sequential I/O. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fastfile.js imported `{ constants }` from "node:fs", which bundlers that stub
the "fs" module for the browser (e.g. snarkjs's rollup config) do not cover,
so the browser build left node:fs as an undefined external global and crashed
with "node_fs is not defined".
Read O_TRUNC/O_CREAT/O_RDWR/O_EXCL/O_RDONLY off `fs.constants` via the existing
`import fs from "fs"` instead. The "fs" module is already stubbed for browser
builds, and these flags are only used on the Node file path (browsers use
MemFile), so `fs.constants || {}` is harmless there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…p_build # Conflicts: # build/main.cjs # src/fastfile.js
_rangeHasDirtyPages / _rangeHasCachedPages walked every page index in
[firstPage, lastPage], so a large direct read/write scanned the whole range
(e.g. ~12.5M iterations for a 100GB span with small pages) just to look for
conflicting cached pages.
Iterate the actually-cached pages via Object.keys(this.pages) and test range
membership instead. The cache holds few pages, so the check is now O(cached
pages) regardless of range size or page size, and order-independent. Using
Object.keys (own enumerable keys) also keeps it immune to prototype pollution.
Result set is unchanged: { cached pages with index in [firstPage, lastPage] }.
Verified byte-exact via a direct-io guard test (large+small mixed writes,
mid-file overwrite with a cached page in range, read-back) plus fastfile 15/15
and the snarkjs node suite 49/49.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n) + tests The direct read/write fast paths hand the buffer straight to fd.read / fd.write, which require a real TypedArray/DataView. fastfile is also called with a BigBuffer (paged, not an ArrayBufferView) for large sections; the cached path handles those via BigBuffer's own .set()/.slice(), but the direct path passed the BigBuffer to fd.read and silently left it unfilled -> corrupted data. This surfaced as `powersoftau verify` on a prepared ptau failing with "Phase2 caclutation does not match with powers of tau": the verify reads section chunks into a BigBuffer, and the direct read returned garbage. The in-memory test suites never caught it because they use MemFile, not OsFile; only on-disk ceremony/zkey I/O hits this path (it's exercised by the snarkjs tutorial). Gate both fast paths on ArrayBuffer.isView(buff): a BigBuffer now falls back to the cached path. Plain Uint8Array sections (e.g. the groth16 prove zkey reads) still get the direct-io speed/memory win. Adds two on-disk OsFile regression tests (read into / write from a 2MB BigBuffer, above the 1MB direct threshold). Verified they fail without the gate and pass with it. fastfile suite 17/17. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
readExisting used `process.browser` to decide whether a string argument is a URL to fetch or a file path to open. `process.browser` is a webpack-ism and is undefined under Vite/esbuild/SES -> ReferenceError. Replace with a non-throwing `isNode` (process.versions.node, true for Node/Bun/Deno). Node-like envs open the file; everything else falls back to fetch, which is universal (Node 18+, Bun, Deno, edge, browser) -- not a browser-only API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… hand-stubbing Browser bundlers got the raw src, which imports `fs` (osfile) and the open-mode flags from "constants" -- unresolved without consumer-side stubs. Add a package.json "browser" field: fs -> false (empty; only used on the Node OsFile path, browsers use MemFile), and constants -> a small browser-stub file with NAMED exports (O_TRUNC etc. = 0, unused in browser). The named stub matters because fastfile.js does a NAMED import; an empty `false` stub has no named exports and would fail that import (and a default import would break snarkjs's named-only virtual constants stub at build time). Node behavior unchanged (browser field ignored by Node; real constants used). Verified: clean browser bundle resolution, full snarkjs build, browser + Node tutorial e2e. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
npm audit fix for the in-range bumps (ajv, brace-expansion, cross-spawn, flatted, ...); serialize-javascript ^7.0.5, diff ^8.0.3 and minimatch ^3.1.4 overridden (mocha 10 pins vulnerable ranges). npm audit clean; 17 tests pass.
readToBuffer's cached-page path (reads below directReadThreshold, i.e. the common case for most zkey/ptau/wtns sections) computed the destination-buffer write offset as `offset + len - r`, where `r` had already been clamped to the EOF-truncated byte count on the previous line. That put the on-disk bytes at a nonzero offset in the output buffer instead of at the start -- the START of the returned buffer came back as zeros (the caller's fresh-allocated buffer contents) while the truncated tail held garbage/misplaced real data, rather than the read failing cleanly or zero-padding only the genuinely missing tail. Reproduced with a section whose declared size exceeds the actual (truncated/corrupted) file: readSection returned bytes shifted by exactly the truncated amount. Fixed by tracking bytes-written with an independent counter instead of deriving the offset from the EOF-clamped remaining-count.
…o close() Two error-propagation bugs in osfile's async page cache, the same class as the recently-fixed ThreadManager hangs: 1. A failed page read rejected only the FIRST waiter; co-readers queued on page.loading were never settled (awaited forever), and the page stayed cached with a dead loading list so every future reader of that page hung too. Now every waiter is rejected and the page is dropped so a retry re-reads it. 2. A failed background page flush was recorded in self.error and only surfaced at close(); page.writing also stayed true, pinning the page. A prover that skipped close() on its error path silently produced a truncated file while 'succeeding'. Now write()/ readToBuffer() fail fast once a flush error is recorded, writing is cleared, and _tryClose no longer falls through to resolve after rejecting. Both covered by regression tests (fail pre-fix: one as a hang, one as a silent success).
readExisting with a URL previously buffered the entire body in memory (browser: any string; Node: unsupported). Now a Range probe (bytes=0-0) routes to a paged read-only backend: large reads stream the 206 body straight into the caller's buffer, small header reads coalesce into cached pages. If-Range with the validator captured at open fails reads if the remote file changes mid-session instead of mixing versions. Servers without range support fall back to the old buffer-it-all path, reusing the probe response body. Blob/File inputs stream via slice(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wo6AVSAwvL9mHTpvnREZPR
Callers (snarkjs) pass 8 MiB pageSize hints meant for the disk cache; over HTTP that turned a 4-byte header read into a whole-file range request for any file under the page size. Cap http pages at 64 KiB and blob pages at 1 MiB; reads at/above the page size already bypass the cache, so large section reads are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wo6AVSAwvL9mHTpvnREZPR
Double-closing an osfile threw "Closing the file twice" synchronously, forcing every consumer's cleanup path (finally blocks in snarkjs) to wrap close() in try/catch. Repeated calls now return the same promise (including the same rejection when a final flush fails), and the http range backend's close() is a no-op on repeat. Reads and writes after close still fail fast. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…wo readString bugs New test/api_surface.test.js (34 tests) covers the dispatcher variants (createOverride/createNoOverride/readExisting/readWriteExisting/ readWriteExistingOrCreate for mem, bigMem, file, blob and invalid types), the ULE32/UBE32/ULE64/readString helpers on every backend, error paths (out-of-bounds reads, writes to read-only backends, read-after-close, short/overlong/misaligned HTTP range responses, failing probes, 416 and unknown-total fallbacks, cache eviction, failed page-load retry). Two genuine readString bugs found by the new tests, both fixed: - memfile: reading a string past the written data on a writable file built a negative-length typed-array view and threw a RangeError; it now returns "" (read-only files still reject out of bounds). - bigmemfile: an unterminated string spun forever re-reading an empty window at the end of the data; it now ends the string at EOF, matching the rangefile backend. c8-ignored with reasons: osfile's logHistory debug instrumentation and httpfile's non-streaming fetch fallbacks (Node's undici always streams). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
A 206 answer to the probe can come from an intermediary -- notably a
browser's HTTP cache, which satisfies range requests out of a cached
full 200 -- while the origin itself ignores Range and answers 200.
The range reader treated any mid-session 200 as fatal ('file changed'),
which broke snarkjs's in-browser URL streaming against st-style static
servers. A 200 whose strong validator still matches now hands its body
to the reader, which buffers it once and serves all further reads from
memory; a changed validator remains a hard error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
…ent-close) Port of the feature/esm-tooling migration (fde21ad) onto the current branch: Rollup -> Vite (build/node/main.cjs + browser esm/iife), Mocha/Chai -> Vitest (node-esm / node-cjs / browser Playwright projects), flat eslint.config.js. Everything since the original fork point is preserved: - src/fastfile.browser.js adopts the explicit browser/node split but routes strings through the HTTP Range-streaming backend and Blobs through the blob backend (the original migration fetched URLs whole, regressing the streaming work); constants.browser.js and the browser field are gone. - Our post-fork suites (api_surface, httpfile, blobfile, osfile additions) converted: chai/chai-as-promised -> vitest expect, rejectedWith -> rejects.toThrow, this.timeout dropped. - The browser URL test now mocks a 200 no-Range server (the backend probes with Range: bytes=0-0 and reuses the full body on fallback). - ffjavascript devDep pinned to 4ac1cba (Vite+Vitest migration landed there); vitest family at ^4.1.11 (GHSA-p63j-vcc4-9vmv); postcss/brace-expansion audit overrides added (npm audit clean). 72 tests pass (64 node incl. CJS-build project, 8 browser in Chromium); lint clean. (cherry picked from commit fde21ad) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
.vitest-attachments/ and test/__screenshots__/ are debris from a failing browser-test run that git add -A swept into the migration commit; with no "files" field they would ship into every consumer's node_modules on a git install. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
The Chromium suite verified URL fetching but not Blob reads; the blob backend (page-cached small reads + direct large slices) is the primary browser use case, so exercise both read shapes against a 64 KiB Blob. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
README now covers the page cache, the five backends (file/mem/bigMem/ http/blob) with examples for each, the API surface, browser resolution via the exports map, and idempotent close. Copyright updated to 2018-2026. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Stream reads over HTTP Range requests and from Blobs
…kend
Prototype of the warm-start capability that chunked-zkey schemes (zkEmail)
get from localforage, without their costs: the standard single-file zkey,
no snarkjs fork, and bounded memory.
readExisting({type: "http", url, persistentCache: true}) wraps the range
reader with a block cache (default 2 MiB blocks) persisted in IndexedDB:
- blocks fully inside a request are fetched straight into the caller's
buffer (no extra copy); only the <=2 boundary blocks use a one-block temp,
so the bounded-memory property of the streaming design is preserved;
- entries are keyed by URL + strong validator + size, and stale blocks are
dropped at open -- without a strong validator nothing is cached, so a
replaced file can never serve mixed or stale bytes;
- total storage is bounded (maxBytes, default 512 MiB) with per-file LRU
eviction at open, never evicting the file being opened;
- everywhere IndexedDB is missing or broken (Node, private windows, denied
storage, quota errors mid-session) the cache degrades to a no-op and
reads keep streaming.
Prototype limitations noted in the module header: no in-flight dedupe for
concurrent cold reads of one block, the degraded full-body path is not
persisted, and eviction runs only at open.
4 new in-Chromium tests (cold correctness across block boundaries, warm
start with zero network requests, validator invalidation, LRU eviction)
plus a Node no-op test; 78 tests total pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
The first realistic benchmark caught the documented prototype limitation being far from theoretical: provers issue concurrent, overlapping reads (MSM read-ahead, coefficient streaming), and every concurrent miss of the same block refetched it whole -- a cold authV3 proof moved 92 MiB for a 28 MiB zkey (46 block fetches for 15 blocks) and took 8.8s vs 3.1s plain streaming. A missing block now registers an in-flight promise before its fetch starts (both the interior-run and boundary paths); every other reader awaits that promise and copies from the resolved block. Entries are removed once persisted (waiters then fall through to IndexedDB), and a failed fetch rejects and clears its entries so a retry can refetch. Interior runs still land in the caller's buffer in one range request each. New in-Chromium regression test: 8 concurrent cold 128 KiB reads inside one block cause at most 2 block fetches (was 8). 79 tests pass; lint clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Browsers cap HTTP/1.1 connections per host (Chrome: 6). A prover's MSM read-ahead can put more range requests in flight; on a bandwidth-limited link the extras queue behind stalled not-yet-consumed responses and the connection pool deadlocks. Observed streaming a 1 GiB zkey in Chromium over a shaped 100 Mbps link: the proof froze with exactly six open responses -- the consumer awaited request 7, Chrome queued it until a connection freed, and connections only free when the first six bodies are consumed. Unshaped links never hit it (the server finishes sending, so connections free regardless of consumption), which is why localhost testing missed it. httpReadRangeInto now takes one of 4 semaphore slots and holds it until the response body is fully consumed. Slotted reads complete independently of one another, so the semaphore itself cannot deadlock. Regression test: 12 concurrent direct reads through a delayed fetch stub never exceed 4 in flight. 80 tests pass; lint clean; bundles rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Previously only the Range/206 streaming path populated and used the IndexedDB cache; a Range-less origin re-shipped the whole file every session (the probe itself is a full download there, so the cache never got a chance). - readExisting probes conditionally when it holds cached meta for the URL (If-None-Match for ETags, If-Modified-Since for Last-Modified): an unchanged file answers a bodyless 304 and the reader is built over the cache -- zero body bytes even from a server that ignores Range. Missing blocks fall back to range requests, or the degrade-to-full path when the origin turns out to ignore Range. - The 200 full-body path persists the downloaded body (persistFullBody: batched block writes, exact meta.bytes) so the next session hits the 304 path. - The streaming builder is extracted (buildStreamingFile) and shared by the fresh-206 and warm-304 paths; a changed validator falls through to fresh content and re-persists as before. New in-Chromium test: cold 200 persists the body; warm open makes exactly one bodyless 304 probe and reads locally across block boundaries and the short tail; a changed ETag serves and re-persists the new content, warm on the following open. 81 tests pass; lint clean; bundles rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
If-None-Match/If-Modified-Since are not CORS-safelisted; a cross-origin server whose preflight does not allow them fails the fetch outright. The cache must never break an open that works without it: on a thrown conditional probe, retry with the plain Range probe (costing only the warm-start for that origin). Server CORS requirements documented in the module header. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Fault injection showed the http backend dies on the first hiccup: one
mid-body connection drop or transient 503 fails the whole proof, and a
stalled response hangs it forever (fetch has no timeout of its own).
Range GETs are idempotent and If-Range already guards consistency, so
transient failures are safely retryable:
- httpReadRangeInto retries up to 3 times with exponential backoff
(300ms doubling). Retries happen inside the connection-cap slot, so a
failing origin is not hammered by other readers meanwhile.
- A stall watchdog (AbortController, bumped on headers and on every body
chunk, default 10s) turns a wedged connection into a bounded, retryable
failure.
- Errors that retrying cannot fix are classified permanent at their throw
site and rethrown immediately: 4xx (except 429) and a changed validator.
The degrade-to-full control-flow signal passes through untouched.
- Tunables exported as httpRetryConfig {retries, backoffMs, stallTimeoutMs}
rather than plumbed through per-call options.
Out of scope, deliberately: retrying the open() probe, resuming a partial
range from its break point, and un-sticking a failed degrade-mode full-body
download.
4 new tests against a fault-injecting server: mid-body drop recovery,
transient-503 recovery vs fail-fast 404, bounded stall recovery, retry
exhaustion. 82 tests pass; lint clean; bundles rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Retry with backoff + stall timeout for HTTP range requests
…xes) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
OBrezhniev
force-pushed
the
feature/direct_rw_optimization
branch
from
August 29, 2026 19:46
ce2e4d5 to
80ede4e
Compare
The b/c arguments of the entry points only applied while coercing
string (and Blob) sources into descriptor objects; a caller passing
{type: "file"|"http"} without explicit sizes silently got the small
built-in defaults even when it supplied tuned hints (snarkjs passes
32 MiB cache / 8 MiB pages for its binfiles). The hints now fill
exactly the fields the descriptor leaves undefined, in every entry
point; explicit fields keep winning and the caller's object is never
mutated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01833VaUEJmrFZ7bVprrWwpp
OBrezhniev
force-pushed
the
feature/direct_rw_optimization
branch
from
August 29, 2026 19:50
80ede4e to
1514b2a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Direct-IO fast paths + page-cache correctness and robustness fixes
Summary
Two performance fast paths for large transfers, followed by a series of
correctness and error-propagation fixes for the page cache, all covered by
new tests. snarkjs/binfileutils drive every zkey/ptau/wtns through this
library, so large sequential section reads/writes dominate its real-world
profile.
Performance
copy straight from disk into the caller's buffer, skipping the page cache
and its page→destination copy (the dominant cost of large sequential
section reads).
cached pages; any cached page in range falls back to the cached path so
nothing goes stale.
paths iterate the actually-cached pages instead of every page index in the
range, keeping them cheap for huge ranges with small pages.
Correctness
fd.read/fd.write, which requires a real TypedArray/DataView. ABigBuffer (paged, not a view) silently corrupted data; both paths are now
gated on
ArrayBuffer.isViewand BigBuffers take the cached path (whichcopies via their own
.set()). Regression-tested with >1 GiB-styledsection reads/writes.
truncated file computed the destination offset from the EOF-clamped
remaining count, silently shifting the valid bytes to the wrong position in
the output buffer. Tracked independently now; the real bytes land at the
start.
first waiter — co-readers of the same page were queued on
page.loadingand awaited forever; the dead page also blocked retries. A failed
background flush left
page.writingset forever (pinning the page andwedging
close()) and surfaced the error only atclose()— which errorpaths often skip, silently truncating the file. Now: every waiter is
rejected and the page dropped for retry; flush errors clear the flag, latch
self.error, fail the nextread/writefast, and rejectclose().Compatibility
process.versions.node) instead of thewebpack-only
process.browser, which is undefined under Vite/esbuild/SES.browserfield stubsfs/constantsso bundlers need no hand-writtenstubs; open flags come from
fs.constantsrather than importingnode:fsin the shared module.
Validation
npm test: 21 passing, including the new regression tests for the BigBufferdirect-io gate, the EOF offset shift, multi-waiter page-read failure, and
background-flush error surfacing. Each fix's test was verified to fail
against the pre-fix code.
Stacked work
The HTTP Range / Blob streaming backends build on this branch and are PR'd
separately from
feature/url-blob-streaming.🤖 Generated with Claude Code
https://claude.ai/code/session_01Wo6AVSAwvL9mHTpvnREZPR