Groth16 prover: memory scoping + MSM acceleration + prover options - #628

Open

OBrezhniev wants to merge 122 commits into

feature/memory-scoping

Commits on May 17, 2024

WIP. Run garbage collection during in zkey_new
OBrezhniev
committed
b021132

Commits on Feb 11, 2025

Improve memory usage in groth16_prove by limiting scope of vars
OBrezhniev
committed
9254928

Commits on Feb 15, 2025

switch to promises and wait for them later. memusage monitoring
OBrezhniev
committed
8e00207

Commits on Sep 13, 2025

Do zero-copy views when possible in buildABC1
OBrezhniev
committed
1df2879

Commits on Sep 14, 2025

Commits on Sep 26, 2025

Switch from awaits to promises and wait for them when data is actually needed. Changed a few anonymous functions to named for easier profiling. Smaller chunks in joinABC to split work in most cases…
OBrezhniev
committed
c09de49

Commits on Feb 10, 2026

Commits on Jun 20, 2026

Rebuild with updated ffjavascript ThreadManager (WorkerSlot refactor)
OBrezhniev
committed
d2b352d

Commits on Jun 21, 2026

Remove debug console.log calls from groth16 prover build

OBrezhniev
and
claude
committed
0b30b7e
Rebuild browser bundle with in-place FFT bit-reversal from ffjavascript

OBrezhniev
and
claude
committed
f83c819
Rebuild browser bundle with pairingEq buffer-transfer fix from ffjavascript

OBrezhniev
and
claude
committed
1764dfb
Rebuild browser bundles: browser-compatible fastfile + bn128 wasm loading

OBrezhniev
and
claude
committed
b922d34

Commits on Jun 23, 2026

compat: feature-detect crypto, browser-API detection for entropy prompt; drop dead taskmanager

OBrezhniev
and
claude
committed
b4be905
Rebuild browser bundles: pick up ffjavascript/fastfile browser-field stubs

OBrezhniev
and
claude
committed
60e32f4
Rebuild browser bundles: vendored prebuilt wasm from ffjavascript

OBrezhniev
and
claude
committed
b3367fd
Rebuild browser bundles: lazy worker source + native-preferred base64 from ffjavascript

OBrezhniev
and
claude
committed
13fdf34

Commits on Jun 24, 2026

groth16 prove: stream multiexp bases from disk in chunks

OBrezhniev
and
claude
committed
d397261

Commits on Jun 25, 2026

groth16 prove: adaptive streaming buildABC (lower, bounded memory)

OBrezhniev
and
claude
committed
d745c04
Rebuild bundles: explicit 1 GiB BigBuffer page/threshold

OBrezhniev
and
claude
committed
e678865
groth16 prove: consume FFT inputs in the abc pipeline

OBrezhniev
and
claude
committed
c714531
buildABCStream: return flat Uint8Array so the IFFT can consume it

OBrezhniev
and
claude
committed
42490ce
buildABC: retire buildABCWASM1, route "wasm1" through streaming

OBrezhniev
and
claude
committed
dd08a23

Commits on Jun 28, 2026

r1cs info: read only the header, not the constraints

OBrezhniev
and
claude
committed
5b66077

Commits on Jul 1, 2026

Commits on Jul 2, 2026

Rebuild bundles: no-carry CIOS mul (via ffjavascript)
OBrezhniev
committed
1b668ed
Rebuild bundles: GLV MSM (via ffjavascript)
OBrezhniev
committed
3215d46
Rebuild bundles: GLS G2 MSM (via ffjavascript)
OBrezhniev
committed
3fa3d0c

Commits on Jul 3, 2026

Rebuild bundles: sync to HEAD (via ffjavascript)
OBrezhniev
committed
2bdd666
browser_tests: instrumented in-browser proving benchmark
OBrezhniev
committed
e02858d
groth16 prove: retire the multi-threaded buildABC and the wasm/wasm1 options
OBrezhniev
committed
452ecba
Rebuild bundles: buildABC cleanup
OBrezhniev
committed
49e1cfc
buildABCStream: gather witness values per chunk instead of shipping the witness
OBrezhniev
committed
9ff9820
Rebuild bundles: gathered-witness buildABCStream
OBrezhniev
committed
141bf59
groth16 prove: msmGls option (boolean, default true)
OBrezhniev
committed
f821097
Rebuild bundles: msmGls option (via ffjavascript)
OBrezhniev
committed
cb40252
groth16 prove: msmGlv option; msmGlv/msmGls take "auto" | "disabled"
OBrezhniev
committed
bb26c1d
Rebuild bundles: msmGlv/msmGls modes (via ffjavascript)
OBrezhniev
committed
b53db00
Rebuild bundles: bls12-381 GLV (via ffjavascript)
OBrezhniev
committed
fc5391b
Rebuild bundles: FF_NO_BATCH removal (via ffjavascript)
OBrezhniev
committed
e799ba1
buildABCStream: typed-lane witness gather (per-element memcpy was 8.5% of the
OBrezhniev
committed
cac4a5c
Rebuild bundles: lane-copy gather
OBrezhniev
committed
5d53ab0
buildABCStream: gather via Uint32 lanes, not Float64
OBrezhniev
committed
c77af73
Rebuild bundles: u32-lane gather
OBrezhniev
committed
2013992
Rollback expose_gc flag
OBrezhniev
committed
71339be
IIFE build: stub out wasmbuilder/wasmcurves
OBrezhniev
committed
adc0448
Rebuild IIFE bundles: wasm toolchain stubbed out
OBrezhniev
committed
88c08d7

Commits on Jul 4, 2026

Update dependencies: noble-hashes 2, rollup 4, eslint 10, mocha 11, ejs 6
OBrezhniev
committed
4be7ec2
Rebuild bundles: rollup 4, noble-hashes 2
OBrezhniev
committed
169eba4
Pin binfileutils/ffjavascript/r1csfile to commit refs instead of file: paths
OBrezhniev
committed
d6ed3de
Remove console.time instrumentation from the groth16 prover
OBrezhniev
committed
4cf02f4
Rebuild bundles: silent prover
OBrezhniev
committed
415e144
Re-pin binfileutils/ffjavascript/r1csfile (debug-output cleanup)
OBrezhniev
committed
f8c16c2
Regenerate lockfile against the pinned commit refs
OBrezhniev
committed
791e625
Add opt-in memory-usage logging: --memlog CLI flag / memoryLogging option
OBrezhniev
committed
9ef1184
Rebuild bundles: --memlog flag
OBrezhniev
committed
1aaa760
Switch binfileutils/ffjavascript/r1csfile pins to git+https, re-pin
OBrezhniev
committed
346a96d
Re-pin binfileutils (blank-line tidy)
OBrezhniev
committed
5a4c4ed
Fix dev-dependency audit findings via overrides
OBrezhniev
committed
d5234b0

Commits on Jul 7, 2026

remove forced gc execution
OBrezhniev
committed
df52fb7

Commits on Jul 8, 2026

Merge branch 'master' into feature/memory-scoping
OBrezhniev
committed
33bc3e9
Rebuild
OBrezhniev
committed
a265fd6
Fix CI
OBrezhniev
committed
65d47c3
Fix CI
OBrezhniev
committed
771e45b
Re-pin binfileutils/ffjavascript/r1csfile to latest commits
OBrezhniev
committed
b2cf078
test: cover msmBatching/msmGlv/msmGls groth16 prove options
OBrezhniev
committed
a1688b8
fix: guard unguarded logger calls in plonk/fflonk verify and wtns check
OBrezhniev
committed
2d2183e
Re-pin binfileutils/ffjavascript/r1csfile to latest commits
OBrezhniev
committed
baa7612
Rebuild bundles against re-pinned deps
OBrezhniev
committed
8118625

Commits on Jul 9, 2026

Commits on Jul 10, 2026

cleanup: trim dead code, stale comments, and always-on memory logging
OBrezhniev
committed
d04d5fc

Commits on Jul 11, 2026

perf: re-pin deps for scalar-size partitioned MSM
OBrezhniev
committed
b7b0c6f
perf: re-pin deps (small-gathered-partition endo cutoff)
OBrezhniev
committed
c82f76b

Commits on Jul 12, 2026

mem: narrow buffer lifetimes in groth16 setup
OBrezhniev
committed
1d5a433
feat: stream zkeys from URLs (browser and Node) via HTTP Range requests

OBrezhniev
and
claude
committed
08a7950
chore: re-pin deps (worker boot-failure give-up in ffjavascript)

OBrezhniev
and
claude
committed
e71be10

Commits on Jul 13, 2026

chore: re-pin deps (wasmcurves ESM/Vite migration merged)

OBrezhniev
and
claude
committed
1dc10ed

Commits on Aug 17, 2026

fix: expX(pol, 1) must return a copy, not an IFFT of the coefficients

OBrezhniev
and
claude
committed
f87302a
test: cover branch changes lacking tests + circom -O0/-O1/-O2 groth16 matrix

OBrezhniev
and
claude
committed
fe74bcc

Commits on Aug 25, 2026

Commits on Aug 29, 2026

fix: close fds on every exit path in setup/prove/verify/import functions

OBrezhniev
and
claude
committed
e2b41e1
chore: re-pin deps (idempotent fastfile close())

OBrezhniev
and
claude
committed
77472d5
chore: re-pin deps (sibling coverage pushes + dormant-bug fixes)

OBrezhniev
and
claude
committed
20e27a5
test: standards vectors (FIPS 180-4, RFC 7693, Keccak-256, transcript mirror); re-pin deps

OBrezhniev
and
claude
committed
b44589c
chore: re-pin deps (wasmcurves coverage push and fixes)

OBrezhniev
and
claude
committed
69736a5
chore: re-pin deps (browser Range degrade, portable worker teardown, Node 20 dispatch)

OBrezhniev
and
claude
committed
681c3e9
chore: restore transitive git deps in the lockfile (npm ci sync)

OBrezhniev
and
claude
committed
d5d421e
Migrate build/test tooling to Vite + Vitest (rebased onto feature/memory-scoping)

OBrezhniev
and
claude
committed
2459c24
Re-pin migrated siblings; restore the old browser-ESM external contract

OBrezhniev
and
claude
committed
0b41088
chore: re-pin migrated siblings (ffjavascript CI-hang fix cascade)

OBrezhniev
and
claude
committed
938d9e5
chore: re-pin siblings (wasmcurves vitest-4.1.11/audit-overrides cascade)

OBrezhniev
and
claude
committed
c05e0d8
chore: re-pin siblings (lts/-1 CI matrices, fastfile Blob browser test)

OBrezhniev
and
claude
committed
322d71c
chore: re-pin siblings (worker unref-on-terminate fix); rebuild bundles

OBrezhniev
and
claude
committed
6c7ee3c
ci: add timeout-minutes to the tutorial workflow; trigger on master, not main

OBrezhniev
and
claude
committed
387ec53
feat: persistentCache prover option (IndexedDB warm start for http sources)

OBrezhniev
and
claude
committed
b46c2e3
chore: re-pin fastfile/binfileutils (in-flight cache dedupe); rebuild bundles

OBrezhniev
and
claude
committed
8ab81b6
chore: re-pin fastfile/binfileutils (range-fetch connection cap); rebuild bundles

OBrezhniev
and
claude
committed
6efa1c6
chore: re-pin fastfile/binfileutils (full-download persistent cache); rebuild bundles

OBrezhniev
and
claude
committed
069bf52
chore: re-pin fastfile/binfileutils (conditional-probe CORS fallback); rebuild bundles

OBrezhniev
and
claude
committed
79d5508
chore: re-pin siblings (ffjavascript fsqrt c=0 hang fix); rebuild bundles

OBrezhniev
and
claude
committed
576f9b2
chore: re-pin siblings (threadman idle-termination dispatch-race fix); rebuild bundles

OBrezhniev
and
claude
committed
51ba284
chore: re-pin fastfile/binfileutils (PR #145: http retry + stall timeout); rebuild bundles

OBrezhniev
and
claude
committed
3ee5862
test: groth16 verify concurrency/memory stress; re-pin siblings (verify-path fixes)

OBrezhniev
and
claude
committed
38a1c07
test: mixed prove+verify concurrency stress on the shared curve

OBrezhniev
and
claude
committed
f8d066e
test: plonk/fflonk prove+verify concurrency stress (cross-protocol)

OBrezhniev
and
claude
committed
d784756
mem: narrow buffer lifetimes in plonk/fflonk setup and provers

OBrezhniev
and
claude
committed
0aa5ee9
mem: bound RSS of the streaming ceremony commands (ptau/zkey contribute)

OBrezhniev
and
claude
committed
b652619
chore: remove the memlog instrumentation

OBrezhniev
and
claude
committed
19b94cf