From d0237a59f5a34cb19bbad79079341b36299e741a Mon Sep 17 00:00:00 2001 From: TimothyBanks Date: Tue, 29 Sep 2026 11:20:22 -0400 Subject: [PATCH 1/7] feat: Using binaryen to optimize wasm sizes --- .github/workflows/docs.yml | 4 + .github/workflows/test.yml | 7 ++ AGENTS.md | 14 ++- scripts/README.md | 5 +- scripts/build.sh | 3 + scripts/setup.sh | 4 + scripts/wasm-opt.sh | 179 +++++++++++++++++++++++++++++++++++++ 7 files changed, 214 insertions(+), 2 deletions(-) create mode 100755 scripts/wasm-opt.sh diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 1364d0a5..dc4385b8 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -22,6 +22,10 @@ jobs: target/ key: ${{ runner.os }}-cargo-docs-${{ hashFiles('**/Cargo.lock') }} + # docs.sh -> build.sh -> wasm-opt + - name: Ensure wasm-opt + run: ./scripts/wasm-opt.sh --ensure-tool + - name: Build documentation # the `--enable-index-page` flag creates an index.html file at the root of the docs # the `-Zunstable-options` flag is needed to use `--enable-index-page` diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8fff3bae..1a9bdc92 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -134,6 +134,10 @@ jobs: target/ key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} + # Compiles Binaryen on a cache miss; lands in the cached ~/.cargo/bin. + - name: Ensure wasm-opt + run: ./scripts/wasm-opt.sh --ensure-tool + - name: Run build and test run: ./scripts/build-and-test.sh @@ -184,6 +188,9 @@ jobs: if: steps.cache-nodemodules.outputs.cache-hit != 'true' run: npm ci + - name: Ensure wasm-opt + run: ./scripts/wasm-opt.sh --ensure-tool + - name: Start rippled in Docker run: ./scripts/docker-rippled.sh start diff --git a/AGENTS.md b/AGENTS.md index f9d0c8f6..560d3fec 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -63,7 +63,7 @@ DEVNET=true ./scripts/run-tests.sh # run against wss://w ./scripts/generate-ledger-objects.sh ``` -Other scripts not part of the primary workflow above: `scripts/benchmark-gas.sh`, `scripts/check-wasm-exports.sh`, `scripts/docs.sh` (builds and deploys the GitHub Pages docs/UI site), `scripts/host-function-audit.sh` (see below), `scripts/run-markdown.sh`, `scripts/validate-ui.sh`, `scripts/cargo-deny.sh` (RustSec advisories on the library workspace; advisory-only in CI until the first clean run). +Other scripts not part of the primary workflow above: `scripts/benchmark-gas.sh`, `scripts/check-wasm-exports.sh`, `scripts/docs.sh` (builds and deploys the GitHub Pages docs/UI site), `scripts/host-function-audit.sh` (see below), `scripts/run-markdown.sh`, `scripts/validate-ui.sh`, `scripts/wasm-opt.sh` (called by `build.sh`; see below), `scripts/cargo-deny.sh` (RustSec advisories on the library workspace; advisory-only in CI until the first clean run). Pre-commit hooks (`.pre-commit-config.yaml`) run `cargo fmt --all` and `cargo clippy --all-targets --all-features -- -Dclippy::all` on staged Rust files, plus prettier with `--no-semi --tab-width 2` for JS/MD/YAML. @@ -151,6 +151,18 @@ panic = "abort" # no_std can't unwind; also avoids pulling in a panic handle The library defines a custom `#[panic_handler]` for `target_arch = "wasm32"` (in `xrpl-common-stdlib/src/lib.rs`) that calls `core::arch::wasm32::unreachable()`. Dev profile uses `panic = "unwind"` so unit tests can run on the host. +`scripts/build.sh` finishes by running `scripts/wasm-opt.sh`, which rewrites the **release** artifacts of both WASM workspaces in place with Binaryen's `wasm-opt` (~30% smaller, which matters because rippled charges gas per instruction and caps contract size). In place, rather than a separate `.opt.wasm`, so `tests/runSingleTest.js`, `ui/embed-wasm.sh`, and the gas benchmark all exercise the bytes that would ship on-ledger; a sidecar `.wasm.opt-stamp` holding the optimized file's hash makes reruns skip modules cargo didn't relink. Debug artifacts are left alone. + +The flag set is `-Oz --mvp-features --enable-sign-ext --enable-mutable-globals --strip-debug --strip-producers`. Starting from `--mvp-features` and re-enabling exactly sign-extension and mutable-globals is deliberate: that pair _is_ `wasm32v1-none`, and it stops wasm-opt from emitting post-MVP instructions (bulk memory, SIMD, reference types) that rippled's engine would reject. Don't widen it without checking what the host actually accepts. + +The step is **mandatory** — it never silently skips. `scripts/wasm-opt.sh` resolves a `wasm-opt` from `$WASM_OPT`, then `PATH`, then `$CARGO_HOME/bin`; finding none it runs `cargo install wasm-opt --version --locked` and fails the build if that fails. The pinned crate version lives in that script and is the single source of truth: `scripts/setup.sh` and both CI workflows call `./scripts/wasm-opt.sh --ensure-tool` to resolve/install up front rather than stalling a build mid-way. CI's `~/.cargo/bin` cache carries the compiled binary between runs. + +The `wasm-opt` crate isn't a reimplementation — its binary links Binaryen's real `wasm_opt_main`, so it's the upstream CLI with upstream flags, just built from source (needs a C++17 toolchain, a few minutes on first install). It tracks Binaryen **116** while upstream is at 133; output measured within 0.4% of 132 on every example, so the lag costs nothing that matters. It's pinned because a different Binaryen means different bytes, which means different numbers out of `scripts/benchmark-gas.sh`. A native Binaryen already on `PATH` wins over the pin — respecting a developer's install beats a multi-minute compile, and CI pins itself by installing the crate version explicitly. + +Note that `cargo install` is invoked with `RUSTFLAGS=""`: the repo-wide `-Dwarnings` would otherwise fail the build on any warning in a third-party crate. + +Other knobs: `SKIP_WASM_OPT=true` (the one deliberate opt-out), `WASM_OPT_NO_INSTALL=true` (fail rather than auto-install), `WASM_OPT=`, `WASM_OPT_FLAGS="..."`. The script also takes explicit dirs/files as arguments. + ## Writing a contract Minimal template (see `examples/smart-escrows/hello_world/src/lib.rs`): diff --git a/scripts/README.md b/scripts/README.md index 460456fd..a2134f3f 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -29,7 +29,7 @@ You can also run individual test suites: ### Main Scripts - **`build-and-test.sh`** - Build and test the native workspace, build WASM examples, and run craft -- **`build.sh`** - Build all examples (accepts `release` argument for release builds) +- **`build.sh`** - Build all examples (accepts `release` argument for release builds), then shrink the release WASM via `wasm-opt.sh` ### Helper Scripts @@ -40,6 +40,7 @@ You can also run individual test suites: - **`docker-rippled.sh`** - Start/stop/check a local rippled node in Docker, pinned to the same image CI uses; also used by CI itself, so this is the only place the container's `docker run`/health-check/logs logic lives (`start`/`stop`/`status`/`logs`) - **`host-function-audit.sh`** - Audit host functions against XRPLd (requires Node.js) - **`cargo-deny.sh`** - Check RustSec advisories on the library workspace (requires `cargo-deny`) +- **`wasm-opt.sh`** - Shrink the built release contracts in place with Binaryen's `wasm-opt` (run automatically at the end of `build.sh`; auto-installs a pinned `wasm-opt` via cargo if none is found, and fails the build if it can't). `--ensure-tool` resolves/installs the tool without optimizing anything. - **`benchmark-gas.sh`** - Measure and compare gas costs of optimized helper functions - **`generate-sfields.sh`** - Generate type-safe SField constants from rippled source (requires Node.js) @@ -177,6 +178,7 @@ Actions workflows, ensuring perfect consistency between local and CI environment - **Pre-commit not found**: Run `./scripts/setup.sh` to install dependencies - **Node.js required**: Install Node.js for the host function audit, or skip that script - **WASM target missing**: The scripts automatically install the `wasm32v1-none` target +- **Build pauses on "installing wasm-opt via cargo"**: expected on a machine with no `wasm-opt`; it compiles Binaryen from source once (a few minutes, needs a C++17 toolchain). Install a native Binaryen instead (`brew install binaryen` / `apt-get install binaryen`) to skip the compile, or set `SKIP_WASM_OPT=true` to build without the optimization step - **Docker not found / daemon not reachable**: Install/start Docker, or set `NO_DOCKER=true` (use your own local rippled) or `DEVNET=true` (use WASM Devnet) ## Script Dependencies @@ -185,6 +187,7 @@ Actions workflows, ensuring perfect consistency between local and CI environment setup.sh (run first) ├── build-and-test.sh ├── build.sh + └── wasm-opt.sh (required; auto-installs a pinned wasm-opt via cargo) ├── check-wasm-exports.sh ├── clippy.sh ├── run-tests.sh diff --git a/scripts/build.sh b/scripts/build.sh index 8d93a7d3..5d8ca2b1 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -56,4 +56,7 @@ else fi cd .. +echo "🗜️ Optimizing release WASM..." +./scripts/wasm-opt.sh + echo "✅ Build completed successfully!" diff --git a/scripts/setup.sh b/scripts/setup.sh index 30abd786..3f5c3fef 100755 --- a/scripts/setup.sh +++ b/scripts/setup.sh @@ -27,6 +27,10 @@ rustup default stable echo "📦 Adding wasm32v1-none target..." rustup target add wasm32v1-none +# Up front, since installing the crate compiles Binaryen from source. +echo "📦 Ensuring wasm-opt (Binaryen) is available..." +./scripts/wasm-opt.sh --ensure-tool + # Note: Pre-commit checks are handled by GitHub Actions using pre-commit/action@v3.0.1 # Local pre-commit installation is optional for development convenience echo "ℹ️ Pre-commit checks are handled by GitHub Actions" diff --git a/scripts/wasm-opt.sh b/scripts/wasm-opt.sh new file mode 100755 index 00000000..6cf3897a --- /dev/null +++ b/scripts/wasm-opt.sh @@ -0,0 +1,179 @@ +#!/bin/bash +# Shrink built release WASM contracts in place with Binaryen's wasm-opt (~30%). +# A sidecar `.wasm.opt-stamp` holds the optimized hash so reruns skip +# modules cargo didn't relink. Mandatory: installs wasm-opt if missing, fails if +# it can't. +# +# Usage: +# ./scripts/wasm-opt.sh # optimize the default release dirs +# ./scripts/wasm-opt.sh path/to/dir a.wasm # optimize specific dirs/files +# ./scripts/wasm-opt.sh --ensure-tool # only resolve/install wasm-opt, optimize nothing +# +# Environment: +# SKIP_WASM_OPT=true explicit opt-out of the whole step +# WASM_OPT= use a specific wasm-opt binary instead of resolving one +# WASM_OPT_NO_INSTALL=true never auto-install; fail if wasm-opt isn't already present +# WASM_OPT_FLAGS="..." replace the default flag set + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +cd "$REPO_ROOT" + +if [[ "${SKIP_WASM_OPT:-false}" == "true" ]]; then + echo "⏭️ SKIP_WASM_OPT set - skipping wasm-opt." + exit 0 +fi + +ENSURE_ONLY=false +if [[ "${1:-}" == "--ensure-tool" ]]; then + ENSURE_ONLY=true + shift +fi + +# Pinned: a different Binaryen means different bytes, and different gas numbers +# out of scripts/benchmark-gas.sh. +WASM_OPT_CRATE_VERSION="0.116.1" + +CARGO_BIN="${CARGO_HOME:-$HOME/.cargo}/bin" +resolve_wasm_opt() { + if [[ -n "${WASM_OPT:-}" ]]; then + command -v "$WASM_OPT" &> /dev/null && return 0 + echo "❌ WASM_OPT is set to '$WASM_OPT', which is not executable." + exit 1 + fi + if command -v wasm-opt &> /dev/null; then + WASM_OPT="$(command -v wasm-opt)" + return 0 + fi + if [[ -x "$CARGO_BIN/wasm-opt" ]]; then + WASM_OPT="$CARGO_BIN/wasm-opt" + return 0 + fi + return 1 +} + +if ! resolve_wasm_opt; then + if [[ "${WASM_OPT_NO_INSTALL:-false}" == "true" ]]; then + echo "❌ wasm-opt not found and WASM_OPT_NO_INSTALL=true." + echo " Install Binaryen (brew install binaryen | apt-get install binaryen)," + echo " or run: cargo install wasm-opt --version $WASM_OPT_CRATE_VERSION --locked" + exit 1 + fi + + echo "📦 wasm-opt not found - installing wasm-opt v$WASM_OPT_CRATE_VERSION via cargo..." + echo " (this compiles Binaryen from source and takes a few minutes the first time;" + echo " it needs a C++17 toolchain. Set WASM_OPT_NO_INSTALL=true to opt out.)" + # Cleared because the repo-wide -Dwarnings would fail on third-party crates. + if ! RUSTFLAGS="" cargo install wasm-opt --version "$WASM_OPT_CRATE_VERSION" --locked; then + echo "❌ Failed to install wasm-opt via cargo." + echo " Install Binaryen another way (brew install binaryen | apt-get install binaryen)" + echo " and re-run, or set SKIP_WASM_OPT=true to build without optimization." + exit 1 + fi + if ! resolve_wasm_opt; then + echo "❌ cargo install succeeded but wasm-opt is still not on PATH or in $CARGO_BIN." + exit 1 + fi +fi + +if [[ "$ENSURE_ONLY" == "true" ]]; then + echo "✅ $($WASM_OPT --version) ready at $WASM_OPT" + exit 0 +fi + +# MVP + sign-ext + mutable-globals is exactly wasm32v1-none. Don't widen it: +# post-MVP instructions (bulk memory, SIMD, reftypes) would be rejected by the +# target and by rippled's engine. +DEFAULT_WASM_OPT_FLAGS=( + -Oz + --mvp-features + --enable-sign-ext + --enable-mutable-globals + --strip-debug + --strip-producers +) +if [[ -n "${WASM_OPT_FLAGS:-}" ]]; then + # shellcheck disable=SC2206 # intentional word splitting of a user-supplied flag string + FLAGS=($WASM_OPT_FLAGS) +else + FLAGS=("${DEFAULT_WASM_OPT_FLAGS[@]}") +fi + +# Release only; debug builds stay unstripped for troubleshooting. +TARGETS=("$@") +if [[ ${#TARGETS[@]} -eq 0 ]]; then + TARGETS=( + "examples/target/wasm32v1-none/release" + "e2e-tests/target/wasm32v1-none/release" + ) +fi + +echo "🗜️ Optimizing WASM with $($WASM_OPT --version)..." + +hash_of() { + if command -v shasum &> /dev/null; then + shasum -a 256 "$1" | cut -d' ' -f1 + else + sha256sum "$1" | cut -d' ' -f1 + fi +} + +# maxdepth 1: deps/ and build/ below the profile dir hold intermediates. +WASM_FILES=() +for target in "${TARGETS[@]}"; do + if [[ -d "$target" ]]; then + while IFS= read -r wasm; do + WASM_FILES+=("$wasm") + done < <(find "$target" -maxdepth 1 -name '*.wasm' | sort) + elif [[ -f "$target" ]]; then + WASM_FILES+=("$target") + else + echo "❌ $target not found - build the WASM workspaces first." + exit 1 + fi +done + +if [[ ${#WASM_FILES[@]} -eq 0 ]]; then + echo "❌ No .wasm files found to optimize in: ${TARGETS[*]}" + exit 1 +fi + +total_before=0 +total_after=0 +optimized=0 +skipped=0 + +for wasm in "${WASM_FILES[@]}"; do + name="$(basename "$wasm")" + stamp="$wasm.opt-stamp" + before=$(wc -c < "$wasm" | tr -d ' ') + + if [[ -f "$stamp" ]] && [[ "$(cat "$stamp")" == "$(hash_of "$wasm")" ]]; then + echo " ⏭️ $name (already optimized, ${before} bytes)" + skipped=$((skipped + 1)) + total_before=$((total_before + before)) + total_after=$((total_after + before)) + continue + fi + + tmp="$wasm.opt.tmp" + if ! "$WASM_OPT" "${FLAGS[@]}" "$wasm" -o "$tmp"; then + rm -f "$tmp" + echo "❌ wasm-opt failed on $wasm" + exit 1 + fi + mv "$tmp" "$wasm" + + after=$(wc -c < "$wasm" | tr -d ' ') + hash_of "$wasm" > "$stamp" + optimized=$((optimized + 1)) + total_before=$((total_before + before)) + total_after=$((total_after + after)) + echo " ✅ $name: ${before} -> ${after} bytes ($(( (before - after) * 100 / (before > 0 ? before : 1) ))% smaller)" +done + +if [[ $total_before -gt 0 ]]; then + echo "🗜️ wasm-opt: ${optimized} optimized, ${skipped} unchanged; ${total_before} -> ${total_after} bytes total" +fi From ee2f4597bcc05022183d2a6cb66ad168d279ce35 Mon Sep 17 00:00:00 2001 From: TimothyBanks Date: Tue, 29 Sep 2026 11:58:48 -0400 Subject: [PATCH 2/7] feat: Using binaryen to optimize wasm sizes --- AGENTS.md | 2 +- scripts/README.md | 2 +- scripts/wasm-opt.sh | 24 ++++++++++++++++-------- 3 files changed, 18 insertions(+), 10 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 560d3fec..6d946769 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -155,7 +155,7 @@ The library defines a custom `#[panic_handler]` for `target_arch = "wasm32"` (in The flag set is `-Oz --mvp-features --enable-sign-ext --enable-mutable-globals --strip-debug --strip-producers`. Starting from `--mvp-features` and re-enabling exactly sign-extension and mutable-globals is deliberate: that pair _is_ `wasm32v1-none`, and it stops wasm-opt from emitting post-MVP instructions (bulk memory, SIMD, reference types) that rippled's engine would reject. Don't widen it without checking what the host actually accepts. -The step is **mandatory** — it never silently skips. `scripts/wasm-opt.sh` resolves a `wasm-opt` from `$WASM_OPT`, then `PATH`, then `$CARGO_HOME/bin`; finding none it runs `cargo install wasm-opt --version --locked` and fails the build if that fails. The pinned crate version lives in that script and is the single source of truth: `scripts/setup.sh` and both CI workflows call `./scripts/wasm-opt.sh --ensure-tool` to resolve/install up front rather than stalling a build mid-way. CI's `~/.cargo/bin` cache carries the compiled binary between runs. +The **tool** is mandatory — `scripts/build.sh` calls `wasm-opt.sh` outside its release/debug branch, so even a debug build fails if `wasm-opt` can't be obtained. It resolves one from `$WASM_OPT`, then `PATH`, then `$CARGO_HOME/bin`; finding none it runs `cargo install wasm-opt --version --locked` and fails the build if that fails. Having no release WASM to optimize is a different matter and exits 0 quietly, so a debug-only build isn't broken by the step — but a dir or file named explicitly on the command line that's missing or holds no `.wasm` is treated as a typo and fails. The pinned crate version lives in that script and is the single source of truth: `scripts/setup.sh` and both CI workflows call `./scripts/wasm-opt.sh --ensure-tool` to resolve/install up front rather than stalling a build mid-way. CI's `~/.cargo/bin` cache carries the compiled binary between runs. The `wasm-opt` crate isn't a reimplementation — its binary links Binaryen's real `wasm_opt_main`, so it's the upstream CLI with upstream flags, just built from source (needs a C++17 toolchain, a few minutes on first install). It tracks Binaryen **116** while upstream is at 133; output measured within 0.4% of 132 on every example, so the lag costs nothing that matters. It's pinned because a different Binaryen means different bytes, which means different numbers out of `scripts/benchmark-gas.sh`. A native Binaryen already on `PATH` wins over the pin — respecting a developer's install beats a multi-minute compile, and CI pins itself by installing the crate version explicitly. diff --git a/scripts/README.md b/scripts/README.md index a2134f3f..b7b7dfb2 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -40,7 +40,7 @@ You can also run individual test suites: - **`docker-rippled.sh`** - Start/stop/check a local rippled node in Docker, pinned to the same image CI uses; also used by CI itself, so this is the only place the container's `docker run`/health-check/logs logic lives (`start`/`stop`/`status`/`logs`) - **`host-function-audit.sh`** - Audit host functions against XRPLd (requires Node.js) - **`cargo-deny.sh`** - Check RustSec advisories on the library workspace (requires `cargo-deny`) -- **`wasm-opt.sh`** - Shrink the built release contracts in place with Binaryen's `wasm-opt` (run automatically at the end of `build.sh`; auto-installs a pinned `wasm-opt` via cargo if none is found, and fails the build if it can't). `--ensure-tool` resolves/installs the tool without optimizing anything. +- **`wasm-opt.sh`** - Shrink the built release contracts in place with Binaryen's `wasm-opt` (run automatically at the end of `build.sh`; auto-installs a pinned `wasm-opt` via cargo if none is found, and fails the build if it can't; exits quietly when there's no release WASM to optimize). `--ensure-tool` resolves/installs the tool without optimizing anything. - **`benchmark-gas.sh`** - Measure and compare gas costs of optimized helper functions - **`generate-sfields.sh`** - Generate type-safe SField constants from rippled source (requires Node.js) diff --git a/scripts/wasm-opt.sh b/scripts/wasm-opt.sh index 6cf3897a..5d6dca67 100755 --- a/scripts/wasm-opt.sh +++ b/scripts/wasm-opt.sh @@ -1,8 +1,8 @@ #!/bin/bash # Shrink built release WASM contracts in place with Binaryen's wasm-opt (~30%). # A sidecar `.wasm.opt-stamp` holds the optimized hash so reruns skip -# modules cargo didn't relink. Mandatory: installs wasm-opt if missing, fails if -# it can't. +# modules cargo didn't relink. Installs wasm-opt if missing and fails if it +# can't, but exits 0 when there's simply no release WASM to optimize. # # Usage: # ./scripts/wasm-opt.sh # optimize the default release dirs @@ -103,15 +103,15 @@ fi # Release only; debug builds stay unstripped for troubleshooting. TARGETS=("$@") +EXPLICIT_TARGETS=true if [[ ${#TARGETS[@]} -eq 0 ]]; then + EXPLICIT_TARGETS=false TARGETS=( "examples/target/wasm32v1-none/release" "e2e-tests/target/wasm32v1-none/release" ) fi -echo "🗜️ Optimizing WASM with $($WASM_OPT --version)..." - hash_of() { if command -v shasum &> /dev/null; then shasum -a 256 "$1" | cut -d' ' -f1 @@ -129,17 +129,25 @@ for target in "${TARGETS[@]}"; do done < <(find "$target" -maxdepth 1 -name '*.wasm' | sort) elif [[ -f "$target" ]]; then WASM_FILES+=("$target") - else - echo "❌ $target not found - build the WASM workspaces first." + elif [[ "$EXPLICIT_TARGETS" == "true" ]]; then + echo "❌ $target not found." exit 1 fi done +# Nothing to do is only an error when the caller named the targets; with the +# defaults it just means no release WASM was built (e.g. a debug-only build). if [[ ${#WASM_FILES[@]} -eq 0 ]]; then - echo "❌ No .wasm files found to optimize in: ${TARGETS[*]}" - exit 1 + if [[ "$EXPLICIT_TARGETS" == "true" ]]; then + echo "❌ No .wasm files found in: ${TARGETS[*]}" + exit 1 + fi + echo " No release WASM found - nothing to optimize." + exit 0 fi +echo "🗜️ Optimizing WASM with $($WASM_OPT --version)..." + total_before=0 total_after=0 optimized=0 From 9503d03288369f7c0ced8db208d1a930c344199d Mon Sep 17 00:00:00 2001 From: Timothy Banks Date: Tue, 29 Sep 2026 14:17:38 -0400 Subject: [PATCH 3/7] ci: Pin all GitHub Actions to full commit SHAs (#315) Mutable tag and branch refs can be silently repointed by the action owner, as happened in the trivy-action and kics-github-action compromises. Pin all 30 `uses:` refs across test.yml, docs.yml, and check-pr-description.yml to 40-character commit SHAs, keeping the original tag as a trailing comment so the intended version stays readable and Dependabot can bump both together. The two XRPLF/actions reusable workflows were already SHA-pinned and are unchanged. `taiki-e/install-action@cargo-deny` and `@cargo-llvm-cov` were the most exposed: those are per-tool aliases the maintainer repoints on every release, not version tags. .github/dependabot.yml already declares the `github-actions` ecosystem, so the pins stay maintained with no config change. Each SHA was resolved from the tag it replaces and independently re-verified against the live tag. Co-authored-by: Claude Opus 5 (1M context) Co-authored-by: David Fuelling --- .github/workflows/check-pr-description.yml | 2 +- .github/workflows/docs.yml | 8 ++-- .github/workflows/test.yml | 50 +++++++++++----------- 3 files changed, 30 insertions(+), 30 deletions(-) diff --git a/.github/workflows/check-pr-description.yml b/.github/workflows/check-pr-description.yml index 7cc2ff19..a6cd352f 100644 --- a/.github/workflows/check-pr-description.yml +++ b/.github/workflows/check-pr-description.yml @@ -26,7 +26,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Write PR body to file env: diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index dc4385b8..c13a5564 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -10,9 +10,9 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Cache dependencies - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 with: path: | ~/.cargo/bin/ @@ -34,7 +34,7 @@ jobs: - name: Upload documentation files as artifact id: docs-artifact - uses: actions/upload-pages-artifact@v5 + uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5 with: path: target/doc retention-days: 10 @@ -51,4 +51,4 @@ jobs: steps: - name: Deploy to Documentation to GitHub Pages id: docs-artifact - uses: actions/deploy-pages@v5 + uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5 diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 1a9bdc92..81cb7cae 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -17,9 +17,9 @@ jobs: pre-commit: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Cache dependencies - uses: actions/cache/restore@v6 + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 with: path: | ~/.cargo/bin/ @@ -28,20 +28,20 @@ jobs: ~/.cargo/git/db/ target/ key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} - - uses: actions-rust-lang/setup-rust-toolchain@v2 + - uses: actions-rust-lang/setup-rust-toolchain@ecabd13d1c56bd1345c230e542e9144811ad706f # v2 with: toolchain: stable components: clippy, rustfmt # This runs the pre-commit hooks defined in .pre-commit-config.yaml # TODO: figure out how to update to use pre-commit.ci - - uses: pre-commit/action@v3.0.1 + - uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1 clippy_linting: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Cache dependencies - uses: actions/cache/restore@v6 + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 with: path: | ~/.cargo/bin/ @@ -57,7 +57,7 @@ jobs: rustfmt: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Run formatting check run: ./scripts/fmt.sh @@ -65,7 +65,7 @@ jobs: check_wasm_exports: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Check WASM contract exports run: ./scripts/check-wasm-exports.sh @@ -73,10 +73,10 @@ jobs: check_ledger_objects_generated: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Use Node.js v24 - uses: actions/setup-node@v7 + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7 with: node-version: 24.x @@ -91,7 +91,7 @@ jobs: runs-on: ubuntu-latest if: false steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Run host function audit run: ./scripts/host-function-audit.sh @@ -103,17 +103,17 @@ jobs: runs-on: ubuntu-latest continue-on-error: true steps: - - uses: actions/checkout@v7 - - uses: actions-rust-lang/setup-rust-toolchain@v2 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 + - uses: actions-rust-lang/setup-rust-toolchain@ecabd13d1c56bd1345c230e542e9144811ad706f # v2 - name: Install cargo-deny - uses: taiki-e/install-action@cargo-deny + uses: taiki-e/install-action@3ce020901a1c1292716d3268bf506c1ae55a84cd # cargo-deny - name: Check RustSec advisories run: ./scripts/cargo-deny.sh validate_ui: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Validate UI run: ./scripts/validate-ui.sh @@ -121,10 +121,10 @@ jobs: build_and_test: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Cache dependencies - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 with: path: | ~/.cargo/bin/ @@ -144,7 +144,7 @@ jobs: run-markdown: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Run code in Markdown files run: ./scripts/run-markdown.sh @@ -153,15 +153,15 @@ jobs: runs-on: ubuntu-latest needs: build_and_test steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Use Node.js v24 - uses: actions/setup-node@v7 + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7 with: node-version: 24.x - name: Cache cargo dependencies - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 with: path: | ~/.cargo/bin/ @@ -173,7 +173,7 @@ jobs: - name: Cache Node dependencies id: cache-nodemodules - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 env: cache-name: cache-node-modules with: @@ -211,9 +211,9 @@ jobs: env: CARGO_TERM_COLOR: always steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - name: Cache dependencies - uses: actions/cache/restore@v6 + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 with: path: | ~/.cargo/bin/ @@ -225,6 +225,6 @@ jobs: - name: Install Rust run: rustup update stable - name: Install cargo-llvm-cov - uses: taiki-e/install-action@cargo-llvm-cov + uses: taiki-e/install-action@72f835688cb6c15903b796f704b01e652a2b8835 # cargo-llvm-cov - name: Generate code coverage run: ./scripts/coverage.sh From 2067e48c94f057e6b145b4a55106c8aab0bc0d03 Mon Sep 17 00:00:00 2001 From: TimothyBanks Date: Tue, 29 Sep 2026 14:48:06 -0400 Subject: [PATCH 4/7] fix: Address ai review comments --- scripts/wasm-opt.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/scripts/wasm-opt.sh b/scripts/wasm-opt.sh index 5d6dca67..ae555267 100755 --- a/scripts/wasm-opt.sh +++ b/scripts/wasm-opt.sh @@ -120,6 +120,14 @@ hash_of() { fi } +# The stamp covers the optimizer and its flags as well as the bytes, so that +# editing WASM_OPT_FLAGS or moving to a different wasm-opt re-optimizes a module +# cargo hasn't relinked. +OPT_SIG="$($WASM_OPT --version) | ${FLAGS[*]}" +stamp_value() { + echo "$(hash_of "$1") | $OPT_SIG" +} + # maxdepth 1: deps/ and build/ below the profile dir hold intermediates. WASM_FILES=() for target in "${TARGETS[@]}"; do @@ -158,7 +166,7 @@ for wasm in "${WASM_FILES[@]}"; do stamp="$wasm.opt-stamp" before=$(wc -c < "$wasm" | tr -d ' ') - if [[ -f "$stamp" ]] && [[ "$(cat "$stamp")" == "$(hash_of "$wasm")" ]]; then + if [[ -f "$stamp" ]] && [[ "$(cat "$stamp")" == "$(stamp_value "$wasm")" ]]; then echo " ⏭️ $name (already optimized, ${before} bytes)" skipped=$((skipped + 1)) total_before=$((total_before + before)) @@ -175,7 +183,7 @@ for wasm in "${WASM_FILES[@]}"; do mv "$tmp" "$wasm" after=$(wc -c < "$wasm" | tr -d ' ') - hash_of "$wasm" > "$stamp" + stamp_value "$wasm" > "$stamp" optimized=$((optimized + 1)) total_before=$((total_before + before)) total_after=$((total_after + after)) From 5e782dbbcff88a9a8682315f8144e587aea9d1bb Mon Sep 17 00:00:00 2001 From: TimothyBanks Date: Tue, 29 Sep 2026 14:59:22 -0400 Subject: [PATCH 5/7] fix: Enforce the wasm-opt pin and drop the ineffective result cache Two review findings on the wasm-opt integration. Verify a cargo-managed wasm-opt against the pin. Tool resolution accepted whatever binary it found. CI caches ~/.cargo/bin under a key derived from Cargo.lock, which bumping WASM_OPT_CRATE_VERSION in this script does not change, so a future pin bump would silently keep using the stale cached binary and the pin would be a lie. A wasm-opt at $CARGO_HOME/bin is now version-checked against the pin, reinstalled on mismatch, and re-checked afterwards so a bad install fails loudly instead of proceeding with the wrong optimizer. An explicit $WASM_OPT and a system Binaryen stay exempt, as documented. Remove the stamp cache. The stamp only hashed the wasm bytes, so editing WASM_OPT_FLAGS or moving to a different wasm-opt silently skipped re-optimization. Rather than extend the stamp to cover both, drop it: writing in place breaks the hardlink cargo makes from deps/, so cargo re-uplifts the unoptimized artifact on every build and the cache could never legitimately hit. It was buying ~0.7s that never materialized while owning a real correctness bug. Each build.sh run is now exactly one pass over pristine cargo output. Co-Authored-By: Claude Opus 5 (1M context) --- AGENTS.md | 8 +++-- scripts/wasm-opt.sh | 85 +++++++++++++++++++++++---------------------- 2 files changed, 49 insertions(+), 44 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6d946769..9f2c4b0f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -151,13 +151,17 @@ panic = "abort" # no_std can't unwind; also avoids pulling in a panic handle The library defines a custom `#[panic_handler]` for `target_arch = "wasm32"` (in `xrpl-common-stdlib/src/lib.rs`) that calls `core::arch::wasm32::unreachable()`. Dev profile uses `panic = "unwind"` so unit tests can run on the host. -`scripts/build.sh` finishes by running `scripts/wasm-opt.sh`, which rewrites the **release** artifacts of both WASM workspaces in place with Binaryen's `wasm-opt` (~30% smaller, which matters because rippled charges gas per instruction and caps contract size). In place, rather than a separate `.opt.wasm`, so `tests/runSingleTest.js`, `ui/embed-wasm.sh`, and the gas benchmark all exercise the bytes that would ship on-ledger; a sidecar `.wasm.opt-stamp` holding the optimized file's hash makes reruns skip modules cargo didn't relink. Debug artifacts are left alone. +`scripts/build.sh` finishes by running `scripts/wasm-opt.sh`, which rewrites the **release** artifacts of both WASM workspaces in place with Binaryen's `wasm-opt` (~30% smaller, which matters because rippled charges gas per instruction and caps contract size). In place, rather than a separate `.opt.wasm`, so `tests/runSingleTest.js`, `ui/embed-wasm.sh`, and the gas benchmark all exercise the bytes that would ship on-ledger. Debug artifacts are left alone. + +There is deliberately **no result caching**. Writing in place breaks the hardlink cargo makes from `deps/`, so cargo re-uplifts the unoptimized artifact on every build — meaning each `build.sh` run is already exactly one `wasm-opt` pass over pristine output, and a skip-cache keyed on the file would never legitimately hit. An earlier stamp-file version of this was removed: it cost ~0.7s of savings that never materialized, and it silently skipped re-optimization when `WASM_OPT_FLAGS` or the wasm-opt version changed. The flag set is `-Oz --mvp-features --enable-sign-ext --enable-mutable-globals --strip-debug --strip-producers`. Starting from `--mvp-features` and re-enabling exactly sign-extension and mutable-globals is deliberate: that pair _is_ `wasm32v1-none`, and it stops wasm-opt from emitting post-MVP instructions (bulk memory, SIMD, reference types) that rippled's engine would reject. Don't widen it without checking what the host actually accepts. The **tool** is mandatory — `scripts/build.sh` calls `wasm-opt.sh` outside its release/debug branch, so even a debug build fails if `wasm-opt` can't be obtained. It resolves one from `$WASM_OPT`, then `PATH`, then `$CARGO_HOME/bin`; finding none it runs `cargo install wasm-opt --version --locked` and fails the build if that fails. Having no release WASM to optimize is a different matter and exits 0 quietly, so a debug-only build isn't broken by the step — but a dir or file named explicitly on the command line that's missing or holds no `.wasm` is treated as a typo and fails. The pinned crate version lives in that script and is the single source of truth: `scripts/setup.sh` and both CI workflows call `./scripts/wasm-opt.sh --ensure-tool` to resolve/install up front rather than stalling a build mid-way. CI's `~/.cargo/bin` cache carries the compiled binary between runs. -The `wasm-opt` crate isn't a reimplementation — its binary links Binaryen's real `wasm_opt_main`, so it's the upstream CLI with upstream flags, just built from source (needs a C++17 toolchain, a few minutes on first install). It tracks Binaryen **116** while upstream is at 133; output measured within 0.4% of 132 on every example, so the lag costs nothing that matters. It's pinned because a different Binaryen means different bytes, which means different numbers out of `scripts/benchmark-gas.sh`. A native Binaryen already on `PATH` wins over the pin — respecting a developer's install beats a multi-minute compile, and CI pins itself by installing the crate version explicitly. +The `wasm-opt` crate isn't a reimplementation — its binary links Binaryen's real `wasm_opt_main`, so it's the upstream CLI with upstream flags, just built from source (needs a C++17 toolchain, a few minutes on first install). It tracks Binaryen **116** while upstream is at 133; output measured within 0.4% of 132 on every example, so the lag costs nothing that matters. It's pinned because a different Binaryen means different bytes, which means different numbers out of `scripts/benchmark-gas.sh`. A native Binaryen already on `PATH` wins over the pin — respecting a developer's install beats a multi-minute compile. + +A **cargo-managed** `wasm-opt` (the one at `$CARGO_HOME/bin`), by contrast, is version-checked against the pin and reinstalled if it doesn't match, then re-checked so a bad install fails loudly. This matters because CI caches `~/.cargo/bin` under a key derived from `Cargo.lock`, which bumping `WASM_OPT_CRATE_VERSION` in the script does not change — without the check, a pin bump would silently keep using the stale cached binary and the pin would be a lie. The expected Binaryen version is derived from the crate version, which is published as `0..`; if that convention ever breaks, fix `PINNED_BINARYEN_VERSION` in the script. An explicit `$WASM_OPT` and a system Binaryen are both exempt from this check. Note that `cargo install` is invoked with `RUSTFLAGS=""`: the repo-wide `-Dwarnings` would otherwise fail the build on any warning in a third-party crate. diff --git a/scripts/wasm-opt.sh b/scripts/wasm-opt.sh index ae555267..28d9a5e0 100755 --- a/scripts/wasm-opt.sh +++ b/scripts/wasm-opt.sh @@ -1,8 +1,11 @@ #!/bin/bash # Shrink built release WASM contracts in place with Binaryen's wasm-opt (~30%). -# A sidecar `.wasm.opt-stamp` holds the optimized hash so reruns skip -# modules cargo didn't relink. Installs wasm-opt if missing and fails if it -# can't, but exits 0 when there's simply no release WASM to optimize. +# Installs wasm-opt if missing or off the pin, and fails if it can't, but exits +# 0 when there's simply no release WASM to optimize. +# +# No result caching: cargo re-uplifts the profile-dir .wasm from deps/ on every +# build (our in-place write breaks its hardlink), so each build.sh run is +# already exactly one pass over pristine output. # # Usage: # ./scripts/wasm-opt.sh # optimize the default release dirs @@ -33,8 +36,12 @@ if [[ "${1:-}" == "--ensure-tool" ]]; then fi # Pinned: a different Binaryen means different bytes, and different gas numbers -# out of scripts/benchmark-gas.sh. +# out of scripts/benchmark-gas.sh. The crate is versioned 0... WASM_OPT_CRATE_VERSION="0.116.1" +PINNED_BINARYEN_VERSION="$(echo "$WASM_OPT_CRATE_VERSION" | cut -d. -f2)" + +WASM_OPT_USER_SET=false +[[ -n "${WASM_OPT:-}" ]] && WASM_OPT_USER_SET=true CARGO_BIN="${CARGO_HOME:-$HOME/.cargo}/bin" resolve_wasm_opt() { @@ -54,17 +61,19 @@ resolve_wasm_opt() { return 1 } -if ! resolve_wasm_opt; then +binaryen_version_of() { + "$1" --version 2>/dev/null | grep -oE '[0-9]+' | head -1 +} + +install_pinned_wasm_opt() { if [[ "${WASM_OPT_NO_INSTALL:-false}" == "true" ]]; then - echo "❌ wasm-opt not found and WASM_OPT_NO_INSTALL=true." + echo "❌ Need wasm-opt v$WASM_OPT_CRATE_VERSION but WASM_OPT_NO_INSTALL=true." echo " Install Binaryen (brew install binaryen | apt-get install binaryen)," echo " or run: cargo install wasm-opt --version $WASM_OPT_CRATE_VERSION --locked" exit 1 fi - - echo "📦 wasm-opt not found - installing wasm-opt v$WASM_OPT_CRATE_VERSION via cargo..." - echo " (this compiles Binaryen from source and takes a few minutes the first time;" - echo " it needs a C++17 toolchain. Set WASM_OPT_NO_INSTALL=true to opt out.)" + echo " (compiles Binaryen from source, a few minutes; needs a C++17 toolchain." + echo " Set WASM_OPT_NO_INSTALL=true to opt out.)" # Cleared because the repo-wide -Dwarnings would fail on third-party crates. if ! RUSTFLAGS="" cargo install wasm-opt --version "$WASM_OPT_CRATE_VERSION" --locked; then echo "❌ Failed to install wasm-opt via cargo." @@ -72,12 +81,35 @@ if ! resolve_wasm_opt; then echo " and re-run, or set SKIP_WASM_OPT=true to build without optimization." exit 1 fi +} + +if ! resolve_wasm_opt; then + echo "📦 wasm-opt not found - installing wasm-opt v$WASM_OPT_CRATE_VERSION via cargo..." + install_pinned_wasm_opt if ! resolve_wasm_opt; then echo "❌ cargo install succeeded but wasm-opt is still not on PATH or in $CARGO_BIN." exit 1 fi fi +# A cargo-managed wasm-opt must actually match the pin. CI caches ~/.cargo/bin +# under a key derived from Cargo.lock, which a pin bump here doesn't change, so +# without this the old binary would silently survive the bump. A system Binaryen +# (brew/apt) is deliberately left alone - see AGENTS.md. +if [[ "$WASM_OPT_USER_SET" == "false" && "$WASM_OPT" -ef "$CARGO_BIN/wasm-opt" ]]; then + have="$(binaryen_version_of "$WASM_OPT")" + if [[ "$have" != "$PINNED_BINARYEN_VERSION" ]]; then + echo "♻️ cargo-installed wasm-opt is Binaryen $have, pin wants $PINNED_BINARYEN_VERSION - reinstalling..." + install_pinned_wasm_opt + have="$(binaryen_version_of "$WASM_OPT")" + if [[ "$have" != "$PINNED_BINARYEN_VERSION" ]]; then + echo "❌ Installed wasm-opt v$WASM_OPT_CRATE_VERSION but got Binaryen $have, expected $PINNED_BINARYEN_VERSION." + echo " If the crate no longer versions as 0.., fix PINNED_BINARYEN_VERSION in this script." + exit 1 + fi + fi +fi + if [[ "$ENSURE_ONLY" == "true" ]]; then echo "✅ $($WASM_OPT --version) ready at $WASM_OPT" exit 0 @@ -112,22 +144,6 @@ if [[ ${#TARGETS[@]} -eq 0 ]]; then ) fi -hash_of() { - if command -v shasum &> /dev/null; then - shasum -a 256 "$1" | cut -d' ' -f1 - else - sha256sum "$1" | cut -d' ' -f1 - fi -} - -# The stamp covers the optimizer and its flags as well as the bytes, so that -# editing WASM_OPT_FLAGS or moving to a different wasm-opt re-optimizes a module -# cargo hasn't relinked. -OPT_SIG="$($WASM_OPT --version) | ${FLAGS[*]}" -stamp_value() { - echo "$(hash_of "$1") | $OPT_SIG" -} - # maxdepth 1: deps/ and build/ below the profile dir hold intermediates. WASM_FILES=() for target in "${TARGETS[@]}"; do @@ -158,22 +174,11 @@ echo "🗜️ Optimizing WASM with $($WASM_OPT --version)..." total_before=0 total_after=0 -optimized=0 -skipped=0 for wasm in "${WASM_FILES[@]}"; do name="$(basename "$wasm")" - stamp="$wasm.opt-stamp" before=$(wc -c < "$wasm" | tr -d ' ') - if [[ -f "$stamp" ]] && [[ "$(cat "$stamp")" == "$(stamp_value "$wasm")" ]]; then - echo " ⏭️ $name (already optimized, ${before} bytes)" - skipped=$((skipped + 1)) - total_before=$((total_before + before)) - total_after=$((total_after + before)) - continue - fi - tmp="$wasm.opt.tmp" if ! "$WASM_OPT" "${FLAGS[@]}" "$wasm" -o "$tmp"; then rm -f "$tmp" @@ -183,13 +188,9 @@ for wasm in "${WASM_FILES[@]}"; do mv "$tmp" "$wasm" after=$(wc -c < "$wasm" | tr -d ' ') - stamp_value "$wasm" > "$stamp" - optimized=$((optimized + 1)) total_before=$((total_before + before)) total_after=$((total_after + after)) echo " ✅ $name: ${before} -> ${after} bytes ($(( (before - after) * 100 / (before > 0 ? before : 1) ))% smaller)" done -if [[ $total_before -gt 0 ]]; then - echo "🗜️ wasm-opt: ${optimized} optimized, ${skipped} unchanged; ${total_before} -> ${total_after} bytes total" -fi +echo "🗜️ wasm-opt: ${#WASM_FILES[@]} contracts, ${total_before} -> ${total_after} bytes total" From 5a470647652551f0e9e3ba3eb70e3981df715f06 Mon Sep 17 00:00:00 2001 From: TimothyBanks Date: Thu, 1 Oct 2026 09:34:12 -0400 Subject: [PATCH 6/7] fix: Fix e2e tests --- AGENTS.md | 6 +++++- scripts/wasm-opt.sh | 6 ++---- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9f2c4b0f..78e69b07 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -155,7 +155,11 @@ The library defines a custom `#[panic_handler]` for `target_arch = "wasm32"` (in There is deliberately **no result caching**. Writing in place breaks the hardlink cargo makes from `deps/`, so cargo re-uplifts the unoptimized artifact on every build — meaning each `build.sh` run is already exactly one `wasm-opt` pass over pristine output, and a skip-cache keyed on the file would never legitimately hit. An earlier stamp-file version of this was removed: it cost ~0.7s of savings that never materialized, and it silently skipped re-optimization when `WASM_OPT_FLAGS` or the wasm-opt version changed. -The flag set is `-Oz --mvp-features --enable-sign-ext --enable-mutable-globals --strip-debug --strip-producers`. Starting from `--mvp-features` and re-enabling exactly sign-extension and mutable-globals is deliberate: that pair _is_ `wasm32v1-none`, and it stops wasm-opt from emitting post-MVP instructions (bulk memory, SIMD, reference types) that rippled's engine would reject. Don't widen it without checking what the host actually accepts. +The flag set is `-Oz --mvp-features --enable-mutable-globals --strip-debug --strip-producers`. Starting from `--mvp-features` stops wasm-opt from emitting post-MVP instructions (bulk memory, SIMD, reference types) that rippled's engine would reject. + +**Sign-extension is deliberately not enabled, even though `wasm32v1-none` allows it.** rippled rejects `i32.extend8_s`/`i64.extend16_s` and friends with `temINVALID_BYTECODE`, so the accepted feature set is narrower than the target's. This is not theoretical: with `--enable-sign-ext` the e2e suite failed on exactly the four contracts where `-Oz` chose to introduce such an instruction (`gas_benchmark`, `trace_escrow_account`, `trace_escrow_finish`, `trace_escrow_ledger_object`) and passed on the twelve where it didn't. Note that rustc itself emits no sign-extension instructions for any current contract — wasm-opt was the only source — so this is latent for hand-written code too: if a future rustc or a new contract starts emitting them, it will fail on-ledger with nothing to do with wasm-opt. Dropping the flag costs nothing; it made output 135 bytes smaller overall, since the `target_features` section shrinks too. + +Don't widen the flag set without checking what the host actually accepts — the target's feature list is not the authority here. The **tool** is mandatory — `scripts/build.sh` calls `wasm-opt.sh` outside its release/debug branch, so even a debug build fails if `wasm-opt` can't be obtained. It resolves one from `$WASM_OPT`, then `PATH`, then `$CARGO_HOME/bin`; finding none it runs `cargo install wasm-opt --version --locked` and fails the build if that fails. Having no release WASM to optimize is a different matter and exits 0 quietly, so a debug-only build isn't broken by the step — but a dir or file named explicitly on the command line that's missing or holds no `.wasm` is treated as a typo and fails. The pinned crate version lives in that script and is the single source of truth: `scripts/setup.sh` and both CI workflows call `./scripts/wasm-opt.sh --ensure-tool` to resolve/install up front rather than stalling a build mid-way. CI's `~/.cargo/bin` cache carries the compiled binary between runs. diff --git a/scripts/wasm-opt.sh b/scripts/wasm-opt.sh index 28d9a5e0..138fd9e8 100755 --- a/scripts/wasm-opt.sh +++ b/scripts/wasm-opt.sh @@ -115,13 +115,11 @@ if [[ "$ENSURE_ONLY" == "true" ]]; then exit 0 fi -# MVP + sign-ext + mutable-globals is exactly wasm32v1-none. Don't widen it: -# post-MVP instructions (bulk memory, SIMD, reftypes) would be rejected by the -# target and by rippled's engine. +# Narrower than wasm32v1-none on purpose: rippled rejects sign-extension with +# temINVALID_BYTECODE, and -Oz will happily introduce it. DEFAULT_WASM_OPT_FLAGS=( -Oz --mvp-features - --enable-sign-ext --enable-mutable-globals --strip-debug --strip-producers From e69de8e3040fe734fed90ae77699bb884fd5208b Mon Sep 17 00:00:00 2001 From: TimothyBanks Date: Thu, 1 Oct 2026 09:39:55 -0400 Subject: [PATCH 7/7] fix: Fix e2e tests --- AGENTS.md | 12 +++++++++--- scripts/wasm-opt.sh | 7 ++++--- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 78e69b07..3182129f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -155,11 +155,17 @@ The library defines a custom `#[panic_handler]` for `target_arch = "wasm32"` (in There is deliberately **no result caching**. Writing in place breaks the hardlink cargo makes from `deps/`, so cargo re-uplifts the unoptimized artifact on every build — meaning each `build.sh` run is already exactly one `wasm-opt` pass over pristine output, and a skip-cache keyed on the file would never legitimately hit. An earlier stamp-file version of this was removed: it cost ~0.7s of savings that never materialized, and it silently skipped re-optimization when `WASM_OPT_FLAGS` or the wasm-opt version changed. -The flag set is `-Oz --mvp-features --enable-mutable-globals --strip-debug --strip-producers`. Starting from `--mvp-features` stops wasm-opt from emitting post-MVP instructions (bulk memory, SIMD, reference types) that rippled's engine would reject. +The flag set is `-Oz --mvp-features --strip-debug --strip-producers` — **pure MVP, deliberately narrower than `wasm32v1-none`.** -**Sign-extension is deliberately not enabled, even though `wasm32v1-none` allows it.** rippled rejects `i32.extend8_s`/`i64.extend16_s` and friends with `temINVALID_BYTECODE`, so the accepted feature set is narrower than the target's. This is not theoretical: with `--enable-sign-ext` the e2e suite failed on exactly the four contracts where `-Oz` chose to introduce such an instruction (`gas_benchmark`, `trace_escrow_account`, `trace_escrow_finish`, `trace_escrow_ledger_object`) and passed on the twelve where it didn't. Note that rustc itself emits no sign-extension instructions for any current contract — wasm-opt was the only source — so this is latent for hand-written code too: if a future rustc or a new contract starts emitting them, it will fail on-ledger with nothing to do with wasm-opt. Dropping the flag costs nothing; it made output 135 bytes smaller overall, since the `target_features` section shrinks too. +The authority for this is `wasm_engine()` in rippled's `crates/xrpl-wasm-vm/src/vm.rs`. Contracts run on **wasmi**, configured for consensus determinism: every post-MVP proposal is explicitly switched off, one line each — `wasm_sign_extension(false)`, `wasm_mutable_global(false)`, `wasm_multi_value(false)`, `wasm_saturating_float_to_int(false)`, `wasm_bulk_memory(false)`, `wasm_reference_types(false)`, `wasm_tail_call(false)`, `wasm_extended_const(false)`, `wasm_multi_memory(false)`, `wasm_wide_arithmetic(false)`, `wasm_custom_page_sizes(false)`, plus `floats(false)` and `allow_start_fn(false)`. The reason is consensus, not engine capability: the accepted module set has to be fixed so every validator accepts exactly the same bytes, so the posture is deny-by-default and a module using anything beyond MVP is `temINVALID_BYTECODE`. **The target's feature list is not the authority — that file is.** Mirror it; don't infer the allowed set from `wasm32v1-none`. -Don't widen the flag set without checking what the host actually accepts — the target's feature list is not the authority here. +Sign-extension is the one that has actually bitten us. `-Oz` introduces `i32.extend8_s`/`i64.extend16_s` on its own, and with `--enable-sign-ext` the e2e suite failed on exactly the four contracts where it chose to (`gas_benchmark`, `trace_escrow_account`, `trace_escrow_finish`, `trace_escrow_ledger_object`) and passed on the other twelve. rustc emits none for any current contract, so wasm-opt was the sole source — but that also makes it latent for hand-written code: `wasm32v1-none` advertises sign-ext, so a future rustc or a new contract could emit it and fail on-ledger with wasm-opt uninvolved. + +`--enable-mutable-globals` was also dropped. Modules do contain one _internal_ mutable global (the shadow stack pointer), which is fine — wasmi's `wasm_mutable_global(false)` only rejects _imported/exported_ mutable globals, and the two globals these contracts export (`__data_end`, `__heap_base`) are immutable. So the flag was a no-op: removing it left output byte-identical. It's gone anyway, because with it present wasm-opt would be permitted to export a mutable global and that would be an on-ledger rejection; without it, the same situation is a loud build-time wasm-opt failure. Same reasoning for floats: `float_tests` contains zero `f32`/`f64` instructions because XRPL float math goes through host functions (`OpaqueFloat`), which is what makes `floats(false)` survivable. + +The `target_features` custom section (36 bytes/module) is deliberately **not** stripped, even though `--strip-target-features` would save ~576 bytes total. wasm-opt rewrites it to match the real feature set, so post-fix it reads exactly `+mutable-globals` and nothing else — a cheap, greppable record of the host-compatibility contract. rippled ignores it (`ignore_custom_sections(true)`). + +`-Oz` specifically is a free choice, not a tuned one: because cargo already applies `opt-level = "s"` + LTO, `-Oz`/`-Os`/`-O3` land within ~0.1% of each other on both size and instruction count, and `-Oz` is never worse on instruction count (on `keylet_exists` it is both smaller and leaner: 10,635 B / 3,891 instrs vs `-O3`'s 11,067 B / 4,088). So the usual "`-Oz` trades gas for size" worry does not apply here — relevant because the engine meters per-instruction fuel. The **tool** is mandatory — `scripts/build.sh` calls `wasm-opt.sh` outside its release/debug branch, so even a debug build fails if `wasm-opt` can't be obtained. It resolves one from `$WASM_OPT`, then `PATH`, then `$CARGO_HOME/bin`; finding none it runs `cargo install wasm-opt --version --locked` and fails the build if that fails. Having no release WASM to optimize is a different matter and exits 0 quietly, so a debug-only build isn't broken by the step — but a dir or file named explicitly on the command line that's missing or holds no `.wasm` is treated as a typo and fails. The pinned crate version lives in that script and is the single source of truth: `scripts/setup.sh` and both CI workflows call `./scripts/wasm-opt.sh --ensure-tool` to resolve/install up front rather than stalling a build mid-way. CI's `~/.cargo/bin` cache carries the compiled binary between runs. diff --git a/scripts/wasm-opt.sh b/scripts/wasm-opt.sh index 138fd9e8..f333c89a 100755 --- a/scripts/wasm-opt.sh +++ b/scripts/wasm-opt.sh @@ -115,12 +115,13 @@ if [[ "$ENSURE_ONLY" == "true" ]]; then exit 0 fi -# Narrower than wasm32v1-none on purpose: rippled rejects sign-extension with -# temINVALID_BYTECODE, and -Oz will happily introduce it. +# Pure MVP, narrower than wasm32v1-none on purpose: rippled's wasmi engine turns +# every post-MVP proposal off for consensus determinism, so anything extra is a +# temINVALID_BYTECODE on-ledger. Mirrors wasm_engine() in rippled's +# crates/xrpl-wasm-vm/src/vm.rs - see AGENTS.md before touching. DEFAULT_WASM_OPT_FLAGS=( -Oz --mvp-features - --enable-mutable-globals --strip-debug --strip-producers )