Skip to content

tests: add tests for recursive functions that require spills - #715

Merged
PavelKopyl merged 4 commits into
mainfrom
kpv-spills-in-recursive-functions
Oct 28, 2026
Merged

PavelKopyl merged 4 commits into
mainfrom
kpv-spills-in-recursive-functions

Conversation

@PavelKopyl

@PavelKopyl PavelKopyl commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

@PavelKopyl
PavelKopyl marked this pull request as draft September 16, 2026 23:22
@PavelKopyl PavelKopyl added ci:integration Trigger integration tests workflow on PR ci:integration-benchmark-full Run the full benchmark matrix (all compilers + comparisons) in integration tests labels Sep 16, 2026
@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

🧪 Integration tests — standard · PR vs main

⚠️ Output changed — 132 of 28,254 size comparisons differ (+170 B total); 74 of 36,563 solx-tester gas comparisons differ. If this PR is meant to be output-preserving, investigate before merging.
✅ No new failures — Foundry's 414 failures already present on main.
⚠️ No baseline — solx-tester: 84 runs (0 failures); Foundry: 151 runs (0 failures) have no main counterpart; their failures are not compared.

Suite New failures Size Δ Gas Δ Report
solx-tester ✅ 0 ⚠️ 124 of 26,494 (+74 B) ⚠️ 74 of 36,563 solx-tester-report.xlsx ↓
Foundry · 26 proj ✅ 0 (414 pre-existing) ⚠️ 8 of 1,760 (+96 B) ⚪ jitter 29 of 3,150, median <0.1% (not gated) foundry-report.xlsx ↓
Hardhat · 3 proj ✅ 0 ⚪ not collected ⚪ not collected hardhat-report.xlsx ↓

solx-tester — largest size changes:

  • tests/solidity/simple/mlir/recursive_fnptr_dispatch.sol::default [EVMLA M3B3 0.8.37, deploy] 1,102 → 1,119 B (+1.5%)
  • tests/solidity/simple/mlir/recursive_fnptr_dispatch.sol::default [EVMLA M3B3 0.8.37, runtime] 1,075 → 1,092 B (+1.6%)
  • solidity/test/libsolidity/semanticTests/structs/conversion/recursive_storage_memory_complex.sol [EVMLA MzB3 0.8.37, deploy] 1,416 → 1,428 B (+0.8%)
  • tests/solidity/simple/mlir/recursive_struct_mutual.sol::default [EVMLA MzB3 0.8.37, deploy] 529 → 541 B (+2.3%)
  • tests/solidity/simple/mlir/recursive_struct_mutual.sol::default [EVMLA MzB3 0.8.37, runtime] 502 → 514 B (+2.4%)
  • +119 more — see solx-tester-report.xlsx

solx-tester — largest gas changes:

  • tests/solidity/simple/recursion/fibonacci.sol::sixteenth[fib:1] [EVMLA M3B3 0.8.37] 249,200 → 253,991 (+1.9%)
  • tests/solidity/simple/recursion/fibonacci.sol::sixteenth[fib:1] [EVMLA MzB3 0.8.37] 249,200 → 253,991 (+1.9%)
  • solidity/test/libsolidity/semanticTests/structs/conversion/recursive_storage_memory.sol[run:1] [EVMLA M3B3 0.8.37] 165,654 → 166,257 (+0.4%)
  • solidity/test/libsolidity/semanticTests/structs/conversion/recursive_storage_memory.sol[run:1] [EVMLA MzB3 0.8.37] 167,589 → 167,790 (+0.1%)
  • tests/solidity/simple/algorithm/tree/treap.sol::one[one:1] [EVMLA M3B3 0.8.37] 24,047 → 24,238 (+0.8%)
  • +69 more — see solx-tester-report.xlsx

Foundry — largest size changes:

  • Bytes32ArraysMock [legacy, deploy] 1,244 → 1,261 B (+1.4%)
  • Bytes32ArraysMock [legacy, runtime] 832 → 849 B (+2.0%)
  • Uint256ArraysMock [legacy, deploy] 2,230 → 2,247 B (+0.8%)
  • Uint256ArraysMock [legacy, runtime] 1,818 → 1,835 B (+0.9%)
  • AddressArraysMock [legacy, deploy] 1,471 → 1,484 B (+0.9%)
  • +3 more — see foundry-report.xlsx

Compile time — wall-clock tripwire, positive = PR slower (authoritative Δ in the compile-time benchmark comment)

Suite legacy (agg / median)
Foundry · 26 proj -0.1% / -0.1%
Hardhat · 3 proj +2.0% / +4.1%

Within noise — no suite ≥ 5%, no project ≥ 15%.

@nebasuke nebasuke added the ci:release Trigger dry-run release workflow on PR label Sep 17, 2026
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from 37e80fa to fcdbc32 Compare September 17, 2026 10:41
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from 0415d02 to c611122 Compare October 22, 2026 10:54
@PavelKopyl
PavelKopyl changed the base branch from main-solc to main October 22, 2026 10:55
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from c611122 to 863c8c1 Compare October 22, 2026 11:04
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Compile-time benchmark (--standard-json)

pr: slang v0.2.0, LLVM-based Solidity compiler for the EVM, Front end: Slang, LLVM build: f8b6e392ba671764eb73efff3db33751f4ce7c0a
main: slang v0.2.0, LLVM-based Solidity compiler for the EVM, Front end: Slang, LLVM build: 92d37cac15a93a3e0d4a127a664c5eba63224324
prerelease: slang v0.2.0, LLVM-based Solidity compiler for the EVM, Front end: Slang, LLVM build: 716613b727e0de5b086556ca69b475aa3e496c06
runner: solx-bench-ams-1
commits: pr 003296e8 merged into main, main 0efe7c8d

ens-verifiable-factory-solx--solx-legacy-dwarf

binary median mean ± σ min … max vs baseline
pr 0.283 s 0.286 s ± 0.005 s 0.282 s … 0.294 s 0.975×
main 0.291 s 0.292 s ± 0.006 s 0.283 s … 0.302 s 1.000×
prerelease 0.497 s 0.496 s ± 0.004 s 0.489 s … 0.501 s 1.711×

openzeppelin-contracts-0.34--solx-legacy-dwarf

binary median mean ± σ min … max vs baseline
pr 2.198 s 2.199 s ± 0.005 s 2.189 s … 2.207 s 0.997×
main 2.205 s 2.205 s ± 0.007 s 2.191 s … 2.216 s 1.000×
prerelease 3.203 s 3.201 s ± 0.018 s 3.175 s … 3.227 s 1.452×

uniswap-v4-core-solx--solx-legacy-dwarf

binary median mean ± σ min … max vs baseline
pr 0.953 s 0.954 s ± 0.011 s 0.943 s … 0.972 s 0.986×
main 0.967 s 0.963 s ± 0.012 s 0.946 s … 0.976 s 1.000×
prerelease 2.133 s 2.128 s ± 0.026 s 2.086 s … 2.162 s 2.206×

@PavelKopyl PavelKopyl added ci:compile-benchmark Run hyperfine compilation benchmark for this PR and removed ci:release Trigger dry-run release workflow on PR ci:integration-benchmark-full Run the full benchmark matrix (all compilers + comparisons) in integration tests labels Oct 22, 2026
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch 3 times, most recently from f9eaa30 to 2e7e3eb Compare October 23, 2026 20:47
@PavelKopyl
PavelKopyl marked this pull request as ready for review October 24, 2026 07:12
@PavelKopyl
PavelKopyl marked this pull request as draft October 24, 2026 07:13
nebasuke added a commit that referenced this pull request Oct 25, 2026
OpenZeppelin builds a `.yul` source since its commit 598d3905, and the slang
compiler cannot compile Yul, so the Foundry pin stops at the commit before it.
The Hardhat pin keeps the newer commit, as hardhat-yul compiles that source
outside solx.

seaport fails on a recursive internal dispatch that is stack too deep, which
#715 addresses; it stays disabled until then, with the
memory-safety check off for the other stack-too-deep errors it reports.
nebasuke added a commit that referenced this pull request Oct 25, 2026
OpenZeppelin builds a `.yul` source since its commit 598d3905, and the slang
compiler cannot compile Yul, so the Foundry pin stops at the commit before it.
The Hardhat pin keeps the newer commit, as hardhat-yul compiles that source
outside solx.

seaport fails on a recursive internal dispatch that is stack too deep, which
#715 addresses; it stays disabled until then, with the
memory-safety check off for the other stack-too-deep errors it reports.
nebasuke added a commit that referenced this pull request Oct 26, 2026
OpenZeppelin builds a `.yul` source since its commit 598d3905, and the slang
compiler cannot compile Yul, so the Foundry pin stops at the commit before it.
The Hardhat pin keeps the newer commit, as hardhat-yul compiles that source
outside solx.

seaport fails on a recursive internal dispatch that is stack too deep, which
#715 addresses; it stays disabled until then, with the
memory-safety check off for the other stack-too-deep errors it reports.
nebasuke added a commit that referenced this pull request Oct 26, 2026
OpenZeppelin builds a `.yul` source since its commit 598d3905, and the slang
compiler cannot compile Yul, so the Foundry pin stops at the commit before it.
The Hardhat pin keeps the newer commit, as hardhat-yul compiles that source
outside solx.

seaport fails on a recursive internal dispatch that is stack too deep, which
#715 addresses; it stays disabled until then, with the
memory-safety check off for the other stack-too-deep errors it reports.
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from 2e7e3eb to c6bf6ba Compare October 26, 2026 21:34
@PavelKopyl PavelKopyl added the ci:sourcify-sweep Run Sourcify corpus on this PR label Oct 26, 2026
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from c6bf6ba to 9e652e9 Compare October 26, 2026 21:51
@PavelKopyl
PavelKopyl marked this pull request as ready for review October 26, 2026 21:51
@PavelKopyl
PavelKopyl requested a review from hedgar2017 October 26, 2026 21:51
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Sourcify sweep

  • candidate: slang (slang v0.2.0, LLVM-based Solidity compiler for the EVM, Front end: Slang, LLVM build: f8b6e392ba671764eb73efff3db33751f4ce7c0a)
  • reference: solx-latest (solx v0.1.8, LLVM-based Solidity compiler for the EVM, Front end: solc, LLVM build: 7d0702e169889fe4f1a2241c57bef7d2c1c68737)
  • contracts: 119117
outcome count share meaning
ok 118920 99.83% compiles: no errors and bytecode for the target source
candidate-fail 35 0.03% candidate fails, reference compiles
both-fail 162 0.14% candidate and reference both fail

Candidate-only failures (35)

kinds: error 35

count kind signature examples
35 error This contract cannot be compiled due to a combination of a memory-unsafe assembly block and a stack-too-deep error. The compiler can automatically fix the stack 10_0x33442fc10a20aad0ddd73f6ae24500f5b370dc51, 10_0x3e7b6997399ec402491c4a049e4cd727d3aa1738, 10_0x467837f44a71e3eab90aecfc995c84dc6b3cfcf7 (+32 more)

Failures shared with the reference (reference diagnostics) (162)

kinds: error 162

count kind signature examples
156 error runtime code LLVM IR generator: This contract cannot be compiled due to a combination of a memory-unsafe assembly block and a stack-too-deep error. 100_0x239d413a6ac5322d3ccaaaf43e34045bdacd7e74, 100_0x27ebdb6cefd590feaf79b20f6e77bf79dc3c04fe, 100_0x325afb837204d46a3d4158ded26a8be2681761b5 (+153 more)
5 error Undeclared identifier. 11155111_0x0dcf68b5416eaaf96e896809a33c404f30a06c7c, 11155111_0x74908cc7dc1416fc57fd4ef65d849020b18253b8, 11155111_0xc1335a802c09380d26202f0178464c3ef4e48fce (+2 more)
1 error Member "UnexpectedInitSelector" not found or not visible after argument-dependent lookup in type(contract ParkERC1967Proxy). 56_0x69d890f783a6662e8554c8f71d1daeda1baddda2

Comment thread solx-dev/src/test/foundry/mod.rs Resolved
Comment thread solx-dev/foundry-tests.toml Resolved
Comment thread solx/tests/cli/stack_too_deep.rs Resolved
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from 9e652e9 to 7f92a6e Compare October 27, 2026 13:35

@hedgar2017 hedgar2017 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment thread solx-dev/src/utils.rs Resolved
Comment thread solx-dev/foundry-tests.toml Resolved
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from 7f92a6e to 7b420f7 Compare October 27, 2026 15:07
@PavelKopyl PavelKopyl removed the ci:sourcify-sweep Run Sourcify corpus on this PR label Oct 27, 2026
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch 2 times, most recently from ee28012 to 098aad4 Compare October 27, 2026 16:11
@PavelKopyl PavelKopyl added the ci:sourcify-sweep Run Sourcify corpus on this PR label Oct 27, 2026
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from 098aad4 to 5ef587c Compare October 27, 2026 17:57

@nebasuke nebasuke left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good!

There is one swap-vm failure (test_TWAP_CumulativeSoldTracking_Multiple_Swaps) but this is an upstream test bug. (I'll follow up with a PR so this is more visible in the integration test comment next time.)

nebasuke added a commit that referenced this pull request Oct 28, 2026
A PR run whose main counterpart failed to build was paired for the build
comparison, but main's test count was never measured, so the PR's test
failures were skipped without appearing anywhere in the summary. #715 hit
this: main could not build swap-vm, the PR could, and the one failing
swap-vm test showed up in neither the new, pre-existing nor unbaselined
counts. Such runs now count as unbaselined, so their failures reach the
"No baseline" line and the table's failure cell.
nebasuke added a commit that referenced this pull request Oct 28, 2026
A PR run whose main counterpart failed to build was paired for the build
comparison, but main's test count was never measured, so the PR's test
failures were skipped without appearing anywhere in the summary. #715 hit
this: main could not build swap-vm, the PR could, and the one failing
swap-vm test showed up in neither the new, pre-existing nor unbaselined
counts. Such runs now count as unbaselined when the PR's tests fail, so
their failures reach the "No baseline" line and the table's failure cell.
A repaired build whose tests all pass leaves nothing uncompared and stays
out of it. The line now says "no `main` baseline", which holds for both
an unpaired run and a paired one whose main side never ran its tests.
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from 5ef587c to d45eb72 Compare October 28, 2026 14:49
nebasuke added a commit that referenced this pull request Oct 28, 2026
A PR run whose main counterpart failed to build was paired for the build
comparison, but main's test count was never measured, so the PR's test
failures were skipped without appearing anywhere in the summary. #715 hit
this: main could not build swap-vm, the PR could, and the one failing
swap-vm test showed up in neither the new, pre-existing nor unbaselined
counts. Such runs now count as unbaselined when the PR's tests fail, so
their failures reach the "No baseline" line and the table's failure cell.
A repaired build whose tests all pass leaves nothing uncompared and stays
out of it. The line now says "no `main` baseline", which holds for both
an unpaired run and a paired one whose main side never ran its tests.
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from d45eb72 to cbafcc5 Compare October 28, 2026 16:06
PavelKopyl and others added 4 commits October 28, 2026 20:42
…ills

The llvm pin is bumped to the version that supports spilling in
recursive functions.
The 0.8.34 Sourcify sweep found twelve deployed contracts that solc
compiles and solx rejected with a stack-too-deep error in a recursive
function. They reduce to four distinct functions, so one deployment per
function is kept.

Each test keeps the mode that reproduced the rejection and asserts that
the standard JSON output holds bytecode and no stack-too-deep message.

Co-authored-by: Bas van Gijzel <bas@nomic.foundation>
A dependency can pin an exact pragma, so a project names the installed
directories to rewrite; everything else keeps the tracked-only rewrite.
@PavelKopyl
PavelKopyl force-pushed the kpv-spills-in-recursive-functions branch from cbafcc5 to 003296e Compare October 28, 2026 19:42
@PavelKopyl
PavelKopyl added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit d95c115 Oct 28, 2026
72 checks passed
@PavelKopyl
PavelKopyl deleted the kpv-spills-in-recursive-functions branch October 28, 2026 22:18
nebasuke added a commit that referenced this pull request Nov 2, 2026
OpenZeppelin builds a `.yul` source since its commit 598d3905, and the slang
compiler cannot compile Yul, so the Foundry pin stops at the commit before it.
The Hardhat pin keeps the newer commit, as hardhat-yul compiles that source
outside solx.

seaport fails on a recursive internal dispatch that is stack too deep, which
#715 addresses; it stays disabled until then, with the
memory-safety check off for the other stack-too-deep errors it reports.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:compile-benchmark Run hyperfine compilation benchmark for this PR ci:integration Trigger integration tests workflow on PR ci:sourcify-sweep Run Sourcify corpus on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants