Skip to content

Fix CTR slowdown with aes 0.9.4's AVX-512 VAES backend - #5

Open
z7vian wants to merge 2 commits into
rjriajul:masterfrom
z7vian:ctr-wide-backend
Open

z7vian wants to merge 2 commits into
rjriajul:masterfrom
z7vian:ctr-wide-backend

Conversation

@z7vian

@z7vian z7vian commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What

CtrProcess (src/lib.rs) keeps the counter in a local u128 while it fills a batch and writes it back to iv once at the end, instead of storing it to iv after every block and loading it again for the next one. Output, iv and state are unchanged byte for byte.

Why

Cargo.toml asks for aes = "0.9.2" with no upper bound, so anyone on Rust 1.89+ gets aes 0.9.4. aes 0.9.4 turns its VAES backends on by default and picks them at runtime. In 0.9.2 they already existed, but only behind --cfg aes_backend="avx256" or "avx512", so master was also slow for anyone building with that flag. On x86 CPUs with AVX-512 and VAES, the backend encrypts 64 blocks per batch (0.9.2's default AES-NI backend does 8). With it, ctr256 and Ctr256 ran about 2.6x slower: 5.9 → 2.3 GB/s at 512 KiB on an Intel Xeon Platinum 8370C, and the same on AMD Zen 4.

Cause: in the batch loop (src/lib.rs lines 247–250 on master, *k = block_from_slice(ctr); ctr_next(ctr);), ctr_next stores the counter as two 8-byte halves and block_from_slice reads it back as one 16-byte load. That load can't be forwarded from two smaller stores, so every block waited for the previous block's store to reach cache, one after another. With 8-block batches LLVM fully unrolls the loop and keeps the counter in registers, so this never showed up on 0.9.2's default backend. With 64 blocks it keeps the loop, and the counter goes through memory (checked in the disassembly of the 0.9.4 build). After this change the AVX-512 backend is 40–56% faster than on 0.9.2, where before it was 60% slower.

Machines with only AES-NI, or with 256-bit VAES and no AVX-512 (0.9.4 uses 30-block batches there), aren't slowed down (the laptop numbers below are noisy; if anything they're faster).

Benchmarks

MB/s, median of 3 passes, release build with lto = "fat", codegen-units = 1. Each column is a separate crate depending on WarpCrypto with aes pinned to =0.9.2 or =0.9.4. "master" is d6adf6f. "this PR" is the first commit (9419218), which cleared the batch buffer once per call. The second commit clears it once per batch again, as master does, so short calls skip the 1 KiB clear. In its 0.9.4 build the counter still stays in registers, and the per-batch clear is 16 512-bit stores per 64 blocks. A rerun on the Zen 3 laptop showed no difference between the two commits beyond its noise. The Zen 4 machine wasn't rerun.

AMD EPYC, Zen 4 (family 25 model 17; AVX-512 + VAES → 64-block backend on 0.9.4), Linux, rustc 1.99.1

op size master, aes 0.9.2 master, aes 0.9.4 this PR, aes 0.9.2 this PR, aes 0.9.4
ctr256 1 KiB 4189 2047 4184 5850
ctr256 64 KiB 5678 2216 5678 8138
ctr256 512 KiB 5706 2218 5693 8184
Ctr256 1 KiB 5546 2301 5523 7908
Ctr256 64 KiB 5712 2303 5687 8869
Ctr256 512 KiB 5712 2306 5696 8893
ige256_encrypt 1 KiB 830 866 830 865
ige256_encrypt 64 KiB 852 872 852 872
ige256_encrypt 512 KiB 852 872 852 872
ige256_decrypt 1 KiB 825 865 824 865
ige256_decrypt 64 KiB 853 872 852 872
ige256_decrypt 512 KiB 852 872 853 872

AMD Ryzen 5 PRO 5675U, Zen 3 (VAES, no AVX-512 → 30-block backend on 0.9.4), Windows, rustc 1.97.1. This is a laptop that was busy with other work, so expect about ±15% noise.

op size master, aes 0.9.2 master, aes 0.9.4 this PR, aes 0.9.2 this PR, aes 0.9.4
ctr256 1 KiB 1774 1694 1732 2034
ctr256 64 KiB 2206 2370 2448 2730
ctr256 512 KiB 2110 2027 2459 2520
Ctr256 1 KiB 2076 1911 2656 2539
Ctr256 64 KiB 2045 2271 2491 2861
Ctr256 512 KiB 2134 2044 2490 2511
ige256_encrypt 512 KiB 502 520 509 508
ige256_decrypt 512 KiB 510 504 516 516

IGE doesn't use the batch path, and its numbers don't change.

Tests

  • New ctr_matches_reference_across_batch_widths test: checks ctr256 and Ctr256 against a one-block-at-a-time CTR built straight on aes. It covers lengths around the 8-, 30- and 64-block batch sizes, three starting states, data starting off 16-byte alignment, and counters that carry into the high 64 bits or wrap past 2^128 in the middle of a batch. The last case also wraps inside the 4-block batches of the software backend. Under Miri it uses a smaller set of lengths.
  • cargo test (debug and --release) and cargo clippy --all-targets -- -D warnings pass with aes 0.9.2 (the locked version) and 0.9.4 on Windows/Zen 3, which exercises the 30-block backend, on both commits. The first commit also passed on Linux/Zen 4, which exercises the 64-block one.
  • cargo +1.85 test --release passes (MSRV), checked on the first commit.
  • RUSTFLAGS='--cfg aes_backend="soft"' cargo +nightly miri test --test rust passed all 14 tests on the first commit, on Linux. On the second commit, the 7 CTR tests (ctr_ filter) also pass under Miri, on Windows.

Cargo.lock stays on aes 0.9.2 so the 1.85 CI job keeps working, and the wheels built from it don't change. The speedup reaches crates that resolve aes 0.9.4 themselves. No public API changes.

If you want this out as a patch release, it would fit as 2.2.1. I've left the version alone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant