Skip to content

arm64: Decode ARMv8.1-A LSE atomics and ARMv8.3-A LDAPR - #96

Open
zardus wants to merge 1 commit into
masterfrom
feature/aarch64-lse-atomics
Open

arm64: Decode ARMv8.1-A LSE atomics and ARMv8.3-A LDAPR#96
zardus wants to merge 1 commit into
masterfrom
feature/aarch64-lse-atomics

Conversation

@zardus

@zardus zardus commented Aug 29, 2026

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

The AArch64 front end decodes no ARMv8.1-A LSE atomic and no ARMv8.3-A LDAPR, so a block containing one lifts to nothing at all:

$ pyvex.lift(bytes.fromhex("4100e0b8"), 0x1000, ArchAArch64(), max_bytes=4)   # ldaddal w0, w1, [x2]
  size=0  instructions=0  jumpkind=Ijk_NoDecode

A corpus census over two multi-architecture collections found 307 objects carrying at least one declared function start that refuses this way, 3,789 such starts, and confirmed 4,549 of them Ijk_NoDecode in context through angr.Project at the real load address -- 100% of what it re-checked. Because the refusal lands on the function's first instruction, the consumer gets no bytes and no address to resume from, and the function disappears from analysis rather than being truncated.

Root cause

priv/guest_arm64_toIR.c forked from Valgrind before these instructions existed; its arm64 front end matches valgrind 3.15.0, and f1cf73455, which added LSE, landed on 2019-11-20, after 3.15.0 shipped. dis_ARM64_load_store therefore falls through to

   vex_printf("ARM64 front end: load_store\n");
   return False;

for every one of them, and disInstr_ARM64 turns that into dres.len = 0 with Ijk_NoDecode.

Fix

Port the decode blocks upstream Valgrind already carries. LDADD, LDCLR, LDEOR, LDSET, LDSMAX, LDSMIN, LDUMAX, LDUMIN and SWP come from f1cf73455 with the swp/ldumax corrections of 6052ee66a; CAS from 2281c8c86; CASP from 9608e8019; LDAPR/LDAPRB/LDAPRH from 41e2f95cf. All four sizes and all four orderings decode. The ported region is byte-identical to upstream master apart from gen_SIGBUS_if_not_XX_aligned, which arrived in 2be719921 after this fork's base and which the neighbouring LDXR and LDAR blocks here still carry as a FIXME rather than as a check; adding it for the new blocks alone would leave the file inconsistent and would give every atomic a side exit on a jumpkind pyvex's consumers do not model. widenSto64 comes along because the LSE block needs it, and the LDAPR disassembly text says ldapr rather than reusing the ldar spelling upstream left in place.

The atomics lift the way upstream models them: a barrier for the acquire variants, a load, the operation, an Ist_CAS, a barrier for the release variants, and an Ijk_Boring exit back to the instruction when the CAS reported a stale value. That is the same shape guest_amd64_toIR.c already emits for a lock-prefixed instruction.

ARMv8.4-A RCpc2 -- LDAPUR, STLUR and their sized forms -- is deliberately not included: upstream has no implementation either, only an open unpatched report at https://bugs.kde.org/show_bug.cgi?id=496477. It is 271 of the 3,789 refusing starts the census measured, so this change closes 92.8% of them.

Testing

The regression lands with the consumer, angr/pyvex#579, which lifts every word of a compiled ARMv8.4-A fixture's .text and asserts that the only words still refused are the RCpc2 group. On the merge base 97 of its words refuse; here 8 do. An A/B over 2,081,200 distinct AArch64 encodings -- the executable sections of every AArch64 ELF in angr/binaries, the LDAR/STLR block's whole free-bit space exhaustively, and a fixed-seed uniform sample of the 32-bit space -- reports a difference on 2,480 encodings, every one of them previously Ijk_NoDecode. Validation: #96 (comment)

Merge order: this lands first, then angr/pyvex#579, whose vex submodule is pinned to this head (846d111). Merging the pyvex half on its own leaves pyvex master's submodule pointing at a commit that is on no vex branch. Nothing in this repository reads a sync: line — .github/workflows/build.yml is the only workflow and it is three build jobs — so the line below is a note to the reader.

sync: angr/pyvex#579

session: sharpen

This front end forked from Valgrind before ARMv8.1-A existed, so every
LD<OP>, SWP, CAS, CASP and LDAPR encoding falls through dis_ARM64_load_store
and the whole block comes back Ijk_NoDecode with size 0.

Port the decode blocks upstream already carries: f1cf73455 with the swp and
ldumax corrections of 6052ee66a for the atomic memory operations, 2281c8c86
for CAS, 9608e8019 for CASP and 41e2f95cf for LDAPR. The ported region is
byte-identical to upstream master except for gen_SIGBUS_if_not_XX_aligned,
which arrived after this fork's base and which the LDXR and LDAR blocks here
still carry as a FIXME rather than as a check.

ARMv8.4-A RCpc2 (LDAPUR, STLUR) is not included; upstream has no
implementation of it either, only https://bugs.kde.org/show_bug.cgi?id=496477.
@zardus

zardus commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 846d11116dd782a3c04a91d4964701d304e9a5b2 against baseline 875f7c9a5f6be621b4f000c29c016e15ddf32207.

Provenance. Both arms are cmake --build of the same tree in one worktree; the baseline arm ran libpyvex.so sha256 648eb18a78537e54a8fe41c0a0b4aef95bdec8391417e93717a7e4e81c65a537, the head arm 531ba2079a862d30903462c07eab60b49e9c1fe0706d8244cc6c61eff9e3324e. Each arm prints the sha256 of the library it actually mapped, read out of /proc/self/maps.

Fidelity to upstream. The ported region -- from ARMv8.1-LSE: Atomic Memory Operations to the end of dis_ARM64_load_store -- is 229 lines on both sides and diffs to zero lines against VEX/priv/guest_arm64_toIR.c at valgrind c7c8188ff, once the four gen_SIGBUS_if_not_XX_aligned calls are normalised against the FIXME comments that replace them.

A/B over the encoding space. 2,081,200 distinct 4-byte encodings, identical population in both arms: every distinct word in the executable sections of all 17 AArch64 ELF objects tracked in angr/binaries at a87538bc248531d2a1d22434143e8b9752459bac, the LDAR/STLR block's entire free-bit space exhaustively (8,192), and a random.Random(20260829) uniform sample of the 32-bit space (2,000,000). Each encoding is lifted with max_bytes=4, opt_level=0 and recorded as size, jumpkind and sha1 of the printed IRSB.

differing encodings                                2480
  base Ijk_NoDecode -> head Ijk_Boring             2480
differing encodings that previously DECODED           0
  class LSE LD<OP>/SWP                             2189
  class LSE CAS                                     251
  class LSE CASP                                     39
  class RCpc LDAPR                                     1
  class OTHER                                          0

LDAPR predicate, exhaustively. The widened LDAR/STLR block's new arm has a 4,096-encoding free-bit space (4 sizes x 32 Rn x 32 Rt); all 4,096 refuse on the baseline and all 4,096 lift at this head. The block's pre-existing arm, 8,192 encodings, is in the A/B above and differs on none of them.

Assembled-encoding probe. 230 distinct encodings assembled by llvm-mc 21.1.8 -triple=aarch64 -mattr=+lse,+rcpc,+rcpc-immo covering every LD<OP>/ST<OP>/SWP/CAS/CASP ordering and size plus the RCpc and RCpc2 loads and stores: 230 of 230 refuse on the baseline, 220 of 230 lift at this head. The 10 that still refuse are exactly the ARMv8.4-A RCpc2 group (LDAPUR, LDAPURB, LDAPURH, LDAPURSB, LDAPURSH, STLUR, STLURB, STLURH and the X-width forms).

Consumer suite. ANGR_FEATURE=lse-atomics nix develop --command ./.agents/skills/angr-validate-workspace/scripts/run-all-tests.sh in a feature instance that has adopted pyvex and binaries: pyvex 68 passed, every configured pre-commit hook over all files passed, and the workspace's own checks, the test-input check and the per-feature-instance suite passed. It skipped archinfo, pypcode, pysoot, cle, claripy, angr, angr-rust and angr-management, naming each, because the instance has not adopted them. Its final worktree-cleanliness check failed on .agents/skills/angr-sweep-corpus/scripts/sweepctl.sh and unit.py in the workspace repository, which another agent edited at 09:00:02 and 09:00:54 inside the run's 08:58:48-09:01:13 window; neither path is touched by this change. The three native libraries a live corpus sweep has mapped -- repos/angr/angr/unicornlib.so, repos/angr/target/release/librustylib.so and repos/pyvex/pyvex/lib/libpyvex.so -- were byte, inode and mtime identical before and after.

Corpus measurement. The census that motivated this ran over datasets/gors-bins and datasets/megabench: 307 objects carry at least one declared function start that libVEX refuses at an ARMv8.1-A LSE, ARMv8.3-A RCpc or ARMv8.4-A RCpc2 instruction, 3,789 such starts were recorded, and of the 277 objects that reached the in-context pass, 4,549 of 4,549 addresses lifted through angr.Project at the real load address came back Ijk_NoDecode. 271 of the 3,789 are RCpc2, which this change does not cover, so it closes 3,518 of 3,789 -- 92.8%. 280 of the 307 objects are redistribution: allowed; the fixture in angr/binaries is a compiled reproducer that needs none of them.

Not covered. ARMv8.4-A RCpc2 (LDAPUR/STLUR), unimplemented upstream as well -- https://bugs.kde.org/show_bug.cgi?id=496477 . The census also found LDXP/STXP and the unprivileged LDTR*/STTR* family refusing, which is baseline ARMv8.0 and a separate gap; upstream tracks it at https://bugs.kde.org/show_bug.cgi?id=444399 . The alignment checks upstream added in 2be719921 are deliberately absent, matching the LDXR and LDAR blocks in this fork.

session: sharpen

@zardus

zardus commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

One encoding per group, lifted with pyvex.lift(data, 0x400000, ArchAArch64(), max_bytes=4, opt_level=0), before and after this change. Each side prints the sha256 of the libpyvex.so the process actually mapped, read out of /proc/self/maps.

Before -- every one comes back as an empty block, so the caller gets no bytes and no address to resume from:

vex at 875f7c9
libpyvex sha256 648eb18a78537e54a8fe41c0a0b4aef95bdec8391417e93717a7e4e81c65a537

---- ldaddal w0, w1, [x2]   (4100e0b8) ----
size=0 instructions=0 jumpkind=Ijk_NoDecode
IRSB {
   

   NEXT: PUT(pc) = 0x0000000000400000; Ijk_NoDecode
}

---- swpal x0, x1, [x2]   (4180e0f8) ----
size=0 instructions=0 jumpkind=Ijk_NoDecode
IRSB {
   

   NEXT: PUT(pc) = 0x0000000000400000; Ijk_NoDecode
}

---- casal w0, w1, [x2]   (41fce088) ----
size=0 instructions=0 jumpkind=Ijk_NoDecode
IRSB {
   

   NEXT: PUT(pc) = 0x0000000000400000; Ijk_NoDecode
}

---- caspal x0, x1, x2, x3, [x4]   (82fc6048) ----
size=0 instructions=0 jumpkind=Ijk_NoDecode
IRSB {
   

   NEXT: PUT(pc) = 0x0000000000400000; Ijk_NoDecode
}

---- ldapr x0, [x1]   (20c0bff8) ----
size=0 instructions=0 jumpkind=Ijk_NoDecode
IRSB {
   

   NEXT: PUT(pc) = 0x0000000000400000; Ijk_NoDecode
}

---- ldapur w0, [x1, #4]   (20404099) ----
size=0 instructions=0 jumpkind=Ijk_NoDecode
IRSB {
   

   NEXT: PUT(pc) = 0x0000000000400000; Ijk_NoDecode
}

After -- the atomics become a barrier, a load, the operation, an Ist_CAS and a retry exit; ldapr becomes an acquiring load; ldapur, the ARMv8.4-A form, is unchanged and still refuses:

vex with this change
libpyvex sha256 531ba2079a862d30903462c07eab60b49e9c1fe0706d8244cc6c61eff9e3324e

---- ldaddal w0, w1, [x2]   (4100e0b8) ----
size=4 instructions=1 jumpkind=Ijk_Boring
IRSB {
   t0:Ity_I64 t1:Ity_I32 t2:Ity_I32 t3:Ity_I32 t4:Ity_I64 t5:Ity_I64 t6:Ity_I32 t7:Ity_I64 t8:Ity_I64 t9:Ity_I1 t10:Ity_I64 t11:Ity_I64 t12:Ity_I64 t13:Ity_I64

   00 | ------ IMark(0x400000, 4, 0) ------
   01 | t0 = GET:I64(x2)
   02 | MBusEvent-Imbe_Fence
   03 | t1 = LDle:I32(t0)
   04 | t7 = GET:I64(x0)
   05 | t6 = 64to32(t7)
   06 | t5 = 32Uto64(t6)
   07 | t8 = 32Uto64(t1)
   08 | t4 = Add64(t8,t5)
   09 | t3 = 64to32(t4)
   10 | t(2,4294967295) = CASle(t0 :: (t1,None)->(t3,None))
   11 | MBusEvent-Imbe_Fence
   12 | t10 = 32Uto64(t1)
   13 | t11 = 32Uto64(t2)
   14 | t9 = CasCmpNE64(t11,t10)
   15 | if (t9) { PUT(pc) = 0x400000; Ijk_Boring }
   16 | t12 = 32Uto64(t2)
   17 | PUT(x1) = t12
   18 | PUT(pc) = 0x0000000000400004
   19 | t13 = GET:I64(pc)
   NEXT: PUT(pc) = t13; Ijk_Boring
}

---- swpal x0, x1, [x2]   (4180e0f8) ----
size=4 instructions=1 jumpkind=Ijk_Boring
IRSB {
   t0:Ity_I64 t1:Ity_I64 t2:Ity_I64 t3:Ity_I64 t4:Ity_I1 t5:Ity_I64

   00 | ------ IMark(0x400000, 4, 0) ------
   01 | t0 = GET:I64(x2)
   02 | MBusEvent-Imbe_Fence
   03 | t1 = LDle:I64(t0)
   04 | t3 = GET:I64(x0)
   05 | t(2,4294967295) = CASle(t0 :: (t1,None)->(t3,None))
   06 | MBusEvent-Imbe_Fence
   07 | t4 = CasCmpNE64(t2,t1)
   08 | if (t4) { PUT(pc) = 0x400000; Ijk_Boring }
   09 | PUT(x1) = t2
   10 | PUT(pc) = 0x0000000000400004
   11 | t5 = GET:I64(pc)
   NEXT: PUT(pc) = t5; Ijk_Boring
}

---- casal w0, w1, [x2]   (41fce088) ----
size=4 instructions=1 jumpkind=Ijk_Boring
IRSB {
   t0:Ity_I64 t1:Ity_I32 t2:Ity_I32 t3:Ity_I64 t4:Ity_I32 t5:Ity_I64 t6:Ity_I64 t7:Ity_I64

   00 | ------ IMark(0x400000, 4, 0) ------
   01 | t0 = GET:I64(x2)
   02 | MBusEvent-Imbe_Fence
   03 | t3 = GET:I64(x0)
   04 | t2 = 64to32(t3)
   05 | t5 = GET:I64(x1)
   06 | t4 = 64to32(t5)
   07 | t(1,4294967295) = CASle(t0 :: (t2,None)->(t4,None))
   08 | MBusEvent-Imbe_Fence
   09 | t6 = 32Uto64(t1)
   10 | PUT(x0) = t6
   11 | PUT(pc) = 0x0000000000400004
   12 | t7 = GET:I64(pc)
   NEXT: PUT(pc) = t7; Ijk_Boring
}

---- caspal x0, x1, x2, x3, [x4]   (82fc6048) ----
size=4 instructions=1 jumpkind=Ijk_Boring
IRSB {
   t0:Ity_I64 t1:Ity_I64 t2:Ity_I64 t3:Ity_I64 t4:Ity_I64 t5:Ity_I64 t6:Ity_I64 t7:Ity_I64

   00 | ------ IMark(0x400000, 4, 0) ------
   01 | t0 = GET:I64(x4)
   02 | MBusEvent-Imbe_Fence
   03 | t3 = GET:I64(x1)
   04 | t4 = GET:I64(x0)
   05 | t5 = GET:I64(x3)
   06 | t6 = GET:I64(x2)
   07 | t(1,2) = CASle(t0 :: (t4,t3)->(t6,t5))
   08 | MBusEvent-Imbe_Fence
   09 | PUT(x0) = t1
   10 | PUT(x1) = t2
   11 | PUT(pc) = 0x0000000000400004
   12 | t7 = GET:I64(pc)
   NEXT: PUT(pc) = t7; Ijk_Boring
}

---- ldapr x0, [x1]   (20c0bff8) ----
size=4 instructions=1 jumpkind=Ijk_Boring
IRSB {
   t0:Ity_I64 t1:Ity_I64 t2:Ity_I64

   00 | ------ IMark(0x400000, 4, 0) ------
   01 | t0 = GET:I64(x1)
   02 | t1 = LDle:I64(t0)
   03 | PUT(x0) = t1
   04 | MBusEvent-Imbe_Fence
   05 | PUT(pc) = 0x0000000000400004
   06 | t2 = GET:I64(pc)
   NEXT: PUT(pc) = t2; Ijk_Boring
}

---- ldapur w0, [x1, #4]   (20404099) ----
size=0 instructions=0 jumpkind=Ijk_NoDecode
IRSB {
   

   NEXT: PUT(pc) = 0x0000000000400000; Ijk_NoDecode
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant