Skip to content

Add an x86_64 ELF whose hash tables are zeroed - #200

Open
zardus wants to merge 2 commits into
masterfrom
gnu-hash-resiliency-fixture
Open

Add an x86_64 ELF whose hash tables are zeroed#200
zardus wants to merge 2 commits into
masterfrom
gnu-hash-resiliency-fixture

Conversation

@zardus

@zardus zardus commented Aug 26, 2026

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

The linked cle change makes the loader survive an ELF whose GNU hash table declares no buckets — the shape some stripping and obfuscation tools leave behind when they blank an executable's DT_HASH and DT_GNU_HASH. Nothing in this repository has it, so that change's regression has no input and its reproducer is a path that does not exist:

ls: cannot access 'tests/x86_64/gnu_hash_resiliency_0': No such file or directory

Root cause

The fixture cannot be built, only derived. No linker emits an empty bucket array: a shared object that exports nothing ({ local: *; };, --hash-style=gnu) links to nbuckets = 1 under GNU ld 2.46 and under LLD 21.1.8 alike, because binutils special-cases a table with nothing to hash. Only corruption produces nbuckets = 0. A struct.pack-assembled header would test a shape no real toolchain emits, which is the failure mode that passes while the real format still fails.

Fix

tests/x86_64/gnu_hash_resiliency_0 is tests/x86_64/test_killing_ref — a real linker's output — with the 96 bytes its two hash tables occupy zeroed. Same 15,960 bytes, same mode, and cmp -l reports exactly 20 differing bytes, all between offsets 0x370 and 0x3cb. tests_src/gnu_hash_resiliency/build_gnu_hash_resiliency.py derives it from the committed input, so a reviewer can regenerate it from material already in this repository and needs no toolchain.

What the fixture makes observable, which its uncorrupted source does not:

test_killing_ref       .gnu.hash: nbuckets=2, symoffset=8, bloom_size=1, bloom=[8454144], buckets=[8, 0]
gnu_hash_resiliency_0  .gnu.hash: nbuckets=0, symoffset=0, bloom_size=0, bloom=[],        buckets=[]

cle.Loader('tests/x86_64/gnu_hash_resiliency_0')
  -> ValueError: max() iterable argument is empty   (elftools/elf/hash.py:160, unpatched cle)

The count a loader has to recover instead is checkable from the file alone, two independent ways: DT_SYMTAB 0x3d0 to DT_STRTAB 0x4a8 at DT_SYMENT 24 is 9 entries with no remainder, and DT_VERSYM 0x5fe plus two bytes per symbol for 9 symbols reaches 0x610, which is exactly DT_VERNEED.

Testing

No test suite in this repository covers fixtures, so nothing is executed here. What was checked instead: re-running the generator in place reproduced the committed file byte for byte and left git status empty, and with the cle change applied the fixture loads with 56 symbols, 30 sections, 11 relocations, 4 PLT entries and one dependency — identical to the uncorrupted test_killing_ref on every one of those, which is what the consumer's regression asserts.

Validation: #200 (comment)

sync: angr/cle#792

session: sharpen

@zardus

zardus commented Aug 27, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 6c822960527ab674459496dc0af9b698f60c422b against baseline
8646be4eafa4f1fc285d787fb2b73426a5e11d19. Reshaped in place on 2026-08-28 so the opening
line names the head, and one paragraph corrected — see the correction at the end. The defect
this fixture exercises, and the paired evidence for the loader change, are on angr/cle#792.

Provenance. tests/x86_64/gnu_hash_resiliency_0 is tests/x86_64/test_killing_ref with
the 96 bytes of its DT_HASH and DT_GNU_HASH tables zeroed. Same size, 15,960 bytes; same
mode; cmp -l reports exactly 20 differing bytes, all between offsets 0x370 and 0x3cb.
Everything else is the original linker's output, so this is a real toolchain artefact with an
observed corruption applied rather than a hand-assembled shape.

Reproducible from the committed input. tests_src/gnu_hash_resiliency/ carries the
generator. Re-running it in place produced a byte-identical file and left git status empty,
so the fixture can be regenerated by a reviewer from material already in this repository. It
needs no toolchain.

Why it could not be built instead. No linker emits an empty bucket array: linking an
empty-export DSO with GNU ld 2.46 and with LLD 21.1.8 both produce nbuckets=1, and binutils
special-cases the empty table. Only corruption produces nbuckets=0, which is why the fixture
is derived rather than compiled, and why a struct.pack-assembled header would have tested a
shape no real toolchain emits.

What it demonstrates. Its .gnu.hash reads nbuckets=0, symoffset=0, bloom_size=0, bloom=[], buckets=[], against nbuckets=2, symoffset=8, bloom_size=1, bloom=[8454144], buckets=[8, 0] on its uncorrupted source. On unpatched cle the load raises
ValueError: max() iterable argument is empty from pyelftools while counting dynamic symbols;
with angr/cle#792 the object loads with 56 symbols, 30 sections, 11 relocations, 4 PLT entries
and one dependency — identical to test_killing_ref on every one of those. The dynamic symbol
count the loader has to recover is checkable from the file alone, two independent ways:
(0x4a8 - 0x3d0) / 24 = 9 exactly, where 0x3d0 is DT_SYMTAB and 0x4a8 is DT_STRTAB,
and DT_VERSYM 0x5fe + 2*9 = 0x610, which is precisely DT_VERNEED.

Not run: no test suite in this repository covers fixtures, so nothing was executed here
beyond regenerating the file, reading its headers back and loading it. The decompilation and
load-comparison evidence is on angr/cle#792.


Correction (2026-08-28). The earlier version of the "What it demonstrates" paragraph gave
37 dynamic symbols, 33 relocations and 32 PLT entries, and derived the symbol count from
(0x4007f8 - 0x400480) / 24 with DT_VERSYM 0x400a16 and DT_VERNEED 0x400a60. Those
figures are not this file's. Re-read from the committed fixture at 6c82296, its dynamic tags
are DT_SYMTAB 0x3d0, DT_STRTAB 0x4a8, DT_VERSYM 0x5fe and DT_VERNEED 0x610, giving 9
dynamic symbols, and the loaded object reports 11 relocations and 4 PLT entries. The corrected
numbers are the ones in the paragraph above, and they are the same ones the validation record
on angr/cle#792 already carried. Nothing else in this record changed.

@zardus

zardus commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

What this fixture makes observable, before and after adding it. A fixture pull request has no test to run, so the two sides are the same reproducer against angr/binaries at master and at this branch: read the .gnu.hash header of tests/x86_64/test_killing_ref and of the derived tests/x86_64/gnu_hash_resiliency_0, then load the derived file through cle.Loader(path, auto_load_libs=False).

Before — the derived file does not exist, so the consuming loader change's regression has no input:

angr/binaries master (8646be4)
test_killing_ref      .gnu.hash: nbuckets=2, symoffset=8, bloom_size=1, bloom=[8454144], buckets=[8, 0]
gnu_hash_resiliency_0 FileNotFoundError: [Errno 2] No such file or directory: 'binaries/tests/x86_64/gnu_hash_resiliency_0'

cle.Loader(gnu_hash_resiliency_0) -> cle.errors.CLEFileNotFoundError: Could not find file binaries/tests/x86_64/gnu_hash_resiliency_0

After — the corruption this fixture carries is one field wide and its effect is total: nbuckets goes from 2 to 0 and unpatched cle loses the whole object. The last two lines are the same load with the linked cle change applied, which is what the fixture exists to pin:

with this branch (6c82296)
test_killing_ref      .gnu.hash: nbuckets=2, symoffset=8, bloom_size=1, bloom=[8454144], buckets=[8, 0]
gnu_hash_resiliency_0 .gnu.hash: nbuckets=0, symoffset=0, bloom_size=0, bloom=[], buckets=[]

cle.Loader(gnu_hash_resiliency_0) -> builtins.ValueError: max() iterable argument is empty
cle.Loader(gnu_hash_resiliency_0) with angr/cle#792 -> <ELF Object gnu_hash_resiliency_0, maps [0x400000:0x404017]>
  entry 0x401070  symbols 56  sections 30  relocations 11  PLT 4  deps ['libc.so.6']

Some stripping and obfuscation tools blank the DT_HASH and DT_GNU_HASH tables
of a finished executable. They only accelerate lookup by name, so the file
otherwise survives it: DT_SYMTAB, DT_STRTAB, the relocations and the version
tables are all still there. What a reader sees is a .gnu.hash header of all
zeroes, which declares nbuckets == 0 and an empty bucket array, and pyelftools
raises ValueError out of DynamicSegment.num_symbols() on exactly that shape.
cle loses the whole load to it.

No linker emits an empty bucket array. GNU ld 2.46 and LLD 21.1.8 both
special-case a table with nothing to hash and write nbuckets = 1, so the fixture
cannot be produced by building something, and hand-assembling an ELF would test
a shape no toolchain emits. tests_src/gnu_hash_resiliency builds it instead from
tests/x86_64/test_killing_ref, which carries both hash tables directly ahead of
its symbol table, by zeroing the 96 bytes they occupy. Twenty bytes actually
change; everything else, including the file size, is the original linker's
output. Rerunning the script on the same input reproduces the fixture byte for
byte.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@zardus
zardus force-pushed the gnu-hash-resiliency-fixture branch from 6c82296 to 898b521 Compare August 29, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant