Add a short-data-directory PE and a large-common ELF object

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

Two header shapes make a consumer index a table past its declared length, and this repository has neither. Searching every ELF and every PE under tests/:

searched 735 ELF objects under tests/ for: object carrying an SHN_X86_64_LCOMMON symbol
0 match

searched 92 PE images under tests/ for NumberOfRvaAndSizes != 16
distribution of NumberOfRvaAndSizes: 16 directories: 92 images
0 match

Every PE image here declares sixteen data directories, and no symbol here uses a processor-specific reserved st_shndx. So the arithmetic that assumes both cannot be tested against an input that breaks it.

Root cause

Neither shape falls out of an ordinary build. No linker flag produces fewer than sixteen data directories — real firmware gets a short one by writing the header in assembly and letting the assembler count the entries — and a symbol only lands in the large-common area under -mcmodel=medium with -fcommon, which gcc has not defaulted to since gcc 10.

Fix

Add two objects, and nothing else.

tests/x86_64/efi_short_data_directory.efi is a UEFI application whose optional header declares six data directories: NumberOfRvaAndSizes 6, SizeOfOptionalHeader 160, which is 112 fixed PE32+ fields plus six eight-byte entries, so the short directory is real rather than padded. efi_header.S derives the count as .long (section_table - .) / 8, the way arch/x86/boot/header.S does, and the build script reads the field back out of the linked image and asserts it.

tests/x86_64/large_common.o carries big_buffer with st_shndx 0xff02, SHN_X86_64_LCOMMON, produced by compiling an oversized common object rather than by retagging a symbol table. Sources and build scripts for both are in tests_src.

Testing

tests_src/pe_short_data_directory/build.sh rebuilds the .efi byte for byte, sha256 dd3d110220f9ab53653be746166f9b5a051b2a93c0998788843e8373a8cc37ab. tests_src/elf_large_common/build.sh does not: under GCC 15.3.0 from nixpkgs it emits a 41-byte .text where the committed object's is 17, at every optimization level tried, so that script records how the fixture was made rather than reproducing it. The property the fixture exists for holds either way — readelf -s prints big_buffer as LARGE_COM. This repository has no suite of its own; angr/cle#732 is the consumer that exercises both.

Validation: #205 (comment)

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 879075612e2d571d0b8156515b410ed59fecfbea.

• tests/x86_64/efi_short_data_directory.efi — 12288 bytes, sha256 dd3d110220f9ab53653be746166f9b5a051b2a93c0998788843e8373a8cc37ab; NumberOfRvaAndSizes is 6 and SizeOfOptionalHeader is 160, read both with pefile and with a raw struct parse of the on-disk field. 160 is 112 fixed PE32+ fields plus 6 eight-byte directory entries, so the short directory is real rather than padded
• tests/x86_64/large_common.o — 1464 bytes, sha256 6d62b5250b2bb62896e1d06fe89c1510bd87881839fa6237c1ef2df7aebc0fce; symbol big_buffer has st_shndx 65282 (0xff02, SHN_X86_64_LCOMMON), which readelf -s prints as LARGE_COM
• Toolchain survey behind the description's claim that no linker flag produces fewer than sixteen data directories: lld-link, GNU ld through mingw, ld --oformat pei-x86-64, tcc, Zig, Go and Open Watcom were each measured at sixteen, and EDK2's GenFw hardcodes it, which is why the fixture derives the count in assembly as .long (section_table - .) / 8. Every one of the 92 PE images already in this repository declares sixteen (corrected 2026-08-28 from 89; re-derived at baseline de38bc3dbd052f7f9d2442cda0e43eee8e4263a8 with a raw struct parse of NumberOfRvaAndSizes and independently with pefile, which agree)
• Consumer behaviour today, cle b58ea02a446106647cdaae32bdf91b7062404cc1: the .efi raises IndexError: list index out of range in cle/backends/pe/pe.py _meta_dd, reached from _meta_iat, because the data directory is assumed to have sixteen entries; large_common.o raises IndexError: list index out of range in cle/backends/regions.py __getitem__ from ELFSymbol.__init__, because st_shndx is used as an ordinary section index

Rebuild, 2026-08-28, superseding the earlier note that the build scripts had not been rerun: tests_src/pe_short_data_directory/build.sh reproduces efi_short_data_directory.efi byte for byte, sha256 unchanged at dd3d110220f9ab53653be746166f9b5a051b2a93c0998788843e8373a8cc37ab, and its own NumberOfRvaAndSizes == 6 assertion passed. tests_src/elf_large_common/build.sh does not reproduce large_common.o: under GCC 15.3.0 from nixpkgs it emits a 41-byte .text where the committed object's is 17 bytes, and -O1, -O2, -Os and -O3 all differ from the committed digest as well. The property the fixture exists for is unaffected — big_buffer still carries st_shndx 0xff02 in the committed object — but that script records how the fixture was made rather than reproducing it.

Caveats: this repository has no test suite, so the record is header verification of the committed artifacts plus the load attempts above.

Re-keyed 2026-08-29. The figures above were taken at head bcd112ad9273fadfba4875a93d865ee38ff70f6b; the branch has since been replayed onto angr/binaries master twice, at 9d9d9e0 and then at d4ffa2f, and is now 879075612e2d571d0b8156515b410ed59fecfbea. git range-diff reports both commits unchanged and the diff against the merge base is byte-identical, so every figure above still describes this patch and the eight files it adds are the same eight. The second replay was for the consumer: angr/cle#732 pins this branch through a sync: line, so its CI checks this tip out instead of master, and it was failing tests/analyses/decompiler/test_block_simplifier.py on a missing tests/i386/deep_sp_chain that had landed on master in the meantime. This repository runs no CI, so there is no check state to quote.

Re-keyed 2026-09-04. Head is now 88bd6404410910d9da33890f14b47aa213981f63, which is 8545e3e with angr/binaries master 003e82a2bfa641530924055695b36cec8af483ab merged in. git diff --raw of this branch against its merge base is identical before and after the merge, so it adds the same eight files and every figure above still describes them. The merge was for the consumer again: angr/cle#732 pins this branch through its sync: line and cle's CI checks the tip out bare, and master had since gained two Mach-O fixtures that cle master's own test suite needs, so cle#732 would have failed for a reason outside either change. This repository runs no CI, so there is no check state to quote.

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Searching every ELF object under tests/ for an SHN_X86_64_LCOMMON symbol, and every PE image for a data directory shorter than sixteen entries, before and after this change.

Before — neither header shape occurs anywhere in the repository:

angr/binaries at the merge base

angr/binaries at baseline de38bc3
searched 735 ELF objects under tests/ for: object carrying an SHN_X86_64_LCOMMON symbol
0 match

angr/binaries at baseline de38bc3
searched 92 PE images under tests/ for NumberOfRvaAndSizes != 16
distribution of NumberOfRvaAndSizes: 16 directories: 92 images
0 match

After — one object per shape, read back out of the committed files:

with this change

angr/binaries at head bcd112a
searched 736 ELF objects under tests/ for: object carrying an SHN_X86_64_LCOMMON symbol
1 match
    tests/x86_64/large_common.o  --  symbol 'big_buffer' has st_shndx 0xff02 (SHN_X86_64_LCOMMON)

angr/binaries at head bcd112a
searched 93 PE images under tests/ for NumberOfRvaAndSizes != 16
distribution of NumberOfRvaAndSizes: 6 directories: 1 images, 16 directories: 92 images
1 match
    tests/x86_64/efi_short_data_directory.efi  --  NumberOfRvaAndSizes=6, SizeOfOptionalHeader=160, magic=0x020b