Skip to content

Fix gcc 16 array-bounds, maybe-uninitialized and enum-conversion warnings - #183

Merged
mstorsjo merged 4 commits into
mstorsjo:masterfrom
heitbaum:gcc
Nov 11, 2026
Merged

mstorsjo merged 4 commits into
mstorsjo:masterfrom
heitbaum:gcc

Conversation

@heitbaum

@heitbaum heitbaum commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Building 3.0.0 with gcc 16.2 at -O2 -Wall produces several warnings that point at real (if unreachable) issues. These four commits fix them:

  • libSBRenc: clear the IPD table by its own size. ipd is an INT array but the clear used sizeof(PS_DELTA), and multiplying the two enumerators triggers -Wdeprecated-enum-enum-conversion. The generated code is unchanged.
  • LATM StreamMuxConfig (libMpegTPEnc and xhe-aac): the layer - 1 lookup is always index -1, since there is only one layer. Only check the core layer for layer > 0.
  • libFDK: pass the Huffman node tables as const SHORT (*)[2] instead of casting each one to a SHORT[168][2] pointer, which gcc flags as reading past the 39-row reshape table. The casts, HANDLE_HUFF_NODE and MAX_ENTRIES go away.
  • SAC encoders: calc_diff_freq()/calc_diff_time() read element 0 before checking num_val. Return early when there are no bands, and in xhe-aac take num_val as int, to match the int dataBands every caller passes.

Tested: test/run-test.sh passes with hashes identical to the references, both before and after the series (x86_64, gcc 16).

FDKmemclear() sized the ipd table as PS_MAX_ENVELOPES * PS_MAX_BANDS *
sizeof(PS_DELTA), but ipd is declared INT ipd[PS_MAX_ENVELOPES]
[PS_MAX_BANDS]. The two element types only happen to have the same
size, and multiplying the two enumerators is arithmetic between
different enumeration types, which gcc warns about with
-Wdeprecated-enum-enum-conversion:

  ps_encode.cpp:1002:34: warning: arithmetic between different
  enumeration types 'PS_CONSTS' and 'PS_BANDS' is deprecated

Use sizeof(hPsOut->ipd) so the clear follows the declaration.
When allStreamsSameTimeFraming is not set, CreateStreamMuxConfig()
reads the configuration of layer - 1 to decide whether to write
coreFrameOffset. Layer 0 has no layer below it, and LATM_MAX_LAYERS
and MAX_LAYERS are both 1, so the index is always -1. gcc flags it:

  tpenc_latm.cpp:391:72: warning: array subscript -1 is below array
  bounds of 'CODER_CONFIG* [1]' [-Warray-bounds=]
  latm_interface.c:361:66: warning: array subscript -1 is below array
  bounds of 'struct CODER_CONFIG_INFO *[1]' [-Warray-bounds=]

coreFrameOffset describes a scalable layer relative to its core layer,
which only exists from layer 1 up, so only make the check for
layer > 0.
HANDLE_HUFF_NODE cast every Huffman node table to a pointer to a
SHORT[168][2] array, whatever size the table really is. With
huff_read() inlined, gcc sees the reshape table, which has 39 rows,
read through that type:

  nlc_dec.cpp:239:37: warning: array subscript 'const SHORT [168][2]
  {aka const short int [168][2]}[0]' is partly outside array bounds of
  'const HUFF_RES_NODES [1]' [-Warray-bounds=]

Every table is a SHORT[N][2], so pass them as const SHORT (*)[2] and
let each one decay to that type. The casts, HANDLE_HUFF_NODE and
MAX_ENTRIES are then unused and are removed.
calc_diff_freq() and calc_diff_time() copy in_data[0] (and prev_data[0])
before looking at num_val. The callers only fill the quantised MSB
arrays for the dataBands bands they code, so with dataBands of 0 the
first element is read uninitialised, and gcc reports it once the
helpers are inlined:

  sacenc_nlc_enc.cpp:181:26: warning: 'quant_data_msb' may be used
  uninitialized [-Wmaybe-uninitialized]
  sac_nlc_enc.c:162:24: warning: 'quant_data_msb' may be used
  uninitialized [-Wmaybe-uninitialized]

Return early when there is nothing to difference.

In the xhe-aac copy, split_lsb(), calc_diff_freq() and calc_diff_time()
also took num_val as a short while every caller passes the int
dataBands, and the quant_offset path fills the arrays with an int loop.
The helpers and the fill loops could then disagree about how many bands
there are, which keeps the warning alive in EcDataSingleEnc(). Take
num_val as an int, as the libSACenc versions already do.

@mstorsjo mstorsjo left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@mstorsjo
mstorsjo merged commit 8fe9f90 into mstorsjo:master Nov 11, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants