Skip to content

fix: hash verification in --dependency-constraints-txt could be silently skipped - #1140

Merged
gaborbernat merged 10 commits into
pypa:mainfrom
manfred-kaiser:fix/constraints-txt-multiline-hash-set-ordering
Aug 1, 2026
Merged

fix: hash verification in --dependency-constraints-txt could be silently skipped#1140
gaborbernat merged 10 commits into
pypa:mainfrom
manfred-kaiser:fix/constraints-txt-multiline-hash-set-ordering

Conversation

@manfred-kaiser

Copy link
Copy Markdown
Contributor

Problem

--dependency-constraints-txt reads the file with:

constraints=set(map(str.strip, dependency_constraints_file))

This treats every physical line as one independent constraint. It does not join
backslash-continued lines, so a file like the one pip-compile --generate-hashes
produces:

hatchling==1.31.0 \
    --hash=sha256:6b48ad...
    --hash=sha256:aac80b...

is split into three separate set elements: the requirement and each --hash line on
its own. When these get written back out to a temporary file for pip/uv
('\n'.join(constraints)), a set has no guaranteed iteration order, so the
requirement and its hashes may or may not end up adjacent again.

Impact

This is silent, not just cosmetic. When the continuation breaks apart, pip's
constraint parser reads the orphaned --hash line as unattached ("has --hash but
no requirement, and will be ignored") and installs the requirement without
checking any hash at all
— no warning surfaces to the build output.

Verified directly against a single-dependency constraints file
(uv_build==0.12.0 --hash=sha256:...) with a deliberately wrong hash, same file,
same command, 8 runs back to back:

$ python -m build --dependency-constraints-txt requirements-build.txt -o dist .

run 1: build failed, wrong hash detected
run 2: build succeeded  ← wrong hash silently accepted
run 3: build succeeded  ← wrong hash silently accepted
run 4: build failed, wrong hash detected
run 5: build succeeded  ← wrong hash silently accepted
run 6: build succeeded  ← wrong hash silently accepted
run 7: build failed, wrong hash detected
run 8: build succeeded  ← wrong hash silently accepted

5 of 8 runs accept the tampered hash with no error and no indication in the output
that anything was skipped. With PYTHONHASHSEED fixed, the outcome becomes 100%
deterministic per seed (5/5 identical for a given seed) — different seeds give
different, opposite outcomes. This isolates the cause to set iteration order, not
to anything timing- or network-related.

  • Against a larger, six-package constraints file (a hatchling build tree with
    multiple multi-hash entries), the failure was consistent rather than
    intermittent, but the symptom varied between runs: sometimes a hash got
    attributed to the wrong package, sometimes two unrelated requirement lines were
    concatenated into one invalid line (e.g. packaging==26.2 hatchling==1.31.0 pluggy==1.6.0).

So depending on the file and the hash seed, this flag can either crash with a
confusing parse error, or silently skip hash verification entirely while reporting
success.

Fix

Join continuation lines into complete logical lines before deduplicating, so every
set element is a self-contained, order-independent constraint (comments and blank
lines are also filtered out explicitly, matching how the file is meant to be read).

Test plan

  • tests/test_main_helpers.py: three new tests —
    test_parse_constraints_txt_single_line,
    test_parse_constraints_txt_ignores_comments_and_blank_lines, and
    test_parse_constraints_txt_joins_backslash_continuations (the regression test,
    using a real pip-compile --generate-hashes-style fixture).
  • Full existing suite (test_main.py, test_env.py, test_main_helpers.py) passes
    unchanged: 211 passed, 2 skipped (platform-specific).

…ints-txt

The constraints file was read with `set(map(str.strip, file))`, treating
each physical line as an independent constraint. Files produced by
`pip-compile --generate-hashes` (and similar tools) wrap a requirement's
--hash options onto backslash-continued lines, so the requirement line and
its hashes ended up as separate set elements. Because set iteration order
depends on the interpreter's hash seed, writing the set back out to a
temporary file for pip/uv sometimes reassembled the continuation correctly
and sometimes didn't, silently dropping the --hash line from its
requirement in the latter case. A tampered or mismatched package hash
could then install without error, depending only on the hash seed of that
particular run.

Join continuation lines into complete logical lines before deduplicating,
so every set element is a self-contained, order-independent constraint.
CI runs with PYTHONWARNDEFAULTENCODING=1 and treats warnings as errors,
so the missing `encoding=` argument on `write_text()` failed the tox jobs
with an EncodingWarning.
Comment thread tests/test_main_helpers.py Outdated
Comment thread src/build/__main__.py Outdated
Test the observable behaviour (constraints passed to env.install) via
build_package + dependency_constraints_txt, matching the existing
test_build_package_with_constraints, instead of calling the private
_parse_constraints_txt helper directly. Also reworded the docstring to
explain why the join is needed rather than restating the code, and
wrapped it to 120 columns.
@manfred-kaiser

Copy link
Copy Markdown
Contributor Author

The two failing jobs (macOS pypy3.10/min, Windows 3.10/min) are unrelated to this change — both are transient network issues in test_integration.py building external packages: a RemoteDisconnected on macOS (Solaar) and a PyPI index serving an unexpected Content-Type on Windows (setuptools resolution). All jobs touching the changed code (test_main.py, test_main_helpers.py) passed. Happy to have a maintainer re-run them if needed — I don't have permission to trigger reruns on this repo myself.

@layday

layday commented Jul 30, 2026

Copy link
Copy Markdown
Member

These aren't valid dependency specifiers, really - requirements.txt is its own minilang. We shouldn't be treating them the same way we do pyproject.toml dependencies or get_requires_for_*. I think the safest thing to do here is forward the constraints file to the installer as is. pip and uv are better equipped to parse it.

…re-parsing

The previous fix correctly joined backslash continuations before deduplicating
the file into a set, but still round-tripped the content through Python (parse
into lines, dedup into a set, rejoin) before writing it back out for pip/uv.
That round trip is what made the original bug possible in the first place.

Read the file once and pass its content through as a single, untouched
collection element instead: the installer backends' '\n'.join(constraints)
then reproduces it byte-for-byte, so a hashed requirement's --hash
continuation lines can no longer be split from it, reordered, or deduplicated
away, regardless of how it's formatted.
@manfred-kaiser

manfred-kaiser commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Updated approach

Instead of re-parsing --dependency-constraints-txt into individual lines (which is what broke backslash-continued --hash options in the first place), this now reads the file once and passes its content through as a single, untouched unit — the installer backends' '\n'.join(constraints) then reproduces it byte-for-byte. No line splitting, no dedup, no reassembly, so the bug class is gone structurally rather than patched around. Net diff is smaller than the previous fix, and only touches _bootstrap_build_env.

On pip/uv and hash verification

Testing this end-to-end (real pip-compile --generate-hashes file, real package, correct vs. tampered hash) surfaced something worth flagging so it isn't mistaken for a regression here: whether a --hash sourced only from a -c constraints file (not repeated on the -r requirement) actually gets enforced depends on the installer version — both pip and uv have had the exact same class of resolver gap, just fixed at different times:

Both are pre-existing installer bugs, not something introduced or fixed by this PR — build passes the constraints file through unchanged either way; what a given installer version does with a hash sourced from a constraint is outside build's control.

Practically: if you rely on --dependency-constraints-txt for hash verification, you need pip ≥ 26.1 or uv ≥ 0.4.6. Below those versions the hash is silently not enforced (uv, pre-#7093) or the build refuses to proceed at all (pip, pre-#13887) — either way, not a build issue, and not something build can or should detect or work around on older installers.

CI's 100% branch-coverage gate caught what the local partial test runs
didn't: nothing exercised the case where --dependency-constraints-txt
points at an empty (or whitespace-only) file, so install() is called
without a constraints override.
Comment thread src/build/__main__.py Outdated
Comment thread src/build/__main__.py
Comment thread src/build/__main__.py
Points to env.py's installer backends instead of the ambiguous
'installer backends' `'\n'.join(constraints)`' phrasing that left
readers unsure who does the joining.
@gaborbernat
gaborbernat merged commit ee2a9cb into pypa:main Aug 1, 2026
109 of 111 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants