fix: hash verification in --dependency-constraints-txt could be silently skipped - #1140
Conversation
…ints-txt The constraints file was read with `set(map(str.strip, file))`, treating each physical line as an independent constraint. Files produced by `pip-compile --generate-hashes` (and similar tools) wrap a requirement's --hash options onto backslash-continued lines, so the requirement line and its hashes ended up as separate set elements. Because set iteration order depends on the interpreter's hash seed, writing the set back out to a temporary file for pip/uv sometimes reassembled the continuation correctly and sometimes didn't, silently dropping the --hash line from its requirement in the latter case. A tampered or mismatched package hash could then install without error, depending only on the hash seed of that particular run. Join continuation lines into complete logical lines before deduplicating, so every set element is a self-contained, order-independent constraint.
CI runs with PYTHONWARNDEFAULTENCODING=1 and treats warnings as errors, so the missing `encoding=` argument on `write_text()` failed the tox jobs with an EncodingWarning.
Test the observable behaviour (constraints passed to env.install) via build_package + dependency_constraints_txt, matching the existing test_build_package_with_constraints, instead of calling the private _parse_constraints_txt helper directly. Also reworded the docstring to explain why the join is needed rather than restating the code, and wrapped it to 120 columns.
|
The two failing jobs (macOS pypy3.10/min, Windows 3.10/min) are unrelated to this change — both are transient network issues in |
|
These aren't valid dependency specifiers, really - |
…docstring" This reverts commit 557fa0a.
…txt tests" This reverts commit b55ede4.
…-constraints-txt" This reverts commit f36fd88.
…re-parsing The previous fix correctly joined backslash continuations before deduplicating the file into a set, but still round-tripped the content through Python (parse into lines, dedup into a set, rejoin) before writing it back out for pip/uv. That round trip is what made the original bug possible in the first place. Read the file once and pass its content through as a single, untouched collection element instead: the installer backends' '\n'.join(constraints) then reproduces it byte-for-byte, so a hashed requirement's --hash continuation lines can no longer be split from it, reordered, or deduplicated away, regardless of how it's formatted.
|
Updated approach Instead of re-parsing On pip/uv and hash verification Testing this end-to-end (real
Both are pre-existing installer bugs, not something introduced or fixed by this PR — Practically: if you rely on |
CI's 100% branch-coverage gate caught what the local partial test runs didn't: nothing exercised the case where --dependency-constraints-txt points at an empty (or whitespace-only) file, so install() is called without a constraints override.
Points to env.py's installer backends instead of the ambiguous 'installer backends' `'\n'.join(constraints)`' phrasing that left readers unsure who does the joining.
Problem
--dependency-constraints-txtreads the file with:This treats every physical line as one independent constraint. It does not join
backslash-continued lines, so a file like the one
pip-compile --generate-hashesproduces:
is split into three separate set elements: the requirement and each
--hashline onits own. When these get written back out to a temporary file for pip/uv
(
'\n'.join(constraints)), asethas no guaranteed iteration order, so therequirement and its hashes may or may not end up adjacent again.
Impact
This is silent, not just cosmetic. When the continuation breaks apart, pip's
constraint parser reads the orphaned
--hashline as unattached ("has--hashbutno requirement, and will be ignored") and installs the requirement without
checking any hash at all — no warning surfaces to the
buildoutput.Verified directly against a single-dependency constraints file
(
uv_build==0.12.0 --hash=sha256:...) with a deliberately wrong hash, same file,same command, 8 runs back to back:
5 of 8 runs accept the tampered hash with no error and no indication in the output
that anything was skipped. With
PYTHONHASHSEEDfixed, the outcome becomes 100%deterministic per seed (5/5 identical for a given seed) — different seeds give
different, opposite outcomes. This isolates the cause to
setiteration order, notto anything timing- or network-related.
hatchlingbuild tree withmultiple multi-hash entries), the failure was consistent rather than
intermittent, but the symptom varied between runs: sometimes a hash got
attributed to the wrong package, sometimes two unrelated requirement lines were
concatenated into one invalid line (e.g.
packaging==26.2 hatchling==1.31.0 pluggy==1.6.0).So depending on the file and the hash seed, this flag can either crash with a
confusing parse error, or silently skip hash verification entirely while reporting
success.
Fix
Join continuation lines into complete logical lines before deduplicating, so every
set element is a self-contained, order-independent constraint (comments and blank
lines are also filtered out explicitly, matching how the file is meant to be read).
Test plan
tests/test_main_helpers.py: three new tests —test_parse_constraints_txt_single_line,test_parse_constraints_txt_ignores_comments_and_blank_lines, andtest_parse_constraints_txt_joins_backslash_continuations(the regression test,using a real
pip-compile --generate-hashes-style fixture).test_main.py,test_env.py,test_main_helpers.py) passesunchanged: 211 passed, 2 skipped (platform-specific).