Skip to content

Pin the current checksum behaviour with tests - #471

Merged
jdiazdelrio merged 11 commits into
NASA-PDS:mainfrom
adrianseco:463-pin-checksum-behaviour-tests
Nov 18, 2026
Merged

jdiazdelrio merged 11 commits into
NASA-PDS:mainfrom
adrianseco:463-pin-checksum-behaviour-tests

Conversation

@adrianseco

Copy link
Copy Markdown
Contributor

🗒️ Summary

Tests were added to record how the checksum code behaves today, so the restructuring planned for it can show that it changes nothing. Nothing under src/ was touched, and where the current behaviour looks wrong, the test keeps it as it is.

Two oddities turned up along the way and were left for later: the check for duplicate MD5 sums never matches, and on Windows the table lines end in CR CR LF.

⚙️ Test Data and/or Report

Regression tests included: test_classes_product_checksum.py, test_pipeline_npb.py and the new integration/test_checksum_coverage.py (24 new test cases, no existing test modified):

Behaviour pinned Test Cases
PDS4 table, byte for byte test_pds4_table_bytes 2 (LF, CRLF)
PDS3 table: mkpdssum.pl order, padding and size values test_pds3_table_bytes_and_counts 2 (LF, CRLF)
Previous checksum file loaded, PDS4 and PDS3 test_previous_checksum_file_is_loaded 4
Corrupted previous file: both error messages test_corrupted_previous_checksum_file_raises 5
Duplicate MD5 warning, and no error raised test_duplicate_md5_is_warned_and_does_not_raise 2 (debug off, on)
Times: only the spice_kernels collection and orbnum labels count test_only_coverage_labels_set_the_times 1
Times: labels of the previous checksum count in an increment test_previous_checksum_entries_set_the_times 1
Times: generate() keeps what set_coverage() found test_generate_keeps_the_times_of_set_coverage 1
Times: mission times used when no label applies test_mission_times_are_used_without_coverage_labels 1
Times: a past release uses the labels of its own release test_history_checksum_uses_the_labels_of_its_release 2
Pipeline: current checksum added once, before the collection VID test_current_checksum_added_before_collection_vid 1
Pipeline: backfill adds to the release collection and to the current one test_past_release_products_added_to_both_collections 1
Pipeline: PDS3 checksum added once test_pds3_checksum_added_once_to_collection 1

The tests were also run against deliberately broken versions of the code (a plain sort for PDS3, labels counted for the times that should not be, an add removed from the pipeline, and similar), and every change that altered the behaviour made at least one test fail.

pytest tests/npb/test_classes_product_checksum.py tests/npb/integration/test_checksum_coverage.py tests/npb/test_pipeline_npb.py --cov-branch -q
249 passed, 1 skipped

Name                                                                  Stmts   Miss Branch BrPart  Cover
-------------------------------------------------------------------------------------------------------
src/pds/naif_pds4_bundler/classes/product/product_checksum.py           214      4    104      1    98%
src/pds/naif_pds4_bundler/pipeline/npb.py                               237      0     72      0   100%

Full merge-gate suite (pytest tests/npb/ --cov-branch):
2433 passed, 6 skipped, 1 xfailed, 99% overall coverage

♻️ Related Issues

Fixes #463

Add a test that builds a PDS4 checksum table from a fixed set of
entries and compares the written file byte for byte, once with LF and
once with CRLF line endings. It fixes the entry order, the two-space
separator and the line terminator as they are today.
Add a test that builds a PDS3 checksum table from names that differ
only by '_' and '.', and compares the written file byte for byte, with
LF and CRLF line endings. It fixes the mkpdssum.pl entry order, the
trailing-space padding, and the bytes, record_bytes and file_records
values.
Add tests that read a real previous checksum file, in PDS4 and PDS3
with LF and CRLF line endings, and check that its entries are loaded
without line endings or name padding. A second test checks the error
messages for a line without two fields, a blank line, and an MD5 that
is shorter or longer than 32 characters.
Add a test with two files that share an MD5 under different names. It
checks that the table keeps both, that the warnings list the shared MD5
with both names, and that nothing raises, in debug mode or not.
Add integration tests, with a real setup and real label files, that
check where the checksum takes its times from: only the spice_kernels
collection and orbnum labels count, labels listed in the previous
checksum count on an increment, generate() keeps the times that
set_coverage() found, the mission times are used when there are no
coverage labels, and a checksum rebuilt for a previous release uses the
labels of that release.
Add tests that record what run_pipeline does with the checksum and its
inventory at each of the three places it builds them, and compare the
whole sequence. The current PDS4 checksum is added once to the
miscellaneous collection before its VID is set, the PDS3 checksum is
added once after being labeled, and each past release in the history
backfill adds its checksum and its inventory once to its own
collection and once to the current one.
Drop the helper that recorded the order of the checksum calls and read
the order from the calls each collection mock already keeps. The tests
still pin where each checksum and inventory is added and when the VID
is set, and now also check that each past checksum is generated from
the history of its own release.
The checksum cuts product names at the archive folder using forward
slashes, and writes its table in text mode. On Windows the tests gave
it backslash names, and every line feed in the table gained a carriage
return. The tests now give forward-slash names and expect the bytes the
writer leaves on each platform.
The collection is now built before the block that expects the error,
so only the product creation can raise there. The generate test now
compares the times with the expected values before and after, instead
of with a copy of its own result.
The times are now checked one by one against their expected values,
before and after the checksum file is written, in the same style as the
other tests of the file.
@jdiazdelrio
jdiazdelrio marked this pull request as ready for review November 18, 2026 12:29
@jdiazdelrio
jdiazdelrio requested a review from a team as a code owner November 18, 2026 12:29

@jdiazdelrio jdiazdelrio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The add/VID calls and the generate/register/label calls are pinned separately, so nothing pins how they interleave. #463 asks for the order of "what is registered, labelled and added". As it stands, moving add(checksum) before generate() in the PDS3 branch, or the backfill adds before the label, still passes every test.

Please record all the calls of each site in one sequence and assert it in full:

  • PDS4 current release (test_current_checksum_added_before_collection_vid): add(checksum) → set_collection_vid() → set_coverage() → generate() → register_product(checksum) → ChecksumPDS4Label.
  • PDS4 history backfill (test_past_release_products_added_to_both_collections), for each past release: generate(history=...) → register_product → ChecksumPDS4Label → add to the release collection → add to the current collection → set_collection_vid() on the release collection.
  • PDS3 (test_pds3_checksum_added_once_to_collection): generate() → register_product(checksum) → ChecksumPDS3Label → add(checksum).

One way to do it is to attach the collection mocks, the checksum mocks and register_product to one parent mock with attach_mock() and compare its mock_calls. Another is to append to the same calls list the existing *_registered_between_generate_and_label tests use. The existing tests can stay as they are.

The calls that generate, register, label and add the checksum are now
recorded in one list and compared in full. This is done for the current
PDS4 release, for each past release rebuilt from the history, and for PDS3,
so moving any of those calls relative to the others now fails a test.
@sonarqubecloud

Copy link
Copy Markdown

@jdiazdelrio
jdiazdelrio removed the request for review from nbachman-jpl November 18, 2026 16:11
@jdiazdelrio jdiazdelrio added the Refactor Used for refactoring code and test cases label Nov 18, 2026
@jdiazdelrio
jdiazdelrio merged commit ea01391 into NASA-PDS:main Nov 18, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Refactor Used for refactoring code and test cases

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pin the current checksum behaviour with characterisation tests

3 participants