The ScanCode conformance suite is useful for regression checks, but it is
weighted toward cases that previously caused trouble in ScanCode. It does not
show how the exact matcher behaves across ordinary package archives.
Build a sampled regression set from real ecosyste.ms archives. Group inputs and
matched text by content hash so repeated notices do not dominate the sample.
Include examples covering:
- Empty results and exact agreement.
- Differences from stored Licensee or ScanCode results.
NOASSERTION and partial identification.- Dual-license files and several licenses in one archive.
- Encoding, SPDX tag, and short-reference policy cases.
Commit small fixtures when their licensing permits it. For inputs that cannot
be committed, store enough metadata to reproduce the sample from the source
archive.
Use the results to measure how much useful text exact matching misses before
adding sequence alignment or an inverted index. Fuzzy matching should remain
out of the default matcher and become opt-in only if the measured gain justifies
its embedded size, startup time, and memory.
The ScanCode conformance suite is useful for regression checks, but it is
weighted toward cases that previously caused trouble in ScanCode. It does not
show how the exact matcher behaves across ordinary package archives.
Build a sampled regression set from real ecosyste.ms archives. Group inputs and
matched text by content hash so repeated notices do not dominate the sample.
Include examples covering:
NOASSERTIONand partial identification.Commit small fixtures when their licensing permits it. For inputs that cannot
be committed, store enough metadata to reproduce the sample from the source
archive.
Use the results to measure how much useful text exact matching misses before
adding sequence alignment or an inverted index. Fuzzy matching should remain
out of the default matcher and become opt-in only if the measured gain justifies
its embedded size, startup time, and memory.