Skip to content

Correction to storage profiler write sample data size #5191 - #5192

Merged
joachimmetz merged 1 commit into
log2timeline:mainfrom
kev365:fix-storage-profiler-write-size
Aug 2, 2026
Merged

Correction to storage profiler write sample data size #5191#5192
joachimmetz merged 1 commit into
log2timeline:mainfrom
kev365:fix-storage-profiler-write-size

Conversation

@kev365

@kev365 kev365 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #5191.

_WriteNewAttributeContainer rebound serialized_data to the compressed blob before
sampling it, so the storage profiler's write_new samples carried the compressed
length in both the Data size and the Compressed data size column. The value bound
for the _data column is now a separate name, leaving serialized_data holding the
serialized container when the sample is taken.

Data size Compressed data size
before 107 107
after 1266 107

compressed_data in the non-compressed branch is also b"" now, matching
_CreateAttributeContainerFromRow.

Storage file contents are unchanged — the compressed blob is still what is written to
the _data column, and the read path was already correct.

tests/storage/sqlite/sqlite_file.py had no case that attached a storage profiler.
testWriteNewAttributeContainerWithStorageProfiler writes an event_data container
through a real StorageProfiler, reads back the storage-*.csv.gz sample file, and
asserts the data size exceeds the compressed data size. It fails on the parent commit
with AssertionError: 89 not greater than 89.

Verified: 57 tests in tests/storage, 44 in the storage and profiler modules, and the
30 CLI tool tests in tests/cli/{psteal,psort,log2timeline}_tool.py. pylint and
black clean on both files. The one CLI failure,
log2timeline_tool.testExtractEventsFromSourcesOnBDEImage, reproduces identically on
the unmodified parent commit and passes when run on its own — a local environment
issue, not related to this change.

Backward compatibility

None affected. The change is confined to which value the profiler samples; the stored
data, the storage format, and the profiler's CSV schema are all unchanged.

_WriteNewAttributeContainer rebound serialized_data to the compressed
blob before the storage profiler sampled it, so the write_new samples
reported the compressed length in both the data size and the compressed
data size column. The value bound for the _data column now uses a
separate name, leaving serialized_data holding the serialized container
when the sample is taken.

Adds a test that writes an event_data container through a real
StorageProfiler and asserts the sampled data size exceeds the compressed
data size.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.10%. Comparing base (6fadae7) to head (5293516).

Files with missing lines Patch % Lines
plaso/storage/sqlite/sqlite_file.py 50.00% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5192   +/-   ##
=======================================
  Coverage   85.10%   85.10%           
=======================================
  Files         456      456           
  Lines       41089    41090    +1     
=======================================
+ Hits        34967    34970    +3     
+ Misses       6122     6120    -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joachimmetz joachimmetz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@joachimmetz
joachimmetz merged commit 1334b76 into log2timeline:main Aug 2, 2026
18 checks passed
@kev365
kev365 deleted the fix-storage-profiler-write-size branch August 2, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Storage profiler write samples report the compressed size in the data size column

2 participants