Repository navigation
fix(export_l5x): repair Region Map truncation and stale-length errors - #48
Merged
hutcheb merged 1 commit intoNov 3, 2026
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Region Map records can be truncated or self-report the wrong length
The patch file covers the complete fix — both parts described below.
Symptom
Exporting certain real-world
.ACDprojects either:no error at all, or
ValueError: Invalid Region Map lengthand aborts the wholeexport, even though the project opens and runs fine in Studio 5000.
Root cause
The "Region Map" comps record is Logix's index from a routine to its rungs.
Each entry is normally a fixed 16 bytes:
(object_id, parent_id, unknown, seq). Two independent forms of drift were found in real project files:12-byte
(parent_id, unknown, seq)triple with theobject_idfieldsimply missing, instead of the usual 16 bytes.
_parse_region_maptreated any 12-byte remainder as an inert trailer/footer and silently
discarded it — so that rung vanished from the export with no warning,
no error, nothing.
own length near the start (at offset
0x4A), which is supposed to equalthe record's total size. On some real files this stored length is
smaller than the record's actual on-disk size, by anywhere from a
handful of bytes to several hundred — the real entries are still present
past where the stale length says the record ends. The existing code
treated any mismatch here as fatal (
raise ValueError("Invalid Region Map length")), aborting the entire file's export.Both look like the same underlying pattern: something in Studio 5000
appends to this record without always rewriting its own bookkeeping
(length prefix / object_id) to match — under-recording in the second case,
omitting outright in the first. Both were reproduced against real,
non-synthetic project files (proprietary customer projects — see the
No test files attached note in the top-level README of this
contribution for why no sample
.ACD/.L5Xis included).Fix, part 1 — Truncated last entry
_parse_region_mapnow recognizes a 12-byte trailer whose first field(
parent_id) matches aparent_idalready seen among the record's realentries — a coincidence an inert footer has no reason to produce — and
surfaces it as an entry with
object_id=Noneinstead of silently droppingit. A new
_repair_region_map_gaps()(called once therungstableexists) recovers the missing
object_idby elimination: for the routinewith the gap, every other rung already has a resolved
object_idand aknown sequence number; if the gap's sequence number falls strictly between
two already-resolved neighbours, the missing rung must be whichever on-disk
rung (by SQLite
rowid, which tracks on-disk insertion order) falls betweenthem. If more than one candidate qualifies, it logs a warning and leaves
that one rung out rather than guessing wrong.
Fix, part 2 — Stale self-described length (with empirical alignment search)
When the stored
0x4Alength doesn't match either the "modern" or "legacy"expected relationship to the record's actual size, the naive fallback
(assume entries start at fixed offset
0x4E) is often wrong — verifiedon real files where the true start was at a different offset, and the
0x4Efallback silently produced garbage entries (worse than theoriginal error, because it looked successful).
The fix uses an empirical alignment search (
_find_region_map_alignment):brute-force every candidate start byte offset (0 to min(on-disk size − 16,
512)), score each by how many of its would-be 16-byte entries have a
object_idfield (the 4th uint32, atentry_start + 12) that matches areal rung object_id from the
rungstable. Only trust an alignment thatmatches ≥ 80% of its entries and has ≥ 3 entries total (a clear signal when
it's there — wrong offsets consistently score 0%, correct ones score ≥
98%). The
rungstable is populated beforepopulate_region_mapruns in__post_init__, so the known rung ids are always available at this point.Falls back to the fixed
0x4Eoffset (with a clearer "no confidentalignment found" warning) when nothing clears that bar, rather than either
raising or silently accepting a plausible-looking wrong answer.
This approach is logged as a
log.warningin both the "found a betteralignment" and "falling back to 0x4E" cases — it's a real data-quality
signal worth knowing about per-file.
Suggested validation
Both code paths are logged (
log.infofor a recovered rung,log.warningfor an unrecoverable gap or a length mismatch) rather than silent, so it's
easy to see on a batch conversion which files hit either path and by how
much. On the fleet of ~770 real project files this was tested against,
roughly 1 in 30 files hit the stale-length path; for those, the empirical
alignment search found the correct offset in all but 2–3 cases (which
received the
0x4Efallback with a warning, rather than garbage data).🤖 Generated with Claude Code