Skip to content

cli: support converting PX4 ULog (.ulg) files to MCAP - #1751

Draft
amacneil wants to merge 2 commits into
mainfrom
cursor/cli-convert-px4-ulog-0a91
Draft

cli: support converting PX4 ULog (.ulg) files to MCAP#1751
amacneil wants to merge 2 commits into
mainfrom
cursor/cli-convert-px4-ulog-0a91

Conversation

@amacneil

@amacneil amacneil commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds PX4 ULog (.ulg) support to mcap convert, dispatched on the file extension alongside ROS 1 bag and ROS 2 db3 inputs (addresses #1675). ULog is self-describing, so the converter uses the file's embedded message formats and needs no external definitions or sourced workspace.

The decode is hand-rolled (no new dependency), in the same spirit as the existing ros1_bag converter. Output uses a new px4 profile.

Mapping

ULog construct MCAP output
uORB data topic (D, described by F formats) protobuf message; schema px4.<message_name>; topic <message_name>/<multi_id>; multi_id in channel metadata
Logged strings (L/C) single log_message topic, schema px4.log_message (timestamp, severity, text, tag)
Parameters (P/Q) px4.parameter messages on a parameters topic (initial snapshot + runtime changes); a default field marks Q defaults vs P actual values
Info (I/M) one MCAP info metadata record (string/scalar fields; binary blobs skipped)

Each schema carries its own minimal FileDescriptorSet (the message plus its transitive nested dependencies), so schema bytes stay O(types) rather than O(types × channels). Schemas are built dynamically from the ULog format definitions, handling nested types, fixed-size arrays, char[] strings, and padding fields. proto3 lacks 8/16-bit integers so those widen to 32-bit; char[] becomes a string.

Naming decisions

  • px4. namespace (not ulog.): these are PX4/uORB message types that ULog merely records — analogous to not namespacing ROS bag messages by the container.
  • Field/type names taken from PX4's own definitions: e.g. the log schema mirrors PX4's log_message uORB topic (timestamp/severity/text), and severity is normalized from the on-disk ASCII digit ('0''7') to the integer kernel level 07. No lossy display mapping is baked in; a Foxglove-side px4.log_message → foxglove.Log converter is intended as a separate workstream.
  • Topics are bare snake_case (no leading slash, matching uORB), always suffixed with the instance (/0, /1, …). multi_id is the uORB instance index (e.g. IMU #N on one vehicle), also surfaced in channel metadata.

Timestamps are recorded relative to system boot (ULog stores microseconds since startup), preserving the raw values; absolute-time signals (time_ref_utc, GPS time_utc_usec) are preserved in the output for any downstream consumer that wants to derive epoch.

Testing

  • Inline unit tests covering nested/array/char/padding decode, log severity normalization and tags, multi-instance topics + metadata, initial/changed parameters, P-vs-Q defaults, info→metadata (including skipping binary blobs), and incompatible-flag rejection vs the tolerated DATA_APPENDED flag.
  • A real-fixture test converting python/examples/ulog2mcap/fixtures/test_ulog.ulg.
  • cargo test -p mcap-cli, cargo clippy -p mcap-cli --all-targets -- --no-deps -D warnings, cargo fmt, and cspell all pass.
Open in Web Open in Cursor 

Add a hand-rolled ULog parser to `mcap convert`, dispatched on the .ulg/.ulog
extension alongside ROS 1 bag and ROS 2 db3 inputs. ULog data is self-describing,
so conversion uses the embedded message formats and needs no external definitions.

Mapping (px4 profile):
- uORB data topics -> protobuf messages, schema px4.<message_name>, topic
  <message_name>/<multi_id>, with multi_id in channel metadata. Schemas are built
  dynamically from ULog format definitions (nested types, fixed arrays, char[]
  strings, padding handling).
- Logged strings (L/C) -> a single log_message topic using px4.log_message
  (timestamp, severity, text, tag); the kernel severity is normalized from the
  ASCII digit to 0-7.
- Parameters (P/Q) -> px4.parameter messages on a parameters topic (initial
  snapshot plus runtime changes).
- Info (I/M) -> an MCAP info metadata record.

Timestamps are recorded relative to system boot.

Co-authored-by: Adrian Macneil <adrian@foxglove.dev>
claude[bot]

This comment was marked as outdated.

- Build a minimal per-schema FileDescriptorSet (the message plus its transitive
  nested deps) instead of embedding every format in every schema record. Cuts
  uncompressed schema bytes from O(types x channels) to O(types) and fixes a
  latent crash when a uORB format collides with the synthetic log_message/parameter
  names (each now lives in its own descriptor set).
- Avoid deep-cloning the FormatDef on every data message by scoping the immutable
  borrow so decode finishes before the &mut self write.
- Distinguish default parameters (Q) from actual values (P) via a 'default' field
  on px4.parameter instead of silently emitting duplicates.
- Drop dead code in parse_param.
- Add tests for info->metadata (incl. skipping binary blobs), incompat flag
  rejection, the tolerated DATA_APPENDED flag, and P-vs-Q defaults; document the
  fixture channel count.

Co-authored-by: Adrian Macneil <adrian@foxglove.dev>

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The five inline threads from the first pass are addressed and resolved. Two open questions from that review still stand:

  • We now ship two ULog→MCAP converters with incompatible output — this CLI and python/examples/ulog2mcap (px4. vs ulog. namespace, log_message vs /log_message topic, protobuf vs JSON params, raw kernel severity 0–7 vs mapped Foxglove Log levels). What's the user story? Is the Python example getting realigned or deprecated, or do we knowingly hand people two tools with divergent output? At minimum, a docs pointer naming the CLI as canonical would save confusion.
  • Timestamps are boot-relative, so the file opens at ~1970 in players. python/examples/ulog2mcap/convert.py grew a --start-date offset precisely because near-epoch timestamps look broken. Is boot-relative-only acceptable for v1, or do we want the offset flag before this leaves draft? It's the first thing a user notices opening the result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants