Tools: pltfile-to-openpmd Converter - #597
Open
ax3l wants to merge 1 commit into
Open
Conversation
5 tasks
ax3l
added a commit
to AMReX-Codes/amrex
that referenced
this pull request
Jul 29, 2026
## Summary Extract the parsing of the per-level grid table (data file index, particle count and byte offset for each grid) from `ParticleContainer::Restart` into the shared `ParticleHeader` (follow-up to #5476), so standalone consumers - e.g. language bindings and format converters - can locate each grid's binary particle data without duplicating the on-disk format logic. - `ParticleHeader` gains a nested `GridEntry {which, count, where}` and `Vector<Vector<GridEntry>> grids`, filled by the new `parse_grid_table()`. - `ParticleHeader::read()` now returns a fully parsed header including the table; `parse()` still consumes exactly the metadata prefix, keeping the existing stream contract. - `Restart` consumes the shared parser instead of inline `HdrFile >>` reads - one source of truth for the Header format. Motivation: pyAMReX tools (runtime-SoA plotfile reading AMReX-Codes/pyamrex#596, AMReX-Codes/pyamrex#581, and a plotfile-to-openPMD converter AMReX-Codes/pyamrex#597) need the grid table for per-grid particle counts and parallel-read planning. ## Testing ``` cmake -S . -B build -DAMReX_ENABLE_TESTS=ON -DAMReX_TEST_TYPE=All -DAMReX_PARTICLES=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo cmake --build build -j6 ctest --test-dir build -R 'CheckpointRestart' -E 'HDF5|AsyncIO' --output-on-failure ``` All pass: `Particles_CheckpointRestart_3d`, `Particles_CheckpointRestartDualGrid_3d`, `Particles_CheckpointRestartDualGridSOA_3d`, `Particles_CheckpointRestartSOA_3d` (plus `Particles_Redistribute_3d` from the Small set). ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [x] include documentation in the code and/or rst files, if appropriate 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
ax3l
force-pushed
the
topic-pltfile-to-openpmd
branch
from
August 4, 2026 17:50
b0edb13 to
ed901a9
Compare
ax3l
marked this pull request as ready for review
August 4, 2026 17:50
Add a converter from native AMReX plotfiles (mesh + particles) to openPMD series, as an importable module (amrex.tools.pltfile_to_openpmd) and a console script (pltfile-to-openpmd). The openPMD backend - HDF5, ADIOS2 or JSON - is chosen by the output extension; openpmd-api is a new optional dependency (extras_require: amrex[openpmd]). The conversion is information-preserving: - field data at on-disk precision, one chunk per AMReX grid, axes reversed to C order, - mesh-refinement levels per the openPMD PatchBasedMeshRefinement extension proposal (openPMD-standard PR AMReX-Codes#252): _lvl<N> record names plus a refinementRatio attribute, - particles per species with verbatim component names, unpacked id and amrex_cpu records, - AMReX-specific metadata (level steps, box arrays, ghost widths, coordinate system) in amrex_-prefixed attributes. The gitignore plotfile pattern is narrowed (plt[0-9]*, plt_*) so the tool's source files are not ignored. Documented under docs/source/usage/workflows/pltfile_to_openpmd.rst, tested by tests/test_pltfile_to_openpmd.py (single- and multi-level mesh round trips, particles, error paths). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ax3l
force-pushed
the
topic-pltfile-to-openpmd
branch
from
August 4, 2026 17:51
ed901a9 to
3918bd0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Top of a PR stack (earlier commits shown here belong to the PRs below):
Summary
pltfile-to-openpmd: convert native AMReX plotfiles (mesh + particles) to openPMD series, as an importable module (amrex.tools.pltfile_to_openpmd) and a console script. Backend (HDF5.h5, ADIOS2.bp, JSON.json) selected by the output extension;openpmd-apibecomes an optional dependency (amrex[openpmd]). Each plotfile becomes one iteration (level-0 step); the plotfile's dimensionality selectsamrex.space{1,2,3}dautomatically.Information-preserving mapping:
store_chunkper AMReX grid, axes reversed to openPMD C order;_lvl<N>+refinementRatioattribute for finer levels;list_particle_species, read viaread_particles): verbatim component names, unpackedid+amrex_cpu, constant zeropositionOffset;amrex_-prefixed attributes for AMReX metadata without an openPMD equivalent (level steps, per-level box arrays, ghost widths, coordinate system, per-species file metadata).Documented limits: particle on-disk ordering is not preserved (identity is, via
id/amrex_cpu); MR level assignment is recoverable from positions + stored box arrays; ghost-cell values not written (width recorded); plotfiles carry no units (unitSI=1, dimensionless).Also narrows the
.gitignoreplotfile pattern (**/plt*->plt[0-9]*,plt_*) which would otherwise ignore the tool's own source files.Testing
Built against AMReX
development+ AMReX-Codes/amrex#5577 + AMReX-Codes/amrex#5578:14 passed: single-level mesh round trip (position-dependent ramp, byte-compared after C-order transpose), two-level hierarchy (
_lvl1,refinementRatio=[2,2,2], sparse chunk read-back), particles (names, values, unique ids, per-level counts), error paths. Smoke-tested withopenpmd-lsand by converting a C++-written plotfile with single-precision particle data in a double-precision Python build (exercises AMReX-Codes/amrex#5578 end to end).🤖 Generated with Claude Code