Add OpenPMD support by pgrete · Pull Request #1050 · parthenon-hpc-lab/parthenon

pgrete · 2024-04-12T14:57:16Z

PR Summary

Leftover todos (potentially for future PRs):

cleaner integration in build system
add logic for selection of var (rather than all)
double check additional logic for restart outputs (like UserWorkBeforeRestart)
Add doc
Add callback to provide/forward standard compatible downstream code info
Add pattern file for Visit/Paraview
Add support for slices

For upcoming PRs:

Adjust python script to read opmd output (or is this not needed because of the existing openpmd-viewer)?
Add support for units
Add support for Metadata::None
Add support for Metadata::Fine
Add support for Metadata::CoordinatesVec

PR Checklist

BenWibking · 2026-05-05T17:47:34Z

The only issue I discovered during manual testing is that the time metadata field seems to be missing from the outputs. Is that written?

Time is not stored in a separate attribute but in the "standard" field (iteration.time)

ts = OpenPMDTimeSeries('/path/to/out/dir')

ts.iterations

array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29])

ts.t


ts.t

array([0.        , 0.10168297, 0.20282794, 0.3000894 , 0.40210582,
       0.50107859, 0.60178476, 0.70180927, 0.80225403, 0.90006754,
       1.00190689, 1.10025895, 1.20133299, 1.30205319, 1.4007721 ,
       1.50098825, 1.60000468, 1.70110938, 1.80133319, 1.90043797,
       2.00050956, 2.10170538, 2.20182487, 2.30114008, 2.40104512,
       2.50118173, 2.60176224, 2.70163334, 2.8007994 , 2.90145011])

Ah, ok. I wasn't reading this correctly in my VisIt plugin. Fixed now, thanks.

BenWibking

Works in my production runs. Throughly audited with Codex. Did not attempt to manually read each line.

pgrete · 2026-05-18T08:09:53Z

ping @Yurlungur for review.

Yurlungur · 2026-05-18T18:56:52Z

ping @Yurlungur for review.

Sorry I know I'm the blocker here... I'm out sick currently. Will review ASAP

Yurlungur

Long time coming, and pretty clean in the end. Glad to see you got to reuse some of the utilities from HDF5. Sorry for the long delay in review. I have some comments but I don't consider any of them blocking.

Yurlungur · 2026-05-24T20:44:46Z

+    //  Current unclear how to properly handle other vectors and tensors, so everything
+    //  that not's a proper vector is a a scalar for now.


@pgrete what's the status of this? How does this work right now for, e.g., a tensor field?

lroberts36

Awesome! I am glad to see this done. Just a few small non-blocking comments.

lroberts36 · 2026-05-27T20:43:15Z

+  // TODO(?) in principle, we could abstract this to a more general WriteAttributes place
+  // and reuse for hdf5 and OpenPMD output with corresponing calls


This would be great (WriteOutputFile is pretty big for both OpenPMD and HDF5), but certainly not critical for this PR.

lroberts36 · 2026-05-27T20:48:05Z

+//----------------------------------------------------------------------------------------
+//! \fn void OpenPMDOutput:::WriteOutputFile(Mesh *pm)
+//  \brief  Write output in OpenPMD format
+template <bool WRITE_SINGLE_PRECISION>


Why not just template on the type itself (i.e. template <class OutT>)?

I'm not sure I follow.

lroberts36 · 2026-05-27T20:53:15Z

+RestartReaderOPMD::SparseInfo RestartReaderOPMD::GetSparseInfo() const {
+  SparseInfo info;
+  // Only read if data exists. Otherwise return default constructed.
+  if (it->containsAttribute("SparseInfo")) {
+    auto sinfo_vec = it->getAttribute("SparseInfo").get<std::vector<int8_t>>();
+    info.labels = it->getAttribute("SparseFields").get<std::vector<std::string>>();
+    info.num_sparse = static_cast<int>(info.labels.size());
+    info.num_blocks = sinfo_vec.size() / info.num_sparse;
+    info.dealloc_count = it->getAttribute("SparseDeallocCount").get<std::vector<int>>();
+
+    // copy "vector" data to bool pointer for compatiblity between output backends
+    info.allocated.reset(new bool[sinfo_vec.size()]);
+    for (int i = 0; i < sinfo_vec.size(); i++) {
+      info.allocated[i] = sinfo_vec.at(i);
+    }
+  }


I don't know what the solution is, but it concerns me a little that we basically have to mirror all of the parameter reading for HDF5 and OpenPMD exactly to ensure exact restart behavior.

I think the solution would be (along the lines of "simplify"/write common WriteOutputFile) to have an abstraction to write arbitrary attributes so that a restart output would just call WriteAttribute that under the hoods then calls the backend specific implementation.

lroberts36 · 2026-05-27T20:58:23Z

+if (PARTHENON_ENABLE_OPENPMD)
+
+  # h5py is also needed for the current test
+  list(APPEND REQUIRED_PYTHON_MODULES openpmd_api h5py)
+
+  # Restart
+  list(APPEND TEST_DIRS restart_opmd)
+  list(APPEND TEST_PROCS ${NUM_MPI_PROC_TESTING})
+  list(APPEND TEST_ARGS "--driver ${PROJECT_BINARY_DIR}/example/advection/advection-example \
+    --driver_input ${CMAKE_CURRENT_SOURCE_DIR}/test_suites/restart_opmd/parthinput.restart \
+    --num_steps 3")
+  list(APPEND EXTRA_TEST_LABELS "")
+
+  endif()
+


Should this just be part of the restart test? I think just adding OpenPMD restart output and restarting from both OpenPMD and HDF5 would test the same things and maybe run a little quicker. (I assume this new test takes as long as the current restart test).

In principle yes, but until OpenPMD is on by default (and more broadly used) I think it makes sense to keep this separate because the logic is simpler in two separate tests than having a configure dependent number of steps and test logic for one (large) test.

pgrete · 2026-06-10T19:57:42Z

Are people happy with the latest changes? I think we'd be good to go then.

Yurlungur · 2026-06-10T20:38:13Z

Ship it.

pgrete and others added 25 commits January 8, 2024 20:58

Add OpenPMD as external lib

49027c7

Add OpenPMD skeleto

4525e86

WIP more Open PMD

4de250c

WIP OpenPMD use file id

b906af7

Merge branch 'develop' into pgrete/pmd-output

108fc7a

Write blocks

79660a2

Merge remote-tracking branch 'origin/develop' into pgrete/pmd-output

372b585

Centralize getting var info for output

8d40c91

WIP openpmd, chunks don't work yet plus check dimensionality

4f20c26

Merge remote-tracking branch 'origin/develop' into pgrete/pmd-output

4dde705

Fix chunk extents

f29e8d1

Write Attributes

a501a5d

Rename restart to restart_hdf5

c0be75c

WIP abstract RestartReader

7f03528

WIP separating RestartReader

56795f2

Make RestartReader abstract

8587303

Merge branch 'pgrete/refactor-restart' into pgrete/pmd-output

8bf955b

Add OpenPMD restart skeleton

e3ea8d7

WIP updating loc logic

33b6261

Merge branch 'develop' into pgrete/pmd-output

788118c

Fix interface from recent changes in develop

62e54da

Read and Write loc

6309663

Houston, we have a build

7224f42

Added OpenPMD restart ReadBlocks

ae1f241

Fix loc level

19863d7

pgrete commented Apr 12, 2024

View reviewed changes

Comment thread src/outputs/openpmd.cpp Outdated

pgrete commented Apr 12, 2024

View reviewed changes

Comment thread src/outputs/restart_opmd.cpp Outdated

pgrete mentioned this pull request Apr 12, 2024

pmb->loc.level() differences in writing and reading restart files #1051

Closed

pgrete added 2 commits April 15, 2024 12:17

Make Series persistent and fix rootlevel typo

e2b2bd1

WIP Read/Write Params

f9373e8

BenWibking approved these changes May 5, 2026

View reviewed changes

Merge branch 'develop' into pgrete/pmd-output

a28377c

Yurlungur approved these changes May 24, 2026

View reviewed changes

Merge branch 'develop' into pgrete/pmd-output

a11c740

lroberts36 approved these changes May 27, 2026

View reviewed changes

pgrete added 6 commits June 4, 2026 17:02

Merge branch 'develop' into pgrete/pmd-output

450a932

Disable OpenPMD by default (for now)

1f96116

Forward sim metadata to opmd

967114a

Remove extra copy for load/store views in openpmd attributes

e8cfcb9

Address Luke's comments

10dae37

Fix vector/tensor component labeling (assisted by Claude)

ace1f7e

pgrete enabled auto-merge (squash) June 10, 2026 19:57

pgrete disabled auto-merge June 10, 2026 19:57

pgrete added 3 commits June 11, 2026 00:13

Skip blocks not included in slices entirely

6b4d3b1

Fix compile

984a06d

Merge branch 'develop' into pgrete/pmd-output

d2e2a15

pgrete enabled auto-merge (squash) June 12, 2026 06:29

Let cmake figure out compiler and wrappers

6ba910d

pgrete disabled auto-merge June 12, 2026 14:30

pgrete enabled auto-merge (squash) June 12, 2026 14:30

pgrete merged commit a686f87 into develop Jun 12, 2026
38 checks passed

pgrete deleted the pgrete/pmd-output branch June 12, 2026 18:48

BenWibking mentioned this pull request Jun 25, 2026

Ext: PatchBasedMeshRefinement openPMD/openPMD-standard#252

Open

11 tasks

pgrete linked an issue Jul 9, 2026 that may be closed by this pull request

ADIOS I/O #946

Closed

pgrete mentioned this pull request Jul 9, 2026

ADIOS I/O #946

Closed

		// Current unclear how to properly handle other vectors and tensors, so everything
		// that not's a proper vector is a a scalar for now.

		// TODO(?) in principle, we could abstract this to a more general WriteAttributes place
		// and reuse for hdf5 and OpenPMD output with corresponing calls

Uh oh!

Conversation

pgrete commented Apr 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

Uh oh!

Uh oh!

BenWibking commented May 5, 2026

Uh oh!

BenWibking left a comment

Choose a reason for hiding this comment

Uh oh!

pgrete commented May 18, 2026

Uh oh!

Yurlungur commented May 18, 2026

Uh oh!

Yurlungur left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lroberts36 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pgrete commented Jun 10, 2026

Uh oh!

Yurlungur commented Jun 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pgrete commented Apr 12, 2024 •

edited

Loading