Skip to content

Allocate Plasma Components Dynamically - #1377

Open
AlexanderSinn wants to merge 42 commits into
Hi-PACE:developmentfrom
AlexanderSinn:Dynamic_Particle_Components
Open

Allocate Plasma Components Dynamically#1377
AlexanderSinn wants to merge 42 commits into
Hi-PACE:developmentfrom
AlexanderSinn:Dynamic_Particle_Components

Conversation

@AlexanderSinn

@AlexanderSinn AlexanderSinn commented Apr 15, 2026

Copy link
Copy Markdown
Member

With this PR, plasma components are only allocated at runtime when needed. This includes aabssq, x_prev, ion_level and the AB5 force terms. This allows to store the aabssq on the plasma particle instead of gathering it in every kernel which has become quite tedious. Additionally using the AB5 pusher is now a runtime option instead of a compile time option.

This leads to significant memory savings for simple Wakefield simulations:
PR:

Device Memory Usage:
--------------------------------------------------------------------
Name                                      Nalloc    AvgMem    MaxMem
--------------------------------------------------------------------
The_Arena::Initialize()                        1   163   B    29 GiB
PlasmaParticleContainer::InitParticles()       4    11 GiB    11 GiB
Fields::AllocData()                            1  6055 MiB  6055 MiB
Hipace::ExplicitMGSolveBxBy()                 49  2105 MiB  2112 MiB
FFTPoissonSolverDirichletQuick::define()       8  1151 MiB  1152 MiB
BeamParticleContainer::resize()              903   203 MiB   395 MiB
ResizeRandomSeed                               1    40 MiB    40 MiB
hpmg::MultiGrid::solve1()                   3874    47 KiB   432 KiB
shiftSlippedParticles()                      760    33   B   216 KiB
MultiBuffer::get_data()                      760    12   B   108 KiB
main()                                        17  4351   B  4352   B
Hipace::InitData()                            13  3321   B  3328   B
DepositCurrent_PlasmaParticleContainer()    1001    48   B   256   B
Hipace::Evolve()                               1   255   B   256   B
--------------------------------------------------------------------

Total GPU global memory (MB) spread across MPI: [40441 ... 40441]
Free  GPU global memory (MB) spread across MPI: [9663 ... 9663]
[The         Arena] max space (MB) allocated spread across MPI: [30331 ... 30331]
[The         Arena] max space (MB) used      spread across MPI: [21279 ... 21279]
[The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8]
[The Managed Arena] max space (MB) used      spread across MPI: [0 ... 0]
[The  Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8]
[The  Pinned Arena] max space (MB) used      spread across MPI: [0 ... 0]
[The   Comms Arena] max space (MB) allocated spread across MPI: [8 ... 8]
[The   Comms Arena] max space (MB) used      spread across MPI: [0 ... 0]

dev:

Device Memory Usage:
--------------------------------------------------------------------
Name                                      Nalloc    AvgMem    MaxMem
--------------------------------------------------------------------
The_Arena::Initialize()                        1  1099   B    29 GiB
PlasmaParticleContainer::InitParticles()       5    14 GiB    14 GiB
Fields::AllocData()                            1  6055 MiB  6055 MiB
Hipace::ExplicitMGSolveBxBy()                 49  2108 MiB  2112 MiB
FFTPoissonSolverDirichletQuick::define()       8  1152 MiB  1152 MiB
BeamParticleContainer::resize()              903   203 MiB   395 MiB
ResizeRandomSeed                               1    40 MiB    40 MiB
hpmg::MultiGrid::solve1()                   3874    46 KiB   432 KiB
shiftSlippedParticles()                      760    33   B   216 KiB
MultiBuffer::get_data()                      760    12   B   108 KiB
main()                                        17  4351   B  4352   B
Hipace::InitData()                            13  3327   B  3328   B
DepositCurrent_PlasmaParticleContainer()    1001    48   B   256   B
Hipace::Evolve()                               1   255   B   256   B
--------------------------------------------------------------------

Total GPU global memory (MB) spread across MPI: [40441 ... 40441]
Free  GPU global memory (MB) spread across MPI: [9663 ... 9663]
[The         Arena] max space (MB) allocated spread across MPI: [30331 ... 30331]
[The         Arena] max space (MB) used      spread across MPI: [24160 ... 24160]
[The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8]
[The Managed Arena] max space (MB) used      spread across MPI: [0 ... 0]
[The  Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8]
[The  Pinned Arena] max space (MB) used      spread across MPI: [0 ... 0]
[The   Comms Arena] max space (MB) allocated spread across MPI: [8 ... 8]
[The   Comms Arena] max space (MB) used      spread across MPI: [0 ... 0]

AlexanderSinn and others added 30 commits April 22, 2025 22:14
@AlexanderSinn AlexanderSinn added component: plasma About the plasma species performance optimization, benchmark, profiling, etc. labels Jul 9, 2026
@AlexanderSinn
AlexanderSinn requested a review from MaxThevenet July 9, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component: plasma About the plasma species performance optimization, benchmark, profiling, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant