SSIDS: front-size- and thread-adaptive CPU block size - #271
Open
haldaas wants to merge 1 commit into
Open
Conversation
The dense supernode factorization tiled every front with a single global cpu_block_size (default 256). The optimum is not constant: it depends on the front's size and on how many threads work it. Small/over-large tiles are both harmful -- tiny tiles shatter a front into a flood of dependent OpenMP tasks and go scheduler-bound (negative thread scaling), while over-large tiles starve parallelism. Add an opt-in adaptive rule, selected when cpu_block_size <= 0, that tiles each front to ~ m/(tau*P) rows per tile clamped to [b_min, b_max], where m is the front row count and P the size of the OpenMP team assigned to the subtree by the topology model (captured once at subtree entry and threaded explicitly through factor_node to the LDLT and Cholesky kernels). Positive cpu_block_size keeps the historic fixed-block behaviour unchanged. All tuning constants are fixed at build time via Meson options (no run-time environment variables): ssids_block_div (0 => thread-scaled), ssids_block_tiles_per_thread (tau), ssids_block_min and ssids_block_max. The floor b_min is, by default, chosen automatically at configure time by a BLAS efficiency probe (meson/blas_block_probe.cxx) -- the smallest square tile at which the local BLAS runs near peak -- so it adapts to the installed BLAS (e.g. 160 for OpenBLAS, 64 for netlib reference on the same machine). INNER_BLOCK_SIZE is now defined once in block_size.hxx and shared by both kernels. The driver honours SPRAL_NFACT to loop the factorization for stable timing of small matrices.
Contributor
Author
|
I might need to change the low and hi values for tuning parameters for the adaptive block selection during the meson build. Even with these potentially suboptimal values, performance gains were observed on an Apple Silicon M4 Max and AMD EPYC 7452. |
Contributor
|
Once again I leave this to @amontoison |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The dense supernode factorization tiled every front with a single global cpu_block_size (default 256). The optimum is not constant: it depends on the front's size and on how many threads work it. Small/over-large tiles are both harmful -- tiny tiles shatter a front into a flood of dependent OpenMP tasks and go scheduler-bound (negative thread scaling), while over-large tiles starve parallelism.
Add an opt-in adaptive rule, selected when cpu_block_size <= 0, that tiles each front to ~ m/(tau*P) rows per tile clamped to [b_min, b_max], where m is the front row count and P the size of the OpenMP team assigned to the subtree by the topology model (captured once at subtree entry and threaded explicitly through factor_node to the LDLT and Cholesky kernels). Positive cpu_block_size keeps the historic fixed-block behaviour unchanged.
All tuning constants are fixed at build time via Meson options (no run-time environment variables): ssids_block_div (0 => thread-scaled), ssids_block_tiles_per_thread (tau), ssids_block_min and ssids_block_max. The floor b_min is, by default, chosen automatically at configure time by a BLAS efficiency probe (meson/blas_block_probe.cxx) -- the smallest square tile at which the local BLAS runs near peak -- so it adapts to the installed BLAS (e.g. 160 for OpenBLAS, 64 for netlib reference on the same machine).
INNER_BLOCK_SIZE is now defined once in block_size.hxx and shared by both kernels. The driver honours SPRAL_NFACT to loop the factorization for stable timing of small matrices.