Forest fire

[jacob-moore22]

Pure-CMake build, Kokkos 5.2.1, swappable precision, host-side parallel macros

Summary

Replaces MATAR's bash build system with a self-contained CMake build so ELEMENTS and Fierro can consume
MATAR + Kokkos via FetchContent, add_subdirectory, or find_package. Trilinos support is removed
(retired under legacy/). On top of that: the bundled Kokkos moves 4.3.0 → 5.2.1 (and the language
standard to C++20), a compile-time-swappable floating-point precision system lands, host-side parallel
macros are added, and set_values is reimplemented on Kokkos::deep_copy.

Build system

One root CMakeLists.txt defines the header-only matar INTERFACE target (+ matar::matar alias). All
defines (HAVE_KOKKOS, HAVE_MPI, backend HAVE_*) and link dependencies attach to the target, so
consumers inherit them through target_link_libraries(app matar::matar).

Option	Default	Effect
MATAR_ENABLE_KOKKOS	ON	Kokkos-backed types (builds the bundled submodule)
MATAR_ENABLE_MPI	OFF	MPICArrayKokkos, CommunicationPlan
MATAR_ENABLE_GPU_AWARE_MPI	OFF	Assume GPU-aware MPI
MATAR_USE_EXTERNAL_KOKKOS	OFF	Use an installed Kokkos instead of the submodule
MATAR_BUILD_EXAMPLES / _TESTS / _BENCHMARKS	OFF	Developer targets
MATAR_REAL / MATAR_HIGH_REAL / MATAR_LOW_REAL	double	Precision tiers (below)

• Kokkos builds from the bundled submodule by default so backend selection stays under MATAR's
  control; external Kokkos is explicit opt-in. If a parent project already provides Kokkos::kokkos,
  MATAR uses it. Backends use standard Kokkos_ENABLE_*/Kokkos_ARCH_* — the --kokkos_build_type enum
  is gone (this also fixes cuda_mpi builds missing Matar_CUDA_BUILD).
• CMakePresets.json replaces the script's backend matrix: serial, openmp, pthreads, cuda,
  hip, *-mpi, *-debug — cmake --preset openmp && cmake --build --preset openmp && ctest --preset openmp.
• Install/export: find_package(Matar) works from an installed prefix; bundled Kokkos installs
  alongside so find_dependency(Kokkos) resolves. Legacy Matar_ENABLE_KOKKOS/MPI map with a deprecation
  warning; Matar_ENABLE_TRILINOS is a hard error pointing at legacy/.
• Dependencies: googletest and google/benchmark via pinned FetchContent. PT-Scotch moved from a
  configure-time shell script to ExternalProject_Add.
• Tests register via gtest_discover_tests(... DISCOVERY_MODE PRE_TEST), so builds never execute the
  test binary. MPI suite runs under mpirun -n 4.

Kokkos 5.2.1 + C++20

Submodule bumped 4.3.0 → 5.2.1; target_compile_features is now cxx_std_20. Source changes for the
Kokkos 5 API, plus suppression of incorrect nvcc warnings in the test sources.

Swappable precision (src/include/precision.h, new)

Three tier names whose meaning is fixed for the whole build:

Type	CMake var	Allowed
real_t	MATAR_REAL	double (default), float, half, bfloat16, quad
high_real_t	MATAR_HIGH_REAL	double (default), float, quad
low_real_t	MATAR_LOW_REAL	double (default), float, half, bfloat16

half/bfloat16 map to the Kokkos types (native 16-bit on CUDA/HIP/SYCL, float-backed elsewhere — reported
by MATAR_FP16_IS_EMULATED/MATAR_BF16_IS_EMULATED); quad is __float128, host backends only, needs
Kokkos_ENABLE_LIBQUADMATH. Non-Kokkos builds support double/float only. MPI types map each tier to the
right MPI_Datatype.

Host-side parallel macros

FOR_ALL_HOST, DO_ALL_HOST, RUN_HOST, FOR_REDUCE_{SUM,MAX,MIN,PRODUCT}_HOST,
DO_REDUCE_{SUM,MAX,MIN}_HOST, their _CLASS variants, and MATAR_FENCE_HOST() /
MATAR_FENCE_DEVICE(). These run on Kokkos::DefaultHostExecutionSpace and capture by reference, so
host-only objects (std::string, std::vector) work inside them and they can sit in a private member
function. On a no-Kokkos build they alias the serial macros.

The supported interop pattern is host macros on the .host() side of a dual type, then update_device()
(TestMacrosHost.DualTypeHostSide). Launching a device kernel and host work back-to-back without an
intervening fence composes correctly (TestMacrosHost.ConcurrentHostAndDeviceWork), but nothing asserts
the two actually overlap, and host/device macros over the same buffer is still a data race.

set_values

• Owning device types use Kokkos::deep_copy(array_, val) instead of a hand-rolled parallel_for.
• Dual types fill both sides and then clear_sync_state() — Kokkos has no deep_copy(DualView, scalar),
  and marking both sides modified trips DualView's concurrent-modification abort. Both sides now hold the
  value with no update_host() required.
• View types are KOKKOS_INLINE_FUNCTION ... const, so a View constructed inside a kernel can fill
  itself there (issue View set_values calls should be standard for loop #146). const is required because KOKKOS_LAMBDA captures by value into a
  non-mutable closure; without it the captured view is const and the call fails to compile.
• New test_set_values_deep_copy.cpp pins the contract the per-type tests could not (they all called
  update_host() before checking .host()).

Consuming MATAR

include(FetchContent)
FetchContent_Declare(matar GIT_REPOSITORY https://github.com/lanl/MATAR GIT_TAG <tag>)
FetchContent_MakeAvailable(matar)      # builds bundled Kokkos with your Kokkos_ENABLE_* flags
target_link_libraries(app PRIVATE matar::matar)

add_subdirectory and installed find_package(Matar) work identically; the plain matar target name is
kept. The tutorials use exactly this pattern and serve as living tests of it.

Bug fixes surfaced by the migration

• matar.h: mpi_types.h included only under HAVE_MPI.
• macros.h serial/no-Kokkos path (never previously CI-tested): _CLASS aliases pointed at nonexistent
  names; added missing serial RUN/RUN_CLASS, FOR_REDUCE_PRODUCT, 2D/3D reduce_prod.
• DynamicRaggedDownArrayKokkos::dims(): missing return path (UB in Release for dims(2)) and wrong
  assert bound.
• CArrayDevice_benchmark.cpp: used the long-removed REDUCE_SUM name.
• examples/matar_mpi.cpp: signed loop bounds (multi-dim FOR_ALL brace-initializes a signed
  MDRangePolicy).

Formatting + CI

.clang-format (Google base, 4-space, 150 cols, SortIncludes: Never) plus
formatting/matar-format.py, a post-processor producing the canonical parallel-macro layout
clang-format cannot. Applied across src/include/, solvers/, examples/, tutorial/, test/;
macros.h is excluded via .clang-format-ignore.

test.yml runs the preset matrix (serial/openmp × debug/release on Ubuntu + Mac, serial-mpi-debug with
the mpirun suite). cmake.yml covers no-Kokkos, bundled-Kokkos + install round-trip with a downstream
find_package(Matar) smoke test, and external-Kokkos.

Verification

• cuda-mpi: 317/317 tests pass, including the mpirun -n 4 suite
• Install round-trip + FetchContent/add_subdirectory consumer projects compile and run
• Tutorials build and run, including the Fortran-interop example

What this may break downstream

Anyone tracking main (ELEMENTS, Fierro) should expect the following:

1. C++20 is now required. matar exports cxx_std_20 as an INTERFACE feature, so it propagates to
   every consumer. Anything pinned to C++17 will fail to configure or hit new standard-conformance errors.
2. u_int is gone. It was a global using u_int = unsigned int; in aliases.h and no header defines
   it any more. Any downstream use fails to compile — the fix is unsigned int or a local typedef.
3. real_t moved to precision.h and is no longer unconditionally double. It is still available
   globally and still defaults to double, but a build that sets MATAR_REAL changes its meaning
   everywhere. Code that assumes real_t == double (MPI_DOUBLE literals, %lf formats, reinterpret
   casts, ABI across a library boundary) is now fragile.
4. Kokkos 4.3.0 → 5.2.1 is a major bump. Deprecated-in-4.x APIs that downstream code still calls may
   be removed. Kokkos_ENABLE_CUDA_LAMBDA is dropped (on by default in 5.x). Downstream projects
   providing their own Kokkos must supply 5.x, since MATAR headers now compile against that API.
5. Trilinos/Tpetra support removed. tpetra_wrapper_types.h, Tpetra_LRMultiVector_{decl,def}.hpp,
   Tpetra_LR_WrappedDualView.hpp, mapped_mpi_types.h, partition_map.h, mpi_types_old.h,
   communication_plan_old.h are deleted from src/include/. Any #include of them breaks;
   Matar_ENABLE_TRILINOS is now a hard CMake error.
6. The bash build scripts are gone (scripts/build-matar.sh and friends, now under legacy/). CI or
   developer workflows invoking them must move to the presets.
7. set_values on Dual types now populates both sides. Code that relied on the host side being
   stale after set_values (e.g. asserting a pending sync, or calling update_host() to detect a
   change) will observe different behavior. The values themselves are correct either way.
8. View set_values is now const. Additive for callers, but an override or an explicit
   void (View::*)(T) member-pointer will no longer match the signature.
9. Legacy CMake option names warn. Matar_ENABLE_KOKKOS/Matar_ENABLE_MPI still map to the new
   names but emit a deprecation warning; builds with -Werror-style CMake policies should update.

Follow-up work (not in this PR)

• ELEMENTS, then Fierro, migration to this build
• Re-test CUDA RDC, which Fierro's device function pointers and cross-TU device functions require
  (Build error with Cuda backend and C++ 20 when relocatable device code is enabled kokkos/kokkos#7779)