Roadmap: free-threaded CPython, abi3t, and alternative Python binding generators

[hjmjohnson]

Roadmap issue, not a work item — nothing here is actionable until Python 3.15 ships. PR #6715 makes ITK Python wrapping abi3 by default while keeping a version-specific build path. This issue records why that path exists, when it can be deleted, and what else in this space ITK does and does not intend to support.

Tracking statement:

The non-abi3 path exists for one documented, time-limited reason, and should be removed when Python 3.15 and CMake 4.4 make abi3t reachable.

Why the framing is "one documented reason" rather than "distributions might want it"

Every candidate justification for keeping a non-abi3 build was checked against primary sources. Exactly one is documented:

Candidate reason	Status
Free-threaded CPython 3.13t / 3.14t	Documented (PEP 803). The only one.
Py_DEBUG builds	Not documented — the CPython stable-ABI page names no debug-build exclusion, only platform/compiler considerations.
CMake < 3.26	Documented in CMake, but a toolchain constraint rather than a Python one.
Linux distributions / conda-forge	No documentation found requiring it. conda-forge documents abi3 support, and conda-forge/itk-feedstock currently builds only ____cpython variants (python 3.10/3.11/3.12 × hdf5 × platform — no free-threaded t variants).
Performance under Py_LIMITED_API	Unmeasured, and judged not a concern for ITK's coarse-grained call pattern.
PyPy / GraalPy	Would require a build that is neither abi3 nor abi3t, but nothing documents ITK needing to support them. Out of scope — see below.

From PEP 803:

"The Stable ABI is currently not available for free-threaded builds. Extensions will fail to build for the Stable ABI on free-threaded Python (that is, when both Py_LIMITED_API and Py_GIL_DISABLED preprocessor macros are defined)."

Combined with abi3t targeting 3.15+, free-threaded CPython 3.13t/3.14t sits in a gap where abi3 is a build failure and abi3t does not yet exist. The only buildable configuration there is version-specific.

Stating it this narrowly is what makes the option deletable later: a maintainer gets a named expiry condition instead of an open-ended "someone might need this."

Phase 1 — now: abi3 available everywhere, default not yet changed

PR #6715 makes ITKVtkGlue stable-ABI clean, removing the last module that
could not be built with ITK_USE_PYTHON_LIMITED_API=ON. Its typemaps use
Limited API calls only, which are valid in a full-API build too, so one
implementation serves both modes.

Not yet done, and deliberately out of scope for #6715:

• Change ITK_USE_PYTHON_LIMITED_API to default ON. Prototyped and
  verified locally (both modes build 8565/8565 and pass 177/177
  ctest -R Python on macOS arm64), but held back as separate work.
  This also lets find_package(Python3 ...) run once instead of twice:
  the current default is derived from the discovered interpreter
  version, which forces a second search once the components are known.
• Record the PEP 803 expiry alongside the option, so a future maintainer
  knows the OFF path is deletable rather than open-ended.
• Note for whoever does this: Modules/Core/Common/src/CMakeLists.txt
  uses _itk_python_target at two sites. Updating only the first
  breaks ITK.macOS.Python with undefined _Py_BuildValue — it fails
  there and nowhere else because only a shared macOS build links
  libITKCommon strictly enough to notice.

Phase 2 — gated on Python 3.15 + CMake 4.4: add abi3t

abi3t is a subset of abi3's allowed APIs and a superset of its compatible interpreters, so the abi3 work in #6715 is a strict prerequisite. It is not a rebuild: PEP 803 requires both Py_LIMITED_API and Py_TARGET_ABI3T, and recommends shipping both variants under the wheel tag abi3.abi3t.

CMake already models this and no new find_package component is needed — Development.SABIModule and USE_SABI are unchanged. From the FindPython3 documentation (added in CMake 4.4):

"When USE_SABI is specified, if Python3_FREE_THREADED is true and Python version is 3.15 or upper, Py_TARGET_ABI3T preprocessor definition will be defined rather than Py_LIMITED_API."

Work items when the gate opens:

• Guard the free-threaded-below-3.15 gap. For a free-threaded 3.13t/3.14t interpreter with USE_SABI specified, CMake still defines Py_LIMITED_API and hands you the failing compile. ITK should detect Python3_FREE_THREADED and select the version-specific path automatically, so users are not required to know the option exists.
• Add a conditional CMake floor of 4.4 inside the abi3t branch only. The 3.26 floor for Development.SABIModule stays for everything else; do not raise ITK_OLDEST_VALIDATED_POLICIES_VERSION (currently 3.22.1).
• Remove ITK_USE_PYTHON_LIMITED_API once every supported interpreter can do abi3 or abi3t.
• ITKPythonPackage: a second artifact per platform for abi3.abi3t, partially offsetting the wheel-count reduction abi3 enables (relevant to Create weekly builds with python wrapping for ITK and its modules, and submit them to pypi #4656).

Phase 3 — gated on a thread-safety audit (the larger piece of work)

Buildable under free-threading is not the same as correct under free-threading. Removing the GIL removes the implicit serialisation that ITK's Python layer relies on today. Un-audited areas, at minimum:

• itk::ObjectFactoryBase registration and the factory registry
• the SWIG type registry
• the itkPyCommand callback path

This is expected to be substantially more work than the ABI change, and should not be conflated with it.

Alternative binding generators — scope

nanobind — gated on raising ITK's wrapping floor to Python 3.12, since nanobind's stable-ABI mode requires 3.12 while ITK_WRAP_PYTHON_MINIMUM_VERSION is 3.11. Not actionable before that floor moves.

pybind11 — no Limited API support.

PyPy / GraalPy — explicitly out of scope unless someone brings a concrete use case. PyPy's benefit is JIT-compiling hot pure-Python loops, but ITK's Python layer is a thin SWIG shim over C++ with no such loop; PyPy runs C extensions through cpyext, whose per-call overhead is worse than CPython's, and the C-API boundary is exactly where ITK spends its time. No Discourse thread, issue, or downstream project (Slicer, SimpleITK, ANTs) has requested it. Recorded here so the question does not need re-litigating in every wrapping discussion.

References: PEP 803 · CPython stable ABI · CMake FindPython3 · quansight: abi3t

Raised from review discussion on PR #6715 (@blowekamp asked about free-threading implications; @thewtex noted abi3t will need a separate build). Related: #6711, #4656.