Add a Tenstorrent backend

[royc6]

Add a Tenstorrent backend

Adds a Tenstorrent backend alongside serial / OpenMP / CUDA, enabled with
-DUSE_TENSTORRENT=ON. gather and scatter are implemented; multi_* and the
atomic variants report unimplemented at run time. Three design choices below are yours to make, and I would rather change them now than after more is built on top of them.

Developed on a Tenstorrent Blackhole p150a with ttnn 0.72.0 / tt-metal v0.72.0,
against ec89237 (tag v2.1).

What was verified

All four runs are from a clean checkout of this branch, not from a working
directory:

Configuration	Result
Default build (backend off), full ctest	38 of 38 passed
-DUSE_TENSTORRENT=ON without -DTT_METAL_INCLUDE_DIRS	Stops at configure with the header-assembly diagnostic
-DUSE_TENSTORRENT=ON -DTT_METAL_INCLUDE_DIRS=..., build	Clean
The same build, ctest -R tenstorrent	6 of 6 passed

The first row is the one I would check first in your position: with the backend
off, the #ifdef USE_TENSTORRENT blocks compile out and nothing in the existing
suite changes.

One gap in that table: all four rows use the pip-wheel install with
hand-assembled headers, because that is the only install a shared cluster
without root allows. The ordinary tt-metal install, which is the path
TenstorrentSupport.cmake tries first, is untested by me, and is probably the
most useful thing for a reviewer with a standard setup to exercise.

Two integration defects were found and fixed during development, both invisible
to the compiler and worth mentioning because they are easy to reintroduce. The
-b validator in Input.hh listed only serial/openmp/cuda, so a valid backend
name was rejected at run time. And the Tenstorrent else if blocks in
Input.hh and JSONParser.cc had been placed inside the #ifdef USE_CUDA
region, so with CUDA off the preprocessor deleted them and the build stayed
clean while the branch did not exist. Both were caught with g++ -E, not by
whether the build succeeded.

Scope of correctness checking

Before the backend was wired in, a standalone binary running the same sequence
(link libtt_metal, build a program from kernel source, allocate device memory,
enqueue a 1x1 mesh workload, read back) was compared element by element against
a host reference over 8192 gathered elements, and matched exactly.

The integrated path is covered by the standard suite, which checks exit status
rather than gathered values. That is the same coverage the CUDA backend has, so
I have not added anything beyond it, but I would rather state it than let the
table above imply more. If you would like a numerical self-check in the backend
(an environment-gated compare against a host reference, so it stays out of the
timed path), it is a small addition and I am happy to write it.

Portability across Tenstorrent parts

Nothing binds to a particular board: there is no serial number, PCIe address or
slot anywhere in the backend. The device is chosen by SPATTER_TT_DEVICE
(default 0) together with TT_VISIBLE_DEVICES.

Most of what could be architecture-specific is queried at run time rather than
assumed: the compute grid from compute_with_storage_grid_size(), the DRAM
capacity used by the allocation guard from num_dram_channels() and
dram_size_per_channel(), and the per-core scratchpad (L1) ceiling implicitly,
since the backend
never references an L1 size and tt-metal's own
validate_circular_buffer_region enforces whatever the part has.

One constant is hardcoded and I would rather you told me how you want it
handled. kSlotBytes = 64 is the DRAM alignment, measured on Blackhole. The
consequences are asymmetric: on a part with a smaller alignment 64 is still a
valid multiple, so the mapping stays correct and only wastes space, but on a
part with a larger alignment it would silently corrupt every slot off the
boundary, which is how the value was found in the first place. tt-metal can
report the alignment, so querying it at device open is the obvious fix. I left
it as a constant because it also appears as a kernel compile-time argument and
because I could only test one alignment; say the word and I will make it a
runtime query.

Two further limits worth stating plainly:

• Single chip by construction. The backend opens a 1x1 unit mesh, so on a
  multi-chip board it uses one chip. Nothing about the design prevents a larger
  mesh, but nothing implements it either.
• Pinned to the tt-metal v0.72.0 application programming interface (API)
  generation. MeshDevice::create_unit_mesh
  and EnqueueMeshWorkload are what that version exposes; EnqueueProgram is
  not public API there, which is worth knowing since most tutorials still show
  it.

All of the above is reasoning from the code and from the tt-metal API, not from
testing: the only hardware available to me was a Blackhole p150a, so portability
to Wormhole or Grayskull is by construction and untested.

Three questions for you

1. Device allocations are opaque handles, not pointers. tt-metal allocates
MeshBuffer objects held by shared_ptr; there is no double* to hand back.
ConfigurationBase stores device allocations as double*& / size_t*, so the
backend keeps buffers alive in a registry and puts the MeshBuffer* in that
slot. Nothing on the host dereferences it, and the payoff is that
Configuration<Tenstorrent> is a line-for-line mirror of
Configuration<CUDA>, which keeps the diff small and reviewable. The
alternative is a typed device-allocation abstraction in ConfigurationBase,
which touches every backend. Happy to do that instead if you prefer it.

2. One element per 64-byte page, which costs both footprint and pattern
length. Blackhole's DRAM allocator aligns pages to 64 bytes, so an 8-byte
element occupies a full 64-byte slot. This has two consequences that are really
one decision:

• Device footprint is 8x the host array. -l 16777216 is 1 GiB on the host
  and about 8 GiB on a 32 GB card. There is a capacity check in
  tt_device_alloc that refuses with the numbers rather than failing inside the
  allocator, plus tenstorrent-ustride.json and tenstorrent-stream.json at
  1/32 the count of the GPU configurations.

• pattern_length is capped. The kernel stages one slot per pattern element
  in L1, so the per-core requirement is 68 * pattern_length + 111104 bytes
  (64-byte slot plus 4-byte index, from the two CreateCircularBuffer calls),
  against 1572864 bytes of L1 per Tensix. That gives pattern_length <= 21496;
  32768 and above stop in validate_circular_buffer_region. The formula matches
  four measured points to the byte, and throughput is flat right up to the limit
  (3224 / 3207 / 3220 MB/s at 4096 / 8192 / 16384), so the cap is not a
  performance cliff, just a ceiling.

  For scale, the largest pattern_length anywhere in standard-suite/ is 256,
  including the application traces, so nothing currently in the repository comes
  within 84x of it.

A larger page with in-kernel offset arithmetic removes the amplification and
raises the cap together, at the cost of an integer divide per element. Since
per-element issue rate is exactly what a gather benchmark measures, the cheap
mapping seemed the right default and the limit seemed better documented than
paid for. Chunking the pattern through L1 would lift the cap on its own if you
would rather keep the mapping and fix only that. Both are written up in
Build.md; tell me which trade you want.

3. How should CMake treat a pip-wheel install? A normal tt-metal install
ships host headers and libtt_metal together and is found in one step. A pip
ttnn wheel ships the library and the device-kernel headers but no host API
headers, so CMake finds the library and then cannot find the API.
TenstorrentSupport.cmake detects that combination, prints the assembly steps
(a sparse tt-metal checkout, the umd submodule, and five header-only
dependencies at their pinned versions, none of which need root) and then stops
at configure time rather than letting the build fail on a missing header. That
felt more useful than a bare "not found", but it is a lot of text for a CMake
message.

Continuous integration

build-tenstorrent.yml and run-crnch-tenstorrent.sh are copies of the CUDA
pair already in the repository (build-cuda.yml / run-crnch-cuda.sh), so a
Tenstorrent job follows the existing convention rather than prescribing new
infrastructure: a self-hosted runner that submits a Slurm batch job to the node
holding the accelerator, since GitHub-hosted runners have no such hardware. The
site-specific parts are placeholders in the same sense the CUDA ones are, to be
replaced with whatever the runner setup actually uses. Hardware-forced
differences:

• The Slurm generic resource (GRES) to request is r5accel:p150a, not gpu.
  The Blackhole cards share a chassis with an L40S, so asking for gpu gets the
  wrong device.
• TT_VISIBLE_DEVICES is what restricts execution to a single chip. It gates
  PCIe visibility ahead of enumeration by the user-mode driver (UMD), so a
  single-card allocation works;
  TT_METAL_VISIBLE_DEVICES does not, it gates only a diagnostic subset.
• TT_METAL_RUNTIME_ROOT is exported before the binary runs.

Question: the CUDA runner submits to partition rg-nextgen-hpc. Can that
partition schedule the Tenstorrent node? If not, I can reduce the workflow to a
build check until it can.

Possible pre-existing issue, flagging rather than asserting

tests/misc/run-crnch-cuda.sh configures with -DBACKEND=cuda -DCOMPILER=nvcc,
but neither BACKEND nor COMPILER appears to be read anywhere in the CMake on
main — BACKEND occurs only inside CudaBackend.hh's include guard. The
option the build consumes is USE_CUDA. If that reading is right, the CUDA
workflow has been configuring and testing the serial backend, which would pass
without exercising any CUDA code. I noticed it because the Tenstorrent script
started as a copy of that one; it uses -DUSE_TENSTORRENT=ON. Happy to send a
separate PR if you confirm.

Separately, CudaBackend.cu:142 sets threads_per_block = min(pattern_length, 1024). For pattern_length < 32 that runs partial warps: at
UNIFORM:8:1:NR on a GB10 the same access stream reads 67.5 GB/s against 272
GB/s at pattern_length >= 64, and 272139 * 8/32 = 68035 accounts for the
measured 67508 to within 0.8%. Short patterns are a common configuration, so
this may be worth a note in the documentation even if the behaviour is
intentional.

Notes for reviewers

• EnqueueProgram is not public API in tt-metal v0.72.0. A single card runs
  as a 1x1 unit mesh: MeshDevice::create_unit_mesh →
  MeshWorkload::add_program → EnqueueMeshWorkload. Most tt-metal tutorials
  still show the older entry point.
• Programs are cached across calls. Device kernels are compiled just in time
  on first
  enqueue, so building inside the timed region would measure the compiler. The
  timed region is exactly EnqueueMeshWorkload + Finish.
• No double-precision (FP64) arithmetic is needed. Blackhole has no FP64
  fabric, but gather and scatter do
  no arithmetic — the CUDA kernel's if (x == 0.5) is dead-code prevention — so
  each element moves as an opaque 8-byte payload.
• 64-byte alignment is load-bearing. A DRAM-to-L1 noc_async_read needs its
  L1 destination on that boundary. This is the origin of the slot stride in the
  kernels; packing tighter silently corrupts every slot that is not on the
  boundary, which is how it was found.
• The backend's caches are deliberately never destroyed. They hold
  MeshWorkload and MeshBuffer objects whose teardown order against
  libtt_metal's own singletons is unspecified, and releasing them from a
  static destructor segfaults at process exit. Exit reclaims the memory and
  closing the driver descriptor releases the card.
• cmake may not be on the system path. On the machine this was developed
  on it comes from the same Python environment as ttnn. Noted in Build.md.