Add secure, transactional CUDA and WebGPU EP bootstrapping

[Baiju Meswani (baijumeswani)]

Summary

This change adds first-class CUDA execution-provider bootstrapping for Windows x64, Windows ARM64, and Linux x64. It also moves WebGPU onto the same reusable bundle installer so both providers follow one secure, transactional installation flow.

CUDA is treated as one compatible bundle assembled from independently reusable artifacts. The large CUDA and cuDNN packages change less frequently than the execution-provider package, so Foundry Local validates each installed artifact before downloading anything. If the CUDA or cuDNN runtime-file hashes are unchanged, those files are reused and only changed artifacts are downloaded. The reused and newly downloaded files are then assembled, verified, and atomically activated as one generation.

On Windows, provider dependencies are made discoverable at model-load time using SetDllDirectoryW. This avoids eagerly loading the full CUDA bundle while still supporting downstream bare-name DLL loads performed by ONNX Runtime GenAI. On Linux x64, libonnxruntime-genai-cuda.so is preloaded with RTLD_NOW | RTLD_GLOBAL and retained for the bootstrapper lifetime.

Bundle installer responsibilities

The shared installer provides one implementation for:

• Archive and runtime-file SHA-256 validation.
• Selective artifact reuse and repair.
• Immutable bundle generations.
• Atomic activation and rollback.
• Cross-process synchronization.
• Cancellation and cleanup.
• Secure archive extraction.

Architecture

The bootstrappers describe their bundles and handle provider-specific registration. The shared installer owns the package lifecycle.

EpDetector::DownloadAndRegisterEps()
|
+-- CudaEpBootstrapper::DownloadAndRegister()
|   +-- NvmlGpuDetector::HasNvidiaGpu()
|   +-- BuildCudaEpManifest(HostCudaEpPlatform())
|   `-- EpBundleInstaller::EnsureInstalled(cuda_manifest)
|
`-- WebGpuEpBootstrapper::DownloadAndRegister()
    +-- BuildWebGpuManifest()
    `-- EpBundleInstaller::EnsureInstalled(webgpu_manifest)

Shared bundle installation

EpBundleInstaller::EnsureInstalled(manifest)
|
+-- ValidateManifest()
|   +-- Validate bundle and artifact identifiers
|   +-- Require HTTPS package URLs
|   +-- Validate archive and runtime-file SHA-256 values
|   `-- Validate all relative paths
|
+-- Acquire the provider's cross-process file lock
|
+-- Read the active-generation marker
|
+-- Clean stale staging and inactive generation directories
|   `-- Reject symlinked or otherwise unsafe managed directories
|
+-- If the active bundle verifies
|   `-- Reuse it without downloading
|
`-- Otherwise create a private staging generation
    |
    +-- For each artifact
    |   +-- Copy its files from the active generation when they still verify
    |   `-- Otherwise download, authenticate, extract, and verify it
    |
    +-- Verify the exact completed runtime-file set
    `-- Atomically move staging into an immutable bundle generation

Activation and provider registration

The install transaction keeps the cross-process lock until registration succeeds or is rolled back.

EpInstallTransaction::Activate()
|
+-- Reverify the completed generation while holding the provider lock
`-- Atomically publish the active-generation marker

Provider registration
|
+-- Register the provider library with ONNX Runtime
|
+-- Success
|   `-- EpInstallTransaction::Finalize()
|       `-- Remove stale generations
|
`-- Failure
    `-- EpInstallTransaction::Rollback()
        `-- Restore the previous active-generation marker

The transaction retains its logger, so destructor-triggered rollback can report recovery failures instead of failing silently.

Provider registration and model loading

Provider registration
|
+-- CUDA on Linux x64
|   +-- Load libonnxruntime-genai-cuda.so with RTLD_NOW | RTLD_GLOBAL
|   +-- Retain the handle transactionally
|   `-- Register CUDAExecutionProvider
|
+-- CUDA on Windows
|   `-- Register CUDAExecutionProvider by absolute provider path
|
`-- WebGPU
    `-- Register Foundry.WebGPU by absolute provider path


ModelLoadManager::LoadModel()
|
+-- Resolve the effective execution provider
|   +-- Explicit provider override, when supplied
|   +-- CUDA for generic-gpu when CUDA is registered
|   +-- WebGPU for generic-gpu when CUDA is unavailable and WebGPU is registered
|   `-- Model-ID requirement for provider-specific variants such as cuda-gpu
|
+-- Verify the required provider is registered
|
+-- EpDetector::PrepareForModelLoad(required_ep)
|   `-- Windows CUDA/WebGPU: SetDllDirectoryW(bundle_directory)
|
`-- Construct the GenAI model

SetDllDirectoryW is process-global, but model loads are serialized and the directory is selected immediately before GenAI model construction. This supports GenAI's bare-name DLL loads without preloading the CUDA and cuDNN bundle.

CUDA platform support

Platform	Package layout	Status
Windows x64	CUDA toolkit, cuDNN, and CUDA EP archives	Supported
Windows ARM64	CUDA toolkit, cuDNN, and CUDA EP archives	Supported
Linux x64	CUDA EP and GenAI CUDA archive	Supported
Linux ARM64	No published bundle	Unsupported

WebGPU behavior

WebGPU uses the shared bundle installer on its supported platforms.

Generic GPU models continue to prefer CUDA when CUDA is registered. If CUDA is unavailable and WebGPU is registered, the model uses WebGPU. The Windows WebGPU bundle directory is prepared with SetDllDirectoryW before model construction so delayed dependencies such as DXCompiler can be discovered.

WebGPU uses the same:

• Archive and runtime-file verification.
• Immutable generations and atomic activation.
• Transactional rollback.
• Cross-process synchronization.
• Cancellation and cleanup.
• Secure extraction.
• Authenticated metadata handling.

Installation and repair behavior

Valid active bundle

Every declared runtime file is rehashed. If all hashes and the exact installed file set match, the existing generation is reused without downloading.

Partially damaged bundle

Each artifact is checked independently. Valid artifact files are copied into a new staging generation, while only invalid artifacts are downloaded again. The combined generation is fully verified before activation.

Forced installation

A forced request downloads every artifact instead of selectively reusing installed files.

Concurrent processes

Each provider uses its own OS-backed cross-process file lock. Only one process can install and register a given EP at a time. Other processes wait for the lock and then reuse the verified active generation. Different providers use different locks and may install concurrently.

The lock is held through provider registration and transaction finalization. OS locks are released automatically when a process exits; the lock file remaining on disk does not indicate that the lock is still held.

Failure or cancellation

Failed downloads, hash mismatches, malformed archives, cancellation, activation failures, and registration failures return failure for that provider. Temporary staging data is removed, and the previous active generation is restored when registration fails after activation.

EpDetector attempts the remaining requested providers after one provider fails. It stops early only when the caller cancels through the progress callback. The result reports both successfully registered and failed providers.

EP artifact downloads currently restart from the beginning after an interruption; resumable range downloads are not part of this change.

Recovery behavior

The installer is designed to recover without manual cache deletion:

• Incomplete downloads and extraction remain in unique staging directories.
• Stale staging directories are removed by the next lock owner.
• A generation is published only after complete verification.
• Unpublished immutable generations are cleaned on the next installation attempt.
• A failed registration restores the previous active marker.
• A process crash releases the OS file lock automatically.

Manual cleanup should only be necessary for external filesystem failures such as broken permissions, antivirus interference, or directories that the operating system refuses to remove.

Security and reliability

• HTTPS-only package manifests.
• Hard-coded SHA-256 values for archives and installed runtime files.
• Exact archive and installed file-set validation.
• Cross-process installation locks.
• Unique private staging directories.
• Immutable generation directories.
• Atomic active-marker replacement.
• Transactional activation and rollback.
• ZIP path-traversal protection.
• Symlink and special-file rejection.
• Duplicate-entry and file/directory collision rejection.
• Entry-count and uncompressed-size limits.
• HTTP body-size enforcement with or without Content-Length.
• Strict Content-Length parsing.
• Cancellation checks before provider registration.
• Logged destructor-triggered transaction rollback.
• WindowsStore-safe NVML behavior.
• No eager Windows CUDA or cuDNN DLL preload.

NVIDIA GPU detection

CUDA eligibility is detected through NVML. Foundry Local requires at least one NVIDIA device with the supported compute capability. NVML is loaded dynamically and released after detection.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview	Aug 7, 2026 9:49pm

[Copilot]

Pull request overview

Adds a shared, transactional installer for CUDA and WebGPU execution-provider bundles.

Changes:

• Adds verified, reusable bundle installation with atomic activation.
• Adds CUDA platform manifests, NVML detection, and dependency ownership.
• Replaces external ZIP extraction and expands security-focused tests.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
sdk_v2/cpp/vcpkg.json	Adds archive dependencies.
sdk_v2/cpp/CMakeLists.txt	Builds and links new components.
sdk_v2/cpp/src/ep_detection/cuda_ep_bootstrapper.cc	Installs and registers CUDA bundles.
sdk_v2/cpp/src/ep_detection/cuda_ep_bootstrapper.h	Updates CUDA bootstrapper ownership.
sdk_v2/cpp/src/ep_detection/cuda_ep_manifest.cc	Defines platform CUDA bundles.
sdk_v2/cpp/src/ep_detection/cuda_ep_manifest.h	Declares CUDA manifest APIs.
sdk_v2/cpp/src/ep_detection/ep_bundle_installer.cc	Implements transactional installation.
sdk_v2/cpp/src/ep_detection/ep_bundle_installer.h	Declares installer transactions.
sdk_v2/cpp/src/ep_detection/ep_bundle_manifest.h	Defines bundle metadata.
sdk_v2/cpp/src/ep_detection/ep_utils.cc	Manages Windows dependencies.
sdk_v2/cpp/src/ep_detection/ep_utils.h	Exposes dependency helpers.
sdk_v2/cpp/src/ep_detection/nvml_gpu_detector.cc	Implements NVML GPU detection.
sdk_v2/cpp/src/ep_detection/nvml_gpu_detector.h	Declares GPU detection APIs.
sdk_v2/cpp/src/ep_detection/webgpu_ep_bootstrapper.cc	Migrates WebGPU to bundles.
sdk_v2/cpp/src/ep_detection/webgpu_ep_bootstrapper.h	Updates WebGPU bootstrapper ownership.
sdk_v2/cpp/src/http/http_download.cc	Adds strict size handling.
sdk_v2/cpp/src/http/http_download.h	Extends downloader contract.
sdk_v2/cpp/src/manager.cc	Updates EP lifecycle and teardown.
sdk_v2/cpp/src/manager.h	Documents revised ownership order.
sdk_v2/cpp/src/util/zip_extract.cc	Adds in-process bounded extraction.
sdk_v2/cpp/src/util/zip_extract.h	Defines extraction limits.
sdk_v2/cpp/test/CMakeLists.txt	Registers new unit tests.
sdk_v2/cpp/test/internal_api/c_api_test.cc	Tests manager recreation.
sdk_v2/cpp/test/internal_api/cuda_ep_bootstrapper_test.cc	Tests CUDA manifests and behavior.
sdk_v2/cpp/test/internal_api/ep_bundle_installer_test.cc	Tests installation transactions.
sdk_v2/cpp/test/internal_api/ep_utils_test.cc	Tests dependency selection.
sdk_v2/cpp/test/internal_api/http_download_test.cc	Tests Content-Length parsing.
sdk_v2/cpp/test/internal_api/nvml_gpu_detector_test.cc	Tests capability filtering.
sdk_v2/cpp/test/internal_api/webgpu_ep_bootstrapper_test.cc	Tests WebGPU registration.
sdk_v2/cpp/test/internal_api/zip_extract_test.cc	Tests secure extraction.
sdk_v2/cpp/test/utils/scoped_environment_variable.h	Adds environment test helper.
sdk_v2/cpp/test/utils/zip_builder.h	Adds ZIP fixture builder.

[Scott McKay (skottmckay)]