[ROCm] Enable seekdb+hipvs on AMD GPU

[zihaomu]

TL;DR

This PR lets seekdb run approximate nearest-neighbor (ANN) vector search on AMD GPUs
(gfx1100 / RDNA3) through hipVS (AMD's port of NVIDIA cuVS), while keeping the existing
CPU (VSAG) engine as the default and the safe fallback.

• OFF by default (OB_BUILD_CUVS=OFF) -> no new dependency, behavior identical to upstream.
• Declarative, per-index opt-in: CREATE VECTOR INDEX ... WITH (..., lib=cuvs).
• SQL-callable batched ANN: CALL dbms_vector.batch_knn(index_tbl, probe_tbl, k, out_tbl).
• The GPU path is always "correct or safe-fallback" (filters / deletes / non-L2 / staleness -> VSAG).

---

1. How it works (support principle)

seekdb's vector index is backed by VSAG (CPU HNSW). All vector operations flow through the
obvsag C-API in src/oblib/lib/vector/ob_vsag_adaptor.*
(create_index / add_index / knn_search / delete_index). We hook that seam.

1) Bridge (isolation layer). libseekdb_cuvs_bridge.so is a thin C shim that exposes plain
symbols (seekdb_cuvs_build/search/free, plus a one-shot batch entry) and internally runs a cuVS
CAGRA graph on the GPU via hipVS libcuvs_c. It hides all ROCm/cuVS headers from seekdb's
compiler (seekdb builds with a bundled clang; cuVS needs hipcc), so seekdb only ever sees C symbols.

2) Adaptor hooks. For an index that opted into the GPU path:

• add_index buffers the incoming vectors per index handle.
• knn_search lazily builds a CAGRA graph from that buffer (>= 256 points) and serves top-k
  from the GPU. The build + search run on a dedicated 32 MB-stack pthread because a cuVS graph
  build overflows the small (~1.5 MB) OB worker-thread stack.
• Correctness guards (the GPU path never returns wrong results):
  • Filters / deletes: cuVS returns the unfiltered top-k, then we post-filter with the same
    semantics VSAG uses; if any of the k rows is excluded, we fall back to VSAG (the true filtered
    top-k may rank beyond k).
  • Metric: the bridge builds an L2 graph, so only L2 indexes use the GPU; IP/cosine -> VSAG.
  • Freshness: the GPU index serves only when it matches the current buffer; if rows were added
    since the last build, we serve from VSAG (always fresh). This yields the split
    streaming delta -> VSAG, stable snapshot -> GPU.

3) Declarative per-index opt-in (lib=cuvs). We add ObVectorIndexAlgorithmLib::VIAL_CUVS and
accept lib=cuvs in the DDL. The storage plugin (ob_plugin_vector_index_adaptor) marks the
index handle for the GPU when the index is lib=cuvs + L2 + HNSW, and the adaptor hooks gate on that
per-handle mark -- not a global switch. So only indexes that declare lib=cuvs use the GPU;
every other index stays on VSAG in the same process.

4) Batch operator (dbms_vector.batch_knn). A PL system-package procedure that reads the probe
and index vectors via SQL, builds one CAGRA, issues one batched GPU search for all probes, and
writes neighbors to an output table (mysql-mode PL has no pipelined table functions, hence the
output table).

5) Build gating. OB_BUILD_CUVS (CMake option, OFF by default) controls everything. When OFF the
bridge .so is not linked (inert stubs satisfy the linker) and the GPU code never runs.

Why this design -- the key finding

We benchmarked end-to-end and found that single-query SQL gets no speedup on GPU: a well-built
CPU HNSW is already sub-millisecond, and the per-call GPU overhead (device alloc + PCIe copy + kernel
launch + sync) erases the edge. Batched ANN, on the other hand, wins by ~100-260x. That is why
the GPU value is exposed two ways -- transparent per-index acceleration for higher recall, and an
explicit batch operator for the workload where GPU decisively wins (similarity joins / bulk scoring).

---

2. Measured performance (AMD Radeon PRO W7900, gfx1100, ROCm 7.2.4)

Single vs batch (this is the whole story):

path	throughput	recall@10	note
single query, 100k CAGRA	2,148 q/s (0.466 ms)	--	~= VSAG-snapshot 0.41 ms -> no win
VSAG-CPU, per query (10k)	4,598 q/s	0.926	best single-query baseline
cuVS-GPU, per query (10k)	470 q/s	0.873	slower (per-call overhead)
cuVS-GPU, batch (10k, nq=100)	54,777 q/s	0.873	116.7x vs per-query; 12x vs best CPU
bridge batch sweep, nq=5000	634,930 q/s	--	263x, GPU saturated

• seekdb-native batch seam (cuvs_knn_search_batch, real adaptor, 10k index / 100 probes):
  per-query 470 q/s -> batch 54,777 q/s = 116.7x, recall identical (0.873), neighbors
  bit-identical (1000/1000).
• dbms_vector.batch_knn (SQL, 10k / 100 probes): recall@10 0.8690 (identical to the bridge
  batch), one GPU call; per-probe cost amortizes 3.3 ms -> 0.165 ms as batch size grows.
• Per-index correctness: 10k rows, lib=cuvs query recall@10 0.89 vs ground truth;
  WHERE / DELETE / cosine all safely fall back to VSAG with correct results.
• Per-index routing verified: with the server started without any env var, a lib=cuvs index
  serves on the GPU (cuvs_serve) while a lib=vsag index stays on VSAG (knn_simple, 0 GPU calls).

---

3. How to use it

Build with the backend on:

./build.sh release \
  -DOB_BUILD_CUVS=ON \
  -DCUVS_BRIDGE_LIB=/path/to/libseekdb_cuvs_bridge.so \
  --make -j64

Runtime: make the bridge and libcuvs reachable (LD_LIBRARY_PATH=/path/to/bridge:/opt/hipvs/lib,
or install + ldconfig). Requires an AMD gfx1100 GPU.

A) Declarative per-index GPU acceleration -- just add lib=cuvs:

ALTER SYSTEM SET ob_vector_memory_limit_percentage = 30;

CREATE TABLE t (
  c1 INT PRIMARY KEY,
  c2 VECTOR(128),
  VECTOR INDEX idx(c2) WITH (distance=l2, type=hnsw, lib=cuvs,
                             m=16, ef_construction=200, ef_search=64)
);
-- inserts + APPROXIMATE queries on this index run on the GPU; all other
-- indexes in the same server stay on VSAG. No environment variable needed.
SELECT c1 FROM t ORDER BY l2_distance(c2, '[...]') APPROXIMATE LIMIT 10;

B) Batched ANN operator -- one GPU call for many probes:

-- index_tbl and probe_tbl are (id INT, vec VECTOR(dim)); out_tbl is pre-created:
CREATE TABLE bk_out(probe_id BIGINT, neighbor_id BIGINT, distance FLOAT, rk INT);

CALL dbms_vector.batch_knn('index_tbl', 'probe_tbl', 10, 'bk_out');
SELECT * FROM bk_out ORDER BY probe_id, rk;

The bridge libseekdb_cuvs_bridge.so is built out-of-tree from hipVS
(hipvs-compile -shared -fPIC seekdb_cuvs_bridge.c); see
docs/gpu-vector-index-hipvs-cuvs/ for the source, the reproduction steps and the smoke tests.

---

4. What's in this PR

• Compile switch OB_BUILD_CUVS (+ CUVS_BRIDGE_LIB, + developer-only OB_BUILD_CUVS_TRACE).
• Adaptor GPU path in ob_vsag_adaptor.* (per-index mark, lazy CAGRA build/serve, safe fallback,
  batch seam, one-shot raw batch).
• DDL: VIAL_CUVS + lib=cuvs accepted for dense L2 HNSW (rejected for IVF).
• Plugin routing: mark lib=cuvs index handles for the GPU path.
• dbms_vector.batch_knn PL procedure.
• CI-safe DDL contract test + a GPU-required smoke template + design/benchmark docs.

When OB_BUILD_CUVS=OFF (default): the bridge is not linked, the GPU code never runs, and the
runtime behavior is identical to upstream VSAG.

---

5. Not included / follow-ups

• Snapshot persistence: rebuild the GPU index after an fdeserialize snapshot reload
  (currently a reloaded snapshot falls back to VSAG until the next in-process rebuild).
• Persistent GPU worker instead of a per-query pthread; reuse device buffers (cut single-query overhead).
• Memory: avoid keeping a second copy of the vectors in the GPU buffer for large tables.
• Stricter DDL: reject lib=cuvs with a non-L2 distance at DDL time; add a
  _enable_gpu_vector_index global kill-switch.
• Cleaner OFF build: fully #ifdef-out the OFF-path code (today it is compiled but gated by an
  inert stub + a runtime-false check).
• CI: regenerate the mysqltest .result via --record and add a GPU-present skip guard for the
  runtime tests; the GPU behavior test needs a GPU runner.
• batch_knn: cache the built index across calls so repeated calls skip the rebuild.
• Portability / perf: broaden GPU support beyond gfx1100 and enable performance-tuned cuVS
  (CK backend); the current bridge is dynamic-only and correctness-validated, not perf-tuned.

cc @zhangnju

Task Description

Solution Description

Passed Regressions

Upgrade Compatibility

Other Information

Release Note

[CLAassistant]

All committers have signed the CLA.