[KV Connector] Canonical KV Cache Allocation for HMA Models - #37885

Open

Etelis wants to merge 38 commits into

vllm-project:mainfrom

Etelis:canonical-kv-caches

Commits on Mar 23, 2026

Add CanonicalKVCaches data classes for HMA KV cache representation
EtelisIBM
committed
52688de
Add WorkerConnectorInitializationData and initialize_worker_connector
EtelisIBM
committed
e25e020
Add canonical KV cache allocation for HMA models
EtelisIBM
committed
01b7897
Wire up canonical KV cache allocation in gpu_model_runner
EtelisIBM
committed
03903f1
Add unit tests for canonical KV cache allocation
EtelisIBM
committed
9697d1c
Fix mypy error: rename shadowed variable in use_canonical_kv_caches
EtelisIBM
committed
d9f7203
Move canonical KV cache dataclasses to connector base
EtelisIBM
committed
f39ae32
Address CR: relax group count check and use per-group spec
EtelisIBM
committed
68ce39a
Address CR: prioritize canonical path and scope initialize_worker_connector
EtelisIBM
committed
5b2b3bc
Address CR: merge loops in allocate_canonical_kv_caches
EtelisIBM
committed
3f90424
Address CR: always call initialize_worker_connector
EtelisIBM
committed
432d002

Commits on Mar 25, 2026

Validate tensor sizes in use_canonical_kv_caches
EtelisIBM
committed
77655ed
Merge branch 'main' into canonical-kv-caches
Etelis
authored
24b90c8

Commits on Mar 26, 2026

Merge branch 'main' into canonical-kv-caches
Etelis
authored
d44b920

Commits on Mar 30, 2026

Refactor canonical KV cache allocation into single-pass loop
EtelisIBM
committed
5a209c2

Commits on Apr 2, 2026

Simplify canonical KV cache allocation using physical buffer
EtelisIBM
committed
61c2e96

Commits on Apr 6, 2026

Move per-layer reshape logic into inner loop
EtelisIBM
committed
94fa930
Merge branch 'main' into canonical-kv-caches
Etelis
authored
2119286

Commits on Apr 12, 2026

Merge branch 'main' into canonical-kv-caches
Etelis
authored
d6fbfbf
Address CR: use single cross-layers int8 tensor for canonical KV caches
EtelisIBM
committed
ebe6311
Merge branch 'main' into canonical-kv-caches
Etelis
authored
16e28c2

Commits on Apr 13, 2026

Merge branch 'main' into canonical-kv-caches
Etelis
authored
fa03305

Commits on Apr 19, 2026

Commits on Apr 23, 2026

Address CR: restore canonical dataclasses in base.py and fix group page size
EtelisIBM
committed
f869fc6

Commits on Apr 26, 2026

Merge branch 'main' into canonical-kv-caches
Etelis
authored
7e6f1e5

Commits on Apr 28, 2026

Commits on May 4, 2026

Merge branch 'main' into canonical-kv-caches
Etelis
authored
18fb58b

Commits on May 12, 2026

Merge branch 'main' into canonical-kv-caches
Etelis
authored
afd03b6

Commits on Jun 2, 2026

Merge remote-tracking branch 'origin/main' into canonical-kv-caches
LucasWilkinson
committed
73ab712
fixes
LucasWilkinson
committed
76cefbd

Commits on Jun 3, 2026

Merge branch 'main' into canonical-kv-caches
Etelis
authored
f567c43
Merge branch 'main' into canonical-kv-caches
Etelis
authored
6220b0b
Merge branch 'main' into canonical-kv-caches
Etelis
authored
99c1c97
Merge branch 'main' into canonical-kv-caches
Etelis
authored
73edcc8
Merge branch 'main' into canonical-kv-caches
Etelis
authored
fba712c
Merge branch 'main' into canonical-kv-caches
Etelis
authored
757e0fc