From d969f49acaf5c1d069fee61dd0b34d23e0eeb215 Mon Sep 17 00:00:00 2001 From: Alex Date: Mon, 10 Aug 2026 17:18:08 +0800 Subject: [PATCH] [Doc] Sync KV event medium terminology after #48123 The KV event medium wire contract for filesystem and object-store secondary tiers was collapsed from FS / OBJ to a single coarse-grained STORAGE value in PR #48123 (commit 77cba0259f, merged 2026-07-27). The user-facing docs/features/kv_offloading_usage.md still described FS and OBJ as the medium emitted by each tier, which would mislead any KV event subscriber implementing against the doc. This change updates the doc to: - state that both filesystem and object-store tiers emit medium STORAGE; - point readers at the optional tier locality field (introduced in #48281) for location semantics instead of the medium; - drop the ambiguous "OBJ does not imply REMOTE" sub-clause, which mixed tier-type terminology with event-medium terminology; - leave the Filesystem (FS) / Object Store (OBJ) section headings, the type: "fs" / type: "obj" config values, and the NIXL OBJ backend name untouched, since those are tier-type or NIXL-backend identifiers, not event-medium values. Refs: #47923, #48123, #48281 No protocol or runtime code change. Doc-only. Co-authored-by: Claude Opus 5 (1M context) Signed-off-by: Alex --- docs/features/kv_offloading_usage.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/features/kv_offloading_usage.md b/docs/features/kv_offloading_usage.md index 83c98c3d019f..41ecab462042 100644 --- a/docs/features/kv_offloading_usage.md +++ b/docs/features/kv_offloading_usage.md @@ -113,9 +113,9 @@ Then set `"eviction_policy": "my_policy"` in `kv_connector_extra_config`, the sa Each entry in `secondary_tiers` is a dict with a required `type` field plus tier-specific fields. -The filesystem and object-store tiers can publish hash-only `BlockStored` KV events for blocks they successfully store, tagged with a stable per-tier `medium` (`FS` for the filesystem tier, `OBJ` for the object-store tier). Set `enable_kv_events: true` in the tier's entry to opt in; events are published only when KV cache events are also enabled globally via `--kv-events-config`. +The filesystem and object-store tiers can publish hash-only `BlockStored` KV events for blocks they successfully store. Both tiers use the coarse-grained wire medium value `STORAGE`; the medium does not distinguish filesystem from object-store storage. To recover location semantics, set the optional `locality` field (`LOCAL` / `REMOTE`) on the tier entry — that field, not the medium, tells consumers whether the tier's blocks are local to the publishing vLLM instance. Set `enable_kv_events: true` in the tier's entry to opt in; events are published only when KV cache events are also enabled globally via `--kv-events-config`. -Set the optional `locality` tier field to `LOCAL` or `REMOTE` to describe the tier's storage location relative to the publishing vLLM instance. `LOCAL` marks storage local to that instance, while `REMOTE` marks storage that is not local to it. When the setting is omitted, locality is unspecified. vLLM does not infer it from the tier type, so an OBJ tier is not implicitly `REMOTE`. A KV event includes `locality` only when the tier explicitly configures it. This metadata describes the tier property without implying that a consumer can already route requests to its blocks. +Set the optional `locality` tier field to `LOCAL` or `REMOTE` to describe the tier's storage location relative to the publishing vLLM instance. `LOCAL` marks storage local to that instance, while `REMOTE` marks storage that is not local to it. When the setting is omitted, locality is unspecified. vLLM does not infer it from the tier type, so an `obj` tier is not implicitly `REMOTE`. A KV event includes `locality` only when the tier explicitly configures it. This metadata describes the tier property without implying that a consumer can already route requests to its blocks. ### Filesystem (FS) @@ -127,7 +127,7 @@ The filesystem tier (`type: "fs"`) writes blocks to a filesystem directory. | `root_dir` | yes | — | Base directory; vLLM creates subdirectories beneath it (see [On-Disk Layout](#on-disk-layout)). | | `n_read_threads` | no | `16` | Read-priority I/O threads (load path). | | `n_write_threads` | no | `16` | Write-priority I/O threads (store path). | -| `enable_kv_events` | no | `false` | Publish `BlockStored` KV events (medium `FS`) for successfully stored blocks. Requires KV cache events to be enabled globally. | +| `enable_kv_events` | no | `false` | Publish `BlockStored` KV events (medium `STORAGE`) for successfully stored blocks. Requires KV cache events to be enabled globally. | | `locality` | no | unspecified | `LOCAL` or `REMOTE` relative to the publishing vLLM instance. Included in the tier's KV events only when explicitly configured. | Each thread group prefers its own queue but pulls from the other when its primary queue is empty, so a write-heavy or read-heavy burst won't leave the off-priority queue waiting. Size the totals to your storage's effective concurrency. @@ -168,8 +168,8 @@ The object-store tier (`type: "obj"`) offloads blocks to an S3-compatible object | `store_config` | yes | — | Object store connection parameters (see below). | | `prefix` | no | `""` | Key prefix prepended to all object keys. | | `io_threads` | no | `4` | Number of NIXL OBJ backend I/O threads. | -| `enable_kv_events` | no | `false` | Publish `BlockStored` KV events (medium `OBJ`) for successfully stored blocks. Requires KV cache events to be enabled globally. | -| `locality` | no | unspecified | `LOCAL` or `REMOTE` relative to the publishing vLLM instance. Included in the tier's KV events only when explicitly configured; OBJ does not imply `REMOTE`. | +| `enable_kv_events` | no | `false` | Publish `BlockStored` KV events (medium `STORAGE`) for successfully stored blocks. Requires KV cache events to be enabled globally. | +| `locality` | no | unspecified | `LOCAL` or `REMOTE` relative to the publishing vLLM instance. Included in the tier's KV events only when explicitly configured. | `store_config` fields: