Skip to content

Add Max Context Length control to instance launch Advanced Options - #2241

Open
alytaphoenix wants to merge 1 commit into
exo-explore:mainfrom
alytaphoenix:feature/context-window-ui
Open

Add Max Context Length control to instance launch Advanced Options#2241
alytaphoenix wants to merge 1 commit into
exo-explore:mainfrom
alytaphoenix:feature/context-window-ui

Conversation

@alytaphoenix

Copy link
Copy Markdown

Summary

Adds a "Max Context Length" control to the instance-launch Advanced Options panel (alongside Sharding Strategy / Interconnect / Minimum Devices), letting a user cap a model's context window below its native max before launching.

  • Model list now carries context_length (already returned by /models, just wasn't in the frontend's local type).
  • New numeric input, defaulting to and clamped to the selected model's native context_length; a "reset" link appears once reduced.
  • The chosen cap is sent as max_context_length on both the placement-preview request (/instance/previews?...&max_context_length=...) and the /place_instance fallback body used when no precomputed preview is available.
  • Resets to "native" whenever a different model is selected (a cap chosen for one model isn't meaningful for another).

Depends on #2240

This is the frontend half of the context-window feature; the backend half (max_context_length on PlaceInstanceParams, get_placement, get_placement_previews, and the placement/memory-validation logic) is in #2240, not yet merged.

Merge-order note: these two PRs are safe to merge in either order. If this one lands first, main's backend simply ignores the extra query param/body field (FastAPI drops unrecognized params silently) — the control renders and clamps, but has no effect on placement until #2240 lands. If #2240 lands first, this PR's control immediately becomes functional.

Test plan

  • npx svelte-check — same 15 pre-existing errors / 6 warnings as main (no new errors introduced; confirmed by diffing against a main baseline run)
  • npm run build — succeeds
  • Manual verification via headless Playwright against a local uv run exo (see caveat below):
    • Selecting a model pre-fills the input with its native context length (e.g. 262,144 for a 256K-context model)
    • Reducing the value triggers a re-fetch of /instance/previews with &max_context_length=<value> in the request, confirmed by intercepting the network request
    • Entering a value above the native max clamps the input back down to it
    • A "reset" link appears once the value differs from native, and clears the cap when clicked
  • Caveat: this branch is off main, which doesn't yet have Add configurable context-window with memory-safety validation #2240's backend changes — so this verification covers request formation and input clamping only, not the cap's actual effect on placement or memory validation (that requires Add configurable context-window with memory-safety validation #2240 merged).
  • nix fmt / nix flake check not run in this environment; ran prettier directly on the touched files instead.

🤖 Generated with Claude Code

https://claude.ai/code/session_011rjSfwDBTkmySmfU6NgHKF

Lets a user cap a model instance's context window below its native max
directly from the launch panel. Adds `context_length` to the fetched
model list, a numeric input (defaulting to and clamped by the model's
native context_length) alongside Sharding Strategy / Interconnect /
Minimum Devices, and threads the chosen value into both the placement-
preview request and the place_instance fallback request.

Depends on the max_context_length field added to PlaceInstanceParams /
get_placement_previews in exo-explore#2240 (not yet merged) — until that backend
change lands, main ignores the extra query param/body field, so this
control has no effect but also can't error.

Verified manually via Playwright against a local `uv run exo` (branched
from main, so the cap's actual placement effect isn't exercised here —
only request formation and input clamping): selecting a model
pre-fills the input with its native context length; reducing the value
re-fetches previews with `&max_context_length=<value>` and shows a
"reset" link; entering a value above the native max clamps back to it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011rjSfwDBTkmySmfU6NgHKF
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant