Add Max Context Length control to instance launch Advanced Options - #2241
Open
alytaphoenix wants to merge 1 commit into
Open
Add Max Context Length control to instance launch Advanced Options#2241alytaphoenix wants to merge 1 commit into
alytaphoenix wants to merge 1 commit into
Conversation
Lets a user cap a model instance's context window below its native max directly from the launch panel. Adds `context_length` to the fetched model list, a numeric input (defaulting to and clamped by the model's native context_length) alongside Sharding Strategy / Interconnect / Minimum Devices, and threads the chosen value into both the placement- preview request and the place_instance fallback request. Depends on the max_context_length field added to PlaceInstanceParams / get_placement_previews in exo-explore#2240 (not yet merged) — until that backend change lands, main ignores the extra query param/body field, so this control has no effect but also can't error. Verified manually via Playwright against a local `uv run exo` (branched from main, so the cap's actual placement effect isn't exercised here — only request formation and input clamping): selecting a model pre-fills the input with its native context length; reducing the value re-fetches previews with `&max_context_length=<value>` and shows a "reset" link; entering a value above the native max clamps back to it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011rjSfwDBTkmySmfU6NgHKF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a "Max Context Length" control to the instance-launch Advanced Options panel (alongside Sharding Strategy / Interconnect / Minimum Devices), letting a user cap a model's context window below its native max before launching.
context_length(already returned by/models, just wasn't in the frontend's local type).context_length; a "reset" link appears once reduced.max_context_lengthon both the placement-preview request (/instance/previews?...&max_context_length=...) and the/place_instancefallback body used when no precomputed preview is available.Depends on #2240
This is the frontend half of the context-window feature; the backend half (
max_context_lengthonPlaceInstanceParams,get_placement,get_placement_previews, and the placement/memory-validation logic) is in #2240, not yet merged.Merge-order note: these two PRs are safe to merge in either order. If this one lands first,
main's backend simply ignores the extra query param/body field (FastAPI drops unrecognized params silently) — the control renders and clamps, but has no effect on placement until #2240 lands. If #2240 lands first, this PR's control immediately becomes functional.Test plan
npx svelte-check— same 15 pre-existing errors / 6 warnings asmain(no new errors introduced; confirmed by diffing against amainbaseline run)npm run build— succeedsuv run exo(see caveat below):/instance/previewswith&max_context_length=<value>in the request, confirmed by intercepting the network requestmain, which doesn't yet have Add configurable context-window with memory-safety validation #2240's backend changes — so this verification covers request formation and input clamping only, not the cap's actual effect on placement or memory validation (that requires Add configurable context-window with memory-safety validation #2240 merged).nix fmt/nix flake checknot run in this environment; ranprettierdirectly on the touched files instead.🤖 Generated with Claude Code
https://claude.ai/code/session_011rjSfwDBTkmySmfU6NgHKF