docs: deploy LitGPT on Nebius Serverless Endpoints - #2342
Open
SalikovAlex wants to merge 2 commits into
Open
SalikovAlex wants to merge 2 commits into
SalikovAlex wants to merge 2 commits into
Conversation
SalikovAlex
marked this pull request as ready for review
September 28, 2026 14:24
SalikovAlex
requested review from
andyland,
k223kim,
lianakoleva and
t-vi
as code owners
September 28, 2026 14:24
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LitGPT's serving guide does not show how to deploy its OpenAI-compatible API to an authenticated Nebius Serverless Endpoint. This adds a linked tutorial and a self-contained container recipe using SmolLM2-135M-Instruct, pinned runtime dependencies and model revision, managed HTTPS with bearer-token authentication, readiness and streaming examples, and cleanup instructions.
Related scope proposal: #2341. This remains a draft for feedback on provider-specific placement.
Live validation exposed a deployment constraint: the endpoint service rejects image references longer than 64 characters with a Compute label error, including normal digest references. The tutorial therefore uses a short unique tag, verifies its remote digest against the build output, and explains that the tag must not be overwritten. It also covers Nebius Container Registry within the same project.
Validation:
/health: HTTP 200ok. Missing and incorrect tokens: HTTP 401 on both health and chat-completions routes.[DONE].git diff --check.This is a single-GPU deployment smoke test, not a throughput or scaling benchmark.