feat(cache): add resumable cache management - #5471
Merged
qinxuye merged 7 commits intoSep 2, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a new 'Download without launching' (cache-only) feature to Xinference, allowing users to download model artifacts without deploying them. It adds several REST endpoints for caching, pausing, resuming, and deleting downloads, along with a dedicated Cache Management page in the frontend. It also updates the permission guard, sidebar, and launch dialog to support this feature. Feedback on the frontend implementation highlights an issue with the polling mechanism in the Cache Management page, recommending a transition from window.setInterval to window.setTimeout with a local cancellation flag to prevent overlapping requests and memory leaks upon unmounting.
Track exact Hub repositories so interrupted downloads can be removed without deleting artifacts shared by cached models or other tasks.
Preserve task state when cancellation cannot finish, protect same-host worker repositories, and track auxiliary model downloads for safe deletion.
Minamiyama
force-pushed
the
FEAT/cache-download-management
branch
from
September 1, 2026 08:34
2404db8 to
7549d2d
Compare
qinxuye
reviewed
Sep 1, 2026
qinxuye
reviewed
Sep 1, 2026
Reject client-controlled internal download metadata before task persistence. Lazily create the cleanup lock for compatible launch paths and preserve original exceptions during guard cleanup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Model download progress, cached models, and virtual environments were previously managed through separate entry points. Interrupted downloads also lacked persistent pause, resume, and cleanup capabilities.
Changes
Cache Management Page
Download Task Management
XINFERENCE_DOWNLOAD_TASK_DB_PATHfor configuring the database path.Safe Cache Cleanup
API
Added or extended:
POST /v1/cache/modelsGET /v1/cache/models/{cache_uid}/progressPOST /v1/cache/models/{cache_uid}/cancelGET /v1/downloadsPOST /v1/downloads/{cache_uid}/pausePOST /v1/downloads/{cache_uid}/resumeDELETE /v1/downloads/{cache_uid}Compatibility
$XINFERENCE_HOME/download_tasks.db.