Skip to content

feat(cache): add resumable cache management - #5471

Merged
qinxuye merged 7 commits into
xorbitsai:mainfrom
Minamiyama:FEAT/cache-download-management
Sep 2, 2026
Merged

feat(cache): add resumable cache management#5471
qinxuye merged 7 commits into
xorbitsai:mainfrom
Minamiyama:FEAT/cache-download-management

Conversation

@Minamiyama

@Minamiyama Minamiyama commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Background

Model download progress, cached models, and virtual environments were previously managed through separate entry points. Interrupted downloads also lacked persistent pause, resume, and cleanup capabilities.

Changes

Cache Management Page

  • Added a new Cache Management page under Model Management.
  • Unified active, incomplete, and completed model downloads in the Model Cache tab.
  • Displayed model format, parameter size, quantization, status, progress, cache paths, and Worker information.
  • Added collapsible download-file details with animation.
  • Standardized action buttons as icons with tooltips.
  • Added a separate tab for virtual environment management.

Download Task Management

  • Added support for downloading model artifacts without launching a model instance.
  • Added pause, resume, cancel, and delete operations.
  • Persisted unfinished tasks so interrupted downloads remain resumable after a Supervisor or Worker restart.
  • Stored task state in SQLite using WAL mode.
  • Added XINFERENCE_DOWNLOAD_TASK_DB_PATH for configuring the database path.

Safe Cache Cleanup

  • Tracked primary, draft, ControlNet, GGUF, and Lightning repositories used by each task.
  • Protected shared repositories used by multiple Workers on the same host.
  • Preserved task state when cancellation times out instead of reporting success or deleting the record.
  • Prevented failed repository resolution or database writes from leaving stale running task IDs.

API

Added or extended:

  • POST /v1/cache/models
  • GET /v1/cache/models/{cache_uid}/progress
  • POST /v1/cache/models/{cache_uid}/cancel
  • GET /v1/downloads
  • POST /v1/downloads/{cache_uid}/pause
  • POST /v1/downloads/{cache_uid}/resume
  • DELETE /v1/downloads/{cache_uid}

Compatibility

  • Existing model launch APIs remain unchanged.
  • Pause and resume apply only to download-only tasks.
  • The default database path is $XINFERENCE_HOME/download_tasks.db.
  • Byte-range continuation depends on the selected model Hub SDK. If unsupported, the incomplete file is downloaded again.
d55867cdd2e63ebf0d754b6a8c0c2b73 6354675584a8ffabaad169b657123f16

@XprobeBot XprobeBot added this to the v3.x milestone Sep 1, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new 'Download without launching' (cache-only) feature to Xinference, allowing users to download model artifacts without deploying them. It adds several REST endpoints for caching, pausing, resuming, and deleting downloads, along with a dedicated Cache Management page in the frontend. It also updates the permission guard, sidebar, and launch dialog to support this feature. Feedback on the frontend implementation highlights an issue with the polling mechanism in the Cache Management page, recommending a transition from window.setInterval to window.setTimeout with a local cancellation flag to prevent overlapping requests and memory leaks upon unmounting.

Comment thread frontend/src/components/pages/cache-management/index.tsx
Track exact Hub repositories so interrupted downloads can be removed without deleting artifacts shared by cached models or other tasks.
Preserve task state when cancellation cannot finish, protect same-host worker repositories, and track auxiliary model downloads for safe deletion.
@Minamiyama
Minamiyama force-pushed the FEAT/cache-download-management branch from 2404db8 to 7549d2d Compare September 1, 2026 08:34
Comment thread xinference/core/supervisor.py
Comment thread xinference/core/worker.py Outdated
Reject client-controlled internal download metadata before task persistence. Lazily create the cleanup lock for compatible launch paths and preserve original exceptions during guard cleanup.

@qinxuye qinxuye left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qinxuye
qinxuye merged commit b86ec7e into xorbitsai:main Sep 2, 2026
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants