BUG: xinference 3.2.0 wheel is unimportable - KeyError: 'model_specs' (an audio model entry is packaged into llm_family.json)

[sfwani]

System Info / 系統信息

Any. Reproduced on python:3.12-slim with a clean pip install xinference==3.2.0.

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

• docker / docker
• pip install / 通过 pip install 安装
• installation from source / 从源码安装

Version info / 版本信息

xinference 3.2.0 (from PyPI)

The command used to start Xinference / 用以启动 xinference 的命令

xinference-local -H 0.0.0.0 -p 9997

Any entrypoint fails identically, because the error happens at import time.

Reproduction / 复现过程

pip install xinference==3.2.0
xinference-local -H 0.0.0.0 -p 9997

File ".../xinference/model/__init__.py", line 25, in _install
    llm_install()
File ".../xinference/model/llm/__init__.py", line 273, in _install
    load_model_family_from_json("llm_family.json", BUILTIN_LLM_FAMILIES)
File ".../xinference/model/llm/__init__.py", line 187, in load_model_family_from_json
    for spec in json_obj["model_specs"]:
                ~~~~~~~~^^^^^^^^^^^^^^^
KeyError: 'model_specs'

Cause

The published wheel's xinference/model/llm/llm_family.json contains one entry that is not an LLM
family and has no model_specs key:

import json
d = json.load(open('.../site-packages/xinference/model/llm/llm_family.json'))
print(len(d))
print([e['model_name'] for e in d if 'model_specs' not in e])

167
['speech_campplus_sv_zh-cn_16k-common']

That entry is an audio model. It is byte-for-byte identical to the entry of the same name in
xinference/model/audio/model_spec.json, where it also correctly appears.

This looks like a release-pipeline problem rather than a source problem. The v3.2.0 git tag is
clean:

Artifact	entries in llm_family.json	entries missing model_specs
v3.2.0 git tag	166	none
xinference-3.2.0-py3-none-any.whl from PyPI	167	1 (speech_campplus_sv_zh-cn_16k-common)

So the audio entry is being appended to the LLM family JSON somewhere between tagging and
publishing, most likely by whatever merges the model JSON updates at build time.

Expected behavior / 期待表现

pip install xinference==3.2.0 should produce a working install. Right now every xinference-*
entrypoint raises KeyError: 'model_specs' on import, so the release is unusable without editing
the packaged JSON by hand.

Workaround

import json
p = '<site-packages>/xinference/model/llm/llm_family.json'
json.dump([x for x in json.load(open(p)) if 'model_specs' in x], open(p, 'w'))

Suggested fix

Two things, independent of each other:

1. Fix whatever appends non-LLM entries to llm_family.json during the release build.
2. Make load_model_family_from_json fail loudly with the offending model name rather than a bare
   KeyError, and consider skipping entries that do not match the expected shape, so one bad entry
   cannot make the whole package unimportable:

for json_obj in json.load(codecs.open(json_path, "r", encoding="utf-8")):
    if "model_specs" not in json_obj:
        raise ValueError(
            f"{json_filename}: entry {json_obj.get('model_name')!r} has no 'model_specs'"
        )

[HongfaZhao]

一样的问题

[cai-chuan-dong]

一样的问题，这个什么时候更新呢

[ebarkhordar]

3.2.1 went up on PyPI at 2026-08-17T10:07:21Z and its wheel does not carry that entry, so an upgrade clears this without hand-editing the packaged JSON.

Checked both published wheels directly (pip download --no-deps, nothing installed for this part):

xinference-3.2.0-py3-none-any.whl   167 entries, 1 without model_specs: ['speech_campplus_sv_zh-cn_16k-common']
xinference-3.2.1-py3-none-any.whl   166 entries, 0 without model_specs

Then both versions installed in a clean python:3.12-slim container. 3.2.0 reproduces your traceback:

$ pip install xinference==3.2.0
$ python -c "import xinference.model"
  File "/usr/local/lib/python3.12/site-packages/xinference/model/__init__.py", line 34, in <module>
    _install()
  File "/usr/local/lib/python3.12/site-packages/xinference/model/llm/__init__.py", line 187, in load_model_family_from_json
    for spec in json_obj["model_specs"]:
KeyError: 'model_specs'

3.2.1, same image:

$ pip install xinference==3.2.1
$ python -c "import xinference; print(xinference.__version__, xinference.__file__)"
3.2.1 /usr/local/lib/python3.12/site-packages/xinference/__init__.py
$ python -c "import xinference.model; print('MODEL_IMPORT_OK_321')"
MODEL_IMPORT_OK_321
$ python -c "from xinference.model.llm import BUILTIN_LLM_FAMILIES as F; print('families:', len(F))"
families: 166

The 166 loaded families match the 166 entries in the packaged file.

One thing worth keeping separate if you are reading this thread for the state of the fix: #5353 adds a build-time check so this class of entry cannot be packaged again, and its own description says it does not change an already-published wheel. It is 3.2.1 that repairs the published artifact.

[github-actions]

This issue is stale because it has been open for 14 days with no activity.