feat: batch text-only MLX VLM requests - #4918

Open

qinxuye wants to merge 13 commits into

xorbitsai:mainfrom

qinxuye:feat/mlx-vlm-batch

Commits on May 30, 2026

feat: batch text-only MLX VLM requests
qinxuye
committed
51a668d
fix: simplify MLX VLM text-only prompt check
qinxuye
committed
96cbe05
fix: guard MLX VLM batching against other modalities
qinxuye
committed
c81454c
fix: isolate MLX batch locks per instance
qinxuye
committed
b99fb5d
test: cap MLX VLM text-only generation length
qinxuye
committed
5ba0d9a
fix: reset MLX batch stream in worker thread
qinxuye
committed
5cb63ed
fix: bind MLX batch stream before batch ops
qinxuye
committed
15bb421
fix: use default MLX stream for batch generation
qinxuye
committed
176862b
fix: avoid mlx-lm batch stream wrapper
qinxuye
committed
d5524a2
fix: run MLX generation on default stream
qinxuye
committed
94cc31b
fix: pin mlx deps for metal stream regression
qinxuye
committed
d9fa727
test: cap mlx parallel generation
qinxuye
committed
a0c9d8b
fix: avoid idle mlx batch generation
qinxuye
committed
9f43d0b