Feature/Qwen3_VL by coder0143 · Pull Request #782 · sgl-project/sglang-jax

coder0143 · 2026-02-06T10:36:57Z

Motivation

Qwen3-VL model support. Issue here: #693

Modifications

Some changes from Qwen 2.5 VL:

Early fusion with deepstack process
3D Interleaved MRope
Learned + Rotary embedding
Timestamp tokens

Accuracy Tests

Benchmarking and Profiling

Checklist

[ x] Please use English, otherwise it will be closed.
[ x] The purpose of the PR, or link existing issues this PR will resolve.
[ x] The test plan, such as providing test command.
[ x] (Optional) The necessary documentation update.

…ect#777) * optim for vae encode and decode

gemini-code-assist · 2026-02-06T10:37:16Z

Summary of Changes

Hello @coder0143, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates the Qwen3-VL multimodal model into SGLang-JAX, significantly enhancing its capabilities to process and generate content based on both visual and textual inputs. The changes introduce a sophisticated vision encoder that leverages DeepStack for efficient feature extraction and a text decoder that incorporates a specialized multimodal rotary position embedding scheme. This foundational work paves the way for advanced multimodal AI applications within the JAX ecosystem.

Highlights

Qwen3-VL Model Support: Introduced comprehensive support for the Qwen3-VL multimodal model, enabling its use within the SGLang-JAX framework for high-performance distributed inference on TPUs.
DeepStack Feature Extraction: Implemented DeepStack functionality within the vision encoder, allowing for the extraction of visual features at intermediate layers for early fusion with the language model.
3D Interleaved Multimodal Rotary Position Embeddings (M-RoPE): Developed a novel M-RoPE mechanism for the text decoder, which partitions the head dimension into temporal, height, and width sections with an interleaved layout to handle multimodal sequences effectively.
New Vision Transformer Architecture: Added a dedicated vision transformer (Qwen3_VL_VisionTransformer) featuring 3D convolutional patch embeddings, bilinear interpolation for position embeddings, and 2D Rotary Position Embeddings for vision attention.
Configurability for Various Model Sizes: Provided configuration classes (Qwen3VLVisionConfig, Qwen3VLTextConfig, Qwen3VLConfig) to support different Qwen3-VL model sizes, including 2B, 4B, 8B, and 32B variants.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

python/sgl_jax/srt/models/registry.py
- Adjusted formatting of assert statements for improved readability and consistency.
python/sgl_jax/srt/multimodal/configs/qwen_vl/qwen3_vl_config.py
- Added new configuration classes (Qwen3VLVisionConfig, Qwen3VLTextConfig, Qwen3VLConfig) to define the architecture and parameters for various Qwen3-VL model sizes (2B, 4B, 8B, 32B).
python/sgl_jax/srt/multimodal/models/qwen3_VL/init.py
- Created an initialization file to expose the Qwen3-VL vision and generation components.
python/sgl_jax/srt/multimodal/models/qwen3_VL/qwen3_vl_generation.py
- Implemented the Qwen3-VL text decoder (Qwen3_VL_Generation) which extends Qwen2Model.
- Introduced MRotaryEmbedding for handling multimodal position embeddings with interleaved T/H/W sections.
- Added logic for loading weights specific to the Qwen3-VL text decoder, including Q/K norms.
python/sgl_jax/srt/multimodal/models/qwen3_VL/qwen3_vl_vit.py
- Implemented the Qwen3-VL vision encoder (Qwen3_VL_VisionModel) and its core Vision Transformer (Qwen3_VL_VisionTransformer).
- Developed 3D convolutional patch embeddings (Qwen3_VLVisionPatchEmbed) for processing visual inputs.
- Integrated bilinear interpolation and 2D Rotary Position Embeddings (Qwen3_VLVisionRotaryEmbedding) for vision attention.
- Included DeepStack feature extraction via Qwen3_VLVisionPatchMerger to merge spatial patches at intermediate layers.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

pengchengneo · 2026-02-06T13:57:02Z

great work @coder0143 ,
Could you please paste your launch server command and test script in this PR, I can help you test on tpu-v6e

coder0143 · 2026-02-06T13:59:30Z

On it!

coder0143 · 2026-02-10T19:28:07Z

Hello @pengchengneo , I have tested the model loading through the inference script:

!python -m sgl_jax.srt.multimodal.models.qwen3_VL.run_inference \
    --model-path Qwen/Qwen3-VL-2B-Instruct \
    --image-url "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.jpeg"

(It works!)

I have written a base script for inference which can be run by:

!python -m sgl_jax --model-path Qwen/Qwen3-VL-2B-Instruct --multimodal

(for multi-tpu), can you test it?

pengchengneo · 2026-02-11T06:24:08Z

@coder0143 , here is the log for the script

uv run python -m sgl_jax.srt.multimodal.models.qwen3_VL.run_inference  --model-path Qwen/Qwen3-VL-2B-Instruct --image-url "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.jpeg"
     Built sglang-jax @ file:///home/gcpuser/sgl-jax/python
Uninstalled 1 package in 0.32ms
Installed 1 package in 0.72ms
============================================================
Qwen3-VL Standalone Inference (sglang-jax)
============================================================
JAX devices: [TpuDevice(id=0, process_index=0, coords=(0,0,0), core_on_chip=0), TpuDevice(id=1, process_index=0, coords=(1,0,0), core_on_chip=0), TpuDevice(id=2, process_index=0, coords=(0,1,0), core_on_chip=0), TpuDevice(id=3, process_index=0, coords=(1,1,0), core_on_chip=0)]
Device count: 4
Backend: tpu
Mesh: Mesh('data': 1, 'tensor': 1, axis_types=(Explicit, Explicit))

1. Downloading model...
 Downloading model: Qwen/Qwen3-VL-2B-Instruct
chat_template.json: 5.50kB [00:00, 35.0MB/s]                                                                                                                                                                                                             | 0/12 [00:00<?, ?it/s]
preprocessor_config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 390/390 [00:00<00:00, 7.17MB/s]
config.json: 1.50kB [00:00, 14.9MB/s]s]
README.md: 7.14kB [00:00, 38.6MB/s]                                                                                                                                                                                                                   | 0.00/390 [00:00<?, ?B/s]
generation_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 269/269 [00:00<00:00, 4.59MB/s]
.gitattributes: 1.52kB [00:00, 17.2MB/s]
merges.txt: 1.67MB [00:00, 34.1MB/s]██████                                                                                                                                                                                                       | 1/12 [00:00<00:01,  8.11it/s]
tokenizer_config.json: 10.9kB [00:00, 70.0MB/s]
video_preprocessor_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 385/385 [00:00<00:00, 5.36MB/s]
tokenizer.json: 7.03MB [00:00, 100MB/s]
vocab.json: 2.78MB [00:00, 46.6MB/s]
model.safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.26G/4.26G [00:03<00:00, 1.22GB/s]
Fetching 12 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:03<00:00,  3.33it/s]
 Model downloaded to: /home/gcpuser/.cache/huggingface/hub/models--Qwen--Qwen3-VL-2B-Instruct/snapshots/89644892e4d85e24eaac8bacfd4f463576704203██████████████████████████████████████████████████████████████████████████████████████████| 4.26G/4.26G [00:03<00:00, 1.71GB/s]

2. Loading configs...
 Detected model size: 2b

3. Loading processor...

4. Loading vision model...
 Vision config: depth=24, hidden=1024, out_hidden=2048
 Initializing vision model...
2026-02-11 06:16:09,784 - sgl_jax.srt.multimodal.models.qwen3_VL.qwen3_vl_vit - INFO - Qwen3_VL_VisionModel initialized with dtype <class 'jax.numpy.bfloat16'>
2026-02-11 06:16:13,119 - sgl_jax.srt.utils.weight_utils - INFO - Scanning metadata for 1 model files (single host only)...
Scanning Metadata: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 44.28file/s]
2026-02-11 06:16:13,144 - sgl_jax.srt.utils.weight_utils - INFO - Starting parallel weight loading via JAX Lazy Loader...
Loading Regular Weights:   0%|                                                                                                                                                                                                                          | 0/316 [00:00<?, ?it/s]
2026-02-11 06:16:13,144 - sgl_jax.srt.utils.weight_utils - WARNING - No file found for weight: model.embed_tokens.weight
Loading Regular Weights: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 316/316 [00:01<00:00, 277.36it/s]
Loading MoE Weights: 0it [00:00, ?it/s]
2026-02-11 06:16:14,311 - sgl_jax.srt.utils.weight_utils - INFO - All weights loaded successfully.
2026-02-11 06:16:14,311 - sgl_jax.srt.multimodal.models.qwen3_VL.qwen3_vl_vit - INFO - Qwen3-VL Vision weights loaded successfully!
 Vision model loaded in 5.74s

5. Loading generation model...
 Text config: layers=28, hidden=2048, vocab=151936
 Initializing generation model...
2026-02-11 06:16:14,560 - sgl_jax.srt.utils.weight_utils - INFO - Scanning metadata for 1 model files (single host only)...
Scanning Metadata: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 45.19file/s]
2026-02-11 06:16:14,584 - sgl_jax.srt.utils.weight_utils - INFO - Starting parallel weight loading via JAX Lazy Loader...
Loading Regular Weights: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 310/310 [00:01<00:00, 170.86it/s]
Loading MoE Weights: 0it [00:00, ?it/s]
2026-02-11 06:16:16,522 - sgl_jax.srt.utils.weight_utils - INFO - All weights loaded successfully.
2026-02-11 06:16:16,523 - sgl_jax.srt.multimodal.models.qwen3_VL.qwen3_vl_generation - INFO - Qwen3-VL (LLM) weights loaded successfully!
 Generation model loaded in 2.21s

--- Text Input Test ---
 Input IDs shape: (1, 15)
 Input IDs (first 20): [151644, 872, 198, 3838, 374, 279, 6722, 315, 9625, 30, 151645, 198, 151644, 77091, 198]
 Total tokens: 15
 Embedding shape: (1, 15, 2048)
 Embedding dtype: bfloat16
 Embedding norm (first token): 0.578125
 ✓ Text embedding lookup successful!

--- Vision Encoding Test ---
 Processing image: https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.jpeg
 Input IDs shape: (1, 1038)
 Pixel values shape: (4096, 1536)
 Pixel values dtype: float32
 Image grid THW: [[ 1 64 64]]
 Running vision encoder...
 Vision embeddings shape: (1024, 2048)
 Vision embeddings dtype: bfloat16
 Vision embeddings norm (mean): 24.875000
 Vision encoding time: 18.45s
 ✓ Vision encoding successful!

============================================================
Summary
============================================================
 ✓ Model weights loaded from safetensors
 ✓ HuggingFace processor can prepare inputs
 ✓ Text decoder embeddings verified
 ✓ Vision encoder produces embeddings

For full auto-regressive generation, use the sglang-jax server:
 python -m sgl_jax --model-path Qwen/Qwen3-VL-2B-Instruct --multimodal
============================================================

pengchengneo · 2026-02-11T06:28:51Z

but when I try run command like
uv run python -m sgl_jax --model-path Qwen/Qwen3-VL-2B-Instruct --multimodal

it will report bug like below, seems it still has tiny bug here

Could please fix it and I will test it again. And could please paste e2e accuracy test result with evalscope here to show your implemention is aligned with Qwen3-VL paper's result, if you don't have resources for that, I can also help you test it

Fetching 12 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 177850.35it/s]
Fetching 8 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 135847.90it/s]
[2026-02-11 06:24:55] Loading stage config from: /home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/models/static_configs/qwen3_vl_stage_config.yaml
[2026-02-11 06:25:01] Stage-1 is initializing, Scheduler:auto_regressive, Params:{}
[2026-02-11 06:25:01] Multimodal mode enabled, disabling overlap schedule
[2026-02-11 06:25:01] Stage-0 is initializing, Scheduler:vit, Params:{}
[2026-02-11 06:25:01] Stage-0 hit exception: Traceback (most recent call last):
  File "/home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/manager/stage.py", line 129, in run_stage
    self._stage_scheduler = scheduler_class(
                            ~~~~~~~~~~~~~~~^
        communication_backend=comm_backend,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        **self.stage_config.scheduler_params,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/manager/scheduler/vit_scheduler.py", line 27, in __init__
    self.vit_worker = VitModelWorker(server_args, mesh=mesh, model_class=model_class)
                      ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/model_executor/vit/vit_model_worker.py", line 15, in __init__
    self.model_runner = VitModelRunner(server_args, self.mesh, model_class=model_class)
                        ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/model_executor/vit/vit_model_runner.py", line 30, in __init__
    self.initialize()
    ~~~~~~~~~~~~~~~^^
  File "/home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/model_executor/vit/vit_model_runner.py", line 33, in initialize
    self.load_model()
    ~~~~~~~~~~~~~~~^^
  File "/home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/model_executor/vit/vit_model_runner.py", line 37, in load_model
    self.model_config = get_qwen_vl_config(self.server_args.model_path)
                        ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/configs/config_registry.py", line 462, in get_qwen_vl_config
    return QwenVLConfigRegistry.get_config(model_path)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/home/gcpuser/sgl-jax/python/sgl_jax/srt/multimodal/configs/config_registry.py", line 411, in get_config
    raise ValueError(
    ...<3 lines>...
    )
ValueError: No VAE config found for model '/home/gcpuser/.cache/huggingface/hub/models--Qwen--Qwen3-VL-2B-Instruct/snapshots/89644892e4d85e24eaac8bacfd4f463576704203'. Available models: ['Qwen/Qwen2.5-VL-3B-Instruct']. You can register new models using QwenVLConfig.register().

[2026-02-11 06:25:01] No quantization config found in HF config or user config
Fetching 8 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 135847.90it/s]

coder0143 · 2026-02-11T12:26:43Z

Hello @pengchengneo , thankyou for testing, I don't have the resources for testing for e2e accuracy, and the sharding conventions in sgl-jax are quite different. Could you please review my model code along with the inference code and suggest changes to add appropriate sharding and test for e2e, thankyou.

pengchengneo · 2026-02-12T06:16:48Z

+        return hidden_states, residual, kv_fused, layer_callback_flag
+
+
+class Qwen3_VL_Model(nnx.Module):


Could this class extend Qwen3 from "python/sgl_jax/srt/models/qwen3.py" ?

pengchengneo · 2026-02-12T06:18:47Z

+    return x_t
+
+
+class MRotaryEmbedding:


This class is similar to Qwen2_5_VL's MRotaryEmbedding; could we reuse that instead?

this maybe optimized by Qwen3-Omni's PR, you can refer to #793

pengchengneo · 2026-02-12T06:19:50Z

+            mesh=self.mesh,
+            dtype=self.dtype,
+        )
+        weight_mappings = self._create_qwen3_weight_mappings()


resuse qwen3's weight mapping ?

pengchengneo · 2026-02-12T06:20:55Z

+            return (weight, weight)
+        return (self.model.embed_tokens.embedding.value, self.lm_head.embedding.value)
+
+    def set_embed_and_head(


this may not by used because it is used for Eagle's MTP algorithm

pengchengneo · 2026-02-12T06:20:59Z

+
+        return mappings
+
+    def get_embed_and_head(self):


this may not by used because it is used for Eagle's MTP algorithm

pengchengneo · 2026-02-12T06:26:00Z

Hello @pengchengneo , thankyou for testing, I don't have the resources for testing for e2e accuracy, and the sharding conventions in sgl-jax are quite different. Could you please review my model code along with the inference code and suggest changes to add appropriate sharding and test for e2e, thankyou.

OK, and Could please fix lint error and the bug I reported previously #782 (comment)

pathfinder-pf · 2026-02-12T06:35:13Z

Can you refer to this PR's test #743 to do some testing? @coder0143

pengchengneo · 2026-02-12T07:24:40Z

Can you refer to this PR's test #743 to do some testing? @coder0143

I think he doesnot have enough resources to test all sizes, I will help him test when he fix all bugs. @pathfinder-pf

Garrybest

Hi @coder0143, could you please test the precision of this model? Some Refs: https://github.com/sgl-project/sglang-jax/blob/main/python/sgl_jax/test/multimodal/test_qwen3_omni_moe_encoder.py#L23-L160

You can test it on CPU in order to make sure your model implement is right. And TPU machine is not compulsory. Make sure the output of forward in JAX is all close to that in torch transformers on CPU.

chaorders1 · 2026-02-17T18:35:56Z

Is this complete? Thanks

coder0143 · 2026-02-17T18:42:26Z

I just had an exam, have resumed working on this, bonsai model is fully done here: jax-ml/bonsai#161 with the sharding and tests. I have started working on some fixes, will then work on tests.

pengchengneo · 2026-02-24T03:50:34Z

@coder0143 any progress about this PR ?

coder0143 · 2026-02-24T04:03:40Z

Yup I'm working on it, will make the changes

pengchengneo · 2026-03-02T09:16:04Z

@coder0143 hello, any porgress? if you have no time , I can help you finish remain works

coder0143 · 2026-03-02T09:45:30Z

@pengchengneo Please do, I have the complete implementation done with optimal sharding configs (fsdp + tp) and multimodal jit support in bonsai(my fork is fully completed: link: https://github.com/coder0143/bonsai/tree/main/bonsai/models/qwen3_vl). You can check the tests here: https://www.kaggle.com/code/prathamshahmldlds/test-bonsai-qwen3-vl

pengchengneo · 2026-03-04T06:49:13Z

@coder0143 hi, please give me your githubid and github's email, once I finish this model, I will merge it myself and add coauthor info for you

coder0143 · 2026-03-05T15:23:54Z

@pengchengneo Thanks for taking it up and adding me as a co-author, my github email is: shahpratham688@gmail.com and github id is: 113518804 , username: @coder0143 . Btw, would love to be connected with you on linkedin!

coder0143 · 2026-03-07T04:39:55Z

@pengchengneo Also, I have started working on Qwen3.5 (hybrid GatedDeltaNet) in bonsai, will let you know once its done!

pengchengneo · 2026-03-07T05:13:57Z

@pengchengneo Also, I have started working on Qwen3.5 (hybrid GatedDeltaNet) in bonsai, will let you know once its done!

get it

Dineshkumar-Anandan-ZS0367 · 2026-03-16T09:18:07Z

Does sglang-jax supports qwen3-vl now for inferencing?

coder0143 · 2026-03-18T20:20:05Z

@Dineshkumar-Anandan-ZS0367 , the work is going on, there is an optimised bonsai implementation you can use here: https://www.kaggle.com/code/prathamshahmldlds/qwen3vl-in-jax?scriptVersionId=304494350

coder0143 and others added 5 commits February 6, 2026 15:21

base commit

1e04973

full revert

b459fe2

[Feat]optim vae decode phase to avoid OOM when large frames (sgl-proj…

691943d

…ect#777) * optim for vae encode and decode

initial commit

0ee885d

Merge branch 'sgl-project:main' into main

01d5bb1

pengchengneo self-requested a review February 6, 2026 13:57

coder0143 and others added 12 commits February 10, 2026 17:15

Running and server utils

e7666e3

Merge branch 'sgl-project:main' into main

ea5be85

add yaml

cf91788

test loading

2283d3b

no mesh

6a15c42

fix

aef9e46

fix

336ffce

fix

13b9cb0

refurbish

ad51d2e

fix

cb3be8e

Merge branch 'sgl-project:main' into main

23b0c2c

fix, verified loading

7f2d691

pengchengneo reviewed Feb 12, 2026

View reviewed changes

Garrybest reviewed Feb 13, 2026

View reviewed changes

pengchengneo mentioned this pull request Mar 11, 2026

[Feature] Qwen3-VL support on TPU #866

Closed

2 tasks

pengchengneo mentioned this pull request Apr 13, 2026

Development Roadmap (2026 Q2) #909

Open

52 tasks

sii-xinglong mentioned this pull request Jul 26, 2026

Development Roadmap (2026 Q3) #1460

Open

46 tasks

		return hidden_states, residual, kv_fused, layer_callback_flag


		class Qwen3_VL_Model(nnx.Module):

Uh oh!

Conversation

coder0143 commented Feb 6, 2026

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Uh oh!

gemini-code-assist Bot commented Feb 6, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

pengchengneo commented Feb 6, 2026

Uh oh!

coder0143 commented Feb 6, 2026

Uh oh!

coder0143 commented Feb 10, 2026

Uh oh!

pengchengneo commented Feb 11, 2026

Uh oh!

pengchengneo commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coder0143 commented Feb 11, 2026

Uh oh!

pengchengneo Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pengchengneo Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

pengchengneo Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

pengchengneo Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

pengchengneo Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

pengchengneo Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

pengchengneo commented Feb 12, 2026

Uh oh!

pathfinder-pf commented Feb 12, 2026

Uh oh!

pengchengneo commented Feb 12, 2026

Uh oh!

Garrybest left a comment

Choose a reason for hiding this comment

Uh oh!

chaorders1 commented Feb 17, 2026

Uh oh!

coder0143 commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pengchengneo commented Feb 24, 2026

Uh oh!

coder0143 commented Feb 24, 2026

Uh oh!

pengchengneo commented Mar 2, 2026

Uh oh!

coder0143 commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pengchengneo commented Mar 4, 2026

Uh oh!

coder0143 commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coder0143 commented Mar 7, 2026

Uh oh!

pengchengneo commented Mar 7, 2026

Uh oh!

Dineshkumar-Anandan-ZS0367 commented Mar 16, 2026

Uh oh!

pengchengneo commented Feb 11, 2026 •

edited

Loading

pengchengneo Feb 12, 2026 •

edited

Loading

coder0143 commented Feb 17, 2026 •

edited

Loading

coder0143 commented Mar 2, 2026 •

edited

Loading

coder0143 commented Mar 5, 2026 •

edited

Loading