Skip to content

Fixed the garbled text issues in Qwen3-8B - #2782

Closed
lizexu123 wants to merge 6 commits into
PaddlePaddle:developfrom
lizexu123:base_attention
Closed

Fixed the garbled text issues in Qwen3-8B#2782
lizexu123 wants to merge 6 commits into
PaddlePaddle:developfrom
lizexu123:base_attention

Conversation

@lizexu123

Copy link
Copy Markdown
Collaborator

修复了Qwen3-8b乱码问题, tie_word_embeddings一直为True的情况,考虑了tie_word_embeddings为False的情况,并且添加了权重lm_head.weight为列切的方式加载。修复后Qwen3-8b精度正确

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@paddle-bot paddle-bot Bot closed this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants