Skip to content

fix: keep developer messages in DeepSeek-V4 thinking drop - #5015

Open
longzhenren wants to merge 1 commit into
InternLM:mainfrom
longzhenren:fix/deepseek-v4-keep-developer
Open

longzhenren wants to merge 1 commit into
InternLM:mainfrom
longzhenren:fix/deepseek-v4-keep-developer

Conversation

@longzhenren

Copy link
Copy Markdown

Root cause

_drop_thinking_messages in deepseek_v4_encoding.py documents that "Developer messages before the last user are dropped entirely", and its keep_roles set indeed lacks developer. But the rest of the module supports the role: the renderer has a developer branch (rendered like a user turn) and find_last_user_index counts it.

So in thinking mode with drop_thinking=True (the default), a developer message before the last user turn is silently removed from the prompt: the instructions never reach the model, and encoding a conversation with or without the developer message produces the same output.

Reproduction:

from lmdeploy.deepseek_v4_encoding import encode_messages

base = [
    {'role': 'system', 'content': 'You are helpful.'},
    {'role': 'user', 'content': 'Real question'},
]
with_dev = [base[0], {'role': 'developer', 'content': 'Always answer in French.'}, base[1]]

a = encode_messages(base, thinking_mode='thinking')
b = encode_messages(with_dev, thinking_mode='thinking')
assert 'Always answer in French.' not in b  # developer instructions silently lost

Fix

Add developer to keep_roles and update the docstring, consistent with how the renderer and find_last_user_index already treat the role.

Test

  • test_deepseek_v4_thinking_drop_keeps_developer_message: [system, developer, user] in thinking mode renders the developer content.
  • test_deepseek_v4_thinking_drop_keeps_developer_in_multiturn: a developer message before the last user turn survives the drop while assistant reasoning_content is still stripped.
  • Both tests fail on the current code and pass with the fix; the existing test file still passes (14 passed).

Copilot AI balanced review requested due to automatic review settings October 1, 2026 18:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants