Skip to content

Fix global LayerNorm mutation in model patching - #1360

Open
vedavyas2727 wants to merge 1 commit into
linkedin:mainfrom
vedavyas2727:vedavyas2727-upstream-quality-contribution
Open

Fix global LayerNorm mutation in model patching#1360
vedavyas2727 wants to merge 1 commit into
linkedin:mainfrom
vedavyas2727:vedavyas2727-upstream-quality-contribution

Conversation

@vedavyas2727

Copy link
Copy Markdown

Summary

Fixes #315.

Prevent class-level LayerNorm patching for Mllama, Gemma3/PaliGemma (SigLIP), InternVL, and SmolVLM from replacing torch.nn.LayerNorm process-wide. Each Transformers modeling module now receives a private torch.nn namespace whose LayerNorm alone points to LigerLayerNorm.

Details

The affected modeling modules import torch.nn as nn, so assigning modeling_module.nn.LayerNorm previously mutated the shared torch.nn module. That could unexpectedly change LayerNorm construction in unrelated models and application code after enabling Liger.

The helper clones the modeling module's nn namespace before overriding LayerNorm. InternVL also caches nn.LayerNorm in NORM2FN, so that model-local factory is updated explicitly to preserve Liger patching for newly constructed vision layers.

No benchmark is applicable because this changes patch isolation only; it does not alter a kernel or runtime computation path.

Testing Done

  • Hardware Type: Windows CPU (torch 2.12.0+cpu)
  • run make test to ensure correctness (CUDA/Triton kernel suite unavailable on this CPU-only host)
  • run checkstyle-equivalent ruff check --output-format=concise . and ruff format --check --diff .
  • run make test-convergence to ensure convergence (requires an accelerator)
  • transformers 5.12.0: pytest test/transformers/test_monkey_patch.py -x -q — 68 passed
  • Minimum supported transformers 4.52.0: targeted class-level regression suite — 6 passed
  • Independent code review completed; its InternVL cached-factory finding was addressed and the final review found no remaining significant issues.

Clone model-local torch.nn namespaces before replacing LayerNorm, and update InternVL's cached norm factory so new model instances still use LigerLayerNorm without mutating torch.nn process-wide.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mllama patch modifies nn.LayerNorm globally

1 participant