Fix global LayerNorm mutation in model patching - #1360
Open
vedavyas2727 wants to merge 1 commit into
Open
Conversation
Clone model-local torch.nn namespaces before replacing LayerNorm, and update InternVL's cached norm factory so new model instances still use LigerLayerNorm without mutating torch.nn process-wide. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #315.
Prevent class-level LayerNorm patching for Mllama, Gemma3/PaliGemma (SigLIP), InternVL, and SmolVLM from replacing
torch.nn.LayerNormprocess-wide. Each Transformers modeling module now receives a privatetorch.nnnamespace whoseLayerNormalone points toLigerLayerNorm.Details
The affected modeling modules import
torch.nn as nn, so assigningmodeling_module.nn.LayerNormpreviously mutated the sharedtorch.nnmodule. That could unexpectedly change LayerNorm construction in unrelated models and application code after enabling Liger.The helper clones the modeling module's
nnnamespace before overridingLayerNorm. InternVL also cachesnn.LayerNorminNORM2FN, so that model-local factory is updated explicitly to preserve Liger patching for newly constructed vision layers.No benchmark is applicable because this changes patch isolation only; it does not alter a kernel or runtime computation path.
Testing Done
torch 2.12.0+cpu)make testto ensure correctness (CUDA/Triton kernel suite unavailable on this CPU-only host)ruff check --output-format=concise .andruff format --check --diff .make test-convergenceto ensure convergence (requires an accelerator)transformers 5.12.0:pytest test/transformers/test_monkey_patch.py -x -q— 68 passedtransformers 4.52.0: targeted class-level regression suite — 6 passed