[GPU] Enable post-op fusion for eltwise with FC/MatMul/Transpose when reshape place in middle - #36879
Open
clee30 wants to merge 4 commits into
Open
[GPU] Enable post-op fusion for eltwise with FC/MatMul/Transpose when reshape place in middle#36879clee30 wants to merge 4 commits into
clee30 wants to merge 4 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a new GPU graph transformation that sinks a unit-dimension Unsqueeze/unsqueeze-equivalent Reshape through a binary eltwise op so the producer (FC/MatMul/Transpose) can fuse the eltwise as a post-op, and integrates the pass into the Intel GPU transformations pipeline.
Changes:
- Implement
SinkUnsqueezeThroughEltwisemodel pass to rewriteEltwise(R, Unsqueeze(P)) -> Unsqueeze(Eltwise(Squeeze(R), P)). - Register the new pass in the Intel GPU
TransformationsPipeline. - Add unit tests covering FC/FC+bias/Transpose producers, plus a negative case.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| src/plugins/intel_gpu/tests/unit/transformations/sink_unsqueeze_through_eltwise_test.cpp | Adds unit tests validating the new sinking rewrite and a no-transform case. |
| src/plugins/intel_gpu/src/plugin/transformations/sink_unsqueeze_through_eltwise.hpp | Declares the new SinkUnsqueezeThroughEltwise pass and documents intended behavior. |
| src/plugins/intel_gpu/src/plugin/transformations/sink_unsqueeze_through_eltwise.cpp | Implements pattern detection and graph rewrite to sink unsqueeze-like reshapes below eltwise. |
| src/plugins/intel_gpu/src/plugin/transformations_pipeline.cpp | Registers the new pass in the GPU transformations pipeline. |
Contributor
|
@v-Golubev , could you please review? |
v-Golubev
reviewed
Jul 23, 2026
github-actions
Bot
added
the
category: transformations
v-Golubev
reviewed
Jul 29, 2026
Contributor
|
Please, do not create 2 matchers inside a single MatcherPass class. This is not how we normally write transformations and it complicates the view. |
Contributor
|
Please do work on writing the transformation. Take other transformations for inspiration. |
CuriousPanCake
requested changes
Aug 3, 2026
…e post-op fusion
Add a new MatcherPass (common transformations) that handles binary elementwise
ops where one input passes through a unit-dimension Unsqueeze/Reshape whose
producer is a "fusable" op (one that can absorb the eltwise as a post-op):
Eltwise(R, Unsqueeze(P, axis))
=> Unsqueeze(Eltwise(Squeeze(R, axis), P), axis)
so the eltwise operates in the lower rank and can be fused as a post-op on P.
This complements the existing MoveEltwiseUpThroughDataMovPerChannel pass, which
only handles the per-channel-constant case:
Eltwise(Reshape(P), Const_per_channel)
=> Reshape(Eltwise(P, Const_lower_rank))
The pass is configurable via constructor parameters:
- fusable_producer_types: list of DiscreteTypeInfo for fusable op types
- check_bias_add: whether to look through one level of bias Add
It is registered in the GPU transformations pipeline for FullyConnected,
MatMul and Transpose producers so the eltwise can be fused as a post-op of
those kernels, which was previously blocked by the rank-changing reshape.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A rank-expanding reshape (a unit-dimension unsqueeze) placed on one input of a binary elementwise Add blocks the producer + Add post-op fusion: the eltwise fusing pass only inspects the Add direct parents and skips rank-changing reshapes, so the Add stays a standalone kernel that re-reads both operands.
This pass rewrites
so the producer P feeds the Add directly in the lower rank and the Add can be fused as a sum post-op on P. The introduced Squeeze is the inverse of the unsqueeze emitted by the preceding op in the chain and is normally removed as a redundant reshape. Only genuine unit-dimension unsqueezes are matched; reshapes that also split or merge non-unit dimensions are left untouched.
The sink is applied when the producer P is a primitive whose GPU kernel can absorb the Add as a sum post-op: FullyConnected / MatMul (or its fused bias Add), and Transpose (Permute). The Transpose case covers the residual add on a transposed attention output, where the add can be fused into the permute kernel instead of running as a separate eltwise; without this the add is emitted as a standalone eltwise_simple_vload8 primitive per block.
Graph before transformation (For MatMul)
Graph after transformation (For MatMul)
Graph after transformation (For Transpose)
Tickets:
AI Assistance: