[GPU][MLIR] MLIR Graph compiler integration by dchigarev · Pull Request #35336 · openvinotoolkit/openvino

dchigarev · 2026-04-14T13:41:55Z

I. Brief description

The PR adds an optional MLIR-based execution path to the GPU plugin. A new stage called transformMLIR is added to the GPU's transformation pipeline that matches suitable subgraphs in ov::Model, converts them to mlir-module(s) using linalg-dialect, and inserts an ov::op::MLIROp operation representing the converted subgraph:

an illustration

ov::Model transformation made by the transformMLIR stage:

Important to note, that an actual compilation and execution of the converted MLIR module happens in a separate project called graph-compiler (GC) - the project is "ingress-agnostic" and doesn't depend on OV specifically, it handles an arbitrary mlir-linalg-code as an input and produces a GPU-binary combined with a cpu-side launching code (using opencl runtime). The Openvino side is only responsible for matching a suitable subgraph in ov::Model, converting it to MLIR as is (all the optimizations are made on the graph-compiler side), and providing the runtime-info on inference (opencl queue/context handlers, buffers, etc).

The ov::op::MLIROp naturally follows the OVs compile/infer semantic: on ov::Model::compile() the MLIR module is fully compiled to a binary (even if the module has dynamic shapes), on infer() it launches the compiled binary (no extra latency).

Supported subgraphs

The main operation that we were focused on was SDPA, we've thoroughly tested this case, spent efforts on tuning its performance and have a good understanding on how it behaves (~30% faster than gpu-native impl on BMG).

SDPA is the only operation that is enabled via MLIR path by default.

The MLIR path supports a lot more operations (full list here), we have unit tests for them, but never tested them on a "real model".

Enabling/disabling certain matching patterns can be controlled via OV_MLIR_PATTERNS env variable. For example OV_MLIR_PATTERNS='mart=MatMul,Add,Reshape,Transpose;rms=Power,ReduceMean,Add,Sqrt,Divide' would match projection subgraphs; Setting the variable to an empty string enables conversion for every supported operation.

Feature enabling

The feature is disabled by default and gated twice:

at build time by -DENABLE_GRAPH_COMPILER=ON (default OFF) - when off, no MLIR related implementations are compiled (no pattern matching/conversion/unit tests/inference logic). The new MLIR-related definitions are still included to the build though (ov::op::MLIROp or cldnn::mlir_primitive header files) to avoid sudden broken includes. We're open to discuss this to decide, how to better gate the mlir-related code in OV.
at runtime by ov::intel_gpu::enable_mlir property (env variable OV_GPU_ENABLE_MLIR) which is also false by default.

Lib size increase

libopenvino_intel_gpu_plugin.so:

master branch: 45300 KiB (~44.2mb)
mlir branch: (mlir disabled): 45360 bytes (~44.2mb) 60kb increase
mlir branch: (mlir enabled): 146112 bytes (~142.6mb) 98.4mb increase

II. Changes in common / existing code

Under spoiler

1. New op: `ov::intel_gpu::op::MLIROp`

src/plugins/intel_gpu/include/intel_gpu/op/mlir_op.hpp — public declaration, next to the
plugin's other internal ops, it's free of any MLIR/GC includes.
src/plugins/intel_gpu/src/plugin/transformations/op/mlir_op.cpp — implementation
(validate_and_infer_types, shape_infer, clone_with_new_inputs, evaluate,
MemRefDescriptor argument packing). Guarded by #ifdef GRAPH_COMPILER.
Registered as an internal factory in primitives_list.hpp
(REGISTER_FACTORY(internal, MLIR), under #ifdef GRAPH_COMPILER) and created by
src/plugins/intel_gpu/src/plugin/ops/mlir_op.cpp.

2. New primitive: `cldnn::mlir_primitive`

include/intel_gpu/primitives/mlir_primitive.hpp — carries the shared_ptr<ov::Node>
(the MLIROp), a shape_infer_function, output count and output types. No MLIR/GC types
cross this boundary, so the whole graph library stays MLIR-free.
src/graph/include/mlir_primitive_inst.h, src/graph/mlir_primitive.cpp
src/graph/impls/common/mlir_primitive.{hpp,cpp} — extracts ocl runtime objects (queue, dependent events, buffers), calls MLIROp::evaluate(...runtime_objects...), and
translates Graph Compiler's result cl_events back into cldnn::events.
Registered in impls/common/register.{cpp,hpp}, registry/registry.hpp and the new
registry/mlir_primitive_impls.cpp.

3. Event / memory / stream native handles (runtime)

To hand OpenCL objects to Graph Compiler, these virtual accessors were added with
default no-op implementations and OCL overrides:

Base	New method	OCL override
`cldnn::memory`	`void* get_native_handle() const`	`gpu_buffer` → `cl_mem`
`cldnn::event`	`void* get_native_handle()`	`ocl_base_event` → `cl_event`
`cldnn::stream`	`void* get_native_handle() const`	`ocl_stream` → `cl_command_queue`
`cldnn::stream`	`event::ptr create_base_event(void* handle)`	`ocl_stream` → wraps a `cl_event`

Non-OCL backends (ze, sycl) keep the defaults; mlir_primitive_impl throws if a handle is
unavailable, so the MLIR path is effectively OCL-only for now.

4. Properties / config

ov::intel_gpu::enable_mlir ("GPU_ENABLE_MLIR") in
src/inference/include/openvino/runtime/intel_gpu/properties.hpp, added to
Plugin::get_supported_properties() (RW).
OV_CONFIG_RELEASE_OPTION(ov::intel_gpu, enable_mlir, false, …) in
include/intel_gpu/runtime/options.inl, i.e. also settable via OV_GPU_ENABLE_MLIR.

5. `transformations_pipeline.cpp`

transformMLIR() is registered as the last stage of the pipeline (just before
ResolveNameCollisions), receiving a loweringContext that carries the cl_context
extracted from the plugin's remote context (ov::intel_gpu::ocl_context).
ScaledDotProductAttentionDecomposition is force-disabled when enable_mlir is on, so
that SDPA reaches the MLIR path as a single node instead of a decomposed subgraph (can be removed in the future, when matching of the 'decomposed' SDPA is implemented).

6. Build system

src/plugins/intel_gpu/CMakeLists.txt — everything under
src/plugin/transformations/mlir/ is always removed from PLUGIN_SOURCES. When
ENABLE_GRAPH_COMPILER=ON those files are built into a dedicated OBJECT library
openvino_intel_gpu_mlir_obj that alone gets the MLIR/GC include dirs and links
GraphCompiler; the object library is then linked into the plugin. This keeps
mlir/*.h and gc/*.h out of every other translation unit.
cmake/developer_package/plugins/plugins.cmake — new LINKABLE flag for
ov_add_plugin(). With BUILD_SHARED_LIBS=ON a plugin is normally a MODULE, which
cannot be linked against; the GPU plugin needs to be a SHARED library when
ENABLE_GRAPH_COMPILER=ON so that ov_gpu_unit_tests (which compiles
transformations_pipeline.cpp directly) can resolve transformMLIR and other symbols from the openvino_intel_gpu_mlir_obj (Open to discuss on how to make it better)

III. How to build OV with MLIR support

We decided no to add gc/mlir as a third-party submodule to the project for now, and agreed that a user/developer would have to manually build a suitable llvm & graph-compiler and then provide cmake-configs during openvino build.

It's an open question on where should we keep the pinned version in ov; for now we've just pinned it in ov-mlir ci job.

steps to build

Actual steps can be taken from OV-MLIR-CI: dev_gpu_linux_mlir.yml

1. Clone suitable graph-compiler and build gc + llvm:

# Repo + revision are the ones pinned in cmake/graph-compiler.cmake and
# .github/workflows/dev_gpu_linux_mlir.yml
git clone --depth 1 --branch ov_pin/0.1.1 \
    https://github.com/dchigarev/graph-compiler.git
cd ~/graph-compiler

# This script clones a suitable llvm to `~/graph-compiler/externals/llvm-project`;
# Builds llvm; Builds graph-compiler
./scripts/compile.sh -r

2. Build Openvino with MLIR support:

GC_INSTALL_DIR="~/graph-compiler/build/install
LLVM_INSTALL_DIR="~/graph-compiler/externals/llvm-project/build"
cmake -S <openvino> -B <build> \
  -DENABLE_GRAPH_COMPILER=ON \
  -DGraphCompiler_DIR="${GC_INSTALL_DIR}/lib/cmake/GraphCompiler" \
  -DMLIR_DIR="${LLVM_INSTALL_DIR}/lib/cmake/mlir" \
  -DLLVM_DIR="${LLVM_INSTALL_DIR}/lib/cmake/llvm" \
  -DENABLE_INTEL_GPU=ON \
  ...

3. Run

export OV_GPU_ENABLE_MLIR=1        # or ov::intel_gpu::enable_mlir(true) via the API

IV. CI and testing

The MLIR path is tested via its own test suites located at intel_gpu/tests/functional/mlir_op (the test suites are not included to the regular build). The suites are mostly copied from the usual OV tests (e.g. functional/single_layer_tests/dynamic/scaled_dot_product_attention.cpp --> functional/mlir_op/sdpa.cpp) but redefine the tests base class (to MlirSubgraphTest), sometimes include additional cases or change the accuracy that's suitable for mlir implementations.

A new CI-job was added dev_gpu_linux.mlir that builds suitable gc + llvm, builds OV with MLIR support, and runs tests from functional/mlir_op folder.

The job is only triggered when mlir-related files are changed.

The LLVM build is quite time consuming (~20-30 mins), so it would be great if we could cache it somewhere in OV's CI infrastructure. The OV-CI docs mention actions/cache which could be a good candidate, but it currently only has only ~1.5gb of free storage (and an LLVM build is ~1gb), so I've decided not to use it for now.

V. Performance of the MLIR path

For now we only tested stable-diffusion3.5 medium on BMG (B580), where only SDPA goes through MLIR path:

Full Table

Although the inference is faster, the compilation time of the whole model degrades significantly with the mlir path. There were no efforts at all to improve the compilation time for now, but we believe that it can be improved significantly:

Full table

- MLIR is used is a new ngraph transformation, compiled together with other transformations and called from CPU plugin transformation pipeline. - The transformation identifies OV Add operation in the graph and replaces it by a new custom MLIROp operation -- a single op to enclose arbitrary MLIR program. - Add op lowering uses linalg::AddOp on tensors. Each op is represented as isolated MLIR Module. - MLIROp::evaluate calls MLIR-compiled partition when graph is inferred. - Limitation: Add op should have no implicit broadcast, it is not supported and not checked. If Add implies implicit broadcast the result is undefined. - Short code to activate the functionality (in Python, using PyTorch as a source for a small model): import torch import openvino as ov class My(torch.nn.Module): def forward(self, a): b = a*a return ((a+a) * (a+b)) / a my = My() input = torch.tensor([1, 2, 3], dtype=torch.float32) print('Expected:', my(input)) ov_model = ov.convert_model(my, example_input=input) print(ov_model) ov_compiled = ov.compile_model(ov_model) print('Result:', ov_compiled(input)[0])

Improves MLIR pipeline to avoid temporary buffer allocation and copy in linalg named binary operation conversion. Changes: - use tensor.empty in outs - add bufferization pre- and post-processing - mark outputs as 'restrict'

This creates a scenario where the Expected<T> doesn't get checked and the execution crashes, even if the engine was created correctly.

Mostly static declarations, but one unnecessarily wide lambda capture.

* Simple graph patritioner * Multi-node MLIR lowering in a correct partition termination, explicit conversion of not broadcastale nodes: Add, Sub, Mul, Div. * Minor: moved elementwise_f32_etc to outer scope, removed debug output * static -> namespace {}

* Moved MLIROp, transformation pipeline and evaluation tools to a separate files mlir_op.hpp/cpp * Moved common conversion for precision nad shape to a separate file * Move SubgraphTracker to a separate file. Not use unordered_ containers and delete hash function for Output<Node>. Cleanup.

Adds cleanup right before bufferization to eliminate temporary buffer creation in multi-node pattern lowering.

* Moved ConversionContext to a separate file * Moved MarkPattern and associated helpers to conversion_context. * FIXME: One particular case of MatMul -> linalg::MatmulTransposeBOp conversion. Currently has a hack in output tensor allocation in common code, works with MatMul of specific size. * Use fill to prepare output for MatMul result, fixed dynamic output dimensions (transpose_b wasn't handled). * Fix getConstant: swap int and real parts * Removed redundant Value(...) * Small clarification in a comment * Generic mapping of dynamic dimensions from input to output of MLIROp during inference, correct output tensor allocation based on that functionality. Removed related hack in MLIROp::evaluate.

* Allow eltwise ops with static shapes * Fix matmul output shape --------- Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>

Adds ReLU op matcher and lowering to MLIR named Linalg ops. Also, adds buffer deallocation passes to prevent memory leaks when temporary buffers are created in larger graphs.

* [GPU] Generic layer draft * mlir op

* Add support for TPP MLIR * Some CMake magic, but not enough for all tools that fail to link * Split enable_tpp_mlir into add_tpp_mlir_includes and add_tpp_mlir_libs. Removed reference of TPP in all places except transformations and main ov library. * Include TPP-MLIR headers * Use MLIR_ALL_LIBS property to link all MLIR libraries * Registering TPP related dialects in injectMLIR * Reference build scripts for MLIR and TPP-MLIR * One more library from tpp as a dependency, reduced includes * Fixed linker problems by properly ordering TPP dependencies * Minor: add more comments --------- Co-authored-by: Renato Golin <rengolin@systemcall.eu>

…_on_model (slyalin#145)

* Postponed of tpp_xsmm_runner_utils load * Added xsmm runner libs copying to the target ov direcotry listing the names of libs explicitly (FIXME)

Manually finds and passes XSMM runner library to the MLIR JIT engine to resolve missing TPP xsmm_* symbols when executing from Python. Works only for Linux currently.

…lyalin#149) This reverts commit a510e01.

* Broadcast support for element-wise ops and more economical way of dynamic dimensions handling based on symbols. * Simpler broadcast dims cacluations, moved to common utils. * Use common function to compute dynamic dimension values in MatMul and Relu. * Element type configurable restriction for the new BinaryEltwisePattern. Forced f32 in the conversion pipeline.

…lyalin#151) * Check MatMul expected attributes in a predicate instead of assert in the transformation callback. Fixes PyTorch addmm layer tests. * Put rank == 2 restriction on MatMul conversion. Fixes PyTorch linear layer tests.

Adds support for `linalg.matmul` and `linalg.matmul_tranpose_a`.

* Check MatMul expected attributes in a predicate instead of assert in the transformation callback. Fixes PyTorch addmm layer tests. * Put rank == 2 restriction on MatMul conversion. Fixes PyTorch linear layer tests. * Enable MLIR and TPP-MLIR activation via environment variables * Added debug macros controlled by OV_MLIR_DEBUG env variable. Disabled debug prints by default.

Relaxes MLIR conversion matchers to accept any element type.

…in return statement in is_dubug function. (slyalin#157)

Collection of MLP benchmarks using combination of OV and TPP-MLIR.

Resizes MLP benchmarks to align with existing MLIR tests. Tweaks model generator to expose full control over M,N,K dimensions of torch Linear layer.

Fewer iterations to speedup testing. Can be changed/reverted later.

Sets OV infer precision same as the data type. Minor debug print and bash fixes.

Adds option for TPP-MLIR benchmark with weights as constants which enables compile-time packing. Also, add utility benchmark runners.

* Integration with GraphCompiler Grapth Compiler is disabled by default, to enable build with -DENABLE_GRAPH_COMPILER=ON * Add suggestions from code review * Apply suggestions from code review Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> --------- Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>

* [GraphCompiler] Use find_package() for CMake < 3.24 * Changed the Graph Compiler git url

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

maxnick · 2026-08-04T15:41:45Z

@EgorDuplensky , could you please review?
@isanghao , @p-durandin FYI

Copilot

Pull request overview

This PR introduces an optional MLIR/Graph-Compiler execution path in the Intel GPU plugin by partitioning supported OpenVINO subgraphs into an internal MLIROp/mlir_primitive and executing them via Graph Compiler (OpenCL runtime objects passed through runtime metadata).

Changes:

Adds MLIR partitioning + conversion infrastructure (patterns, converters, subgraph tracking) and integrates it as an optional GPU transformation stage gated by ENABLE_GRAPH_COMPILER + ov::intel_gpu::enable_mlir.
Adds runtime plumbing to pass native OpenCL handles (queue/memory/events) to Graph Compiler and wrap returned cl_events back into plugin events.
Adds a dedicated MLIR functional test suite + CI workflow(s) for MLIR-enabled builds.

Reviewed changes

Copilot reviewed 78 out of 78 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/plugins/intel_gpu/tests/unit/CMakeLists.txt	Adjusts unit test source lists/linking when Graph Compiler is enabled.
src/plugins/intel_gpu/tests/functional/CMakeLists.txt	Excludes MLIR functional tests when Graph Compiler is disabled.
src/plugins/intel_gpu/tests/functional/mlir_op/mlir_test_env.hpp	Adds MLIR test fixture utilities (env + runtime-graph checks).
src/plugins/intel_gpu/tests/functional/mlir_op/sdpa.cpp	Adds MLIR SDPA functional tests and MLIR execution verification.
src/plugins/intel_gpu/tests/functional/mlir_op/matmul.cpp	Adds MLIR MatMul tests (static + dynamic shapes).
src/plugins/intel_gpu/tests/functional/mlir_op/matmul_rms_norm_concat.cpp	Adds MLIR MatMul+RMSNorm(+Concat) tests and benchmarks.
src/plugins/intel_gpu/tests/functional/mlir_op/rms.cpp	Adds MLIR RMS op functional tests.
src/plugins/intel_gpu/tests/functional/mlir_op/reduction.cpp	Adds MLIR reduction op functional tests.
src/plugins/intel_gpu/tests/functional/mlir_op/concat.cpp	Adds MLIR Concat tests (incl. transpose+concat).
src/plugins/intel_gpu/tests/functional/mlir_op/transpose.cpp	Adds MLIR Transpose and Reshape+Transpose tests.
src/plugins/intel_gpu/tests/functional/mlir_op/unary_eltwise.cpp	Adds MLIR unary elementwise tests.
src/plugins/intel_gpu/tests/functional/mlir_op/binary_eltwise.cpp	Adds MLIR binary elementwise tests (with some currently skipped).
src/plugins/intel_gpu/src/runtime/ocl/ocl_stream.hpp	Exposes OpenCL queue handle + base-event wrapping API on OCL stream.
src/plugins/intel_gpu/src/runtime/ocl/ocl_stream.cpp	Implements base-event creation from an existing `cl_event`.
src/plugins/intel_gpu/src/runtime/ocl/ocl_memory.hpp	Exposes `cl_mem` native handle from OCL GPU buffer.
src/plugins/intel_gpu/src/runtime/ocl/ocl_base_event.hpp	Exposes `cl_event` native handle from OCL events.
src/plugins/intel_gpu/include/intel_gpu/runtime/stream.hpp	Adds generic `get_native_handle()` + `create_base_event(void*)` hooks.
src/plugins/intel_gpu/include/intel_gpu/runtime/memory.hpp	Adds generic `memory::get_native_handle()` hook.
src/plugins/intel_gpu/include/intel_gpu/runtime/event.hpp	Adds generic `event::get_native_handle()` hook.
src/plugins/intel_gpu/include/intel_gpu/runtime/options.inl	Adds `ov::intel_gpu::enable_mlir` runtime option registration.
src/inference/include/openvino/runtime/intel_gpu/properties.hpp	Adds public GPU property `enable_mlir`.
src/plugins/intel_gpu/src/plugin/plugin.cpp	Exposes `enable_mlir` in supported property list.
src/plugins/intel_gpu/src/plugin/transformations_pipeline.cpp	Hooks `transformMLIR` into the GPU pipeline and disables SDPA decomposition when MLIR is enabled.
src/plugins/intel_gpu/include/intel_gpu/op/mlir_op.hpp	Declares internal `ov::intel_gpu::op::MLIROp` node API.
src/plugins/intel_gpu/src/plugin/transformations/op/mlir_op.cpp	Implements `MLIROp` shape inference and evaluation (packed args, runtime metadata).
src/plugins/intel_gpu/include/intel_gpu/primitives/mlir_primitive.hpp	Declares `cldnn::mlir_primitive` wrapper around `MLIROp`.
src/plugins/intel_gpu/src/plugin/ops/mlir_op.cpp	Registers factory + builds `mlir_primitive` from `MLIROp` during program build.
src/plugins/intel_gpu/include/intel_gpu/plugin/primitives_list.hpp	Registers internal MLIR factory under `GRAPH_COMPILER`.
src/plugins/intel_gpu/src/graph/include/mlir_primitive_inst.h	Adds graph-side primitive instantiation for `mlir_primitive`.
src/plugins/intel_gpu/src/graph/mlir_primitive.cpp	Adds layout inference + JSON description for `mlir_primitive`.
src/plugins/intel_gpu/src/graph/registry/registry.hpp	Registers implementation set for `mlir_primitive`.
src/plugins/intel_gpu/src/graph/registry/mlir_primitive_impls.cpp	Wires `mlir_primitive` to common implementation managers.
src/plugins/intel_gpu/src/graph/impls/common/register.hpp	Registers common impl for `mlir_primitive`.
src/plugins/intel_gpu/src/graph/impls/common/register.cpp	Registers common impl for `mlir_primitive`.
src/plugins/intel_gpu/src/graph/impls/common/mlir_primitive.hpp	Declares common impl manager for `mlir_primitive`.
src/plugins/intel_gpu/src/graph/impls/common/mlir_primitive.cpp	Implements runtime execution: extracts OCL handles, calls `MLIROp::evaluate`, wraps result events.
src/plugins/intel_gpu/src/plugin/transformations/mlir/interface/convert.hpp	Declares `transformMLIR()` entry point.
src/plugins/intel_gpu/src/plugin/transformations/mlir/interface/mlir_evaluate_base.hpp	Defines MLIR evaluation interface used by `MLIROp`.
src/plugins/intel_gpu/src/plugin/transformations/mlir/interface/properties.hpp	Defines evaluation-context keys for wait lists/result events/USM flags.
src/plugins/intel_gpu/src/plugin/transformations/mlir/mlir_evaluate.hpp	Declares Graph Compiler-backed MLIR executor for GPU.
src/plugins/intel_gpu/src/plugin/transformations/mlir/mlir_evaluate.cpp	Implements Graph Compiler module build + execution (static/dynamic) with OCL context wiring.
src/plugins/intel_gpu/src/plugin/transformations/mlir/subgraph_tracker.hpp	Adds subgraph tracking structure for partitioning marked nodes.
src/plugins/intel_gpu/src/plugin/transformations/mlir/subgraph_tracker.cpp	Implements partitioning bookkeeping and subgraph termination logic.
src/plugins/intel_gpu/src/plugin/transformations/mlir/graph_converter.hpp	Declares OV→MLIR graph converter infrastructure + marking pass wrapper.
src/plugins/intel_gpu/src/plugin/transformations/mlir/graph_converter.cpp	Implements convertor storage in RT info + subgraph mark helpers.
src/plugins/intel_gpu/src/plugin/transformations/mlir/conversion/patterns.hpp	Declares MLIR marking patterns for supported ops.
src/plugins/intel_gpu/src/plugin/transformations/mlir/conversion/patterns.cpp	Implements patterns and match predicates (SDPA, matmul, broadcastable eltwise, etc.).
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/README.md	Documents converter structure and conversion context responsibilities.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/typedefs.hpp	Adds local typedefs for MLIR conversion code.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/convert_common.hpp	Adds shared MLIR conversion utilities and debug macros.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/convert_common.cpp	Implements type/shape import, broadcast helpers, and debug env toggle.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/conversion_context.hpp	Declares conversion context (inputs + dynamic-dimension value hooks).
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/conversion_context.cpp	Implements dynamic dimension value collection.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/binary_eltwise.hpp	Implements MLIR lowering for broadcastable binary elementwise ops.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/unary_eltwise.hpp	Implements MLIR lowering for unary elementwise ops.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/concat.hpp	Implements MLIR lowering for Concat.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/floor.hpp	Implements MLIR lowering for Floor.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/gather.hpp	Implements MLIR lowering for Gather with negative-index handling.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/matmul.hpp	Implements MLIR lowering for MatMul/BatchMatMul variants with rank adjustments.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/reduce.hpp	Implements MLIR lowering for Reduce ops (including ReduceMean division).
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/relu.hpp	Implements MLIR lowering for Relu.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/reshape.hpp	Implements MLIR lowering for Reshape via (expand/collapse) reassociation.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/rms.hpp	Implements MLIR lowering for internal RMS normalization op.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/sdpa.hpp	Implements MLIR lowering for SDPA via linalgx::AttentionOp.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/shape_of.hpp	Implements MLIR lowering for ShapeOf.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/slice.hpp	Implements MLIR lowering for Slice via ExtractSliceOp (limited semantics).
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/squeeze.hpp	Implements MLIR lowering for Squeeze via CollapseShapeOp.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/transpose.hpp	Implements MLIR lowering for Transpose via linalg::TransposeOp.
src/plugins/intel_gpu/src/plugin/transformations/mlir/common/converters/unsqueeze.hpp	Implements MLIR lowering for Unsqueeze via ExpandShapeOp.
src/plugins/intel_gpu/CMakeLists.txt	Adds Graph Compiler build option and MLIR OBJECT library wiring into GPU plugin.
cmake/features.cmake	Adds `ENABLE_GRAPH_COMPILER` build option.
cmake/llvm.cmake	Adds LLVM/MLIR discovery helper for Graph Compiler integration.
cmake/graph-compiler.cmake	Adds Graph Compiler discovery/fetch logic and build-time toggles.
cmake/developer_package/plugins/plugins.cmake	Adds `LINKABLE` option to build plugins as SHARED when needed by tests.
.github/workflows/dev_gpu_linux_mlir.yml	Adds dedicated MLIR GPU CI workflow (build LLVM+GC, build OV, run MLIR tests).
.github/workflows/graph-compiler.yml	Adds a workflow intended to validate Graph Compiler GPU backend integration.
.github/dockerfiles/docker_tag	Updates CI docker tag reference.

dchigarev · 2026-08-05T12:56:39Z

+on:
+  push:
+  pull_request:
+
+permissions:
+  contents: read
+
+env:
+  GC_REPO: https://x-access-token:${{ secrets.GC_TOKEN }}@github.com/intel-sandbox/graph-compiler
+  GC_TAG: main
+  BUILD_DIR: ${{ github.workspace }}-gc-build
+  OUTPUT_DIR: ${{ github.workspace }}-gc-bin
+
+jobs:
+  ci:
+    runs-on: ${{ vars.RUNNER }}
+


This workflow is only used in our internal OV-fork's CI. It's not launched in the main ov-repo and will be removed just before the merge

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

Copilot

Pull request overview

Copilot reviewed 78 out of 78 changed files in this pull request and generated 4 comments.

Suppressed comments (1)

src/plugins/intel_gpu/tests/functional/mlir_op/mlir_test_env.hpp:36

[MEDIUM] MlirMatchAllEnv uses setenv/unsetenv directly, which is not available on Windows. Other OpenVINO tests use _putenv_s under _WIN32 to keep tests portable.

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

Copilot

Pull request overview

Copilot reviewed 78 out of 78 changed files in this pull request and generated 3 comments.

Suppressed comments (2)

src/plugins/intel_gpu/src/plugin/transformations/mlir/conversion/patterns.cpp:134

[BLOCKER] Equal Q/K/V rank is insufficient for this lowering. OpenVINO SDPA explicitly allows NumPy broadcasting of the leading dimensions (see src/core/shape_inference/include/scaled_dot_product_attention_shape_inference.hpp:41-65), while ConvertSDPA indexes every Q/K/V with the same batch/head induction variables and never broadcasts them. A valid case such as Q [2,7,3,4], K [2,1,5,4], V [1,7,5,6] can therefore index singleton dimensions incorrectly. Restrict the matcher to identical supported leading dimensions (and equivalent supported mask broadcasting), or materialize the required broadcasts before AttentionOp.
src/plugins/intel_gpu/src/plugin/transformations/mlir/conversion/patterns.cpp:83
[HIGH] This predicate still accepts valid MatMul variants that the converter cannot lower: rank-1 operands reach drop_back(2), and NumPy-broadcasted batch dimensions are passed directly to linalg::BatchMatmulOp despite the converter's explicit TODO: Support broadcasts. Such models will be marked for MLIR and fail IR construction/verification instead of falling back to the native GPU path. Require operand ranks of at least 2 and identical supported batch dimensions here, or implement vector promotion and batch broadcasting in the converter.

This issue also appears on line 130 of the same file.

+        auto shape_of = shape::ShapeOfOp::create(builder, loc, mlir::ValueRange{input});
+        auto cast = arith::IndexCastOp::create(builder, loc, indices_type, mlir::ValueRange{shape_of});
+
+        auto empty_add = tensor::EmptyOp::create(builder, loc, indices_expanded.getType(), dynamic_index_dims);
+        auto add = linalg::AddOp::create(builder, loc, mlir::ValueRange{cast.getResult(), indices_expanded}, mlir::ValueRange{empty_add});


Presumably after LLVM comit 55b38aeec8cf, there is a precision degradation.

p-durandin · 2026-08-11T06:50:11Z

build_jenkins

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

* Bump LLVM version to 24 * Fixed errors afetr changes in GC interfaces * Update cmake/llvm.cmake Co-authored-by: Dmitry Chigarev <dmitry.chigarev@intel.com> * Removed tests from exclude --------- Co-authored-by: Dmitry Chigarev <dmitry.chigarev@intel.com>

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

slyalin and others added 30 commits January 27, 2026 16:15

Assert doesn't get compiled in Release mode (slyalin#131)

2d1955b

This creates a scenario where the Expected<T> doesn't get checked and the execution crashes, even if the engine was created correctly.

Reduce warnings (slyalin#133)

6f554c9

Mostly static declarations, but one unnecessarily wide lambda capture.

Pre-bufferization cleanup (slyalin#138)

e30e919

Adds cleanup right before bufferization to eliminate temporary buffer creation in multi-node pattern lowering.

Allow static input (slyalin#141)

b5370bd

* Allow eltwise ops with static shapes * Fix matmul output shape --------- Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>

Match and lower ov::Relu (slyalin#143)

1e1d807

Adds ReLU op matcher and lowering to MLIR named Linalg ops. Also, adds buffer deallocation passes to prevent memory leaks when temporary buffers are created in larger graphs.

[GPU] Draft of MLIR and Generic ops (slyalin#140)

a4dbd98

* [GPU] Generic layer draft * mlir op

Fixed double free due to missing return statement in Partitioner::run…

ff4d018

…_on_model (slyalin#145)

Fixing xsmm runner dynamic load (slyalin#146)

863f139

* Postponed of tpp_xsmm_runner_utils load * Added xsmm runner libs copying to the target ov direcotry listing the names of libs explicitly (FIXME)

Pass XSMM runner lib to MLIR execution engine (slyalin#147)

d44f7c9

Manually finds and passes XSMM runner library to the MLIR JIT engine to resolve missing TPP xsmm_* symbols when executing from Python. Works only for Linux currently.

Revert "Pass XSMM runner lib to MLIR execution engine (slyalin#147)" (s…

c72dc8a

…lyalin#149) This reverts commit a510e01.

More matmul variants (slyalin#153)

f8bbb6a

Adds support for `linalg.matmul` and `linalg.matmul_tranpose_a`.

Match all element types (slyalin#156)

ea8e4a3

Relaxes MLIR conversion matchers to accept any element type.

Fix graph partitioner when two MLIROp instances go side-by-side. Fix …

acbd925

…in return statement in is_dubug function. (slyalin#157)

MLP benchmarks (slyalin#152)

df9b3e0

Collection of MLP benchmarks using combination of OV and TPP-MLIR.

Align MLP sizes with TPP-MLIR benchmarks (slyalin#158)

1f9a555

Resizes MLP benchmarks to align with existing MLIR tests. Tweaks model generator to expose full control over M,N,K dimensions of torch Linear layer.

Lower number of benchmark iterations (slyalin#159)

80dd502

Fewer iterations to speedup testing. Can be changed/reverted later.

MLP bench - control OV infer precision (slyalin#160)

77f5ea3

Sets OV infer precision same as the data type. Minor debug print and bash fixes.

TPP const weights and benchmark runners (slyalin#161)

e93f1a6

Adds option for TPP-MLIR benchmark with weights as constants which enables compile-time packing. Also, add utility benchmark runners.

[GraphCompiler] Use find_package() for CMake < 3.24 (slyalin#162)

ee1d2b4

* [GraphCompiler] Use find_package() for CMake < 3.24 * Changed the Graph Compiler git url

Fixed build failure caused by slyalin#155 (slyalin#163)

21c61af

Fix mlir workflow file

585ff3d

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

dchigarev changed the title ~~[WIP][MLIR][GPU] MLIR feature development (CI only)~~ [MLIR][GPU] MLIR feature development Aug 4, 2026

maxnick requested a lite review from Copilot August 4, 2026 15:40

maxnick assigned EgorDuplensky Aug 4, 2026

Copilot started reviewing on behalf of maxnick August 4, 2026 15:41 View session

maxnick added this to the 2026.4 milestone Aug 4, 2026

maxnick changed the title ~~[MLIR][GPU] MLIR feature development~~ [GPU][MLIR] MLIR Graph compiler integration Aug 4, 2026

maxnick requested a review from EgorDuplensky August 4, 2026 15:43

Copilot AI reviewed Aug 4, 2026

View reviewed changes

dchigarev added 4 commits August 5, 2026 12:02

Merge remote-tracking branch 'origin/master' into HEAD

c27479d

Do not print generated values in SDPA test

cb92724

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

Gate unsupported slice patterns

d4ec2b3

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

Fix 'rank' packing into 'void*' vector

e4b3cc5

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

maxnick requested a lite review from Copilot August 5, 2026 13:39

Copilot started reviewing on behalf of maxnick August 5, 2026 13:40 View session

Copilot AI reviewed Aug 5, 2026

View reviewed changes

dchigarev added 3 commits August 6, 2026 15:02

Fix MemrefDescriptor constructor

b94dc3f

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

Gate unsupported cases for Transpose/Unsqueeze converters

c565fff

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

Guard out-of-bounds access in ConvertReshape

f55a50d

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

maxnick requested a balanced review from Copilot August 10, 2026 09:31

Copilot started reviewing on behalf of maxnick August 10, 2026 09:32 View session

Copilot AI reviewed Aug 10, 2026

View reviewed changes

Increased rel_threshold for UnaryElementwiseExpTest

a53e9e5

Presumably after LLVM comit 55b38aeec8cf, there is a precision degradation.

dchigarev and others added 5 commits August 11, 2026 10:36

Restrict mlir_primitive serialization

07761bb

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

fix clang-tidy

1dc5942

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

add notes regarding integer support

1c9c6eb

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

Update gc-pin to 'ov_pin/0.1.1'

cbf4429

Signed-off-by: dchigarev <dmitry.chigarev@intel.com>

Navigation Menu

Uh oh!

[GPU][MLIR] MLIR Graph compiler integration - #35336

[GPU][MLIR] MLIR Graph compiler integration#35336
dchigarev wants to merge 139 commits into
openvinotoolkit:masterfrom
dchigarev:mlir-gpu-gc-integration

dchigarev commented Apr 14, 2026 •

edited

Loading

Uh oh!

maxnick commented Aug 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

dchigarev Aug 5, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

p-durandin commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

Uh oh!

Conversation

dchigarev commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I. Brief description

Supported subgraphs

Feature enabling

Lib size increase

II. Changes in common / existing code

1. New op: ov::intel_gpu::op::MLIROp

2. New primitive: cldnn::mlir_primitive

3. Event / memory / stream native handles (runtime)

4. Properties / config

5. transformations_pipeline.cpp

6. Build system

III. How to build OV with MLIR support

1. Clone suitable graph-compiler and build gc + llvm:

2. Build Openvino with MLIR support:

3. Run

IV. CI and testing

V. Performance of the MLIR path

Uh oh!

maxnick commented Aug 4, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

dchigarev Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

p-durandin commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

dchigarev commented Apr 14, 2026 •

edited

Loading

1. New op: `ov::intel_gpu::op::MLIROp`

2. New primitive: `cldnn::mlir_primitive`

5. `transformations_pipeline.cpp`