[GGUF FE] Stateful conversion, op-translator fixes and test infrastructure

[mvafin]

Details:

• Groundwork on the existing GGUF frontend. Everything here is reachable through the GgmlOvDecoder path master already ships; the native .gguf builder is a separate, stacked change (see below) and does not appear in this PR.
• MakeStateful pass. The frontend always converts to a stateless graph — every KV cache an explicit Parameter/Result pair, mirroring how optimum-intel exports — and statefulness becomes the consumer's choice, registered as a DecoderTransformationExtension so it runs ahead of the built-in stateless lowering. The pass also takes ownership of beam_idx: it is a beam-search index into an OpenVINO state with no ggml counterpart, so declaring it in a decoder would leave a consumer-less input on the stateless graph.
• Op translators. Keep the output port when passing a value between translators — taking .get_node_shared_ptr() silently resolves to output 0, which throws for multi-output ops such as TopK. Keep the static head layout in permute op_case 4; drop builder-only op_case numbering from RESHAPE/VIEW; give each attention Transpose its own order constant; make the graph valid under both the SDPA and PagedAttention layouts by deriving the leading dims instead of pinning them; share the TopK-indices construction between ARGSORT and TOP_K, and let TOP_K tolerate a dynamic k rather than throwing.
• Quantization. Support the Q2_0 (ternary) type used by the Bonsai family.
• Decoder interface. Drop get_model_weights, which nothing calls.
• Tests. Add an op-coverage gate so a newly registered op cannot ship without a conversion test, and check the activation translators against captured output from real ggml rather than a numpy reimplementation — a numpy oracle can only confirm the formula the author already guessed, which is how the GELU_QUICK error survived. ov_gguf_frontend_tests: 137/137.
• CI. Add a GGUF_FE component so frontend changes scope their own jobs.

Note

This is part 1 of 2. Part 2 (#37421) adds the native .gguf graph builder and stacks on this branch. Splitting it this way keeps the cross-cutting pieces — the decoder interface change and the pass architecture that the llama.cpp ggml-openvino backend integrates against — reviewable without ~4.8k lines of builder alongside.

Tickets:

• CVS-188634

AI Assistance:

• AI assistance used: yes
• AI wrote the code and tests. Human-validated by building and running: the frontend unit tests pass (137/137), and the changes were additionally exercised end-to-end through the stacked builder branch against real checkpoints before being split out.

[Copilot]

Pull request overview

This PR evolves the existing GGUF frontend conversion pipeline to support a “stateless-by-default” graph with an opt-in stateful lowering, improves correctness/robustness of multiple ggml op translators (multi-output handling, layout-polymorphic reshapes/permutes, shared TopK-indices helper, additional quant type), and significantly strengthens the GGUF FE test suite and CI scoping.

Changes:

• Introduces ov::frontend::gguf::pass::MakeStateful as a DecoderTransformationExtension to convert KV-cache SetRows patterns into OpenVINO state (and manage beam_idx).
• Fixes/extends multiple GGUF op translators (TopK/Argsort indices, VIEW/RESHAPE/PERMUTE layout handling, RoPE fusion-friendly decomposition, MoE MUL_MAT_ID lowering) and adds Q2_0 quant support plus weight-path refactoring.
• Expands GGUF FE tests (op-kernel vs real-ggml references, op-coverage gate) and adds a dedicated GGUF_FE CI component/test entry.

Reviewed changes

Copilot reviewed 40 out of 47 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/frontends/gguf/tests/test_ops.cpp	Extends op tests (unary inputs/domains, real-ggml oracle cases, additional op-case coverage for permute/view/reshape/topk).
src/frontends/gguf/tests/test_op_coverage.cpp	Adds a runtime op-coverage gate ensuring every registered op has a conversion test.
src/frontends/gguf/tests/test_extensions.cpp	Adds DecoderTransformationExtension coverage (stateless vs stateful cache behavior, IO contract checks).
src/frontends/gguf/tests/test_dequant_vs_ggml.cpp	Updates comments/formatting and adds Q2_0 dequant test coverage.
src/frontends/gguf/tests/op_test_utils.hpp	Tracks converted op types for coverage gating; includes cleanups.
src/frontends/gguf/tests/CMakeLists.txt	Updates GGUF test target sources/includes and installs reference data (includes an accidental duplicate source entry).
src/frontends/gguf/src/utils.hpp	Adds helpers (permute, make_topk_indices, etc.) and extends make_sin_cos signature.
src/frontends/gguf/src/utils.cpp	Implements new helpers, extends RoPE sin/cos building, and clarifies VIEW offset units.
src/frontends/gguf/src/translate_session.cpp	Updates translate flow to use model decoder, support extra inputs, weight detection, transformation ordering, and post-passes.
src/frontends/gguf/src/quant/weights.hpp	Adds zero-point type selector and extracted-weight split helpers for builder interop.
src/frontends/gguf/src/quant/weights.cpp	Implements zero-point type selection and weight extraction/splitting utilities; refines error strings.
src/frontends/gguf/src/quant/gguf.hpp	Adjusts quant API comments and adds get_shape helper declaration.
src/frontends/gguf/src/quant/gguf_quants.cpp	Documents/implements Q2_0 fill behavior details.
src/frontends/gguf/src/pass/make_stateful.cpp	Implements MakeStateful model pass converting cache SetRows into Variable/ReadValue/Assign state.
src/frontends/gguf/src/op/weight.cpp	Supports weight leaves from both ingest paths (pre-extracted tensors vs raw ggml bytes).
src/frontends/gguf/src/op/view.cpp	Unifies VIEW case handling across ingest paths and adds builder-only case 104 reshape-to-reference behavior.
src/frontends/gguf/src/op/top_k.cpp	Makes TOP_K tolerate dynamic k and shares TopK-indices construction helper.
src/frontends/gguf/src/op/set_rows.cpp	Uses context.get_output_type() and reshapes with special_zero for layout polymorphism.
src/frontends/gguf/src/op/rope.cpp	Improves layout-polymorphic reshapes and switches NEOX to decomposition pattern suitable for RoPE fusion.
src/frontends/gguf/src/op/rms_norm.cpp	Include/order adjustments; no semantic change intended.
src/frontends/gguf/src/op/reshape.cpp	Unifies op_case numbering and makes key reshapes layout-polymorphic via special_zero.
src/frontends/gguf/src/op/permute.cpp	Uses get_op_case(), preserves static head dims under dynamic reshape patterns, clarifies view offset units.
src/frontends/gguf/src/op/mul_mat_id.cpp	Adds GatherMatmul-based lowering for constant expert weights with generic fallback path.
src/frontends/gguf/src/op/glu_geglu.cpp	Ensures GEGLU uses TANH GELU approximation to match ggml semantics.
src/frontends/gguf/src/op/get_rows.cpp	Adds MoE gating gather case via GatherElements and uses context.get_output_type().
src/frontends/gguf/src/op/flash_attn_ext.cpp	Supports ingest-path layout differences, optional attention sinks, and soft-cap decomposition.
src/frontends/gguf/src/op/argsort.cpp	Reuses shared TopK-indices helper.
src/frontends/gguf/src/op_table.hpp	Reorganizes translator declarations and adds missing unary/math ops in the table header.
src/frontends/gguf/src/op_table.cpp	Clarifies GGML_OP_NONE weight semantics for both ingest paths and minor ordering tweaks.
src/frontends/gguf/src/node_context.hpp	Adds accessors (get_input_names, get_op_case, get_output_type) and clarifies view offsets in elements.
src/frontends/gguf/src/input_model.hpp	Exposes underlying model decoder via get_model_decoder().
src/frontends/gguf/src/input_model.cpp	Implements get_model_decoder().
src/frontends/gguf/src/frontend.cpp	Updates discoverability docs and supported_impl behavior for decoder-based loading.
src/frontends/gguf/src/CMakeLists.txt	Updates GGUF frontend build description/comments.
src/frontends/gguf/include/openvino/frontend/gguf/make_stateful.hpp	Adds public header for MakeStateful pass.
src/frontends/gguf/include/openvino/frontend/gguf/frontend.hpp	Documents DecoderTransformationExtension + (currently mismatched) file-path ingest support.
src/frontends/gguf/include/openvino/frontend/gguf/decoder.hpp	Adds optional model-scope accessors (extra inputs) and clarifies decoder contract.
.github/workflows/ubuntu_24.yml	Enables gguf fixture generation in Ubuntu 24 workflow inputs.
.github/workflows/job_cxx_unit_tests.yml	Gates GGUF frontend tests on GGUF_FE affected-components flag.
.github/coverage/tests_cpp.yml	Adds ov_gguf_frontend_tests to coverage test list.
.github/components.yml	Adds GGUF_FE component definition and CPU build/revalidate mapping.

Suppressed comments (1)

src/frontends/gguf/include/openvino/frontend/gguf/frontend.hpp:63

• [MEDIUM] The load_impl docstring describes a second ingest path via a .gguf file path, but load_impl currently errors out unless variants[0] is a std::shared_ptr<GgufDecoder>. Please update the documentation (or add the file-path ingest in the same PR) so the public header matches behavior.

    /// \brief Load the input model, from either of the frontend's two ingest paths.
    /// \param variants A single element, holding either:
    ///        - a `std::shared_ptr<GgufDecoder>` — a decoder supplied by a direct linker, wrapping
    ///          an already-built ggml graph (the llama.cpp cgraph path); or
    ///        - a path to a `.gguf` file — parsed here, with the transformer graph built
    ///          per-architecture by the native builder.
    ///        Both yield a GgufDecoder, so conversion past this point is identical.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.