Skip to content

[GPU] Prevent unsafe in-place crop optimization for 2D-to-1D bbox splits - #37414

Merged
e-ddykim merged 1 commit into
openvinotoolkit:masterfrom
zhanmyz:cvs-192143-fix-regression-issue-v1
Aug 18, 2026
Merged

[GPU] Prevent unsafe in-place crop optimization for 2D-to-1D bbox splits#37414
e-ddykim merged 1 commit into
openvinotoolkit:masterfrom
zhanmyz:cvs-192143-fix-regression-issue-v1

Conversation

@zhanmyz

@zhanmyz zhanmyz commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Details:

  • Prevent unsafe runtime in-place crop optimization for 2D-to-1D rank-reducing reshape patterns.
  • Add a regression unit test for the Faster RCNN / SSD bbox-coordinate pattern.

Description of the issue (symptom, root cause, and resolution)

  • Symptom:
    After commit 3f24992895e41aa669b3743c8b6a7f499f3d789a, several GPU object-detection models produced incorrect detection results. The original reproducer was:

    • TF_V2_Faster_RCNN_Inception_ResNet_v2_atrous_coco
    • Expected detections: 5
    • GPU inference detections: 25
    • Failing output layer: tf_detections

    The same regression pattern was also observed for:

    • TF_V2_Faster_RCNN_ResNet50_v1_atrous_coco
    • TF_V2_Mask_RCNN_ResNetv2_atrous_coco
    • ONNX_Runtime_ssd_mobilenet_V1_coco_mlperf_opset10
  • Root Cause
    Commit 3f24992895e41aa669b3743c8b6a7f499f3d789a enabled runtime padding propagation for the TransposeSplitMatcher QKV pattern. The new condition in reshape_node::is_runtime_propagatable_padding() accepted any axis-1 crop with a size-1 dimension followed by a rank-reducing reshape:

    axis == 1 && input_pshape[1] == 1 && output_rank + 1 == input_rank

    This condition was intended for the QWen-VL pattern:

    [-1, 3, H, S] -> [-1, 1, H, S] -> [-1, H, S]
             4D           4D              3D
    

    However, it also accepted the object-detection bbox-coordinate pattern:

    [N, 4] -> [N, 1] -> [N]
       2D       2D      1D
    

    The latter is not safe for runtime padding propagation. In the affected models, Split([N, 4]) produces four coordinate crops [N, 1], which are squeezed to one-dimensional vectors [N] and consumed by bbox decode arithmetic. The in-place crop optimization exposed a strided/padded view to downstream kernels that expected a contiguous one-dimensional vector. This corrupted decoded box coordinates and caused NMS to return an incorrect number of detections.
    The existing unit tests added by commit 3f24992895e covered the valid QKV/TransposeSplitMatcher path and other padding propagation cases, but did not cover the invalid 2D-to-1D bbox-coordinate boundary case.

  • Resolution:
    Restrict the axis-1 rank-reducing padding propagation rule to outputs with rank at least 2:

    if (prim->output_partial_shape.size() >= 2 &&
        prim->output_partial_shape.size() + 1 == input_pshape.size()) {
        return true;
    }

    The same guard is applied in the vl_sdpa path so both build-time and runtime decisions use the same safety contract.
    This preserves the intended QWen-VL optimization (4D -> 3D) while rejecting the unsafe bbox-coordinate case (2D -> 1D). The change is generic and is not tied to a specific model.

Performance and compatibility

  • The fix only disables the newly introduced padding optimization for unsafe 2D-to-1D rank reduction. This path was not optimized before commit 3f24992895e, so the change restores the previous behavior for these models and avoids an accuracy regression. The valid QWen-VL 4D-to-3D optimization remains enabled.

The code and line that caused this issue (if it is not changed directly)

if (axis == 1 && !input_pshape[1].is_dynamic() && input_pshape[1].get_length() == 1) {
if (prim->output_partial_shape.size() + 1 == input_pshape.size()) {
return true;
}
}

Reproduction step

python -m pytest test_skip_ir_generation.py \
    --tb=native \
    --env_conf=.automation/env_config.yml \
    --test_conf=.automation/test_configs/desktop_test_config.yml \
    -m "not launch_only_if_manually_specified " \
    --pregen_irs=irs_mapping.csv \
    --tf_models_version=2 \
    --modules pipelines/production/tf_2x/heavy \
    -k "TF_V2_Faster_RCNN_Inception_ResNet_v2_atrous_coco_batch_1_device_GPU_precision_FP32" \
    --dynamism_type=None \
    --skip_ir_generation \
    --log-cli-level INFO 

Checklist

  • Is it a proper fix? (not a workaround)
  • Did you include test case for this fix, if necessary?
  • Did you review existing test that can be extended to cover this scenario? Which test did you review?

Tickets:

@zhanmyz zhanmyz added do_not_review do-not-merge Do not merge this PR labels Aug 13, 2026
@zhanmyz zhanmyz changed the title Try to fix CVS-192143 issue [GPU] Try to fix CVS-192143 issue Aug 13, 2026
@github-actions github-actions Bot added the category: GPU OpenVINO GPU plugin label Aug 13, 2026
@ceciliapeng2011

Copy link
Copy Markdown
Contributor

@zhanmyz The PR LGTM. Please open it for official review and merge.

Commit 3f24992 introduced runtime padding
propagation for the TransposeSplitMatcher QKV pattern. Its axis-1
rank-reducing reshape check also matched object-detection bbox splits:

    [N, 4] -> [N, 1] -> [N]

The resulting in-place crop exposed a padded/strided one-dimensional view
to bbox decode arithmetic, producing incorrect box coordinates and NMS
results. For example, TF_V2_Faster_RCNN_Inception_ResNet_v2_atrous_coco
returned 25 detections instead of the reference 5 detections on GPU.

Restrict axis-1 runtime padding propagation to reshape outputs with rank >= 2.
This keeps the intended TransposeSplitMatcher path enabled:

    [-1, 3, H, S] -> [-1, 1, H, S] -> [-1, H, S]

while rejecting the unsafe 2D-to-1D bbox-coordinate path. Apply the same
rank guard in the VLSDPA-specific check to keep build-time and runtime
behavior consistent.

Add unit test:

    prepare_buffer_fusing.in_place_crop_axis1_rank_reducing_to_1d_not_optimized

  - CVS-192143, CVS-192144, CVS-192235

Signed-off-by: zhanmyz <yazhan.ma@intel.com>
@zhanmyz
zhanmyz force-pushed the cvs-192143-fix-regression-issue-v1 branch from 977326f to 7974de9 Compare August 17, 2026 07:30
@zhanmyz zhanmyz changed the title [GPU] Try to fix CVS-192143 issue [GPU] Prevent unsafe in-place crop optimization for 2D-to-1D bbox splits Aug 17, 2026
@zhanmyz
zhanmyz marked this pull request as ready for review August 17, 2026 07:53
@zhanmyz
zhanmyz requested review from a team as code owners August 17, 2026 07:53
@zhanmyz

zhanmyz commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

@zhanmyz The PR LGTM. Please open it for official review and merge.

Sure, Added unit test to cover the changed code.

@e-ddykim e-ddykim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please check if the issue still exists in the latest master? I want to check if the issue was resolved by #37267

@zhanmyz

zhanmyz commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Could you please check if the issue still exists in the latest master? I want to check if the issue was resolved by #37267

I’ve tested with the latest OV PR #37160 as well as PR #37267. Unfortunately, the issue still persists.
So it is not resolved by #37267.

@e-ddykim
e-ddykim added this pull request to the merge queue Aug 18, 2026
Merged via the queue into openvinotoolkit:master with commit 11d482c Aug 18, 2026
342 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: GPU OpenVINO GPU plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants