Skip to content

parallel test fixes - #1308

Draft
vickeybrown wants to merge 1 commit into
ComplianceAsCode:masterfrom
vickeybrown:parallel-test-improvements
Draft

parallel test fixes#1308
vickeybrown wants to merge 1 commit into
ComplianceAsCode:masterfrom
vickeybrown:parallel-test-improvements

Conversation

@vickeybrown

Copy link
Copy Markdown
Collaborator

Fix race conditions and silent failures in parallel e2e tests

Several parallel e2e tests are flaky due to race conditions from shared
namespace resource contention and bugs in the test framework helpers.

Framework fixes:

  • CheckPodPriorityClass silently passed when the priority class didn't match —
    it logged the mismatch but returned success. It now returns an error so tests
    actually fail on unexpected priority class values.
  • CheckPodLimit had a copy-paste bug where the memory limit mismatch error
    message reported CPU values instead of memory values.
  • WaitForPrometheusMetricTargets used a hardcoded "metrics-test" pod name,
    causing oc run collisions when multiple tests invoke it concurrently. Changed
    to a unique timestamp-based name, matching the fix already applied to
    AssertMetricsEndpointUsesHTTPVersion and getMetricResults.

Test race condition fixes:

TestScheduledSuitePriorityClass, TestScheduledSuiteInvalidPriorityClass, and
TestScanSettingBinding all listed scanner pods using the broad "workload":
"scanner" label, which matches pods from any concurrent test in the namespace.
Two of them also listed pods before the scan had started, so the list was
often empty and the checks were silently skipped. The third listed pods after
PhaseDone, when pods may have already been garbage collected.

All three now wait for PhaseRunning to ensure scanner pods exist, then use
GetPodsForScan() to list only pods belonging to the specific scan under test.

@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: vickeybrown
Once this PR has been reviewed and has the lgtm label, please assign yuumasato for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@vickeybrown

Copy link
Copy Markdown
Collaborator Author

/test all

@github-actions

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1308-cbdea4cc424fdc7e608352907cf015972574321c

@vickeybrown
vickeybrown marked this pull request as ready for review July 15, 2026 21:06
@openshift-ci
openshift-ci Bot requested review from jhrozek and rhmdnd July 15, 2026 21:06
@vickeybrown

Copy link
Copy Markdown
Collaborator Author

/test e2e-aws-parallel

@vickeybrown

Copy link
Copy Markdown
Collaborator Author

/retest

@vickeybrown
vickeybrown marked this pull request as draft July 16, 2026 18:43
@vickeybrown
vickeybrown force-pushed the parallel-test-improvements branch from cbdea4c to d19bcd3 Compare July 16, 2026 18:55
@github-actions

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1308-d19bcd31d7e95f7fa2dbdf7ec247e1f38d604845

@vickeybrown

Copy link
Copy Markdown
Collaborator Author

/test e2e-aws-parallel

@vickeybrown
vickeybrown force-pushed the parallel-test-improvements branch from d19bcd3 to a1f2acc Compare July 17, 2026 13:52
@vickeybrown

Copy link
Copy Markdown
Collaborator Author

/test e2e-aws-parallel

@github-actions

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1308-a1f2acc328502f5701e2c6223a0a4e79dbcae6ae

@vickeybrown
vickeybrown removed request for jhrozek and rhmdnd July 17, 2026 14:47
@openshift-ci

openshift-ci Bot commented Jul 17, 2026

Copy link
Copy Markdown

@vickeybrown: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-parallel a1f2acc link true /test e2e-aws-parallel

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant