Run MultiNetworkPolicy test 20x in parallel for flake detection#79572
Run MultiNetworkPolicy test 20x in parallel for flake detection#79572weliang1 wants to merge 6 commits into
Conversation
Configure the nightly-4.22 e2e-aws-ovn-dedicated-serial-techpreview periodic job to execute the [sig-network][Feature:MultiNetworkPolicy] test 20 times in parallel. Changes: - Set shard_count to 20 (was 2) to create 20 parallel job instances - Add TEST_ARGS filter to run only MultiNetworkPolicy tests - Each shard runs the same test filter concurrently Purpose: Detect flaky tests and ensure test stability across multiple concurrent executions for OCPBUGS-85529. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
/pj-rehearse |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds ChangesTest Job Configuration
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 12✅ Passed checks (12 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/pj-rehearse ack |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
- Add TEST_SUITE: openshift/conformance/serial to env section - Ensures test suite is explicitly defined for MultiNetworkPolicy filtering - Matches configuration pattern used in multus-networkpolicy repo Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fix Attempt: Explicitly Set TEST_SUITEIssue Found: Initial rehearsal runs showed that MultiNetworkPolicy tests were not executed despite the TEST_ARGS filter being configured. Fix Applied: Explicitly set Rationale: While the workflow Next Steps:
/pj-rehearse |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Remove TEST_SUITE to properly filter MultiNetworkPolicy tests. The previous config ran all 2290 serial conformance tests instead of just the filtered MultiNetworkPolicy tests because TEST_SUITE loaded the entire openshift/conformance/serial suite before applying the --run filter. Changes: - Remove TEST_SUITE: openshift/conformance/serial - Reduce shard_count from 20 to 2 (only running filtered tests now) - Keep TEST_ARGS: --run \[sig-network\]\[Feature:MultiNetworkPolicy\] This will significantly reduce test execution time by only running the MultiNetworkPolicy tests across 2 shards instead of 2290 tests across 20 shards. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This regenerates the periodic jobs to match the updated config where shard_count was reduced from 20 to 2 for the MultiNetworkPolicy test job. Generated changes: - Removed shards 3-20 (18 jobs no longer needed) - Updated to 2 shards: 1of2 and 2of2 - Total reduction: 1536 lines removed Generated with: make jobs Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
/pj-rehearse |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@weliang1: no rehearsable tests are affected by this change |
|
/pj-rehearse periodic-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@weliang1: job(s): periodic-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview either don't exist or were not found to be affected, and cannot be rehearsed |
|
/pj-rehearse periodic-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-1of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@weliang1: job(s): periodic-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-1of2 either don't exist or were not found to be affected, and cannot be rehearsed |
This adds a TEMPORARY presubmit version of the periodic job that can be rehearsed for pre-merge testing. Jobs created: - pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-1of2 - pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-2of2 Configuration (same as periodic): - shard_count: 2 - TEST_ARGS: --run \[sig-network\]\[Feature:MultiNetworkPolicy\] - TEST_SUITE: removed (filtering only via TEST_ARGS) Usage: - /pj-rehearse nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test - or /test nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-1of2 **NOTE: This is TEMPORARY for testing only. Will be removed before final merge.** Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: weliang1 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
/pj-rehearse nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/test nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-1of2 |
|
@weliang1: The specified target(s) for The following commands are available to trigger optional jobs: Use DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-1of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-2of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-2of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-2of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse abort |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Removed DEDICATED_HOST and aws-edge-zones-quota-slice lease requirements from the temporary presubmit test job to enable faster testing. These resources have very limited capacity causing 2+ hour queue times. The simplified configuration allows immediate rehearsal testing to verify that TEST_ARGS filtering works correctly (should run ~10-20 MultiNetworkPolicy tests instead of 2290 serial conformance tests). Changes: - Removed DEDICATED_HOST: "yes" environment variable - Removed aws-edge-zones-quota-slice lease - Removed ipi-deprovision-aws-dedicated-hosts post step - Kept core test configuration: TEST_ARGS filter and 2-shard setup This is a temporary test-only job that will not be merged.
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-1of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-2of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-1of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@weliang1: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-2of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
5e83324 to
ee19d33
Compare
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-1of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-dedicated-serial-techpreview-presubmit-test-2of2 |
|
@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@weliang1: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Summary
Configure the nightly-4.22 e2e-aws-ovn-dedicated-serial-techpreview periodic job to execute the
[sig-network][Feature:MultiNetworkPolicy]test 20 times in parallel for improved flake detection and stability testing.Changes
--run \[sig-network\]\[Feature:MultiNetworkPolicy\]to run only MultiNetworkPolicy testsPurpose
This configuration provides 20x test coverage for the MultiNetworkPolicy feature to:
Modified Files
ci-operator/config/openshift/release/openshift-release-main__nightly-4.22.yaml- Source configurationci-operator/jobs/openshift/release/openshift-release-main-periodics.yaml- Generated Prow jobsTesting Plan
/pj-rehearseto validate job configurationRelated
/assign @weliang1
Changes to OpenShift Release CI Configuration
This PR updates OpenShift release CI (openshift/release) configuration for nightly-4.22 e2e runs to exercise the sig-network Feature:MultiNetworkPolicy tests in a filtered, sharded workflow and to provide a rehearsal path for that filtered job.
Practical effects
Why this was done
Validation and troubleshooting performed / recommended
Files affected
Overall conclusion