KEP-9270: Introduce configuration for the MultiKueue Incremental Dispatcher

keps/9270-multikueue-incremental-step-size/README.md

@@ -0,0 +1,144 @@
+
+# KEP-9270: Introduce configuration for the MultiKueue Incremental Dispatcher
+<!-- toc -->
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [API Changes](#api-changes)
+  - [Execution Loop](#execution-loop)
+  - [Test Plan](#test-plan)
+    - [Prerequisite testing updates](#prerequisite-testing-updates)
+    - [Unit tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+    - [e2e tests](#e2e-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Possible Follow-ups](#possible-follow-ups)
+- [Drawbacks](#drawbacks)
+<!-- /toc -->
+
+## Summary
+
+This proposal introduces a new configuration structure for the MultiKueue Incremental Dispatcher, starting with a stepSize field. This setting will control the number of worker clusters that the Manager Cluster will attempt to dispatch a workload to concurrently.
+
+## Motivation
+
+Currently, the MultiKueue Incremental Dispatcher has a hardcoded batch size `batchSize := 3` when attempting to schedule a workload across multiple worker clusters. The primary limitation is that this value is not configurable. For administrators managing a massive fleet of worker clusters, a fixed batch size of 3 is too slow and delays job execution. Conversely, if administrators were to theoretically increase this to query all clusters simultaneously to speed things up, it would generate heavy API traffic and increase the risk of race conditions. By making this step size configurable, administrators can find the optimal balance between dispatch speed and Manager Cluster resource usage for their specific environment.
+Introducing a `stepSize` allows cluster administrators to configure a "batch size" for this process, finding the perfect balance between speed and system stability.
+
+### Goals
+
+* Add an `IncrementalDispatcherConfig` struct to the `MultiKueueConfig` API containing a `stepSize` parameter
+* Update the Workload Controller's dispatching loop to process worker clusters in batches defined by the `stepSize`.
+
+### Non-Goals
+
+* We will not change the way worker clusters are scored or prioritized.
+* We will not alter the `AllAtOnce` dispatcher behavior, as this proposal strictly targets the Incremental Dispatcher.
+
+## Proposal
+
+We propose updating the `MultiKueueConfig` API to introduce an `IncrementalDispatcherConfig` struct containing an optional integer for the step size, and updating the Workload Controller to respect this batch limit when iterating through the list of available worker clusters.
+
+### Risks and Mitigations
+
+**Risk:** An administrator might set the `stepSize` to an extremely high number (e.g., equal to the total number of clusters), which effectively recreates the heavy load issues of the `AllAtOnce` dispatcher and increases the chance of race conditions where multiple clusters admit the job at the exact same time.
+
+**Mitigation:** Provide clear API documentation regarding best practices. The Workload Controller's conflict resolution logic will naturally handle race conditions, just as it already does for the `AllAtOnce` strategy.
+
+## Design Details
+
+### API Changes
+
+We will update the global `Configuration` API (specifically the `MultiKueue` struct) to include a new configuration struct, rather than placing it in `MultiKueueConfigSpec`. This keeps the incremental dispatcher configuration co-located with the `DispatcherName` selection.
+
+The field will be named `IncrementalDispatcherConfig` and will contain a `StepSize` field. To ensure backwards compatibility and preserve existing system behavior, the default value for `StepSize` will be `3`.
+
+We will also use Kubebuilder validation markers (e.g., CEL rules) to enforce that this struct can only be populated if `DispatcherName` is set to the incremental dispatcher.
+
+```go
+type MultiKueue struct {
+       // Other existing fields...
+
+	  // DispatcherName defines the type of dispatcher to use.
+      DispatcherName *string `json:"dispatcherName,omitempty"`
+
+      // IncrementalDispatcherConfig contains the configuration for the incremental dispatcher.
+      // This field is only valid when DispatcherName is set to the incremental dispatcher.
+	  // Note: This field is going to be ignored when the MultiKueueIncrementalDispatcher feature gate is disabled.
+      // +optional
+      IncrementalDispatcherConfig *IncrementalDispatcherConfig `json:"incrementalDispatcherConfig,omitempty"`
+}
+
+type IncrementalDispatcherConfig struct {
+      // StepSize defines the number of worker clusters the Incremental 
+      // Dispatcher will query simultaneously. 
+      // Minimum value is 1. If not set, it defaults to 3.
+      // +optional
+      // +kubebuilder:default=3
+      // +kubebuilder:validation:Minimum=1
+      StepSize *int32 `json:"stepSize,omitempty"`
+}
+```
+
+### Execution Loop
+The core logic change is minimal because the batching mechanism is already implemented in the Incremental Dispatcher (`pkg/controller/workloaddispatcher/incrementaldispatcher.go`).
+
+The update will simply modify the existing logic to:
+1. Read the `stepSize` value from the `MultiKueueConfig`.
+2. Replace the hardcoded `batchSize := 3` variable with this dynamically configured `stepSize` parameter.
+
+The existing loop and conflict resolution mechanisms will naturally handle the rest of the execution based on this new size, meaning no major architectural changes are required to the dispatching loop itself.
+
+### Test Plan
+
+ I/we understand the owners of the involved components may require updates to existing tests to make this code solid enough prior to committing the changes necessary to implement this enhancement.
+
+#### Prerequisite testing updates
+
+No major prerequisite testing updates are required. Current mock worker cluster frameworks in EnvTest are sufficient.
+
+#### Unit tests
+
+- `pkg/controller/core/workload_controller_test.go`: Verify that the worker cluster list is correctly divided into chunks matching the `stepSize`.
+- `pkg/controller/core/workload_controller_test.go`: Verify that the default value remains `3` if the user does not specify a `stepSize`.
+
+#### Integration tests
+
+- `TestIncrementalDispatcherStepSize`: Simulate a Manager Cluster with 5 Worker Clusters. Set `stepSize` to `2`. Submit a workload and verify via the API server that exactly 2 proxy workloads are created in the first step. Simulate a rejection on the first 2 clusters, and verify that the controller correctly creates the next 2 proxy workloads in the second step.
+
+#### e2e tests
+
+Standard E2E multi-cluster tests will be updated to include an iteration where `MultiKueueConfig` is deployed with a `stepSize > 1` to ensure workloads execute successfully end-to-end without duplication.
+
+### Graduation Criteria
+
+* **Alpha:**
+    * API fields added.
+    * Workload Controller updated to support batching.
+    * Unit tests and EnvTest integration tests passing.
+* **Beta:** * Gather feedback from the community.
+    * E2E tests added and passing consistently.
+* **Stable:**
+    * Feature has been in Beta for at least one release cycle with no major bugs reported.
+
+## Implementation History
+
+- 2026-05-01: KEP proposed and initial draft created.
+
+## Possible Follow-ups
+
+By introducing the `IncrementalDispatcherConfig` structure, we pave the way for extending the configuration of the incremental dispatcher in the future without breaking API compatibility.
+
+Possible future configurations that could be added to this struct include:
+
+* `timeout`: To configure how long the dispatcher waits before moving to the next batch of clusters.
+
+* `stepSizeAsPercent`: To define the batch size as a percentage of the total available worker clusters rather than a static integer.
+## Drawbacks
+
+This adds slight code complexity to the existing Incremental Dispatcher loop and introduces one additional configuration parameter for administrators to manage and tune.

keps/9270-multikueue-incremental-step-size/kep.yaml

@@ -0,0 +1,33 @@
+title: Introduce configuration for the MultiKueue Incremental Dispatcher
+kep-number: 9270
+authors:
+  - "@Mostafahassen1"
+status: provisional
+creation-date: "2026-05-01"
+reviewers:
+  - olekzabl
+  - mimowo
+approvers:
+  - mimowo
+  - olekzabl
+
+# The target maturity stage in the current dev cycle for this KEP.
+stage: alpha
+
+# The most recent milestone for which work toward delivery of this KEP has been
+# done. Kueue typically uses 0.x versioning.
+latest-milestone: "v0.19"
+
+# The milestone at which this feature was, or is targeted to be, at each stage.
+milestone:
+  alpha: "v0.19"
+  beta: "TBD"
+  stable: "TBD"
+
+# The following PRR answers are required at alpha release
+# List the feature gate name and the components for which it must be enabled
+feature-gates:
+  - name: MultiKueueIncrementalDispatcherConfig
+    components:
+      - kueue-controller-manager
+disable-supported: true