Honor AzureCluster ControlPlaneEndpoint.Port in API server LB#6278
Honor AzureCluster ControlPlaneEndpoint.Port in API server LB#6278mboersma wants to merge 1 commit into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6278 +/- ##
=======================================
Coverage 43.88% 43.88%
=======================================
Files 289 289
Lines 25351 25353 +2
=======================================
+ Hits 11125 11127 +2
Misses 13448 13448
Partials 778 778 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
willie-yao
left a comment
There was a problem hiding this comment.
Looks good overall, thanks for the fix! Just had a few comments
| port := strconv.Itoa(int(s.APIServerPort())) | ||
|
|
||
| missingAPIPort := subnet.GetSecurityRuleByDestination(port) == nil | ||
| if missingAPIPort { | ||
| subnet.SecurityGroup.SecurityRules = append(subnet.SecurityGroup.SecurityRules, infrav1.SecurityRule{ | ||
| Name: "allow_apiserver", | ||
| Description: "Allow K8s API Server", | ||
| Priority: 2201, | ||
| Protocol: infrav1.SecurityGroupProtocolTCP, | ||
| Direction: infrav1.SecurityRuleDirectionInbound, | ||
| Source: ptr.To("*"), | ||
| SourcePorts: ptr.To("*"), | ||
| Destination: ptr.To("*"), | ||
| DestinationPorts: ptr.To(port), | ||
| Action: infrav1.SecurityRuleActionAllow, | ||
| }) |
There was a problem hiding this comment.
One weird thing that copilot discovered is that this adds a security rule to allow apiserver based on s.APIServerPort(), and clusters from before this fix would return 6443 erroneously. If someone upgrades their cluster with a custom port to a version that inclues this fix, now APIServerPort() will return 443 or whatever custom port was set and cause a duplicate security rule to be created. This would be a problem because Azure rejects NSGs with duplicate rule names/priorities. Do you think this is something we need to address in this PR or a follow-up?
| expectAPIServerPort int32 | ||
| name string | ||
| clusterName string | ||
| clusterNetowrk clusterv1.ClusterNetwork |
There was a problem hiding this comment.
| clusterNetowrk clusterv1.ClusterNetwork | |
| clusterNetwork clusterv1.ClusterNetwork |
What type of PR is this?
/kind bug
What this PR does / why we need it:
ClusterScope.APIServerPort()only consultedCluster.Spec.ClusterNetwork.APIServerPort, falling back to 6443. As a result, when a user setAzureCluster.Spec.ControlPlaneEndpoint.Port(e.g. to 443) the API server load balancer rule, health probe, and NSG rule were still constructed for port 6443, even though the AzureCluster spec doc string explicitly tells users they may set this field and CAPZ will honor it.This change updates
APIServerPort()to consultAzureCluster.Spec.ControlPlaneEndpoint.Portfirst, so a user-supplied port flows through to:getLoadBalancingRules)getProbes)ControlPlaneEndpoint.Portin the AzureCluster reconciler (already idempotent — only writes when the field is 0)Which issue(s) this PR fixes:
Fixes #5781
Special notes for your reviewer:
No e2e change is included. None of the existing flavors uses a non-6443 port. Modifying
prow-apiserver-ilb(the closest candidate) would also require coordinating the API server bind port in the KubeadmControlPlane and updating the test's probe assertions, which is non-trivial. Coverage is provided at the unit level instead:azure/scope/cluster_test.go::TestAPIServerPortgains cases for the new precedence.azure/services/loadbalancers/spec_test.go::TestAPIServerLBPortPropagationverifies that a non-defaultLBSpec.APIServerPortpropagates to LB rule ports and the probe port.Adding a dedicated e2e flavor for non-default control plane port would be a reasonable follow-up.
TODOs:
Release note: