Skip to content

feat: configure service-account-issuer on workload cluster apiserver in prow CI templates#6268

Closed
andyzhangx wants to merge 111 commits into
kubernetes-sigs:release-1.23from
andyzhangx:fix-oidc-issuer-prow-templates
Closed

feat: configure service-account-issuer on workload cluster apiserver in prow CI templates#6268
andyzhangx wants to merge 111 commits into
kubernetes-sigs:release-1.23from
andyzhangx:fix-oidc-issuer-prow-templates

Conversation

@andyzhangx
Copy link
Copy Markdown
Member

@andyzhangx andyzhangx commented May 3, 2026

What this PR does

Adds service-account-issuer: ${SERVICE_ACCOUNT_ISSUER} to the apiServer.extraArgs in prow CI cluster templates.

Why

The prow CI templates create workload clusters with apiServer.extraArgs: {}, causing kube-apiserver to use the default --service-account-issuer=https://kubernetes.default.svc.cluster.local. This is an internal URL that AAD cannot reach from the public internet.

When CSI drivers (blob-csi-driver, azurefile-csi-driver) run workload identity e2e tests, they:

  1. Create a Federated Identity Credential (FIC) with the OIDC issuer URL
  2. Mount volumes using workload identity tokens
  3. AAD validates the SA token by calling back the OIDC issuer URL

Since the issuer URL is internal, AAD returns AADSTS50166: Request to External OIDC endpoint failed.

How it works

kind-with-registry.sh already creates a public OIDC storage account (capzoidc*) and exports SERVICE_ACCOUNT_ISSUER pointing to it. This PR makes the workload cluster apiserver use that same issuer URL via ${SERVICE_ACCOUNT_ISSUER} in the template.

The default value falls back to the original kubernetes.default.svc.cluster.local if SERVICE_ACCOUNT_ISSUER is not set, preserving backward compatibility.

Templates updated

  • cluster-template-prow.yaml
  • cluster-template-prow-ci-version.yaml
  • cluster-template-prow-machine-pool.yaml
  • cluster-template-prow-machine-pool-ci-version.yaml

Related issues

/kind feature
/area provider/azure

Release note:

configure service-account-issuer on workload cluster apiserver in prow CI templates

mboersma and others added 30 commits October 2, 2025 11:37
Signed-off-by: Vishal Anarase <iamvishalanarase@gmail.com>
…mplates

Signed-off-by: William Yao <william2000yao@gmail.com>
Update link to Google Artifact Registry staging
Signed-off-by: William Yao <william2000yao@gmail.com>
…ty-scan

Update branches in security scanner workflow for release v1.23
Signed-off-by: William Yao <william2000yao@gmail.com>
…rmance

Enable DRADeviceTaints and DRADeviceTaintRules feature gate in DRA templates
…5291

Updated Makefile to check and warn if az cli if unavailable in local
Signed-off-by: William Yao <william2000yao@gmail.com>
…t-metadata

Update test metadata and versions for CAPZ v1.23.0
Bumps the all-github-actions group with 3 updates: [github/codeql-action](https://github.com/github/codeql-action), [actions/setup-go](https://github.com/actions/setup-go) and [codecov/codecov-action](https://github.com/codecov/codecov-action).


Updates `github/codeql-action` from 4.34.1 to 4.35.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@3869755...c10b806)

Updates `actions/setup-go` from 6.3.0 to 6.4.0
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](actions/setup-go@4b73464...4a36011)

Updates `codecov/codecov-action` from 5.5.3 to 6.0.0
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@1af5884...57e3a13)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.35.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all-github-actions
- dependency-name: actions/setup-go
  dependency-version: 6.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all-github-actions
- dependency-name: codecov/codecov-action
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: all-github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: William Yao <william2000yao@gmail.com>
…ot/github_actions/all-github-actions-cdc0901d3f

dependabot(deps): bump the all-github-actions group with 3 updates
…ance-tests

Add alpha/beta feature gates to conformance test
…-sigs#6203)

* Bump Ray and KubeRay versions to latest releases

* Fix KubeRay e2e: increase head resources and cap object store memory

* Reduce head memory request to fit on self-managed nodes

* Label KubeRay tests with [KubeRay] instead of [OPTIONAL]

Use a dedicated [KubeRay] Ginkgo label so these tests only run from
the pull-cluster-api-provider-azure-e2e-kuberay presubmit job and
are excluded from the general e2e-optional job.
Bumps the all-github-actions group with 1 update: [step-security/harden-runner](https://github.com/step-security/harden-runner).


Updates `step-security/harden-runner` from 2.16.0 to 2.16.1
- [Release notes](https://github.com/step-security/harden-runner/releases)
- [Commits](step-security/harden-runner@fa2e9d6...fe10465)

---
updated-dependencies:
- dependency-name: step-security/harden-runner
  dependency-version: 2.16.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
…ot/github_actions/all-github-actions-bd09a0e644

dependabot(deps): bump step-security/harden-runner from 2.16.0 to 2.16.1 in the all-github-actions group
* Bump Go toolchain to v1.25.8

* Bump golang.org/x/net to v0.51.0

* Update trivy version to v0.69.2

* Fix go_install.sh to use project toolchain for tool builds
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.34.0 to 0.35.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](golang/text@v0.34.0...v0.35.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-version: 0.35.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…ot/go_modules/golang.org/x/text-0.35.0

dependabot(deps): bump golang.org/x/text from 0.34.0 to 0.35.0
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.33.0 to 0.34.0.
- [Commits](golang/mod@v0.33.0...v0.34.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-version: 0.34.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…ot/go_modules/golang.org/x/mod-0.34.0

dependabot(deps): bump golang.org/x/mod from 0.33.0 to 0.34.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.48.0 to 0.49.0.
- [Commits](golang/crypto@v0.48.0...v0.49.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.49.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…ot/go_modules/golang.org/x/crypto-0.49.0

dependabot(deps): bump golang.org/x/crypto from 0.48.0 to 0.49.0
* Add e2e spec for self-managed kuberay

* Use CI K8s builds for self-managed KubeRay tests

Follow the conformance CI pattern: resolve the latest CI Kubernetes
version from dl.k8s.io and use the ci-version flavor template which
downloads K8s binaries from source. This ensures the self-managed
KubeRay tests exercise the in-development Kubernetes at main rather
than only stable releases.

* Use objectStoreMemory constant for Ray object store size

* Extract shared rayClusterSpec helper to reduce duplication
k8s-ci-robot and others added 16 commits May 6, 2026 05:50
…onfig

Add AKS E2E scenario for maintenance configurations via ASO
Use uuid.Validate over uuid.Parse where appropriate
…lidation-coverage

Add unit tests for azuremachine_validation
Signed-off-by: William Yao <william2000yao@gmail.com>
Signed-off-by: William Yao <william2000yao@gmail.com>
…-1.24

Add release notes for release 1.24
Signed-off-by: William Yao <william2000yao@gmail.com>
…rclass

Update RKE2 ClusterClass to v1beta2 and bump provider to v0.24.3
…escriptions

Strip OpenAPI description fields from vendored ASO CRDs
…fault

Switch default Windows image to windows-2022
Update security scanner for release 1.24
@andyzhangx andyzhangx force-pushed the fix-oidc-issuer-prow-templates branch from 2c7a4b5 to 3cae9d8 Compare May 7, 2026 00:50
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 7, 2026
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 7, 2026
@andyzhangx andyzhangx force-pushed the fix-oidc-issuer-prow-templates branch 2 times, most recently from 06782c2 to 0367def Compare May 7, 2026 01:34
Set service-account-issuer: ${SERVICE_ACCOUNT_ISSUER} in the base
cluster template (templates/flavors/base/cluster-template.yaml).

This ensures projected service account tokens are signed with a
discoverable issuer URL. The default value falls back to the
kube-apiserver default (https://kubernetes.default.svc.cluster.local),
so existing deployments are unaffected.

When SERVICE_ACCOUNT_ISSUER is set (e.g., to a public OIDC endpoint),
workload identity flows (CSI drivers, pod identity) will work correctly
because AAD can discover and validate the token issuer.

/kind bug
@andyzhangx andyzhangx force-pushed the fix-oidc-issuer-prow-templates branch from 0367def to ca6f966 Compare May 7, 2026 01:49
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

@andyzhangx: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cluster-api-provider-azure-apidiff-v1beta1 ca6f966 link false /test pull-cluster-api-provider-azure-apidiff-v1beta1
pull-cluster-api-provider-azure-test-v1beta1 ca6f966 link true /test pull-cluster-api-provider-azure-test-v1beta1
pull-cluster-api-provider-azure-build-v1beta1 ca6f966 link true /test pull-cluster-api-provider-azure-build-v1beta1
pull-cluster-api-provider-azure-e2e-v1beta1 ca6f966 link true /test pull-cluster-api-provider-azure-e2e-v1beta1
pull-cluster-api-provider-azure-apiversion-upgrade-v1beta1 ca6f966 link true /test pull-cluster-api-provider-azure-apiversion-upgrade-v1beta1
pull-cluster-api-provider-azure-e2e-aks-v1beta1 ca6f966 link true /test pull-cluster-api-provider-azure-e2e-aks-v1beta1
pull-cluster-api-provider-azure-capi-e2e-v1beta1 ca6f966 link false /test pull-cluster-api-provider-azure-capi-e2e-v1beta1
pull-cluster-api-provider-azure-verify-v1beta1 ca6f966 link true /test pull-cluster-api-provider-azure-verify-v1beta1
pull-cluster-api-provider-azure-ci-entrypoint-v1beta1 ca6f966 link true /test pull-cluster-api-provider-azure-ci-entrypoint-v1beta1
pull-cluster-api-provider-azure-e2e-workload-upgrade-v1beta1 ca6f966 link true /test pull-cluster-api-provider-azure-e2e-workload-upgrade-v1beta1

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@andyzhangx
Copy link
Copy Markdown
Member Author

addressed by #6288

@andyzhangx andyzhangx closed this May 7, 2026
@github-project-automation github-project-automation Bot moved this from Wait-On-Author to Done in CAPZ Planning May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/provider/azure Issues or PRs related to azure provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

7 participants