Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 165 additions & 0 deletions docs/book/src/plugins/extending/multicluster-runtime-plugin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# multicluster-runtime/v1-alpha Plugin

The `multicluster-runtime/v1-alpha` plugin adds
[sigs.k8s.io/multicluster-runtime](https://github.com/kubernetes-sigs/multicluster-runtime)
support to a Kubebuilder project. Instead of reconciling objects in a single cluster, your
controller will reconcile objects across all clusters registered with the chosen provider.

This plugin is built into the `kubebuilder` binary — no separate installation is required.

## When to use this plugin

Use `multicluster-runtime/v1-alpha` when you need to:

- Manage resources across **multiple Kubernetes clusters** from a single operator binary
- React to cluster lifecycle events (clusters joining or leaving the fleet)
- Run **fleet-wide controllers** while still using familiar controller-runtime patterns

## Prerequisites

- Kubebuilder v4+
- Go 1.22+

## Project initialization

Chain `multicluster-runtime/v1-alpha` after `go/v4`:

```bash
kubebuilder init \
--plugins go/v4,multicluster-runtime/v1-alpha \
--domain example.com \
--repo github.com/example/myop \
--provider kubeconfig
```

The plugin rewrites `cmd/main.go` to use `mcmanager.New(...)` instead of `ctrl.NewManager(...)`.

## Provider selection guide

The `--provider` flag (default: `kubeconfig`) controls how clusters are discovered.

| Provider | Flag value | Use case |
|---|---|---|
| **Kubeconfig secrets** | `kubeconfig` | Dynamic fleet — clusters join/leave at runtime by creating kubeconfig Secrets |
| **Namespace** | `namespace` | Single cluster, namespace-per-tenant; each namespace is treated as a "cluster" |
| **Cluster API** | `cluster-api` | Fleet managed by [Cluster API](https://cluster-api.sigs.k8s.io/) controllers |
| **File** | `file` | Static cluster list; one kubeconfig file per cluster in a directory (great for CI) |

### Kubeconfig provider (default)

```bash
kubebuilder init --plugins go/v4,multicluster-runtime/v1-alpha \
--domain example.com --repo github.com/example/myop \
--provider kubeconfig
```

Add a cluster at runtime by creating a Secret with the kubeconfig:

```yaml
apiVersion: v1
kind: Secret
metadata:
name: my-cluster
namespace: default
labels:
# label recognized by the kubeconfig provider
multicluster.x-k8s.io/cluster-name: my-cluster
type: Opaque
data:
kubeconfig: <base64-encoded-kubeconfig>
```

### Namespace provider

```bash
kubebuilder init --plugins go/v4,multicluster-runtime/v1-alpha \
--domain example.com --repo github.com/example/myop \
--provider namespace
```

The generated `cmd/main.go` starts the provider and manager concurrently using `errgroup`.

### File provider

```bash
kubebuilder init --plugins go/v4,multicluster-runtime/v1-alpha \
--domain example.com --repo github.com/example/myop \
--provider file \
--kubeconfig-dir /etc/kubeconfig
```

Place one kubeconfig file per cluster in `--kubeconfig-dir`. Each file name becomes the
cluster name.

## Create a multicluster controller

```bash
kubebuilder create api \
--plugins go/v4,multicluster-runtime/v1-alpha \
--group foo --version v1 --kind Foo \
--controller --resource
```

The generated controller (`internal/controller/foo_controller.go`) uses:

- `mcreconcile.Request` — carries `ClusterName` in addition to the usual `NamespacedName`
- `mcbuilder.ControllerManagedBy(mgr)` — watches objects across all registered clusters
- `mcmanager.Manager` — the multicluster-aware manager type

### Using `req.ClusterName`

```go
func (r *FooReconciler) Reconcile(ctx context.Context, req mcreconcile.Request) (ctrl.Result, error) {
log := log.FromContext(ctx).WithValues("cluster", req.ClusterName)

// Fetch the object from the correct cluster's cache.
foo := &foov1.Foo{}
if err := r.Get(ctx, req.NamespacedName, foo); err != nil {
return ctrl.Result{}, client.IgnoreNotFound(err)
}

log.Info("Reconciling Foo", "name", foo.Name)
// ... your business logic ...
return ctrl.Result{}, nil
}
```

## Webhooks

Webhooks register with the **local cluster's** API server — they do not need multicluster
changes. You can scaffold them normally:

```bash
kubebuilder create webhook \
--plugins go/v4,multicluster-runtime/v1-alpha \
--group foo --version v1 --kind Foo \
--defaulting --programmatic-validation
```

The `multicluster-runtime/v1-alpha` plugin does not modify webhook files. The webhook
scaffolding is handled entirely by `go/v4` and the output is identical to a single-cluster
project. Webhooks register with the **local** cluster's API server and do not need
multicluster changes.

## Switching providers

Use `kubebuilder edit` to replace the provider in an existing project:

```bash
kubebuilder edit --plugins multicluster-runtime/v1-alpha --provider namespace
```

This rewrites `cmd/main.go` while preserving all `// +kubebuilder:scaffold:*` markers so
that future `kubebuilder create api` and `kubebuilder create webhook` commands still work.

## Plugin chain note

This plugin is designed to run **after** `go/v4`. The plugin chain `go/v4,multicluster-runtime/v1-alpha`
means:

1. `go/v4` scaffolds the standard project structure
2. `multicluster-runtime/v1-alpha` rewrites `cmd/main.go` to use the multicluster manager

If `go/v4` is absent from the chain, the scaffolded `cmd/main.go` will not compile
because the standard project structure (`api/`, `internal/controller/`, `Makefile`, etc.)
will be missing. Always chain `go/v4` first.
2 changes: 2 additions & 0 deletions internal/cli/cmd/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ import (
grafanav1alpha "sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/grafana/v1alpha"
helmv1alpha "sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/helm/v1alpha" //nolint:staticcheck // Deprecated
helmv2alpha "sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/helm/v2alpha"
multiclusterruntimev1alpha "sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/multicluster-runtime/v1alpha"
)

// Run bootstraps & runs the CLI
Expand Down Expand Up @@ -78,6 +79,7 @@ func Run() {
&helmv1alpha.Plugin{},
&helmv2alpha.Plugin{},
&autoupdatev1alpha.Plugin{},
&multiclusterruntimev1alpha.Plugin{},
),
cli.WithPlugins(externalPlugins...),
cli.WithDefaultPlugins(cfgv3.Version, gov4Bundle),
Expand Down
68 changes: 68 additions & 0 deletions pkg/plugins/optional/multicluster-runtime/v1alpha/api.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/*
Copyright 2026 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha

import (
"fmt"

"sigs.k8s.io/kubebuilder/v4/pkg/config"
"sigs.k8s.io/kubebuilder/v4/pkg/machinery"
"sigs.k8s.io/kubebuilder/v4/pkg/model/resource"
"sigs.k8s.io/kubebuilder/v4/pkg/plugin"
"sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/multicluster-runtime/v1alpha/scaffolds"
)

var _ plugin.CreateAPISubcommand = &createAPISubcommand{}

type createAPISubcommand struct {
config config.Config
resource *resource.Resource
}

func (p *createAPISubcommand) UpdateMetadata(cliMeta plugin.CLIMetadata, subcmdMeta *plugin.SubcommandMetadata) {
subcmdMeta.Description = `Overwrite the controller scaffolded by go/v4 with a multicluster-aware version.

The generated controller uses:
- mcreconcile.Request (carries ClusterName alongside NamespacedName)
- mcbuilder.ControllerManagedBy(mgr) (watches across all registered clusters)
- mcmanager.Manager (multicluster manager type)`
subcmdMeta.Examples = fmt.Sprintf(` %[1]s create api \
--plugins go/v4,%[2]s \
--group foo --version v1 --kind Foo --controller --resource`, cliMeta.CommandName, plugin.KeyFor(Plugin{}))
}

func (p *createAPISubcommand) InjectConfig(c config.Config) error {
p.config = c
return nil
}

func (p *createAPISubcommand) InjectResource(res *resource.Resource) error {
p.resource = res
return nil
}

func (p *createAPISubcommand) Scaffold(fs machinery.Filesystem) error {
if p.resource == nil || !p.resource.HasController() {
return nil
}
s := scaffolds.NewAPIScaffolder(p.config, *p.resource)
s.InjectFS(fs)
if err := s.Scaffold(); err != nil {
return fmt.Errorf("failed to scaffold api: %w", err)
}
return nil
}
103 changes: 103 additions & 0 deletions pkg/plugins/optional/multicluster-runtime/v1alpha/api_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
/*
Copyright 2026 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha

import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/spf13/afero"

"sigs.k8s.io/kubebuilder/v4/pkg/config"
cfgv3 "sigs.k8s.io/kubebuilder/v4/pkg/config/v3"
"sigs.k8s.io/kubebuilder/v4/pkg/machinery"
"sigs.k8s.io/kubebuilder/v4/pkg/model/resource"
)

const controllerPath = "internal/controller/bar_controller.go"

var _ = Describe("createAPISubcommand", func() {
var (
subCmd *createAPISubcommand
cfg config.Config
)

BeforeEach(func() {
subCmd = &createAPISubcommand{}
cfg = cfgv3.New()
_ = cfg.SetRepository("github.com/example/myop")
_ = cfg.SetDomain("example.com")
Expect(subCmd.InjectConfig(cfg)).To(Succeed())
})

Context("InjectConfig", func() {
It("should store the config", func() {
Expect(subCmd.config).To(Equal(cfg))
})
})

Context("InjectResource", func() {
It("should store the resource", func() {
res := &resource.Resource{
GVK: resource.GVK{Group: "foo", Version: "v1", Kind: "Bar"},
}
Expect(subCmd.InjectResource(res)).To(Succeed())
Expect(subCmd.resource).To(Equal(res))
})
})

Context("Scaffold", func() {
var memFS machinery.Filesystem

BeforeEach(func() {
memFS = machinery.Filesystem{FS: afero.NewMemMapFs()}
})

It("should be a no-op when resource is nil", func() {
subCmd.resource = nil
Expect(subCmd.Scaffold(memFS)).To(Succeed())
})

It("should be a no-op when resource has no controller", func() {
subCmd.resource = &resource.Resource{
GVK: resource.GVK{Group: "foo", Version: "v1", Kind: "Bar"},
Controller: false,
}
Expect(subCmd.Scaffold(memFS)).To(Succeed())
})

It("should write a controller when resource has a controller", func() {
subCmd.resource = &resource.Resource{
GVK: resource.GVK{Group: "foo", Version: "v1", Kind: "Bar"},
Plural: "bars",
Path: "github.com/example/myop/api/v1",
Controller: true,
}
Expect(subCmd.Scaffold(memFS)).To(Succeed())

exists, err := afero.Exists(memFS.FS, controllerPath)
Expect(err).NotTo(HaveOccurred())
Expect(exists).To(BeTrue())

content, err := afero.ReadFile(memFS.FS, controllerPath)
Expect(err).NotTo(HaveOccurred())
Expect(string(content)).To(ContainSubstring("mcreconcile.Request"))
Expect(string(content)).To(ContainSubstring("mcbuilder.ControllerManagedBy"))
Expect(string(content)).To(ContainSubstring("mcmanager.Manager"))
Expect(string(content)).To(ContainSubstring(`"sigs.k8s.io/multicluster-runtime/pkg/reconcile"`))
})
})
})
Loading