diff --git a/docs/guide/gateway/assets/migration/dryrun_workflow.png b/docs/guide/gateway/assets/migration/dryrun_workflow.png new file mode 100644 index 0000000000..5bc850341a Binary files /dev/null and b/docs/guide/gateway/assets/migration/dryrun_workflow.png differ diff --git a/docs/guide/gateway/assets/migration/migration_flow.png b/docs/guide/gateway/assets/migration/migration_flow.png new file mode 100644 index 0000000000..eb65745bfc Binary files /dev/null and b/docs/guide/gateway/assets/migration/migration_flow.png differ diff --git a/docs/guide/gateway/assets/migration/service_reuse.png b/docs/guide/gateway/assets/migration/service_reuse.png new file mode 100644 index 0000000000..3180d82d73 Binary files /dev/null and b/docs/guide/gateway/assets/migration/service_reuse.png differ diff --git a/docs/guide/gateway/assets/migration/traffic_strategies.png b/docs/guide/gateway/assets/migration/traffic_strategies.png new file mode 100644 index 0000000000..f93cacf73e Binary files /dev/null and b/docs/guide/gateway/assets/migration/traffic_strategies.png differ diff --git a/docs/guide/gateway/in_cluster_console.md b/docs/guide/gateway/in_cluster_console.md new file mode 100644 index 0000000000..b49d4915b6 --- /dev/null +++ b/docs/guide/gateway/in_cluster_console.md @@ -0,0 +1,278 @@ +# In-Cluster Migration Console + + + +!!! warning "Under Development" + The migration tool and its console are under active development. + Features may change. + +The migration console is a local web UI bundled into the `lbc-migrate` binary. +It compares the ingress controller's dry-run plan against the gateway controller's +dry-run plan side by side, field by field, so you can confirm the generated +Gateway manifests will create same AWS resources as the current Ingres before switching traffic. + +The console is read-only. It connects to your Kubernetes cluster using the +current kubeconfig context (the same config used by `kubectl`). It reads Gateway +and Ingress resources and their annotations. It never creates, updates, or +deletes cluster or AWS resources. + +## What it supports + +- **Cluster-wide discovery** of namespaces that hold dry-run Gateways, + each paired with its source Ingress via the `gateway.k8s.aws/migrated-from` + tag. +- **Side-by-side comparison** of the full built model stack — LoadBalancer, + Listeners, ListenerRules, TargetGroups, TargetGroupBindings, SecurityGroups — + produced by each controller. +- **Side-by-side comparison** with field-level diff and status filtering. +- **Field-level diff** with four statuses: `same`, `changed`, `added`, `removed`. + Slice fields (e.g. SecurityGroup ingress rules) are compared as multisets. +- **Known-change filtering** that hides known migration artifacts (migrated-from + tag, controller-generated names, health-check default drift, forward weights, + `targetGroupARN.$ref` pointer churn) so genuine user-visible changes stand out. +- **Resource correlation across controllers**: `TargetGroup` and + `TargetGroupBinding` are keyed by `serviceRef.name:port` so the same backing + service shows as one correlated row instead of a removed+added pair, even + though the two controllers generate different raw stack IDs. +- **IngressGroup resolution** including cross-namespace groups, using the + `migrated-from` tag plus a cluster-wide list of Ingresses to locate the single + group member that carries the plan annotation. +- **Export** — download a self-contained HTML report or raw JSON to share the + diff with team members who do not have cluster access. + +What it does not do: + +- It does not call AWS or check live resource state. It only diffs the two + dry-run plans. +- It does not have a namespace-scoped mode. Cross-namespace ingress groups + require cluster-wide list permission on Ingresses and Gateways. +- It does not modify any Kubernetes or AWS resource. + +## How to launch + +The console is part of the dry-run verification workflow (Step 2 of the +[Migration Guide](migrate_from_ingress.md#step-2-preview-with-dry-run)). In +summary: + +1. Enable `IngressPlanAnnotation=true` feature gate on the controller +2. Generate Gateway manifests with `lbc-migrate` (dry-run is on by default) +3. Apply the manifests — both controllers write their plans as annotations +4. Launch the console to compare them + +```bash +lbc-migrate --console +# or bind to a different port +lbc-migrate --console --port 9000 +``` + +The console binds to `http://localhost:8080` by default and operates cluster-wide +using your current kubeconfig context. Open that URL in a browser. + +See the [Migration Guide](migrate_from_ingress.md) for the full step-by-step +instructions including prerequisites and feature gate setup. + +## Using the console + +### Landing page + +The landing page lists every namespace that has at least one Gateway carrying +a `gateway.k8s.aws/dry-run-plan` annotation, with a count of those Gateways. +Namespaces without dry-run plans do not appear. + +An info alert at the top reminds you that the console reads from the cluster +using your current kubeconfig context and that all operations are read-only. + +![Landing page showing namespaces with dry-run plans](assets/console/landing.png) + +Click a namespace to see its gateways. + +### Gateway list + +After selecting a namespace, the console lists all Gateways with dry-run plans +in that namespace. Each gateway row shows a summary of its diff status (how +many fields are same, changed, removed, or added). + +![Gateway list showing summary pills per gateway](assets/console/gateway-list.png) + +Click a gateway to enter the comparison view. + +### Comparison view + +The comparison view shows a side-by-side diff for the selected gateway. It is +organized into these regions: + +![Comparison view with filter controls and resource columns](assets/console/comparison-overview.png) + +- **Segmented filter control** — buttons for `All`, `Same`, `Changed`, + `Removed`, `Added` with counts. Click a button to scope the view to only + resources in that status. Clicking an active filter resets back to All. + Hover over any filter for a description of what it shows. +- **Hide known changes toggle** — filters out diffs classified as known + migration artifacts (see [How diffs are classified](#how-diffs-are-classified) + below). On by default. +- **Export buttons** — download the diff as a self-contained HTML report or raw + JSON. A confirmation dialog warns that the exported file can be viewed without + cluster access. +- **Ingress model** (left) and **Gateway model** (right) — each resource in the + stack appears as a card. When a status filter is active, each card shows a + status tag next to the resource ID for accessibility. + +![Resource cards with Changed filter active showing status tags](assets/console/diff-list.png) + +Click a card to open the detail drawer. The drawer lists every field with its +ingress-side value, gateway-side value, and status. It carries its own +"Hide known changes" checkbox for per-resource filtering. When a known cause is +identified, it appears in the "Known Cause" column. + +![Detail drawer showing field-level diff with status and Known Cause columns](assets/console/detail-drawer.png) + +### Breadcrumb navigation + +The top bar shows breadcrumbs for your current location: +`Namespaces / / `. Click any breadcrumb to navigate back +to that level. The "← Back" button goes up one level. + +### How resources are correlated + +Resources are matched across the two plans by a correlation key: + +- For most resource types the raw stack ID is stable, so the key is the ID itself. +- For `TargetGroup` and `TargetGroupBinding` the two controllers generate + different raw IDs even when pointing at the same backing service. The console + keys these on the TGB's `serviceRef.name:port`, producing a single correlated + row with field-level diffs instead of a removed+added pair. + +### How diffs are classified + +Every field diff is assigned one of four statuses: + +- **same** — both sides produce the same value. Slice fields are compared as + multisets (`[80, 81]` equals `[81, 80]`) because ALB treats things like + SecurityGroup ingress rules as unordered. +- **changed** — both sides have the field, values differ. +- **added** — only the gateway side has the field. +- **removed** — only the ingress side has the field. + +The `Hide known changes` toggle filters out the following known-artifact cases: + +- **`migrated-from` tag added to any resource** — the migration tool stamps + `spec.tags.gateway.k8s.aws/migrated-from` on every generated resource, so an + `added` entry for this tag is expected on the gateway side. +- **Controller-generated name change on ALB-family resources** — `spec.name` on + `LoadBalancer` and `TargetGroup`, `spec.groupName` on `SecurityGroup`, and + `spec.template.metadata.name` on `TargetGroupBinding`. Marked expected only + when both sides match the controller-generated format (`k8s-<...>-<10 hex>`, + two or three dash sections before the suffix). A custom name set via + annotation on either side still surfaces as a real changed diff. +- **Health-check default drift on TargetGroup** — + `spec.healthCheckConfig.healthyThresholdCount`, + `spec.healthCheckConfig.unhealthyThresholdCount`, and + `spec.healthCheckConfig.matcher.httpCode`. The ingress controller defaults + to 2 / 2 / 200; the gateway controller defaults to 3 / 3 / 200–399. +- **`spec.actions` array change on ListenerRule** — when the entire `spec.actions` + field differs only because of different `targetGroupARN.$ref` strings (naming + artifact) and an added `weight` field (gateway controller always emits + `weight: 1` on every forward target group; the ingress controller omits it), + the diff is classified as known. If any other part of the actions structure + differs (action type, status code, etc.), it surfaces as a real change. +- **`targetGroupARN.$ref` string change** — on ListenerRule + (`forwardConfig.targetGroups[...].targetGroupARN.$ref`) and on + TargetGroupBinding (`spec.template.spec.targetGroupARN.$ref`). These `$ref` + values are JSON pointers into another resource's raw stack ID; the stack + IDs differ per controller even when they point at the same backing service, + so the string always differs. Real target-group differences surface on the + correlated `TargetGroup` row. + +Everything not matching these rules is shown as-is, so genuine user-visible +changes are never silently hidden. + +### Exporting results + +Click **Export HTML** or **Export JSON** in the toolbar to share the diff: + +- **HTML** — a self-contained report with embedded styles and a full table of + all diff entries. Open in any browser with no dependencies. +- **JSON** — the raw diff payload including namespace, gateway, timestamp, and + all entries. Useful for storing in a ticket or processing programmatically. + +Both exports trigger a confirmation dialog warning that the file contains full +resource configurations viewable without cluster access. + +### Resolving the ingress source for a Gateway + +Each Gateway is paired with the Ingress that holds its dry-run plan. The +console derives the pairing from the `gateway.k8s.aws/migrated-from` tag on +the LoadBalancer resource of every generated plan: + +- `ingress//` — standalone Ingress, direct pointer. +- `ingress-group/` — the console lists Ingresses cluster-wide, + filters by `alb.ingress.kubernetes.io/group.name == `, and + returns whichever member currently carries a non-empty `dry-run-plan` + annotation. + +On a healthy group, exactly one member holds the plan. If the console finds +zero or more than one, it surfaces an error on the Gateway card. + +## RBAC + +The console needs these permissions in the context it runs under: + +- Cluster-wide `list` on `gateways.gateway.networking.k8s.io` — for the landing + page and per-namespace gateway lists. +- Cluster-wide `list` on `ingresses.networking.k8s.io` — for resolving group + plan holders, since ingress groups can span namespaces. +- `get` on `ingresses.networking.k8s.io` in any namespace that appears on the + landing page — to read the plan annotation once the holder is resolved. + +## Troubleshooting + +**"could not determine ingress plan holder: no migrated-from tag found on +LoadBalancer in gateway model"** — the Gateway's dry-run plan lacks a +`migrated-from` tag, which means it was not generated by `lbc-migrate`. +Confirm you applied the output of `lbc-migrate` and not a hand-authored +Gateway. + +**"no ingress in group `` carries a dry-run-plan annotation"** — the +ingress controller has not yet written the plan annotation for any member of +that group. Confirm the `IngressPlanAnnotation` feature gate is enabled and +the ingress controller has reconciled the group at least once. + +**"multiple ingresses in group `` carry a dry-run-plan annotation"** — +usually a stale annotation left behind after group membership changed. +Manually clear the annotation from all but one member and refresh the console. + +## Limitations + +- The console does not verify AWS resource state. It only compares the two + dry-run plans. +- Cross-namespace explicit groups require cluster-wide list permission on + Ingresses (see [RBAC](#rbac)). There is no namespace-scoped mode. diff --git a/docs/guide/gateway/lbc_migrate_reference.md b/docs/guide/gateway/lbc_migrate_reference.md new file mode 100644 index 0000000000..bd5527e0d8 --- /dev/null +++ b/docs/guide/gateway/lbc_migrate_reference.md @@ -0,0 +1,359 @@ +# Migration Tool (lbc-migrate) + +!!! warning "Under Development" + This tool is under active development. + Features may change, and not all annotation translations are implemented yet. + +`lbc-migrate` is a CLI tool that translates AWS Load Balancer Controller (LBC) Ingress resources to Gateway API equivalents. It reads Ingress, Service, IngressClass, and IngressClassParams resources from YAML/JSON files or a live Kubernetes cluster, translates annotations to Gateway API CRD fields, and writes the output manifests. + +For the end-to-end migration workflow, see **[Migrate from Ingress to Gateway API](migrate_from_ingress.md)**. + +## Installation + +Build from source (requires Go): +```bash +# From the root of the aws-load-balancer-controller repo +make lbc-migrate +``` + +The binary will be at `bin/lbc-migrate`. To install it on your PATH (creates a symlink, so future `make lbc-migrate` rebuilds are picked up automatically): +```bash +make install-lbc-migrate +``` + +Alternatively, use `go run` directly without building: +```bash +go run ./cmd/lbc-migrate/ [flags] +``` + +## Usage + +``` +lbc-migrate [flags] +``` + +### Input Modes + +The tool supports three input modes. You must provide at least one. + +#### Option 1: Individual files +```bash +lbc-migrate -f ingress1.yaml,ingress2.yaml +``` + +#### Option 2: Directory of files +```bash +lbc-migrate --input-dir ./my-manifests/ +``` + +#### Option 3: Read from a live cluster + +!!! note "Read-only access" + The `--from-cluster` mode only performs read operations (list/get). It never creates, updates, or deletes any cluster resources. We recommend using a user or service account with read-only RBAC permissions (e.g. a ClusterRole with only `get` and `list` verbs on Ingress, Service, IngressClass, and IngressClassParams resources). + +```bash +lbc-migrate --from-cluster --all-namespaces +lbc-migrate --from-cluster --namespaces production +lbc-migrate --from-cluster --namespaces production --ingress-name my-ingress +``` + +You can combine `-f` and `--input-dir`, but `--from-cluster` cannot be used with file-based input. + +### Flags + +| Flag | Required | Description | Default | +|------|----------|-------------|---------| +| `-f, --file` | One of `-f`, `--input-dir`, or `--from-cluster` is required | Comma-separated input YAML/JSON file paths (e.g. `-f file1.yaml,file2.yaml`) | | +| `--input-dir` | One of `-f`, `--input-dir`, or `--from-cluster` is required | Directory containing YAML/JSON files | | +| `--from-cluster` | One of `-f`, `--input-dir`, or `--from-cluster` is required | Read resources from a live Kubernetes cluster | | +| `--namespaces` | Optional | Comma-separated namespaces to read from (e.g. `--namespaces ns-a,ns-b`). Only valid with `--from-cluster`. Mutually exclusive with `--all-namespaces` | | +| `--all-namespaces` | Optional | Read from all namespaces. Only valid with `--from-cluster`. Mutually exclusive with `--namespaces` | | +| `--ingress-name` | Optional | Name of a specific Ingress to migrate. Requires exactly one `--namespaces` value. Only valid with `--from-cluster`. Mutually exclusive with `--all-namespaces` | | +| `--kubeconfig` | Optional | Path to kubeconfig file. Only valid with `--from-cluster` | `$KUBECONFIG` or `~/.kube/config` | +| `--output-dir` | Optional | Directory to write output manifests | `./gateway-output` | +| `--output-format` | Optional | Output format: `yaml` or `json` | `yaml` | +| `--split` | Optional | Split output layout. Empty (default) writes one combined file; `namespace` writes one file per namespace plus a `gatewayclass` file for cluster-scoped resources | `""` | +| `--dry-run` | Optional | Add `gateway.k8s.aws/dry-run` annotation to generated Gateway manifests so LBC previews the plan without creating AWS resources. Pass `--dry-run=false` to generate live Gateway manifests. | `true` | +| `--console` | Optional | Launch the migration console web UI to compare ingress and gateway dry-run models | `false` | +| `--port` | Optional | Local port for the console web server (only with `--console`) | `8080` | + +## Output Resources + +By default the tool generates a single manifest file (`gateway-resources.yaml` or `.json`) containing all translated Gateway API resources separated by `---`. To split the output into one file per namespace, see [Split output by namespace](#split-output-by-namespace) below. + +| Output Kind | API Group | Description | +|---|---|---| +| `GatewayClass` | `gateway.networking.k8s.io` | Static, always `controllerName: gateway.k8s.aws/alb`. One per run. | +| `Gateway` | `gateway.networking.k8s.io` | One per Ingress (or per `group.name` group, when supported). Listeners from `listen-ports`. | +| `HTTPRoute` | `gateway.networking.k8s.io` | One or more per Ingress. Routes from `spec.rules`. When the Ingress has a `defaultBackend` and host-based rules, a separate catch-all HTTPRoute is generated (see below). | +| `LoadBalancerConfiguration` | `gateway.k8s.aws` | LB-level settings. Only generated when LB-level annotations are present. | +| `TargetGroupConfiguration` | `gateway.k8s.aws` | Per-service TG settings. Only generated when TG-level annotations are present. | +| `ListenerRuleConfiguration` | `gateway.k8s.aws` | Auth, fixed-response, source-ip conditions. | + +Existing `Deployment` and `Service` resources are reused as-is and are not generated by the tool. Gateway API HTTPRoute `backendRefs` point directly to your existing Services by name — no changes to your application workload are needed. You keep your existing Deployment and Service manifests and replace only the Ingress manifest with the generated Gateway API resources. + +### Split output by namespace + +By default the tool writes a single `gateway-resources.` file containing every generated resource. Pass `--split=namespace` to produce one file per namespace plus a file for cluster-scoped resources: + +```bash +lbc-migrate --from-cluster --all-namespaces --output-dir ./gw/ --split=namespace +``` + +Resulting layout: + +``` +gw/ +├── gatewayclass.yaml # GatewayClass (cluster-scoped) +├── team-a/gateway-resources.yaml # Gateway resources scoped to team-a +└── team-b/gateway-resources.yaml # Gateway resources scoped to team-b +``` + +Apply everything recursively: + +```bash +kubectl apply -R -f ./gw/ +``` + +For cross-namespace IngressGroups (Ingresses in different namespaces sharing a `group.name`), the generated resources naturally split across namespace directories: the shared `Gateway` and `LoadBalancerConfiguration` land in the primary member's namespace (determined by `group.order`, then lexical `namespace/name`), while each member's `HTTPRoute` lands in its own namespace. No `ReferenceGrant` is required because HTTPRoutes only reference backends in their own namespace and the generated Gateway sets `allowedRoutes.namespaces.from: All`. + +## Default Backend Handling + +Gateway API has no `defaultBackend` equivalent ([upstream docs](https://gateway-api.sigs.k8s.io/guides/getting-started/migrating-from-ingress/#default-backend)). When an Ingress has both a `defaultBackend` and host-based rules, the tool generates a separate catch-all HTTPRoute with no hostnames or match conditions. This results in one additional ALB listener rule compared to the original Ingress, which is the expected Gateway API behavior. Additionally, the gateway controller scopes target groups per route, so the separate default-backend HTTPRoute creates its own target group even if it points to the same service as other rules. This means the migrated setup may have one extra target group compared to the Ingress setup, where a single target group is shared across all rules for the same service. + +## IngressGroup Support + +The migration tool detects `alb.ingress.kubernetes.io/group.name` and produces one shared Gateway + LoadBalancerConfiguration per group, with separate HTTPRoutes per member Ingress (preserving team ownership). + +LB-level annotations (scheme, subnets, security groups, tags, etc.) must be consistent across all Ingresses in a group — the tool errors if conflicting values are detected. TLS certificates (`certificate-arn`) are unioned across members. Tags and load-balancer-attributes are unioned with per-key conflict detection. + +Each member's `listen-ports` are unioned to build the shared Gateway's listeners. Each member's HTTPRoutes are scoped to only the listeners that member declared via `sectionName` in `parentRefs`. When `ssl-redirect` is set on any member, it applies group-wide: all members' routes attach to the HTTPS listener only, and a single redirect HTTPRoute is generated for HTTP listeners. + +Cross-namespace groups (Ingresses in different namespaces with the same `group.name`) are supported. When detected, the migration tool generates the Gateway in the primary member's namespace (determined by `group.order` annotation, then lexical `namespace/name` — matching LBC's runtime behavior) and sets `allowedRoutes.namespaces.from: All` on each listener, which permits HTTPRoutes from any namespace to attach. You can move the Gateway to a different namespace after generation if needed. + +For cross-namespace groups, use `--namespaces ns-a,ns-b` (listing all namespaces the group spans) or `--all-namespaces` to ensure all members are included. Using a single `--namespaces` that omits some members will produce incomplete output without warning. + +!!! warning "Security consideration" + `From: All` allows HTTPRoutes from any namespace to attach to the Gateway. If you need tighter scoping, you can manually change `From: All` to `From: Selector` with a label selector after generation. + +## Known Differences from Ingress + +### Rule Priority and `group.order` + +The `alb.ingress.kubernetes.io/group.order` annotation has no equivalent in Gateway API. In the Ingress model, `group.order` gives explicit control over ALB listener rule priority — rules from a lower-order Ingress always get lower priority numbers (higher precedence). In Gateway API, rule precedence is determined by the [Gateway API specification](https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io%2fv1.HTTPRouteRule): hostname specificity, path match type, path length, header/query param count, route creation timestamp, and route name (alphabetical) etc. There is no annotation or field to override this ordering. + +For most configurations — where grouped Ingresses have non-overlapping hosts or paths — this produces equivalent behavior. If your Ingresses rely on `group.order` to resolve overlapping rules (same host + same path across different Ingresses), the migrated Gateway API rules may be evaluated in a different order. Review the generated HTTPRoutes and verify with the dry-run feature before switching traffic. + +The migration tool emits a warning when `group.order` is detected. + +### ALB Listener Rule Count and Priority Order + +The migrated Gateway API manifests may produce more ALB listener rules than the original Ingress. This happens because ALB supports OR within a single condition (e.g., one `http-request-method` condition with values `[GET, HEAD]`), while Gateway API represents OR as separate `HTTPRouteMatch` entries — each becoming its own ALB rule. The routing behavior is functionally equivalent, but the rule count and priority numbers may differ. This affects conditions with multiple values for `path-pattern`, `http-request-method`, and `query-string`. + +Additionally, ALB listener rule priority order may differ between Ingress and Gateway. The Ingress controller assigns priorities based on Ingress spec rule ordering, while the gateway controller follows the [Gateway API precedence specification](https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io%2fv1.HTTPRouteRule) (path type, path length, creation timestamp etc). For most configurations this produces equivalent behavior, but users with overlapping rules that depend on specific priority ordering should verify after migration. + +### TargetGroupBinding + +AWS Load Balancer Controller uses `TargetGroupBinding` (TGB) internally to register pod targets for Ingress, Service, and Gateway resources. These controller-managed TGBs are created and deleted automatically — when you delete an Ingress, its TGBs are cleaned up, and the Gateway controller creates new ones when it reconciles. You do not need to migrate these. + +If you have user-created TGBs that register pods into externally-managed AWS Target Groups (see [TargetGroupBinding documentation](../targetgroupbinding/targetgroupbinding.md)), these are independent of Ingress and Gateway resources. They are reconciled by the TargetGroupBinding controller regardless of how traffic is routed. No changes needed during migration. + +### External Target Group References in Actions + +If your Ingress uses `actions.*` annotations that reference external target groups (via `targetGroupARN` or `targetGroupName`), the migration tool translates these to Gateway API `backendRefs` with `kind: TargetGroupName`. When the original annotation uses an ARN, the tool extracts the target group name from it. See [Specify out-of-band Target Groups](gateway.md#specify-out-of-band-target-groups) for how the gateway controller handles these references. + +Note: external target groups can only be associated with one ALB at a time. During side-by-side migration (Step 3-5), the Gateway ALB cannot attach the same external TG that the Ingress ALB is using. You must either delete the Ingress rules referencing the external TG before applying Gateway manifests (cutover), or create a duplicate TG and update the generated manifest to use the new name. + +### Limited Support for Certain Ingress Annotations +The following Ingress annotation types have limited support in the migration tool and Gateway API: + +- `host-header` conditions — values are passed through to Gateway API hostnames. Since hostnames are route-level in Gateway API (not per-rule), rules with host-header conditions are split into separate HTTPRoutes with their own hostnames. Complex wildcards (e.g., `www.*.example.com`) or regex values that don't conform to Gateway API hostname format will be rejected by the K8s API server when the manifest is applied. +- `url-rewrite` and `host-header-rewrite` transforms with regex capture group references (e.g., `replace: "/$1"` or `replace: "$1.example.org"`) — Gateway API's `URLRewrite` filter only supports static replacements (`ReplaceFullPath` for paths, `PreciseHostname` for hostnames). Transforms with static replacements (no `$N` references) are translated; transforms with capture group references are skipped. Additionally, the original Ingress transform regex is discarded during migration — the gateway controller generates its own fixed regex from the Gateway API filter type (`^([^?]*)` for path rewrites, `.*` for hostname rewrites). This means the ALB transform will always match all paths or all hostnames for the rule, regardless of what the original Ingress regex was. This is functionally equivalent because the ALB rule's conditions (from the HTTPRoute match) already narrow the traffic before the transform is applied. Verify the generated output if your Ingress uses transforms. + +## Annotation Priority Chain + +LBC resolves annotations with a priority chain (highest wins): + +1. **IngressClassParams** (cluster-level policy) +2. **Service annotations** (per-backend override) +3. **Ingress annotations** (per-Ingress config) +4. **LBC controller defaults** + +The migration tool applies this same priority chain when translating. For the most accurate results, use `--from-cluster` which automatically fetches all referenced resources (Services, IngressClasses, IngressClassParams). + +## Examples + +### Basic: single Ingress file +```bash +lbc-migrate -f ingress.yaml --output-dir ./gateway-output/ +``` + +### Directory of manifests +```bash +lbc-migrate --input-dir ./k8s-manifests/ --output-dir ./gateway-output/ +``` + +### From a live cluster (all namespaces) +```bash +lbc-migrate --from-cluster --all-namespaces --output-dir ./gateway-output/ +``` + +### From a live cluster (specific namespace) +```bash +lbc-migrate --from-cluster --namespaces production --output-dir ./gateway-output/ +``` + +### From a live cluster (multiple namespaces) +```bash +lbc-migrate --from-cluster --namespaces team-a,team-b --output-dir ./gateway-output/ +``` + +### From a live cluster (single Ingress) +```bash +lbc-migrate --from-cluster --namespaces production --ingress-name my-api --output-dir ./gateway-output/ +``` + +## Missing Resource Warnings + +When using file-based input (`-f` or `--input-dir`), the tool checks for missing referenced resources and emits warnings to stderr. For example, if an Ingress references a Service that was not provided in the input files: + +``` +WARNING: Ingress "default/my-ingress" references Service "api-service" + but it was not provided. Service-level annotation overrides may be missing. + +WARNING: Ingress "default/my-ingress" uses IngressClass "alb" + but it was not provided. IngressClassParams overrides may be missing. + +Tip: Use --from-cluster to automatically read all referenced resources, + or include Service/IngressClass/IngressClassParams files in your input. +``` + +These warnings are informational — the tool still generates output. For the most accurate results, use `--from-cluster` which automatically fetches all referenced resources. + +## Dry-Run Mode + +### How it works + +When a Gateway is annotated with `gateway.k8s.aws/dry-run: "true"`, LBC builds its built model stack and writes the serialized plan back to the Gateway as an annotation — **without creating any AWS resources**. + +`lbc-migrate` adds this annotation by default, so the first apply always previews the plan: + +```bash +lbc-migrate -f ingress.yaml --output-dir ./gw/ +``` + +To generate manifests for live deployment: + +```bash +lbc-migrate -f ingress.yaml --output-dir ./gw/ --dry-run=false +``` + +### Manual dry-run annotation + +You can add the annotation manually to any Gateway manifest: + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: my-gateway + namespace: default + annotations: + gateway.k8s.aws/dry-run: "true" +spec: + gatewayClassName: aws-alb + listeners: + - name: https + port: 443 + protocol: HTTPS + tls: + certificateRefs: + - name: tls-secret +``` + +### Viewing the plan + +```bash +kubectl get gateway my-gateway \ + -o jsonpath='{.metadata.annotations.gateway\.k8s\.aws/dry-run-plan}' | jq . +``` + +### Deploying after review + +Remove the dry-run annotation to let LBC create the actual AWS resources: + +```bash +kubectl annotate -n gateway gateway.k8s.aws/dry-run- +``` + +This triggers normal reconciliation: LBC creates the ALB, listeners, target groups, and attaches routes. + +### What dry-run does NOT do + +Dry-run is only effective on Gateways that have not yet been deployed. If the Gateway already has AWS resources (indicated by the presence of a controller finalizer), the `gateway.k8s.aws/dry-run` annotation is ignored and normal reconciliation proceeds. + +Dry-run intentionally skips every action that would touch AWS or cluster state beyond the Gateway annotation/status it owns: + +- No AWS resources are created, updated, or deleted. +- No finalizer is added to the Gateway. +- No backend security group is allocated or released. +- No Kubernetes Secrets are monitored for certificate rotation. +- No add-on state (WAF, Shield) is persisted. +- No `serviceReferenceCounter` relations are updated. + +### Annotation reference + +| Annotation | Set by | Description | +| --------------------------------------- | ---------- | --------------------------------------------------------------------------- | +| `gateway.k8s.aws/dry-run` | User | Set to `"true"` to enable dry-run mode on a Gateway. | +| `gateway.k8s.aws/dry-run-plan` | Controller | Serialized stack JSON written by LBC when dry-run is enabled. Do not edit. | + +## Annotation Support + +The tool translates the following Ingress annotations to Gateway API equivalents. Annotations not listed here are not yet supported. + +| Ingress Annotation | Gateway API Equivalent | Status | +|---|---|---| +| `alb.ingress.kubernetes.io/scheme` | `LoadBalancerConfiguration.spec.scheme` | Supported | +| `alb.ingress.kubernetes.io/ip-address-type` | `LoadBalancerConfiguration.spec.ipAddressType` | Supported | +| `alb.ingress.kubernetes.io/subnets` | `LoadBalancerConfiguration.spec.subnetMappings` | Supported | +| `alb.ingress.kubernetes.io/security-groups` | `LoadBalancerConfiguration.spec.securityGroups` | Supported | +| `alb.ingress.kubernetes.io/certificate-arn` | `Gateway.spec.listeners[].tls.certificateRefs` | Supported | +| `alb.ingress.kubernetes.io/listen-ports` | `Gateway.spec.listeners[]` | Supported | +| `alb.ingress.kubernetes.io/ssl-policy` | `LoadBalancerConfiguration.spec.listenerConfigs[].sslPolicy` | Supported | +| `alb.ingress.kubernetes.io/ssl-redirect` | Redirect HTTPRoute | Supported | +| `alb.ingress.kubernetes.io/group.name` | Shared Gateway per group | Supported | +| `alb.ingress.kubernetes.io/group.order` | Used to determine primary group member (Gateway placement) and sort order. No Gateway API equivalent for ALB rule priority — a warning is emitted. See [Rule Priority and group.order](#rule-priority-and-grouporder). | No equivalent (priority) | +| `alb.ingress.kubernetes.io/target-type` | `TargetGroupConfiguration.spec.targetType` | Supported | +| `alb.ingress.kubernetes.io/backend-protocol` | `TargetGroupConfiguration.spec.protocolVersion` | Supported | +| `alb.ingress.kubernetes.io/healthcheck-*` | `TargetGroupConfiguration.spec.healthCheck.*` | Supported | +| `alb.ingress.kubernetes.io/tags` | `LoadBalancerConfiguration.spec.tags` | Supported | +| `alb.ingress.kubernetes.io/load-balancer-attributes` | `LoadBalancerConfiguration.spec.loadBalancerAttributes` | Supported | +| `alb.ingress.kubernetes.io/target-group-attributes` | `TargetGroupConfiguration.spec.targetGroupAttributes` | Supported | +| `alb.ingress.kubernetes.io/actions.*` | HTTPRoute filters / `ListenerRuleConfiguration` | Supported | +| `alb.ingress.kubernetes.io/conditions.*` | HTTPRoute matches / `ListenerRuleConfiguration` | Supported | +| `alb.ingress.kubernetes.io/auth-type` | `ListenerRuleConfiguration.spec.authConfig` | Supported | +| `alb.ingress.kubernetes.io/auth-idp-*` | `ListenerRuleConfiguration.spec.authConfig` | Supported | +| `alb.ingress.kubernetes.io/wafv2-acl-arn` | `LoadBalancerConfiguration.spec.wafv2ACLArn` | Supported | +| `alb.ingress.kubernetes.io/wafv2-acl-name` | `LoadBalancerConfiguration.spec.wafv2ACLName` | Supported | +| `alb.ingress.kubernetes.io/shield-advanced-protection` | `LoadBalancerConfiguration.spec.shieldAdvancedProtection` | Supported | +| `alb.ingress.kubernetes.io/mutual-authentication` | `LoadBalancerConfiguration.spec.mutualAuthentication` | Supported | +| `alb.ingress.kubernetes.io/load-balancer-name` | `LoadBalancerConfiguration.spec.name` | Supported | +| `alb.ingress.kubernetes.io/customer-owned-ipv4-pool` | `LoadBalancerConfiguration.spec.customerOwnedIPv4Pool` | Supported | +| `alb.ingress.kubernetes.io/ipam-ipv4-pool-id` | `LoadBalancerConfiguration.spec.ipamIPv4PoolId` | Supported | +| `alb.ingress.kubernetes.io/manage-security-group-rules` | `LoadBalancerConfiguration.spec.manageSecurityGroupRules` | Supported | +| `alb.ingress.kubernetes.io/inbound-cidrs` | `LoadBalancerConfiguration.spec.inboundCIDRs` | Supported | +| `alb.ingress.kubernetes.io/security-group-prefix-lists` | `LoadBalancerConfiguration.spec.securityGroupPrefixLists` | Supported | +| `alb.ingress.kubernetes.io/load-balancer-capacity-reservation` | `LoadBalancerConfiguration.spec.capacityReservation` | Supported | +| `alb.ingress.kubernetes.io/backend-protocol-version` | `TargetGroupConfiguration.spec.protocolVersion` | Supported | +| `alb.ingress.kubernetes.io/target-node-labels` | `TargetGroupConfiguration.spec.targetNodeLabels` | Supported | +| `alb.ingress.kubernetes.io/target-control-port` | `TargetGroupConfiguration.spec.targetControlPort` | Supported | +| `alb.ingress.kubernetes.io/use-regex-path-match` | HTTPRoute path match type (`RegularExpression`) | Supported | +| `alb.ingress.kubernetes.io/auth-scope` | `ListenerRuleConfiguration.spec.authConfig.scope` | Supported | +| `alb.ingress.kubernetes.io/auth-session-cookie` | `ListenerRuleConfiguration.spec.authConfig.sessionCookieName` | Supported | +| `alb.ingress.kubernetes.io/auth-session-timeout` | `ListenerRuleConfiguration.spec.authConfig.sessionTimeout` | Supported | +| `alb.ingress.kubernetes.io/auth-on-unauthenticated-request` | `ListenerRuleConfiguration.spec.authConfig.onUnauthenticatedRequest` | Supported | +| `alb.ingress.kubernetes.io/jwt-validation` | `ListenerRuleConfiguration.spec.authConfig.jwtValidation` | Supported | +| `alb.ingress.kubernetes.io/waf-acl-id` | WAF Classic — deprecated. Use WAFv2 (`wafv2-acl-arn`) instead. | Not supported | +| `alb.ingress.kubernetes.io/web-acl-id` | Deprecated alias of `waf-acl-id` | Not supported | +| `alb.ingress.kubernetes.io/frontend-nlb-*` | All Frontend NLB annotations (`enable-frontend-nlb`, `frontend-nlb-scheme`, `frontend-nlb-subnets`, etc.) are not yet supported in the migration tool. | Not supported | diff --git a/docs/guide/gateway/migrate_from_ingress.md b/docs/guide/gateway/migrate_from_ingress.md index 6c08cefcfe..d0277a7269 100644 --- a/docs/guide/gateway/migrate_from_ingress.md +++ b/docs/guide/gateway/migrate_from_ingress.md @@ -1,29 +1,23 @@ -# Migrate from Ingress +# Migrate from Ingress to Gateway API !!! warning "Under Development" This tool is under active development. Features may change, and not all annotation translations are implemented yet. -`lbc-migrate` is a CLI tool that helps migrate AWS Load Balancer Controller (LBC) Ingress resources to Gateway API equivalents. It reads Ingress, Service, IngressClass, and IngressClassParams resources from YAML/JSON files or a live Kubernetes cluster, translates annotations to Gateway API CRD fields, and writes the output manifests. +This guide covers migrating AWS Load Balancer Controller (LBC) Ingress resources to Gateway API, step by step. The migration is designed to be safe and non-disruptive — new ALBs are created alongside existing ones, so current workloads stay running throughout the process. -## Prerequisites +Two tools are provided to help: -- The Gateway API CRDs must be installed in your cluster at a compatible version before applying the generated manifests. See the [Gateway API installation guide](https://gateway-api.sigs.k8s.io/guides/getting-started/#installing-gateway-api) and the [AWS LBC Gateway API documentation](gateway.md) for supported versions. -- The tool assumes input Ingress resources are valid and currently working with the AWS Load Balancer Controller. It does not re-validate Ingress annotations or enforce the same constraints as the ingress controller (e.g., ALB rule limits, mutually exclusive annotation combinations). If an Ingress has invalid or conflicting annotations, the output may be incorrect or incomplete without warning. +- **[`lbc-migrate` CLI](lbc_migrate_reference.md)** — translates Ingress manifests (annotations, rules, groups) into equivalent Gateway API YAML. +- **[Migration Console](in_cluster_console.md)** — a local web UI that compares the AWS resource plans produced by both the ingress and gateway controllers field by field, to verify equivalence before applying Gateway manifests for real. -## Installation +## Overview -Build from source (requires Go): -```bash -# From the root of the aws-load-balancer-controller repo -make lbc-migrate -``` +The migration follows six steps. Each step is safe to pause at — you can stop and resume at any point. -The binary will be at `bin/lbc-migrate`. To install it on your PATH (creates a symlink, so future `make lbc-migrate` rebuilds are picked up automatically): -```bash -make install-lbc-migrate -``` +![Migration flow overview](assets/migration/migration_flow.png) +<<<<<<< Updated upstream Alternatively, use `go run` directly without building: ```bash go run ./cmd/lbc-migrate/ [flags] @@ -171,6 +165,16 @@ The following Ingress annotation types have limited support in the migration too - `host-header` conditions — values are passed through to Gateway API hostnames. Since hostnames are route-level in Gateway API (not per-rule), rules with host-header conditions are split into separate HTTPRoutes with their own hostnames. Complex wildcards (e.g., `www.*.example.com`) or regex values that don't conform to Gateway API hostname format will be rejected by the K8s API server when the manifest is applied. - `url-rewrite` and `host-header-rewrite` transforms with regex capture group references (e.g., `replace: "/$1"` or `replace: "$1.example.org"`) — Gateway API's `URLRewrite` filter only supports static replacements (`ReplaceFullPath` for paths, `PreciseHostname` for hostnames). Transforms with static replacements (no `$N` references) are translated; transforms with capture group references are skipped. Additionally, the original Ingress transform regex is discarded during migration — the gateway controller generates its own fixed regex from the Gateway API filter type (`^([^?]*)` for path rewrites, `.*` for hostname rewrites). This means the ALB transform will always match all paths or all hostnames for the rule, regardless of what the original Ingress regex was. This is functionally equivalent because the ALB rule's conditions (from the HTTPRoute match) already narrow the traffic before the transform is applied. Verify the generated output if your Ingress uses transforms. +======= +| Step | Action | What happens | Rollback | +|------|--------|--------------|----------| +| 1 | Translate manifests | `lbc-migrate` converts Ingress YAML to Gateway API YAML | Delete generated files | +| 2 | Preview with dry-run | Ingress controller writes its plan to annotation (requires feature gate). Gateway manifests applied with dry-run — gateway controller writes its plan without creating AWS resources. Console compares both plans. | Delete the dry-run Gateway | +| 3 | Apply Gateway manifests | LBC creates new ALBs for the Gateway resources alongside existing Ingress ALBs | Delete Gateway resources | +| 4 | Verify Gateway ALBs | Confirm new ALBs are healthy, configurations are correct, and metrics look normal | — | +| 5 | Shift traffic | Gradually move traffic from Ingress ALBs to Gateway ALBs | Shift traffic back | +| 6 | Cleanup | Delete Ingress and old resources | — | +>>>>>>> Stashed changes !!! important "What changes and what stays the same" The generated output **replaces only your Ingress resource**. Everything else stays untouched: @@ -179,18 +183,20 @@ The following Ingress annotation types have limited support in the migration too - **Deployment** — unchanged, continues running your application pods. - **Service** — unchanged, the new HTTPRoute `backendRefs` reference it by name. - Once equivalent gateway manifest is generated from ingress manifests, apply the generated Gateway API manifest alongside your existing workload. +![Service reuse during migration](assets/migration/service_reuse.png) +## Prerequisites -### Annotation Priority Chain +- AWS Load Balancer Controller installed in the cluster with Gateway API support enabled +- The [Gateway API CRDs](https://gateway-api.sigs.k8s.io/guides/getting-started/#installing-gateway-api) installed at a compatible version (see [Gateway API documentation](gateway.md)) +- `lbc-migrate` binary built (see [Installation](lbc_migrate_reference.md#installation)) +- The tool assumes input Ingress resources are valid and currently working with the AWS Load Balancer Controller. It does not re-validate Ingress annotations. -LBC resolves annotations with a priority chain (highest wins): +--- -1. **IngressClassParams** (cluster-level policy) -2. **Service annotations** (per-backend override) -3. **Ingress annotations** (per-Ingress config) -4. **LBC controller defaults** +## Step 1: Translate Manifests +<<<<<<< Updated upstream The migration tool applies this same priority chain when translating. For the most accurate results, use `--from-cluster` which automatically fetches all referenced resources (Services, IngressClasses, IngressClassParams). ## Examples @@ -275,149 +281,412 @@ lbc-migrate --from-cluster --namespaces production --output-dir ./gw/ --dry-run Alternatively, add the `gateway.k8s.aws/dry-run: "true"` annotation manually to any Gateway manifest: +======= +First, install the `lbc-migrate` CLI if you haven't already: -```yaml -apiVersion: gateway.networking.k8s.io/v1 -kind: Gateway -metadata: - name: my-gateway - namespace: default - annotations: - gateway.k8s.aws/dry-run: "true" -spec: - gatewayClassName: aws-alb - listeners: - - name: https - port: 443 - protocol: HTTPS - tls: - certificateRefs: - - name: tls-secret +```bash +# Build from the aws-load-balancer-controller repo +make lbc-migrate + +# (Optional) Install on your PATH +make install-lbc-migrate ``` -Apply it: +See [Installation](lbc_migrate_reference.md#installation) for details. + +Then run `lbc-migrate` to convert your Ingress resources to Gateway API equivalents. + +### Quick start ```bash -kubectl apply -f my-gateway.yaml +# From YAML files +lbc-migrate -f ingress.yaml --output-dir ./gw/ + +# From a directory of manifests +lbc-migrate --input-dir ./k8s-manifests/ --output-dir ./gw/ + +# From a live cluster (recommended — automatically fetches referenced Services, IngressClass, etc.) +lbc-migrate --from-cluster --namespaces production --output-dir ./gw/ +``` + +The tool generates Gateway, HTTPRoute, GatewayClass, and optional CRD resources (LoadBalancerConfiguration, TargetGroupConfiguration, ListenerRuleConfiguration). + +!!! tip "Use `--from-cluster` for best results" + File-based input may miss Service-level annotations and IngressClassParams overrides. `--from-cluster` automatically fetches all referenced resources for the most accurate translation. + +**Output:** A directory of Gateway API YAML files ready to apply. By default, `--dry-run=true` is set, so the generated Gateway manifests include the `gateway.k8s.aws/dry-run: "true"` annotation. + +For the full CLI reference (all flags, output formats, split modes, IngressGroup handling, annotation support table), see **[Migration Tool (lbc-migrate)](lbc_migrate_reference.md)**. + +--- + +## Step 2: Preview with Dry-Run + +Before creating any AWS resources, verify that the generated Gateway manifests will produce the same ALB configuration as your current Ingress. + +![Dry-run verification workflow](assets/migration/dryrun_workflow.png) + +### 2a. Enable the Ingress plan annotation + +On the AWS Load Balancer Controller, enable the feature gate: + ``` +--feature-gates=IngressPlanAnnotation=true +``` + +Once enabled, the ingress controller writes its built model stack to each reconciled Ingress as `alb.ingress.kubernetes.io/dry-run-plan`. -### Viewing the dry-run plan +!!! note "IngressGroup behavior" + For an IngressGroup, all members share one plan. The controller writes the annotation only to the primary member (lowest `group.order`, tie-broken by lexical `namespace/name`). -Check that the dry-run plan annotation is populated: +### 2b. Apply the generated Gateway manifests + +```bash +kubectl apply -f ./gw/gateway-resources.yaml +``` + +Because the Gateways carry `gateway.k8s.aws/dry-run: "true"`, the gateway controller builds its model but does **not** create an ALB. It writes the plan back to the Gateway as `gateway.k8s.aws/dry-run-plan`. **No AWS resources are created.** +>>>>>>> Stashed changes + +### 2c. Launch the migration console + +```bash +lbc-migrate --console +# or bind to a different port +lbc-migrate --console --port 9000 +``` + +Open `http://localhost:8080` in your browser. The console reads both plan annotations and shows a field-by-field comparison of every AWS resource (LoadBalancer, Listeners, ListenerRules, TargetGroups, SecurityGroups) that each controller would create. + +The console is read-only and uses your current kubeconfig context — it never modifies any cluster or AWS resources. + +Look for: + +- **"Changed" diffs** — fields that differ between ingress and gateway plans. Known migration artifacts (naming changes, health-check defaults) are filtered by default. +- **"Added" / "Removed"** — resources or fields present on only one side. + +!!! success "When to proceed" + Review the diffs and confirm they are expected. Some differences are intentional (e.g., you deliberately changed a health-check path). As long as you understand and accept all listed changes, you can proceed to Step 3. + +For the full console walkthrough (UI guide, diff classification, export, RBAC, troubleshooting), see **[Migration Console](in_cluster_console.md)**. + +### 2d. (Optional) Inspect the raw plan + +You can also inspect the plan annotation directly: ```bash kubectl get gateway my-gateway \ -o jsonpath='{.metadata.annotations.gateway\.k8s\.aws/dry-run-plan}' | jq . ``` -Example output: - -```json -{ - "id": "default/my-gateway", - "resources": { - "AWS::ElasticLoadBalancingV2::LoadBalancer": { - "LoadBalancer": { - "spec": { - "name": "k8s-default-mygatew-abc123", - "type": "application", - "scheme": "internet-facing", - "ipAddressType": "ipv4", - "subnetMapping": [ - {"subnetID": "subnet-aaa"}, - {"subnetID": "subnet-bbb"} - ], - "securityGroups": ["sg-xxx"] - } - } - }, - "AWS::ElasticLoadBalancingV2::Listener": { - "443": { - "spec": { - "port": 443, - "protocol": "HTTPS", - "sslPolicy": "ELBSecurityPolicy-TLS-1-2-2017-01", - "certificates": [ - {"certificateARN": "arn:aws:acm:us-west-2:123:cert/abc"} - ] +--- + +## Step 3: Apply Gateway Manifests + +Once you've confirmed the dry-run plan matches, regenerate the manifests without dry-run and apply: + +```bash +# Regenerate without dry-run +lbc-migrate --from-cluster --namespaces production --output-dir ./gw/ --dry-run=false + +# Apply +kubectl apply -f ./gw/gateway-resources.yaml +``` + +Alternatively, remove the dry-run annotation from the existing Gateway: + +```bash +kubectl annotate -n gateway gateway.k8s.aws/dry-run- +``` + +**What happens:** + +- LBC creates **new ALBs** for the Gateway resources (one per Gateway object) +- LBC creates new Target Groups pointing to the **same** Services/Pods +- Both your existing Ingress ALBs and the new Gateway ALBs now route to the same backend Pods +- Existing Ingress ALBs continue working as before — no disruption + +!!! note "Service reuse" + Gateway API HTTPRoute `backendRefs` point directly to your existing Kubernetes Services. No new Services, Deployments, or Pods are created. Both sets of ALBs register the same pod IPs. + +--- + +## Step 4: Verify Gateway ALBs + +Before shifting traffic, confirm the new Gateway ALBs are healthy and correctly configured. + +### Check Gateway status + +```bash +kubectl get gateway -n +``` + +Look for: + +- `Programmed: True` condition — the ALB is fully provisioned +- `status.addresses` — contains the new ALB DNS name + +### Verify target group health + +```bash +# Get the ALB ARN from Gateway status or AWS console +aws elbv2 describe-target-health --target-group-arn +``` + +All targets should show `healthy`. + +### Verify ALB configuration + +In the AWS Console or via CLI, confirm: + +- Listeners match expected ports and protocols +- Security groups are correct +- SSL certificates are attached +- WAF/Shield settings are in place (if applicable) +- Tags are correct + +### Monitor metrics + +Check CloudWatch metrics for the new ALBs: + +- `HealthyHostCount` — all targets registered and healthy +- `HTTPCode_ELB_5XX_Count` — no unexpected 5xx errors +- `TargetResponseTime` — latency within expected range + +### Test directly + +```bash +# Get Gateway ALB DNS +GATEWAY_ALB=$(kubectl get gateway -n \ + -o jsonpath='{.status.addresses[0].value}') + +# Test a request +curl -H "Host: your-app.example.com" http://$GATEWAY_ALB/health +``` + +!!! warning "Do not skip verification" + Confirm the new ALBs serve correct responses and all configurations look right before shifting any production traffic. + +--- + +## Step 5: Shift Traffic + +Choose a traffic migration strategy based on your requirements. For each Ingress being migrated, you'll shift traffic from the Ingress ALB to the corresponding Gateway ALB. + +![Traffic migration strategies](assets/migration/traffic_strategies.png) + +| Strategy | Gradual | Requires Domain | Extra Cost | Risk Level | +|----------|---------|-----------------|------------|------------| +| **A: Sudden Cutover** | No | No | None | HIGH (dev/test only) | +| **B: Route 53 Weighted** | Yes | Yes (Route 53) | None | LOW | +| **C: Global Accelerator** | Yes | No | **Yes** ([AGA pricing](https://aws.amazon.com/global-accelerator/pricing/)) | LOW | + +### Option A: Sudden Cutover + +**Best for:** Dev/test environments where downtime is acceptable. + +If you have a custom domain, update DNS to point to the new ALB: + +```bash +# Get the Gateway ALB DNS +GATEWAY_ALB=$(kubectl get gateway -n \ + -o jsonpath='{.status.addresses[0].value}') + +# Update Route 53 (or your DNS provider) +aws route53 change-resource-record-sets --hosted-zone-id --change-batch '{ + "Changes": [{ + "Action": "UPSERT", + "ResourceRecordSet": { + "Name": "api.example.com", + "Type": "CNAME", + "TTL": 60, + "ResourceRecords": [{"Value": "'$GATEWAY_ALB'"}] + } + }] +}' +``` + +If clients access the ALB DNS directly (no custom domain), update all clients to use the new Gateway ALB DNS name. + +### Option B: Route 53 Weighted Routing + +**Best for:** Production workloads with a custom domain in Route 53. + +Gradually shift traffic using weighted DNS records: + +```bash +INGRESS_ALB="k8s-default-myingress-abc123.us-west-2.elb.amazonaws.com" +GATEWAY_ALB=$(kubectl get gateway -n \ + -o jsonpath='{.status.addresses[0].value}') + +# Start: 90% Ingress, 10% Gateway +aws route53 change-resource-record-sets --hosted-zone-id --change-batch '{ + "Changes": [ + { + "Action": "UPSERT", + "ResourceRecordSet": { + "Name": "api.example.com", + "Type": "A", + "SetIdentifier": "ingress-legacy", + "Weight": 90, + "AliasTarget": { + "HostedZoneId": "", + "DNSName": "'$INGRESS_ALB'", + "EvaluateTargetHealth": true } } }, - "AWS::ElasticLoadBalancingV2::TargetGroup": { - "default-api-service-8080": { - "spec": { - "name": "k8s-default-apisvc-xyz789", - "targetType": "ip", - "port": 8080, - "protocol": "HTTP", - "healthCheckConfig": { - "path": "/health", - "intervalSeconds": 15 - } + { + "Action": "UPSERT", + "ResourceRecordSet": { + "Name": "api.example.com", + "Type": "A", + "SetIdentifier": "gateway-new", + "Weight": 10, + "AliasTarget": { + "HostedZoneId": "", + "DNSName": "'$GATEWAY_ALB'", + "EvaluateTargetHealth": true } } } - } -} + ] +}' +``` + +Gradually increase the Gateway weight: 10% → 25% → 50% → 100%. Monitor error rates and latency at each step. + +!!! note "DNS TTL" + Route 53 weighted routing splits traffic at the DNS query level, not per-request. Set a low TTL (e.g., 60s) so changes take effect quickly. Rollback speed is limited by TTL propagation. + +### Option C: Global Accelerator + +**Best for:** Production workloads without a custom domain, or when you need per-request traffic splitting. + +!!! warning "Cost" + Global Accelerator incurs additional AWS charges (per-accelerator hourly fee + data transfer premium). Review [AWS Global Accelerator pricing](https://aws.amazon.com/global-accelerator/pricing/) before choosing this option. The accelerator can be deleted after migration is complete. + +This uses the LBC Global Accelerator CRD for precise traffic control: + +```yaml +apiVersion: aga.k8s.aws/v1beta1 +kind: GlobalAccelerator +metadata: + name: migration-aga + namespace: +spec: + name: "ingress-gateway-migration" + ipAddressType: IPV4 + listeners: + - protocol: TCP + portRanges: + - fromPort: 80 + toPort: 80 + - fromPort: 443 + toPort: 443 + endpointGroups: + - endpoints: + - type: Ingress + name: + namespace: + weight: 100 + - type: Gateway + name: + namespace: + weight: 0 ``` -Confirm no AWS resources were created: +**Migration steps:** + +1. Apply the GlobalAccelerator CRD (100% to Ingress, 0% to Gateway) +2. Update DNS to point to the Global Accelerator endpoint +3. Gradually shift weights: `100/0` → `90/10` → `50/50` → `0/100` +4. Monitor at each step + +See the [Global Accelerator documentation](../globalaccelerator/aga-controller.md) for installation and prerequisites. + +--- + +## Step 6: Cleanup + +After 100% traffic is on the Gateway ALBs and has been stable for your desired monitoring period: + +### Delete the Ingress ```bash -aws elbv2 describe-load-balancers --names k8s-default-mygatew-abc123 -# → "LoadBalancer not found" (expected: dry-run did not deploy) +kubectl delete ingress -n ``` -### Troubleshooting +This triggers LBC to delete the Ingress ALB and its associated Target Groups. -If the `gateway.k8s.aws/dry-run-plan` annotation is empty after applying the Gateway, the -model build failed before reaching the dry-run step. Debug with: +### Remove the Global Accelerator (if used) -1. Check Gateway events for build errors +```bash +kubectl delete globalaccelerator migration-aga -n +``` -2. Check Gateway status conditions for validation errors +### Disable the feature gate -3. Check controller logs for detailed errors +Remove `IngressPlanAnnotation=true` from the controller's feature gates if no other migrations are in progress. -### Deploying after review +### Clean up DNS (if using Option B) -When the plan looks correct, remove the dry-run annotation to let LBC create the actual AWS -resources. The trailing `-` on the annotation key is `kubectl annotate` syntax for removing -an annotation: +Remove the old weighted record set: ```bash -# The trailing `-` tells kubectl to remove (not set) the annotation. -kubectl annotate -n gateway gateway.k8s.aws/dry-run- +aws route53 change-resource-record-sets --hosted-zone-id --change-batch '{ + "Changes": [{ + "Action": "DELETE", + "ResourceRecordSet": { + "Name": "api.example.com", + "Type": "A", + "SetIdentifier": "ingress-legacy", + "Weight": 90, + "AliasTarget": { + "HostedZoneId": "", + "DNSName": "'$INGRESS_ALB'", + "EvaluateTargetHealth": true + } + } + }] +}' ``` -This triggers a normal reconciliation: +Update the remaining Gateway record to a standard (non-weighted) record. + +--- + +## Troubleshooting + +### Dry-run plan annotation is empty + +The model build failed before reaching the dry-run step: + +1. Check Gateway events: `kubectl describe gateway ` +2. Check Gateway status conditions for validation errors +3. Check controller logs for detailed build errors + +### Gateway stuck in "not Programmed" + +After removing dry-run, if the Gateway doesn't reach `Programmed: True`: -- The `gateway.k8s.aws/dry-run-plan` annotation is cleared. -- LBC creates the ALB, listeners, target groups, and attaches routes. -- Once the ALB is active, the `Programmed` condition is set to `True` and the ALB DNS appears - in `status.addresses`. +1. Check events for AWS API errors (permissions, quotas) +2. Verify subnet tags and security group configurations +3. Check controller logs: `kubectl logs -n kube-system deployment/aws-load-balancer-controller` -### What dry-run does NOT do +### Console shows unexpected diffs -Dry-run is only effective on Gateways that have not yet been deployed. If the Gateway already -has AWS resources (indicated by the presence of a controller finalizer), the -`gateway.k8s.aws/dry-run` annotation is ignored and normal reconciliation proceeds. -This prevents accidentally freezing a live or provisioning ALB by adding the dry-run annotation -after deployment. +If the migration console shows real (non-artifact) differences: -Dry-run intentionally skips every action that would touch AWS or cluster state beyond the -Gateway annotation/status it owns: +1. Review which annotations on your Ingress may not be fully supported yet (see [annotation support](lbc_migrate_reference.md#annotation-support)) +2. Check for IngressClassParams overrides that may need manual translation +3. File an issue if you believe the translation is incorrect -- No AWS resources are created, updated, or deleted. -- No finalizer is added to the Gateway. -- No backend security group is allocated or released. -- No Kubernetes Secrets are monitored for certificate rotation. -- No add-on state (WAF, Shield) is persisted. -- No `serviceReferenceCounter` relations are updated. +--- -### Annotation reference +## Further Reading -| Annotation | Set by | Description | -| --------------------------------------- | ---------- | --------------------------------------------------------------------------- | -| `gateway.k8s.aws/dry-run` | User | Set to `"true"` to enable dry-run mode on a Gateway. | -| `gateway.k8s.aws/dry-run-plan` | Controller | Serialized stack JSON written by LBC when dry-run is enabled. Do not edit. | +- **[Migration Tool (lbc-migrate)](lbc_migrate_reference.md)** — full flag reference, annotation support table, output format details +- **[Migration Console](in_cluster_console.md)** — UI walkthrough, diff classification, export, RBAC +- **[Gateway API Overview](gateway.md)** — LBC's Gateway API support documentation +- **[Global Accelerator](../globalaccelerator/aga-controller.md)** — AGA CRD for traffic splitting diff --git a/mkdocs.yml b/mkdocs.yml index 10b8bd1aee..2a7fe4e40c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -42,6 +42,10 @@ nav: - Gateway Chaining: guide/gateway/gateway_chaining.md - ListenerSets: guide/gateway/listenersets.md - Specification: guide/gateway/spec.md + - Migration: + - Migration Guide: guide/gateway/migrate_from_ingress.md + - Migration Tool (lbc-migrate): guide/gateway/lbc_migrate_reference.md + - Migration Console: guide/gateway/in_cluster_console.md - AWS Global Accelerator (New): - Overview: guide/globalaccelerator/aga-controller.md - Installation: guide/globalaccelerator/installation.md