Skip to content

Set override_path = true to namespaced hosts.toml entries in hosts.toml#10765

Open
rahulbabu95 wants to merge 1 commit into
aws:mainfrom
rahulbabu95:rahulgab/fix-hosts-toml-double-v2
Open

Set override_path = true to namespaced hosts.toml entries in hosts.toml#10765
rahulbabu95 wants to merge 1 commit into
aws:mainfrom
rahulbabu95:rahulgab/fix-hosts-toml-double-v2

Conversation

@rahulbabu95
Copy link
Copy Markdown
Member

Issue #, if available:
#10212 introduced a regression while migrating the registry mirror config from config.toml (registry.mirrors endpoint format) to hosts.toml format. When oci namespaces are configured, ToAPIEndpoints() prepends /v2/ to t
he namespace path in the host URL. In the old config.toml format this was used as-is. In hosts.toml format, containerd's parser auto-appends an additional /v2/ to the host path, causing a double /v2/ in requests: /v2/< namespace>/v2/<repo>/manifests/<tag> resulting in 404.

Description of changes:
Add override_path = true to namespaced hosts.toml entries. This instructs containerd to use the host path as is without auto-appending /v2/. The base mirror entry (no path) does not have this flag set so containerd correctly adds /v2/ for direct repo references.

Testing (if applicable):
Pre-fix (setup a test ns rahul-tst in our CI registry and validate the image pull URIs from the node):
Hosts.toml:

server = "https://public.ecr.aws"

[host."https://10.80.148.51:443/v2/rahul-test"]
  capabilities = ["pull", "resolve"]
  ca = "/etc/containerd/certs.d/10.80.148.51:443/ca.crt"
  [host."https://10.80.148.51:443/v2/rahul-test".header]
    authorization = "Basic YWRtaW46eENnd3hqN0Y="

Image pull logs:
May 08 23:02:02 containerd[1404]: level=debug msg="fetch response received" response.status="404 Not Found" url="https://10.80.148.51:443/v2/rahul-test/v2/eks/cilium/cilium/manifests/v1.19.1-0?ns=public.ecr.aws"

Post-fix:
Hosts.toml:

server = "https://public.ecr.aws"

[host."https://10.80.148.51:443/v2/rahul-test"]
  capabilities = ["pull", "resolve"]
  override_path = true
  ca = "/etc/containerd/certs.d/10.80.148.51:443/ca.crt"
  [host."https://10.80.148.51:443/v2/rahul-test".header]
    authorization = "Basic YWRtaW46eENnd3hqN0Y="

response.status="200 OK" content-type=application/vnd.oci.image.manifest.v1+json url="https://10.80.148.51:443/v2/rahul-test/eks/cilium/cilium/manifests/v1.19.1-0?ns=public.ecr.aws"

Documentation added/planned (if applicable):

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@eks-distro-bot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign ahreehong for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@eks-distro-bot eks-distro-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 11, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

Codecov Report

❌ Patch coverage is 60.00000% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.31%. Comparing base (c8c965a) to head (bd4a7c1).

Files with missing lines Patch % Lines
internal/test/registrymirror.go 0.00% 8 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #10765   +/-   ##
=======================================
  Coverage   72.30%   72.31%           
=======================================
  Files         608      608           
  Lines       39388    39402   +14     
=======================================
+ Hits        28481    28493   +12     
- Misses       9173     9176    +3     
+ Partials     1734     1733    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rahulbabu95 rahulbabu95 requested a review from snarkychef May 11, 2026 19:48
… v2 migration

URLs are used as-is) to hosts.toml format (where containerd auto-appends
/v2/ to the host path). Since ToAPIEndpoints() already prepends /v2/ before
the namespace, containerd produced a broken double /v2/ in requests:
/v2/<namespace>/v2/<repo>/manifests/<tag> → 404.

Fix: Always set override_path = true in hosts.toml entries so containerd
uses the host path as-is without auto-appending /v2/. Also update
ToAPIEndpoint to always produce a /v2 path prefix (even for pathless URLs)
ensuring all entries have a well-formed OCI API root.

Signed-off-by: Rahul Ganesh <rahulgab@amazon.com>
@rahulbabu95 rahulbabu95 force-pushed the rahulgab/fix-hosts-toml-double-v2 branch from bcd55c6 to bd4a7c1 Compare May 12, 2026 23:52
@eks-distro-bot eks-distro-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants