feat: add custom port support for sandbox

.codespellrc

@@ -0,0 +1,6 @@
+[codespell]
+skip = .git,*.pb,*.pdf,*.svg,go.sum,go.mod,*requirements.txt,gen,monodocs-environment.lock.yaml,*.lock,*.lock.yaml,*.swagger.json
+check-hidden = true
+# Ignore camelCase and PascalCase identifiers (common in Go code)
+ignore-regex = \b[a-z]+[A-Z]\w*\b|\b[A-Z][a-z]+[A-Z]\w*\b
+ignore-words-list = astroid,bootup,decorder,fo,lightyear,nd,notin,ser,te

.github/dco.yml

@@ -0,0 +1,5 @@
+require:
+  members: false
+allowRemediationCommits:
+  individual: true
+  thirdParty: true

.github/workflows/codespell.yml

@@ -22,8 +22,6 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v4
+      # Configuration is in .codespellrc
       - name: Codespell
         uses: codespell-project/actions-codespell@v2
-        with:
-          skip: "*.pb,monodocs-environment.lock.yaml,.git,*.pdf,*.svg,go.sum,go.mod,*requirements.txt,gen"
-          ignore_words_file: .github/codespell-ignored-words

.github/workflows/single-binary.yml

@@ -22,8 +22,9 @@ jobs:
         with:
           go-version-file: go.mod
       - name: golangci-lint
-        uses: golangci/golangci-lint-action@v3
+        uses: golangci/golangci-lint-action@v9
         with:
+          version: v2.11.4
           working-directory: docker/sandbox-bundled/bootstrap
       - name: Check formatting
         working-directory: docker/sandbox-bundled/bootstrap

.github/workflows/validate-helm-charts.yaml

@@ -53,8 +53,8 @@ jobs:
             --chart-repos spark-operator=https://kubeflow.github.io/spark-operator \
             --chart-repos dask=https://helm.dask.org \
             --chart-repos bitnami=https://charts.bitnami.com/bitnami \
-            --chart-repos twuni=https://helm.twun.io \
-            --chart-repos kubernetes-dashboard=https://kubernetes.github.io/dashboard
+            --chart-repos twuni=https://twuni.github.io/docker-registry.helm \
+            --chart-repos kubernetes-dashboard=https://kubernetes-retired.github.io/dashboard
 
   validate-manifests:
     needs:

.gitignore

@@ -39,3 +39,7 @@ docs/_src
 docs/_projects
 docs/tests
 empty-config.yaml
+.claude/
+flyteidl2/
+gen/
+.kube/

CHANGELOG/CHANGELOG-v1.16.0.md

@@ -0,0 +1,75 @@
+# Flyte 1.16.0
+
+## Added
+
+- Agent/Connector Functionality: Renamed agent to connector ([#6400](https://github.com/flyteorg/flyte/pull/6400)) with improved task retry support in Flyte Connectors ([#6486](https://github.com/flyteorg/flyte/pull/6486))
+- Storage Configuration: Support for configuring storage from secrets to avoid exposing sensitive information ([#6419](https://github.com/flyteorg/flyte/pull/6419))
+- Deployment Annotations: Configurable deployment annotations in Helm charts ([#6385](https://github.com/flyteorg/flyte/pull/6385))
+- Pod Customization: Support for affinity & topologySpreadConstraints in webhook component ([#6431](https://github.com/flyteorg/flyte/pull/6431)), and configurable rollout strategy ([#6447](https://github.com/flyteorg/flyte/pull/6447))
+- Labels and Annotations: Apply labels and annotations set via task decorator to pods and CR objects ([#6421](https://github.com/flyteorg/flyte/pull/6421))
+- GRPC Configuration: MaxConcurrentStreams setting for flyteadmin grpc server ([#6448](https://github.com/flyteorg/flyte/pull/6448))
+- ArrayNode Configuration: max-delta-timestamp configuration option for FlytePropeller ArrayNode ([#6453](https://github.com/flyteorg/flyte/pull/6453))
+- Cloud Events: NATS support as cloud events sender ([#6507](https://github.com/flyteorg/flyte/pull/6507))
+- Signal Handling: SignalWatcher functionality in Copilot ([#6501](https://github.com/flyteorg/flyte/pull/6501))
+- Workflow Control: Workflow concurrency control feature ([#6475](https://github.com/flyteorg/flyte/pull/6475))
+- Monitoring: Service monitor support for flyte scheduler ([#6558](https://github.com/flyteorg/flyte/pull/6558)) and datacatalog ([#6571](https://github.com/flyteorg/flyte/pull/6571))
+- OpenTelemetry: OTEL configuration support in helm chart ([#6543](https://github.com/flyteorg/flyte/pull/6543))
+- Scaling: HPA (Horizontal Pod Autoscaler) support for flyte admin ([#6615](https://github.com/flyteorg/flyte/pull/6615)) and datacatalog ([#6616](https://github.com/flyteorg/flyte/pull/6616))
+- RBAC: Option to configure RBAC rules as namespace scoped ([#6534](https://github.com/flyteorg/flyte/pull/6534))
+- Pre-init Containers: preInitContainers value support in Flyte-Binary charts ([#6559](https://github.com/flyteorg/flyte/pull/6559))
+- Task Configuration: Task config support in flytectl config file ([#6538](https://github.com/flyteorg/flyte/pull/6538))
+- Probes: Overrides for flyteadmin readiness and liveness probes ([#6484](https://github.com/flyteorg/flyte/pull/6484))
+- Bug Reporting: Version details to bug report template ([#6479](https://github.com/flyteorg/flyte/pull/6479))
+- Documentation: Kafka version field clarification in cloud events ([#6502](https://github.com/flyteorg/flyte/pull/6502))
+
+## Changed
+
+- Container Naming: Renamed container_image to image for improved UX ([#6211](https://github.com/flyteorg/flyte/pull/6211))
+- Default Configuration: flyte-core now defaults propeller rawoutput-prefix to use storage.bucketName ([#6433](https://github.com/flyteorg/flyte/pull/6433))
+- Log Context: Updated IDL with log context ([#6443](https://github.com/flyteorg/flyte/pull/6443))
+- Error Handling: Treat kubelet NodeAffinity status.reason as retryable system error ([#6461](https://github.com/flyteorg/flyte/pull/6461))
+- Code Refactoring: Multiple refactoring improvements including replacing HasPrefix+TrimPrefix with CutPrefix ([#6456](https://github.com/flyteorg/flyte/pull/6456)), making createMsgChan private ([#6467](https://github.com/flyteorg/flyte/pull/6467)), and adjusting CoPilot init and Pod status logic ([#6523](https://github.com/flyteorg/flyte/pull/6523))
+- Go Version: Upgraded to Go 1.23 ([#6249](https://github.com/flyteorg/flyte/pull/6249))
+- Repository Updates: Updated multiple repositories to use mockery ([#6477](https://github.com/flyteorg/flyte/pull/6477), [#6607](https://github.com/flyteorg/flyte/pull/6607), [#6608](https://github.com/flyteorg/flyte/pull/6608))
+- CoPilot Improvements: Better folder handling ([#6620](https://github.com/flyteorg/flyte/pull/6620)) and multi-part blob support ([#6617](https://github.com/flyteorg/flyte/pull/6617))
+- Service Account: Set service account in base ray pod spec, allow override by pod template ([#6514](https://github.com/flyteorg/flyte/pull/6514))
+- Ray Job: Set name max length for rayJob plugin ([#6611](https://github.com/flyteorg/flyte/pull/6611))
+- Container Logs: Allow using container name in kubeflow plugin log links ([#6524](https://github.com/flyteorg/flyte/pull/6524))
+  Fixed
+- GitHub Actions: Updated and fixed "stale" GitHub Action ([#6449](https://github.com/flyteorg/flyte/pull/6449)) and PR stale comments ([#6458](https://github.com/flyteorg/flyte/pull/6458))
+- NodeShutdown: Updated DemystifyFailure to respect NodeShutdown ([#6452](https://github.com/flyteorg/flyte/pull/6452))
+- Connector Discovery: Resolved conflicts between agent auto discovery and explicit task type mapping ([#6464](https://github.com/flyteorg/flyte/pull/6464))
+- Release Scripts: Fixed release script for beta versions ([#6489](https://github.com/flyteorg/flyte/pull/6489)) and corrected file paths in release workflows ([#6487](https://github.com/flyteorg/flyte/pull/6487))
+- Resource Management: Fixed PodTemplate resource state pollution between workflow runs ([#6530](https://github.com/flyteorg/flyte/pull/6530)) and compile-time podTemplate resources override issues ([#6483](https://github.com/flyteorg/flyte/pull/6483))
+- Workflow Processing: Fixed workflow equality check ([#6521](https://github.com/flyteorg/flyte/pull/6521)) and enqueue correct work item in node execution context ([#6526](https://github.com/flyteorg/flyte/pull/6526))
+- Grafana Dashboards: Fixed Grafana dashboard queries for dynamic workflow metrics ([#6546](https://github.com/flyteorg/flyte/pull/6546))
+- Scheduler: Fixed invalid cron date schedule creating infinite loop in flytescheduler ([#6555](https://github.com/flyteorg/flyte/pull/6555))
+- Plugin Safety: Made plugin metric registration thread safe ([#6532](https://github.com/flyteorg/flyte/pull/6532))
+- Server Shutdown: Fixed flyteadmin not shutting down servers gracefully ([#6289](https://github.com/flyteorg/flyte/pull/6289))
+- Strict Mode: Disabled strict mode in flytectl ([#6619](https://github.com/flyteorg/flyte/pull/6619))
+
+## Removed
+
+- CI/CD: Removed docs job from GitHub actions workflow ([#6432](https://github.com/flyteorg/flyte/pull/6432))
+- Helm Cleanup: Cleaned up helm template since metrics path is not configurable ([#6444](https://github.com/flyteorg/flyte/pull/6444))
+- Legacy Plugins: Deleted hive plugin ([#6580](https://github.com/flyteorg/flyte/pull/6580)) and cleaned up flyteplugins ([#6515](https://github.com/flyteorg/flyte/pull/6515))
+- Annotations: Removed projectcontour annotations from helm charts by default ([#6564](https://github.com/flyteorg/flyte/pull/6564)) and primary pod annotation from raw container task ([#6618](https://github.com/flyteorg/flyte/pull/6618))
+- JWT: Removed usage of golang-jwt v3 ([#6596](https://github.com/flyteorg/flyte/pull/6596))
+
+## Security
+
+- Permissions: Removed SYS_PTRACE and sharenamespace settings for improved security ([#6509](https://github.com/flyteorg/flyte/pull/6509))
+
+## Dependencies
+
+- Dependency Updates: Multiple security and maintenance updates including:
+  golang.org/x/net updates ([#6338](https://github.com/flyteorg/flyte/pull/6338), [#6418](https://github.com/flyteorg/flyte/pull/6418), [#6587](https://github.com/flyteorg/flyte/pull/6587), [#6588](https://github.com/flyteorg/flyte/pull/6588), [#6589](https://github.com/flyteorg/flyte/pull/6589), [#6591](https://github.com/flyteorg/flyte/pull/6591), [#6593](https://github.com/flyteorg/flyte/pull/6593))
+  golang.org/x/oauth2 updates ([#6535](https://github.com/flyteorg/flyte/pull/6535), [#6594](https://github.com/flyteorg/flyte/pull/6594), [#6595](https://github.com/flyteorg/flyte/pull/6595), [#6597](https://github.com/flyteorg/flyte/pull/6597), [#6598](https://github.com/flyteorg/flyte/pull/6598), [#6599](https://github.com/flyteorg/flyte/pull/6599), [#6601](https://github.com/flyteorg/flyte/pull/6601))
+- mapstructure updates ([#6512](https://github.com/flyteorg/flyte/pull/6512), [#6575](https://github.com/flyteorg/flyte/pull/6575))
+- Docker and crypto updates ([#6573](https://github.com/flyteorg/flyte/pull/6573), [#6414](https://github.com/flyteorg/flyte/pull/6414))
+
+## Contributors
+
+Special thanks to new contributors: @ppeerttu ([#6385](https://github.com/flyteorg/flyte/pull/6385)), @studystill ([#6456](https://github.com/flyteorg/flyte/pull/6456)), @daadc ([#6502](https://github.com/flyteorg/flyte/pull/6502)), @mattiadevivo ([#6507](https://github.com/flyteorg/flyte/pull/6507)), @gopherorg ([#6511](https://github.com/flyteorg/flyte/pull/6511)), @jingchanglu ([#6528](https://github.com/flyteorg/flyte/pull/6528)), @diranged ([#6534](https://github.com/flyteorg/flyte/pull/6534)), @thomasjhuang ([#6475](https://github.com/flyteorg/flyte/pull/6475)), and @hylje ([#6559](https://github.com/flyteorg/flyte/pull/6559)), along with all returning contributors who made this release possible.
+
+Full Changelog: https://github.com/flyteorg/flyte/compare/v1.15.3...v1.16.0

CHANGELOG/CHANGELOG-v1.16.1.md

@@ -0,0 +1,7 @@
+# Flyte 1.16.1
+
+## Changed
+
+- Fix OTEL config map in Helm Chart ([#6624](https://github.com/flyteorg/flyte/pull/6624))
+
+Full Changelog: https://github.com/flyteorg/flyte/compare/v1.16.0...v1.16.1

CHANGELOG/CHANGELOG-v1.16.2.md

@@ -0,0 +1,38 @@
+# Flyte 1.16.2
+
+## Added
+
+- Added support for configuring horizontal pod autoscaling for flyte admin in flyte-core helm chart ([#6625](https://github.com/flyteorg/flyte/pull/6625))
+- Added support for typing.Union in flytecopilot / container tasks ([#6674](https://github.com/flyteorg/flyte/pull/6674))
+
+## Changed
+
+- Only return connector/agent plugins if enabled ([#6644](https://github.com/flyteorg/flyte/pull/6644))
+- Support downloading old versions of flytectl by install.sh ([#6668](https://github.com/flyteorg/flyte/pull/6668))
+- Improve datacatalog query performance with composite index on tags table ([#6672](https://github.com/flyteorg/flyte/pull/6672))
+- Handle child node timeouts in conditional branch nodes ([#6678](https://github.com/flyteorg/flyte/pull/6678))
+- Fix auto refresh cache iteration bug causing parent workflows to get stuck in a running state ([#6725](https://github.com/flyteorg/flyte/pull/6725))
+
+## Removed
+
+- Removed unused node selector functions and legacy environment variables ([#6614](https://github.com/flyteorg/flyte/pull/6614))
+- Remove unused DefaultWorkflowActiveDeadline config ([#6688](https://github.com/flyteorg/flyte/pull/6688))
+
+## Dependencies / Security 
+
+- Bump urllib3 from 2.2.3 to 2.5.0 in /flytectl/docs ([#6728](https://github.com/flyteorg/flyte/pull/6728))
+- Bump requests from 2.32.3 to 2.32.4 in /flytectl/docs ([#6727](https://github.com/flyteorg/flyte/pull/6727))
+
+## Housekeeping
+
+- Removed unused replace directives ([#6638](https://github.com/flyteorg/flyte/pull/6638))
+- Removed use of deprecated pod phase ([#6640](https://github.com/flyteorg/flyte/pull/6640))
+- Executor code cleanup ([#6629](https://github.com/flyteorg/flyte/pull/6629))
+- Updated docker images to use Bitnami legacy repo ([#6631](https://github.com/flyteorg/flyte/pull/6631))
+- Updated helm charts to use new twun.io helm repo ([#6726](https://github.com/flyteorg/flyte/pull/6726))
+
+## Contributors
+
+Special thanks to new contributors: @hefeiyun ([#6629](https://github.com/flyteorg/flyte/pull/6629)), @ihvol-freenome ([#6672](https://github.com/flyteorg/flyte/pull/6672)), @Sally-Yang-Jing-Ou ([#6725](https://github.com/flyteorg/flyte/pull/6725)), along with all returning contributors who made this release possible.
+
+Full Changelog: https://github.com/flyteorg/flyte/compare/v1.16.1...v1.16.2

CHANGELOG/CHANGELOG-v1.16.3.md

@@ -0,0 +1,36 @@
+# Flyte 1.16.3
+
+## Added
+
+- Add start- and end-time log link template vars to Ray plugin ([#6751](https://github.com/flyteorg/flyte/pull/6751))
+- Add metric to track how long nodes are stuck blocked by max parallelism being reached ([#6737](https://github.com/flyteorg/flyte/pull/6737))
+
+## Changed
+
+- Update Flyte to Go 1.24 ([#6603](https://github.com/flyteorg/flyte/pull/6603))
+
+## Fixed
+
+- Upstream array node fixes + update mockery ([#6763](https://github.com/flyteorg/flyte/pull/6763))
+- Handle etcd request too large error ([#6752](https://github.com/flyteorg/flyte/pull/6752))
+- Immediately fail if plugin.BuildResource fails instead of retrying until system retry budget is exhausted ([#6740](https://github.com/flyteorg/flyte/pull/6740))
+- Avoid merging pod template twice if template container name matches task container name ([#6733](https://github.com/flyteorg/flyte/pull/6733))
+
+## Dependencies / Security
+
+- Update x/crypto to v0.45.0 to fix security warnings ([#6774](https://github.com/flyteorg/flyte/pull/6774))
+- Bump golang.org/x/crypto from 0.36.0 to 0.45.0 in /datacatalog ([#6764](https://github.com/flyteorg/flyte/pull/6764))
+- Bump golang.org/x/crypto from 0.36.0 to 0.45.0 in /flyteadmin ([#6770](https://github.com/flyteorg/flyte/pull/6770))
+- Pin neoeinstein-prost to v0.4.0 ([#6773](https://github.com/flyteorg/flyte/pull/6773))
+- Bump stow to latest ([#6741](https://github.com/flyteorg/flyte/pull/6741))
+
+## Housekeeping
+
+- Update Flyte Components ([#8e488c4b6](https://github.com/flyteorg/flyte/commit/8e488c4b6))
+- Update docs version ([#424d7775f](https://github.com/flyteorg/flyte/commit/424d7775f))
+
+## Contributors
+
+Thanks to all the contributors who made this release possible: @fg91 ([#6733](https://github.com/flyteorg/flyte/pull/6733), [#6740](https://github.com/flyteorg/flyte/pull/6740), [#6751](https://github.com/flyteorg/flyte/pull/6751)), @Sovietaced ([#6603](https://github.com/flyteorg/flyte/pull/6603), [#6773](https://github.com/flyteorg/flyte/pull/6773), [#6774](https://github.com/flyteorg/flyte/pull/6774)).
+
+Full Changelog: https://github.com/flyteorg/flyte/compare/v1.16.2...v1.16.3

CHANGELOG/CHANGELOG-v1.16.4.md

@@ -0,0 +1,27 @@
+# Flyte 1.16.4
+
+## What's Changed
+* Bump flyteidl 1 to python 3.13 by @wild-endeavor in https://github.com/flyteorg/flyte/pull/6782
+* Remove unused replace directives and have boilerplate use local flytestdlib by @Sovietaced in https://github.com/flyteorg/flyte/pull/6785
+* Bump urllib3 from 2.5.0 to 2.6.0 in /flytectl/docs by @dependabot[bot] in https://github.com/flyteorg/flyte/pull/6788
+* refactor: omit unnecessary reassignment by @rifeplight in https://github.com/flyteorg/flyte/pull/6768
+* [Feat] add user annotations to k8s objects by @ttitsworth-lila in https://github.com/flyteorg/flyte/pull/6710
+* chore: execute goimports to format the code by @findfluctuate in https://github.com/flyteorg/flyte/pull/6801
+* Fix issue with assertions with time zones by @Sovietaced in https://github.com/flyteorg/flyte/pull/6812
+* [FlyteCTL] only get v1 sandbox image by @machichima in https://github.com/flyteorg/flyte/pull/6834
+* Unpin k8s client library version by @Sovietaced in https://github.com/flyteorg/flyte/pull/6835
+* Unpin controller runtime dependency by @Sovietaced in https://github.com/flyteorg/flyte/pull/6843
+* Fix: Correctly handle malformed dynamic workflows to avoid 'failed + succeeded + running' Schroedinger state by @fg91 in https://github.com/flyteorg/flyte/pull/6854
+* fixed: Remove print that breaks json output by @honnix in https://github.com/flyteorg/flyte/pull/6872
+* Add test to decode access token from cookie values by @pmahindrakar-oss in https://github.com/flyteorg/flyte/pull/6883
+* Update Kubernetes Dashboard URL to retired GitHub repository by @kevinliao852 in https://github.com/flyteorg/flyte/pull/6882
+* Support flag in ray plugin config to optionally disable ingress for Ray cluster by @mickjermsurawong-openai in https://github.com/flyteorg/flyte/pull/6905
+* Prevent panic when retry attempt value in map task gets larger than bitarray was allocated for by @fg91 in https://github.com/flyteorg/flyte/pull/6802
+
+## New Contributors
+* @rifeplight made their first contribution in https://github.com/flyteorg/flyte/pull/6768
+* @ttitsworth-lila made their first contribution in https://github.com/flyteorg/flyte/pull/6710
+* @findfluctuate made their first contribution in https://github.com/flyteorg/flyte/pull/6801
+* @mickjermsurawong-openai made their first contribution in https://github.com/flyteorg/flyte/pull/6905
+
+**Full Changelog**: https://github.com/flyteorg/flyte/compare/v1.16.3...v1.16.4

CHANGELOG/CHANGELOG-v1.16.5.md

@@ -0,0 +1,16 @@
+# Flyte 1.16.5
+
+## What's Changed
+* Fix: Prevent dropping of empty string args when constructing RayJob entrypoint by @fg91 in https://github.com/flyteorg/flyte/pull/6947
+* don't reuse parent wf state for array node subnodes by @pvditt in https://github.com/flyteorg/flyte/pull/6929
+* Feat: Make RayCluster head node ingress optional by @fg91 in https://github.com/flyteorg/flyte/pull/6852
+* Flyte 2 update by @kumare3 in https://github.com/flyteorg/flyte/pull/6961
+* Add RBAC support for cross-namespace secret reading by @rohitrsh in https://github.com/flyteorg/flyte/pull/6919
+* set run_all_sub_nodes for array node idl by @pvditt in https://github.com/flyteorg/flyte/pull/6966
+* Add Config struct with DisableConfigEndpoint option to profutils profiling server. Register config section under "prof" key with pflags generation and conditionally skip the /config HTTP handler when disabled. by @EngHabu in https://github.com/flyteorg/flyte/pull/7016
+* Modernize codespell config: move from inline workflow to .codespellrc by @yarikoptic in https://github.com/flyteorg/flyte/pull/7104
+
+## New Contributors
+* @rohitrsh made their first contribution in https://github.com/flyteorg/flyte/pull/6919
+
+**Full Changelog**: https://github.com/flyteorg/flyte/compare/v1.16.4...v1.16.5