Skip to content
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
528c10e
Remove min_len validation from LoggingContext pod_name and container_…
pingsutw May 4, 2026
f62e8de
Regenerate gen files: remove min_len from LoggingContext pod_name and…
pingsutw May 4, 2026
e115523
Add ContainerIdentifier to TailLogsResponse.Logs
pingsutw May 4, 2026
62c161c
Merge branch 'main' of https://github.com/flyteorg/flyte into fix/log…
pingsutw May 5, 2026
799e1b9
feat: TailLogsRequest pod_selector oneof (primary_pod, all_pods, pod_…
pingsutw May 6, 2026
c28e11b
make gen
pingsutw May 6, 2026
37455e2
feat: add start_time and end_time to GetActionLogContextResponse
pingsutw May 6, 2026
bb6141a
make gen
pingsutw May 6, 2026
b127084
fix: restore min_len=1 validation on kubernetes_pod_name and kubernet…
pingsutw May 7, 2026
969bbb5
rebase
pingsutw May 7, 2026
ff91693
feat(logs): add execution_attempt field to LoggingContext proto
pingsutw May 8, 2026
c216761
mkae generate
pingsutw May 12, 2026
0bf459c
make gen
pingsutw May 12, 2026
6c7c456
Merge branch 'main' of https://github.com/flyteorg/flyte into fix/log…
pingsutw May 13, 2026
c45e2e7
Merge remote-tracking branch 'origin/main' into fix/logging-context-o…
pingsutw May 13, 2026
42d8c94
mkae gen
pingsutw May 14, 2026
03ce2e7
add prefix
pingsutw May 14, 2026
03bd31b
merged main
pingsutw May 14, 2026
fb3990a
make gen
pingsutw May 14, 2026
ff1ef8a
make generate
pingsutw May 14, 2026
f606cfc
update
pingsutw May 15, 2026
d88a33a
add comments
pingsutw May 19, 2026
635acdc
merge main
pingsutw May 19, 2026
212e97e
Merge branch 'main' of https://github.com/flyteorg/flyte into fix/log…
pingsutw May 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions flyteidl2/dataproxy/dataproxy_service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,15 @@ message TailLogsRequest {
// The attempt number.
uint32 attempt = 2 [(buf.validate.field).uint32.gt = 0];

// The pod name to tail logs from. If not provided, attempt to find the primary pod, else assume the first pod.
string pod_name = 3;
// Selects which pod(s) to tail. If unset, defaults to primary_pod.
oneof pod_filter {
// Tail logs from the primary pod only. This is the default when pod_filter is unset.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I see, if this is the default totally fine keeping the oneof field name pod_selector then

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated it

bool primary_pod = 6;
// Tail logs from all pods.
bool all_pods = 5;
// Tail logs from the named pod only.
string pod_name = 3;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would we want to support pod_name_prefix here too?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about other filters (repeated pod_names for example)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessary now. If pod selector is equal all_pods, the log service will use the podNamePrefix in the log context to fetch the logs from all the workers

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add them in the future when needed

}

// +optional. The dataproxy uses this to stream logs from the connector via
// AsyncConnectorService.GetTaskLogs instead of pod logs.
Expand All @@ -217,6 +224,8 @@ message TailLogsResponse {
message Logs {
// Structured log lines.
repeated flyteidl2.logs.dataplane.LogLine lines = 1;
// Which container these lines are from.
Comment thread
pingsutw marked this conversation as resolved.
flyteidl2.logs.dataplane.ContainerIdentifier container = 2;
Comment thread
pingsutw marked this conversation as resolved.
}

// One or more batches of logs.
Expand Down
4 changes: 4 additions & 0 deletions flyteidl2/logs/dataplane/payload.proto
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ message LoggingContext {
map<string, string> kubernetes_pod_labels = 9;
common.Sort.Direction order = 10;
uint64 number_of_batches = 11;
// Attempt number for the execution. Used by log sources to compute pod uniqueIDs
// when pod names are hashed (e.g. CloudWatch stream name prefix narrowing).
// +optional, defaults to 1
uint32 execution_attempt = 12;
}

// Parameters of environment in which logs were collected. Should contain everything
Expand Down
6 changes: 6 additions & 0 deletions flyteidl2/workflow/run_service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,12 @@ message GetActionLogContextResponse {

// The cluster where the action attempt is running.
string cluster = 2;

// The start time of the action attempt.
google.protobuf.Timestamp start_time = 3;

// The end time of the action attempt. Unset if the attempt is still running.
google.protobuf.Timestamp end_time = 4;
}

// Request message for listing runs.
Expand Down
245 changes: 161 additions & 84 deletions gen/go/flyteidl2/dataproxy/dataproxy_service.pb.go

Large diffs are not rendered by default.

70 changes: 69 additions & 1 deletion gen/go/flyteidl2/dataproxy/dataproxy_service.pb.validate.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

222 changes: 118 additions & 104 deletions gen/go/flyteidl2/logs/dataplane/payload.pb.go

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions gen/go/flyteidl2/logs/dataplane/payload.pb.validate.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

683 changes: 355 additions & 328 deletions gen/go/flyteidl2/workflow/run_service.pb.go

Large diffs are not rendered by default.

58 changes: 58 additions & 0 deletions gen/go/flyteidl2/workflow/run_service.pb.validate.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 10 additions & 10 deletions gen/python/flyteidl2/dataproxy/dataproxy_service_pb2.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading