Skip to content

Better config-time validation for overlapping flow metrics configuration#2983

Merged
jmacd merged 6 commits into
open-telemetry:mainfrom
wbutler:wbutler/feat-flow-metrics-early-validation
May 18, 2026
Merged

Better config-time validation for overlapping flow metrics configuration#2983
jmacd merged 6 commits into
open-telemetry:mainfrom
wbutler:wbutler/feat-flow-metrics-early-validation

Conversation

@wbutler
Copy link
Copy Markdown
Contributor

@wbutler wbutler commented May 15, 2026

Closes #2784.

This change implements a BFS walk of the processor pipeline at construction time to validate that flow metrics do not collide.

  • In runtime_pipeline.rs, introduce a helper func to capture a flat edge list vector of pipeline connections and pass the result as a new param on build_flow_metric_state. Saves us from having to pass in some part of _config into that function and keeps data types better separated.
  • In flow_metrics.rs:
    • In build_flow_metric_state, add a new data structure that captures start\end pairs for registered flow metrics.
    • at the end of the main loop, iff the count of flow metrics > 1
      • convert the list of edges into a one-to-many adjacency data structure
      • for each flow metric
        • do a BFS to find all the nodes between the start and the end
        • check for nodes in this set that also appear as a start node in the data structure above.
        • If found, throw an error, halting pipeline construction.

Tests

Within flow_metrics.rs, this change adds 10 unit tests, as follows:

  • 7 tests exercise the interleaving detection logic on synthetically created pipeline topologies.
    • Three expected-pass tests include:
      • Linear topology with disjoint ranges
      • Branching topology with disjoint ranges
      • Linear topology with a single flow metric
    • Four expected-reject tests include:
      • Linear topology with interleaving
      • Linear topology with interleaving, reverse declaration order
      • Diamond topology with interleaving
      • Linear topology with fully nested ranges
  • 3 tests exercise a helper function directly.

Validation

The following commands pass cleanly:

cargo check -p otap-df-engine
cargo test -p otap-df-engine (contains new tests)
cargo clippy -p otap-df-engine --all-targets -- -D warnings
cargo fmt --all -- --check
cargo xtask quick-check

@github-actions github-actions Bot added the rust Pull requests that update Rust code label May 15, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.93%. Comparing base (f562f90) to head (d34bba0).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main    #2983    +/-   ##
========================================
  Coverage   85.93%   85.93%            
========================================
  Files         725      725            
  Lines      275782   275980   +198     
========================================
+ Hits       236996   237172   +176     
- Misses      38262    38284    +22     
  Partials      524      524            
Components Coverage Δ
otap-dataflow 87.06% <100.00%> (+<0.01%) ⬆️
query_abstraction 80.61% <ø> (ø)
query_engine 89.57% <ø> (ø)
otel-arrow-go 52.45% <ø> (ø)
quiver 92.25% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@wbutler wbutler marked this pull request as ready for review May 16, 2026 00:05
@wbutler wbutler requested a review from a team as a code owner May 16, 2026 00:05
@wbutler
Copy link
Copy Markdown
Contributor Author

wbutler commented May 16, 2026

@drewrelmas should get a look at this since it bubbles up to his flow metrics work. Feedback & comments are very welcome.

@wbutler wbutler changed the title [WIP] Better config-time validation for overlapping stopwatch configuration Better config-time validation for overlapping stopwatch configuration May 16, 2026
@wbutler wbutler changed the title Better config-time validation for overlapping stopwatch configuration Better config-time validation for overlapping flow metrics configuration May 16, 2026
Comment thread rust/otap-dataflow/crates/engine/src/flow_metrics.rs
Copy link
Copy Markdown
Member

@lalitb lalitb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@wbutler
Copy link
Copy Markdown
Contributor Author

wbutler commented May 18, 2026

Created #3009 to capture the follow-up hardening for unreachable end nodes.

@wbutler
Copy link
Copy Markdown
Contributor Author

wbutler commented May 18, 2026

Ready for merge, IMO.

@jmacd jmacd added this pull request to the merge queue May 18, 2026
Merged via the queue into open-telemetry:main with commit f894282 May 18, 2026
86 checks passed
@wbutler wbutler deleted the wbutler/feat-flow-metrics-early-validation branch May 18, 2026 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rust Pull requests that update Rust code

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Better config-time validation for overlapping stopwatch configuration

3 participants