Skip to content

Add workers to tcp, udp, and forward input plugins #11757

@lecaros

Description

@lecaros

Is your feature request related to a problem? Please describe.

The in_tcp, in_udp, and in_forward plugins don't support the workers configuration that HTTP-based plugins have. This means these plugins can only use one thread to handle incoming traffic, which can become a bottleneck when processing high volumes of data.

HTTP plugins like in_http, in_splunk, and in_opentelemetry already support workers (recently added). This allows them to use multiple threads and better utilize CPU cores.

Describe the solution you'd like

Add a workers option to in_tcp, in_udp, and in_forward plugins, similar to what exists for HTTP plugins.

This would:

  1. Allow users configure multiple worker threads to handle connections.
  2. Better utilize multi-core CPUs when handling high traffic.
  3. Keep the current behavior by default (workers=1).

Describe alternatives you've considered

  • Running multiple Fluent Bit instances with different ports (or scale in k8s deployments) - adds complexity and needs external load balancing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions