feat(new sink): new journald sink#23593
Conversation
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
|
Hi @wiktorsikora, thank you for your submission! All new sinks/sources need documentation. Your would be in |
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
|
Thanks for the tips, I've added the documentation. I'd appreciate it if you could double-check it as I'm not entirely sure I've done everything correctly, especially the |
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
Signed-off-by: Wiktor Sikora <wiktorsikora7@gmail.com>
|
@vectordotdev/vector just a gentle reminder about this PR :) |
pront
left a comment
There was a problem hiding this comment.
Hi @wiktorsikora, this looks good! Could we add tests here? Or a new integration test.
| } | ||
|
|
||
| /// Sanitize a key and convert it to uppercase. | ||
| fn sanitize_key(&mut self, key: &str) { |
There was a problem hiding this comment.
One small edge case I noticed: this may still produce field names that journald rejects.
The native receiver validates both plain and length-encoded fields with journal_field_valid(..., false): plain fields, length-encoded fields.
A few examples that look worth handling:
🔥can sanitize to an empty name, but empty names are rejected:l <= 0_SYSTEMD_UNITkeeps the leading_, but client-provided protected fields are rejected:p[0] == '_'123abckeeps the leading digit, but digit-prefixed names are rejected:ascii_isdigit(p[0])- if an escape/fallback prefix is added, the final name still needs to stay within 64 bytes:
l > 64
| pub mod humio; | ||
| #[cfg(any(feature = "sinks-influxdb", feature = "prometheus-integration-tests"))] | ||
| pub mod influxdb; | ||
| #[cfg(feature = "sinks-journald")] |
There was a problem hiding this comment.
Looks like this should be Linux-only too, since the implementation depends on Linux/systemd-specific pieces such as memfd_create.
| #[cfg(feature = "sinks-journald")] | |
| #[cfg(all(target_os = "linux", feature = "sinks-journald"))] |
| let bytes_sent = self.send_payload(&self.buf).await?; | ||
| // Clear the buffer after sending | ||
| // We could also keep the buffer for reuse, but by doing this we ensure that | ||
| // we don't allocate too much memory for long time in case of rare large payloads. | ||
| self.buf = vec![]; | ||
| Ok(bytes_sent) |
There was a problem hiding this comment.
If send_payload returns an error here, self.buf is not cleared, so the next event may append to the previous failed payload.
| let bytes_sent = self.send_payload(&self.buf).await?; | |
| // Clear the buffer after sending | |
| // We could also keep the buffer for reuse, but by doing this we ensure that | |
| // we don't allocate too much memory for long time in case of rare large payloads. | |
| self.buf = vec![]; | |
| Ok(bytes_sent) | |
| let payload = std::mem::take(&mut self.buf); | |
| let bytes_sent = self.send_payload(&payload).await?; | |
| Ok(bytes_sent) |
Summary
This PR introduces support for journald sink as discussed in #19177.
Vector configuration
How did you test this PR?
I tested the implementation locally on a Fedora 42 system.
Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.References