OBI causes SIGSEGV on Go binaries compiled with -buildmode=pie + -extldflags='-static-pie' (static-PIE, ET_DYN without PT_INTERP)

[yuvalshi0]

Summary

OBI (and its upstream, Grafana Beyla) causes a SIGSEGV (exit code 139) on any Go binary compiled with -buildmode=pie -extldflags='-static-pie' — i.e. a static-PIE executable. The crash occurs at process startup, before the Go runtime initialises its own signal handlers, so the crashing process produces zero log output.

Affected binary

aws-load-balancer-controller starting from v3.2.0, whose Dockerfile builds the controller with:

go build \
  -buildmode=pie \
  -tags 'osusergo,netgo,static_build' \
  -ldflags='-s -w -linkmode=external -extldflags "-static-pie"' \
  -o /out/controller main.go

v3.1.0 used a different build (no -static-pie), which is why v3.1.0 works fine and v3.2.0+ crashes.

Symptoms

• Second (or later) pod of aws-load-balancer-controller on a node where OBI's network monitor is running exits instantly with exit code 139.
• kubectl logs --previous returns empty output — crash happens before any Go logging initialises.
• Even setting GOTRACEBACK=crash produces no output — the SIGSEGV fires before the Go runtime signal handler is registered.
• OBI network monitor logs show the process being detected as Go and instrumented immediately before the crash:

{"msg":"OBI instrumented process","process":"controller","pid":85707,"language":1}
{"msg":"instrumenting process","cmd":"/usr/lib/systemd/systemd-coredump","pid":85722}
{"msg":"error reading itab section in Go program, manual spans will not work",
 "error":"accessing symbols table: no symbol section"}
{"msg":"process ended for already instrumented executable","cmd":"/controller"}

The systemd-coredump entry confirms a kernel-level crash triggered right after OBI attaches.

Root Cause

Static-PIE binaries are ELF type ET_DYN (same as a regular PIE) but have no PT_INTERP program header (no dynamic linker). They self-contain the C runtime and are fully position-independent.

The issue is in pkg/internal/goexec/instructions.go → findFuncOffset:

// For more info on this calculation: stackoverflow.com/a/40249502
off := f.Value - prog.Vaddr + prog.Off

For a binary built with -linkmode=external -extldflags='-static-pie', the external linker (GNU ld/lld) restructures ELF segments. The runtimeText value extracted from .gopclntab (lines ~160-167 in instructions.go) can diverge from prog.Vaddr of the executable PT_LOAD segment. The resulting off is a garbage file offset. OBI attaches a uprobe at that wrong location; when the process starts executing code there, the eBPF handler fires in an unexpected context, corrupting the Go runtime startup → SIGSEGV.

Additionally, -s -w strips both .symtab and .gosymtab, so findInterfaceImpls correctly warns "no symbol section", but instrumentationPoints still proceeds to attach uprobes via the .gopclntab path with the broken offset.

No static-PIE detection currently exists in OBI. The NodeJS path in pkg/internal/nodejs/signal_check.go already handles ET_DYN vs ET_EXEC correctly; the Go path never received the same treatment for the -static-pie edge case.

Proposed Fix

Add static-PIE detection in pkg/internal/goexec/instructions.go and bail out of Go-specific uprobe attachment:

// isStaticPIE returns true for Go binaries built with -buildmode=pie
// -extldflags='-static-pie': ET_DYN with no PT_INTERP (no dynamic linker).
// Uprobe offset calculation is unreliable for these binaries because the
// external linker can reorder ELF segments in ways that break the
// f.Value - prog.Vaddr + prog.Off formula.
func isStaticPIE(ef *elf.File) bool {
    if ef.Type != elf.ET_DYN {
        return false
    }
    for _, prog := range ef.Progs {
        if prog.Type == elf.PT_INTERP {
            return false
        }
    }
    return true
}

Then in isSupportedGoBinary:

func isSupportedGoBinary(elfF *elf.File) error {
    if isStaticPIE(elfF) {
        return errors.New("static-PIE binary: uprobe offset calculation unsafe, skipping Go-specific instrumentation")
    }
    // ... existing Go version check
}

This causes InspectOffsets to return an error → typer.go falls back to generic kprobe/tracepoint instrumentation only (no uprobes) → no crash. Network-level visibility is preserved; only Go TLS/HTTP uprobe visibility is lost for these binaries.

Historical context

Beyla PR #309 (2023) added support for statically-linked Go executables (ET_EXEC, -extldflags='-static'). Static-PIE (ET_DYN, -extldflags='-static-pie') is a distinct case that was not covered by that fix.

Workaround

Until a fix is released, exclude the binary by exe path:

# OBI / Beyla discovery config
discovery:
  exclude_instrument:
    - exe_path: /controller

Environment

• OBI version: v0.8.0
• Affected controller: aws-load-balancer-controller v3.2.0, v3.2.2
• Kernel: Linux 5.10 (AWS EKS, Amazon Linux 2)
• Architecture: amd64

[grcevski]

Hi @yuvalshi0, thanks for reporting this. The fallback to the generic tracer is not a good option, I think we will try to find a fix to make thing work.

[rafaelroquetto]

@yuvalshi0 could you please retry with OBI 0.9.0 - this releases includes #1851 which I believe might have addressed it.