Getting Started with eBPF in Go¶
In this guide, we'll walk you through building a new eBPF-powered Go application from scratch. We'll introduce the toolchain, write a minimal eBPF C example and compile it using bpf2go. Then, we'll put together a Go application that loads the eBPF program into the kernel and periodically displays its output.
The application attaches an eBPF program to an XDP hook that counts the number of packets received by a physical interface. Filtering and modifying packets is a major use case for eBPF, so you'll see a lot of its features being geared towards it. However, eBPF's capabilities are ever-growing, and it has been adopted for tracing, systems and application observability, security and much more.
eBPF C program¶
Dependencies
To follow along with the example, you'll need:
Let's begin by writing our eBPF C program, as its structure will be used as the basis for generating Go boilerplate.
Click the annotations in the code snippet for a detailed explanation of the individual components.
-
When putting C files alongside Go files, they need to be excluded by a Go build tag, otherwise
go buildwill complain withC source files not allowed when not using cgo or SWIG. The Go toolchain can safely ignore our eBPF C files. -
Include headers containing the C macros used in the example. Identifiers such as
__u64andBPF_MAP_TYPE_ARRAYare shipped by the Linux kernel, with__uint,__type,SECand BPF helper definitions being provided by libbpf. -
Declare a BPF map called
pkt_count, an Array-type Map holding a single u64 value. Seeman bpfor the online bpf man pages for an overview of all available map types.
For this example, we went with an array since it's a well-known data structure you're likely familiar with. In BPF, arrays are preallocated and zeroed, making them safe and ready to use without any initialization. -
The Map definition is placed in the
.mapsELF section, which is whereebpf-goexpects to find it. -
In BPF, not all programs are equal. Some act on raw packets, some execute within the context of kernel or user space functions, while others expect to be run against an
__sk_buff. These differences are encoded in the Program Type. libbpf introduced a set of conventions around which ELF sections correspond to which type. In this example, we've chosenxdpsince we'll attach this program to the XDP hook later. -
There's only one possible element in
pkt_countsince we've specified amax_entriesvalue of 1. We'll always access the 0th element of the array. -
Here, we're asking the BPF runtime for a pointer to the 0th element of the
pkt_countMap.
bpf_map_lookup_elemis a BPF helper declared indocs.h. Helpers are small pieces of logic provided by the kernel that enable a BPF program to interact with its context or other parts of the kernel. Discover all BPF helpers supported by your kernel usingman bpf-helpersor the online bpf-helpers man pages. -
All Map lookups can fail. If there's no element for the requested
keyin the Map,countwill hold a null pointer. The BPF verifier is very strict about checking access to potential null pointers, so any further access tocountneeds to be gated by a null check. -
Atomically increase the value pointed to by
countby 1. It's important to note that on systems with SMP enabled (most systems nowadays), the same BPF program can be executed concurrently.
Even though we're loading only one 'copy' of our Program, accompanied by a singlepkt_countMap, the kernel may need to process incoming packets on multiple receive queues in parallel, leading to multiple instances of the program being executed, andpkt_counteffectively becoming a piece of shared memory. Use atomics to avoid dirty reads/writes. -
XDP allows for dropping packets early, way before it's passed to the kernel's networking stack where routing, firewalling (ip/nftables) and things like TCP and sockets are implemented. We issue the
XDP_PASSverdict to avoid ever interfering with the kernel's network stack. -
Since some BPF helpers allow calling kernel code licensed under GPLv2, BPF programs using specific helpers need to declare they're (at least partially) licensed under GPL. Dual-licensing is possible, which we've opted for here with
Dual MIT/GPL, sinceebpf-gois MIT-licensed.
Create an empty directory and save this file as counter.c. In the next step,
we'll set up the necessary bits to compile our eBPF C program using bpf2go.
Compile eBPF C and generate scaffolding using bpf2go¶
With the counter.c source file in place, create another file called gen.go
containing a //go:generate statement. This invokes bpf2go when running go
generate in the project directory.
Aside from compiling our eBPF C program, bpf2go will also generate some scaffolding code we'll use to load our eBPF program into the kernel and interact with its various components. This greatly reduces the amount of code we need to write to get up and running.
Using a dedicated file for your package's //go:generate statement(s) is
neat for keeping them separated from application logic. At this point in the
guide, we don't have a main.go file yet. Feel free to include it in
existing Go source files if you prefer.
Before using the Go toolchain, Go wants us to declare a Go module. This command should take care of that:
% go mod init ebpf-test
go: creating new go.mod: module ebpf-test
go: to add module requirements and sums:
go mod tidy
% go mod tidy
First, add bpf2go as a tool dependency to your Go module. This ensures the
version of bpf2go used by the Go toolchain always matches your version of the
library.
Now we're ready to run go generate:
% go generate
Compiled /home/timo/getting_started/counter_bpfel.o
Stripped /home/timo/getting_started/counter_bpfel.o
Wrote /home/timo/getting_started/counter_bpfel.go
Compiled /home/timo/getting_started/counter_bpfeb.o
Stripped /home/timo/getting_started/counter_bpfeb.o
Wrote /home/timo/getting_started/counter_bpfeb.go
bpf2go built counter.c into counter_bpf*.o behind the scenes using
clang. It generated two object files and two corresponding Go source files
based on the contents of the object files. Do not remove any of these, we'll
need them later.
Let's inspect one of the generated .go files:
| counter_bpfel.go | |
|---|---|
Neat! Looks like bpf2go automatically generated a scaffolding for interacting
with our count_packets Program from Go. In the next step, we'll explore how to
load our program into the kernel and put it to work by attaching it to an XDP
hook!
The Go application¶
Finally, with our eBPF C code compiled and Go scaffolding generated, all that's left is writing the Go code responsible for loading and attaching the program to a hook in the Linux kernel.
Click the annotations in the code snippet for some of the more intricate details. Note that we won't cover anything related to the Go standard library here.
-
Linux kernels before 5.11 use RLIMIT_MEMLOCK to control the maximum amount of memory allocated for a process' eBPF resources. By default, it's set to a relatively low value. See Resource Limits for a deep dive.
-
counterObjectsis a struct containing nil pointers to Map and Program objects. A subsequent call toloadCounterObjectspopulates these fields based on the struct tags declared on them. This mechanism saves a lot of repetition that would occur by checking a Collection for Map and Program objects by string.
As an added bonus,counterObjectsadds type safety by turning these into compile-time lookups. If a Map or Program doesn't appear in the ELF, it won't appear as a struct field and your Go application won't compile, eliminating a whole class of runtime errors. -
Close all file descriptors held by
objsright before the Go application terminates. See Object Lifecycle for a deep dive. -
Associate the
count_packets(stylized in the Go scaffolding asCountPackets) eBPF program witheth0. This returns aLinkabstraction. -
Close the file descriptor of the Program-to-interface association. Note that this will stop the Program from executing on incoming packets if the Link was not
Link.Pined to the bpf file system. -
Load a uint64 stored at index 0 from the
pkt_countMap (stylized in the Go scaffolding asPktCount). This corresponds to the logic incounter.c.
Save this file as main.go in the same directory alongside counter.c and
gen.go.
Building and running the Go application¶
Now main.go is in place, we can finally compile and run our Go application!
% go build && sudo ./ebpf-test
2023/09/20 17:18:43 Counting incoming packets on eth0..
2023/09/20 17:18:47 Received 0 packets
2023/09/20 17:18:48 Received 4 packets
2023/09/20 17:18:49 Received 11 packets
2023/09/20 17:18:50 Received 15 packets
Generate some traffic on eth0 and you should see the counter increase.
Iteration Workflow¶
When iterating on the C code, make sure to keep generated files up-to-date. Without re-running bpf2go, the eBPF C won't be recompiled, and any changes made to the C program structure won't be reflected in the Go scaffolding.
What's Next?¶
Congratulations, you've just built your (presumably) first eBPF-powered Go app! Hopefully, this guide piqued your interest and gave you a better sense of what eBPF can do and how it works.
With XDP, we've only barely scratched the surface of eBPF's many use cases and applications. For more easily-accessible examples, see the main repository's examples/ folder. It demonstrates use cases like tracing user space applications, extracting information from the kernel, attaching eBPF programs to network sockets and more.
Follow our other guides to continue on your journey of shipping a portable eBPF-powered application to your users.
-
Use
clang --versionto check which version of LLVM you have installed. Refer to your distribution's package index to finding the right packages to install, as this tends to vary wildly across distributions. Some distributions shipclangandllvm-stripin separate packages. ↩ -
For Debian/Ubuntu, you'll typically need
libbpf-dev. On Fedora, it'slibbpf-devel. ↩ -
On AMD64 Debian/Ubuntu, install
linux-headers-amd64. On Fedora, installkernel-devel.On Debian, you may also need
ln -sf /usr/include/asm-generic/ /usr/include/asmsince the example expects to find<asm/types.h>. ↩
Authored by