eBPF Developer Tutorial¶
Summary
The eBPF Developer Tutorial (eunomia.dev/tutorials, source in eunomia-bpf/bpf-developer-tutorial) is an MIT-licensed, example-driven course in CO-RE (Compile Once, Run Everywhere) eBPF development. Each of its 65 tutorial directories is an independent, runnable tool, starting from about 20 lines of C and running up to sched_ext schedulers, GPU tracing and features from Linux 6.16-7.0. Unlike BCC-based material, it teaches libbpf, cilium/ebpf, libbpf-rs and the eunomia-bpf toolchain, in C, Go and Rust.
Key Facts¶
| Fact | Value |
|---|---|
| Maintainer | eunomia-bpf organization (research-affiliated, bpftime OSDI '25 authors) |
| License | MIT |
| Latest Version | No tagged releases. Rolling main, last commit 2026-09-05 |
| Size | 65 tutorials: lessons 0-54, 6 features/*, 3 xpu/*, cgroup |
| Newest lessons | 51-54 (2026-07-20): TCP quarantine, fsession latency (Linux 7.0), BPF qdisc egress pacer (6.16), exec image inspector (6.19) |
| Kernel range | 4.8 (lesson 1) to 7.0 (lesson 52) |
| Verification | Generated compatibility matrix: 23 CI runtime, 31 CI build, 8 not in CI, 3 docs only |
| Toolchain | clang/llvm, vendored libbpf 1.7.0 and bpftool v7.7.0 (per lesson 52), eunomia-bpf ecc/ecli v1.0.38 (2026-03-08) |
| Languages | English and Chinese README per lesson |
Evaluation¶
- Why it is better: Every example is a real, runnable tool rather than a fragment. Lessons carry per-lesson metadata that generates a compatibility matrix with minimum kernel, config options and CI status, which is rare rigor for tutorial material. Coverage goes past most free tutorials: sched_ext scheduling (mainline since Linux 6.12), TCX links, BPF qdisc, fsession, BPF arena/dynptr/workqueues/token, HID-BPF, CUDA and NPU tracing.
- When it fits: Developers who want to write eBPF tools (tracing, networking, security enforcement), systems engineers moving beyond ready-made tools like bpftrace, and teams evaluating the libbpf/CO-RE stack over BCC.
- When it does not: It deliberately skips deep conceptual theory ("does not cover complex concepts and scenario introductions"). Pair it with ebpf.io and docs.ebpf.io. Learners without C basics or a root-capable Linux host will struggle before lesson 1.
Pros and Cons¶
| Pros | Cons |
|---|---|
| 65 runnable tutorials, each an independent tool | Requires root-capable Linux. macOS and Windows users need VMs |
| Generated matrix with documented minimum kernels (4.8 to 7.0) and CI evidence | Only 23 of 65 are executed in CI. 31 are build-only |
| Multi-framework exposure: libbpf, cilium/ebpf (Go), libbpf-rs (Rust), eunomia-bpf | Most lessons use C. Rust appears in lessons 12 and 37, Go mainly via the starter template |
| Actively maintained (commits through September 2026), MIT licensed | Newest lessons need 6.16, 6.19 or 7.0 kernels that few distros ship yet |
| Curated research-paper gateway (lesson 18) | No tagged releases, video course or certification track |
Index drift
Lesson 32 (wall-clock profiler) exists and is built in CI but is missing from the README table of contents, so readers browsing the README see 64 entries instead of 65.
Common Use Cases¶
- Learning eBPF primitives: kprobe/fentry/uprobe tracing, hash maps, perf event arrays, ring buffers, histograms (lessons 0-10).
- Building libbpf user-space tools: process lifecycle monitoring, TCP latency, memory-leak detection, Java GC via USDT (lessons 11-17).
- Security engineering: BPF LSM detection, process hiding and countermeasures, syscall argument rewriting, signal-based response, exec inspection (lessons 19, 24-28, 34, 51, 54).
- Network engineering: XDP packet processing and load balancing, tc and TCX, sockops acceleration, BPF qdisc pacing (lessons 20-21, 29, 41-42, 46, 50, 53).
- Kernel and XPU experimentation: sched_ext schedulers, custom kfuncs, GPU and NPU driver tracing, energy monitoring (lessons 43-45, 47-48,
xpu/*).
Licensing and Commercial Use¶
MIT license ("Copyright (c) 2022 eunomia-bpf"). Free for commercial and internal training use with the license notice kept.
Ecosystem and Connections¶
The tutorial is the educational front door for the eunomia-bpf organization's toolchain:
- eunomia-bpf (~900 stars): compiler (
ecc) and loader (ecli) for kernel-code-only authorship. Artifacts are distributed as JSON packages, Wasm modules or OCI images. Latest release v1.0.38 (2026-03-08). The remote HTTP mode was removed in March 2026. - bpftime (~1.6k stars): userspace eBPF runtime claiming up to 10x lower uprobe overhead than kernel uprobes, with GPU hooks that convert eBPF to PTX. Published at OSDI '25.
- wasm-bpf (~440 stars): library, toolchain and runtime for writing eBPF user space as Wasm modules.
- Starter templates: libbpf-starter-template (C), cilium-ebpf-starter-template (Go), libbpf-rs-starter-template (Rust) and eunomia-template. Each ships a Makefile, Dockerfile and GitHub Actions build/release automation.
- Upstream dependencies: libbpf, bpftool and blazesym are vendored as submodules. Lesson 11 is derived from libbpf-bootstrap.
Compatibility and Requirements¶
- Linux kernel 4.8 baseline. Lesson 1 suggests 5.15+ or 6.2+. Advanced lessons need 6.x features (HID-BPF 6.3, TCX 6.6, BPF arena and token 6.9, kfuncs lesson 6.11, sched_ext 6.12, BPF qdisc 6.16, file dynptr 6.19) and lesson 52 needs 7.0 (released April 2026).
CONFIG_DEBUG_INFO_BTF=yis required by most lessons from lesson 2 onward. Root is required for every runnable lesson.- Architectures: x86_64 and arm64 for most lessons. Lessons 22, 28, 31, 47, 51-54 and
xpu/*are x86_64 only. Lesson 3 has split fentry baselines (x86_64 5.5, arm64 6.0). - Hardware-specific lessons need an NVIDIA CUDA GPU (47,
xpu/flamegraph), an Intel NPU, a network interface (XDP, tc, TCX, qdisc lessons) or a block device (17). - Build hosts: upstream CI uses Ubuntu 24.04 with distro clang/llvm. Details in Reference: Toolchain Requirements.
Latest Versions¶
No tagged releases: the main branch is rolling. Last commit 2026-09-05 (sslsniff GnuTLS/NSS fix). July 2026 brought the generated compatibility matrix (2026-07-14), CI coverage of all libbpf Makefile lessons, Rust 1.97 support, lessons 51-54 (2026-07-20) and repository-local agent skills.
Alternatives¶
- Kernel samples (
samples/bpf/,tools/testing/selftests/bpf): authoritative but minimally explained. - libbpf-bootstrap: scaffold plus a handful of examples. Good once fundamentals are known.
- BCC Python developer tutorial: the inspiration for this project. Runtime compilation makes it heavier to deploy, and the tutorial argues libbpf is now the better choice.
- bpftrace one-liners tutorial (also ported into this repo as
src/bpftrace-tutorial): fastest introduction, but not a software-development path. - ebpf.io and docs.ebpf.io: concept-first references that lesson 0 recommends reading first.
Migration and Lock-in Risks¶
Pure documentation, so no lock-in. Lessons 1-10 use the eunomia-bpf ecc/ecli toolchain and its conventions, but the kernel-side code is ordinary libbpf-style C and maps cleanly to plain libbpf (lesson 11 onward).
Community Health¶
~4.25k GitHub stars and ~600 forks (August 2026 figures), sustained commits through September 2026, and issues and discussions open for learner questions. The maintainer group publishes systems research (bpftime at OSDI '25), which gives the sched_ext, userspace-runtime and GPU material unusual depth.
Topic Map¶
The diagram shows how the course's paths feed into the same kernel machinery.
flowchart LR
L0["Lessons 0-10<br/>ecc + ecli"]
L11["Lessons 11-21<br/>libbpf + bpftool skeletons"]
DEEP["In-depth tracks<br/>networking, tracing, security"]
FEAT["features/*, 43-45<br/>kfuncs, arena, token, sched_ext"]
NEW["50-54<br/>TCX, fsession, BPF qdisc, file dynptr"]
KERNEL["Linux kernel<br/>verifier, JIT, hooks, maps"]
L0 --> L11
L11 --> DEEP
L11 --> FEAT
FEAT --> NEW
L0 --> KERNEL
L11 --> KERNEL
DEEP --> KERNEL
FEAT --> KERNEL
NEW --> KERNEL
- How-to Guides: set up a host, build eunomia-bpf and libbpf lessons, check kernel prerequisites, run sched_ext, debug and clean up.
- Reference: full lesson index with minimum kernels and CI status, toolchain requirements,
SEC()program types, kernel config options, capability and dual-use tables. - Explanation: curriculum structure, the CO-RE pipeline, data plumbing, 2026 kernel features, the compatibility-matrix system and the security model.
Related Topics¶
- Comparison: eBPF Developer Tutorial vs libbpf-bootstrap vs bpftrace: when to learn, scaffold or probe interactively.
- bpftrace: the high-level tracing language taught in lesson 0's learning plan.
- Grafana and Coroot: eBPF-based observability products. Use this tutorial to understand their instrumentation layer.
- Observability 2.0: wide-event paradigm, with eBPF as a zero-instrumentation event source.
- Cilium: production eBPF networking, and an early TCX adopter per lesson 50.
- Calico: eBPF data plane option.
- Tools Catalogue: registered under Learning & Reference > Courses & Learning Paths.
Sources¶
- bpf-developer-tutorial repository: README, LICENSE,
.gitmodules,src/compatibility.md, CI workflow and lesson READMEs (read from raw.githubusercontent.com, 2026-09-25) - Commit history: last commit 2026-09-05, lessons 51-54 on 2026-07-20 (commit feed, 2026-09-25)
- Tutorial hub and compatibility matrix
- Lesson 0: Introduction and Lesson 1: Hello World
- Lesson 52: fsession latency
- Further reading: papers
- eunomia-bpf releases: v1.0.38 on 2026-03-08 (release feed), cross-checked with the Zenodo record for v1.0.37
- libbpf releases: v1.7.0 (March 2026)
- bpftime and its OSDI '25 paper
- Linux 7.0 released (9to5Linux) and KernelNewbies: Linux 7.0
Questions¶
- Which lessons will be ported end-to-end to Rust (libbpf-rs) beyond lessons 12 and 37, and will Go lessons use cilium/ebpf directly rather than only the starter template?
- Does bpftime's GPU path (eBPF compiled to PTX) change how lesson 47 and the
xpu/*lessons are taught? - When will the vendored
vmlinux.hbe regenerated from a 6.19+/7.0+ kernel so lessons 52 and 54 can drop their declaration workarounds? - Will lesson 32 be added to the README table of contents, and will more "CI build" lessons gain runtime tests (for example with virtme-ng or a newer-kernel runner)?
- Is there upstream guidance for verifying the provenance of OCI-published compiled tools run through
ecli? - Does BPF token (
features/bpf_token) offer a practical least-privilege workflow for classroom or shared-lab settings?
Answered Questions¶
- Lesson 49 declares a 6.3 minimum (HID-BPF's first release) but uses the
struct_ops/hid_device_eventform. Which kernel first supports that form? Linux 6.11.drivers/hid/bpf/hid_bpf_struct_ops.c, which registershid_bpf_opsas a BPF struct_ops type with ahid_device_eventcallback, first appears in the v6.11 tree; it is absent in v6.10. On 6.3-6.10 HID-BPF used the olderfmod_retattachment, so the lesson as written needs 6.11+ (checked 2026-09-28).