eBPF Developer Tutorial vs libbpf-bootstrap vs bpftrace¶
Summary
Three entries in the modern eBPF ecosystem that are often weighed against each other but sit at different layers of the workflow: the eBPF Developer Tutorial (a 65-lesson CO-RE curriculum), libbpf-bootstrap (the libbpf project's scaffold of small demo apps) and bpftrace (a high-level tracing language). The practical question is which layer you need. Facts match the topic pages as of 2026-09-25. libbpf-bootstrap has no topic page in this vault; its facts come from its README.
Which One Should I Pick?¶
The flowchart gives a first cut; the three usually chain together rather than compete.
flowchart TD
Q1{"Need an answer about a live<br/>kernel or process right now?"}
Q2{"Kernel 6.1+ with BTF<br/>on the target host?"}
Q3{"Building a tool that must ship<br/>and run long-term (daemon, agent)?"}
Q4{"Team already fluent in libbpf,<br/>CO-RE and skeletons?"}
BT["bpftrace 0.27<br/>(one-liners, .bt scripts)"]
OLD["Older bpftrace release,<br/>or bcc tools"]
LB["libbpf-bootstrap<br/>(or eunomia-bpf starter templates)"]
TUT["eBPF Developer Tutorial<br/>(65 lessons, then a scaffold)"]
TUT2["eBPF Developer Tutorial<br/>(structured learning path)"]
Q1 -->|"Yes"| Q2
Q2 -->|"Yes"| BT
Q2 -->|"No"| OLD
Q1 -->|"No"| Q3
Q3 -->|"Yes"| Q4
Q4 -->|"Yes"| LB
Q4 -->|"No"| TUT
Q3 -->|"No, learning"| TUT2
TL;DR¶
| eBPF Developer Tutorial | libbpf-bootstrap | bpftrace | |
|---|---|---|---|
| What it is | 65-lesson hands-on curriculum; each lesson is a runnable tool | Scaffold repository of small libbpf demo apps to copy and rename | High-level tracing language and CLI for one-liners and scripts |
| Primary output | A learning path ending in self-written tools | A starting project (Makefile, vmlinux.h, skeleton, user-space loader) |
Immediate trace output from .bt scripts or -e one-liners |
| Code surface | C kernel side; C, Go and Rust user space | C examples plus Rust (libbpf-rs) examples | bpftrace DSL compiled through LLVM |
| Best for | Structured skill building, from basics to sched_ext and GPU tracing | Starting your next production tool | "What is my kernel doing right now?" questions |
| Release / activity | No tagged releases; rolling main, last commit 2026-09-05 |
No tagged releases listed; rolling master |
0.27.0 (2026-09-10); two minors a year |
| Stars (2026-08 snapshot) | ~4.25k | ~1.5k | ~10.3k |
| License | MIT | BSD-3-Clause | Apache-2.0 (generated BPF programs load with a GPL-compatible license string) |
Layer Mapping¶
Mapping the three onto explicit layers resolves most confusion:
flowchart TB
subgraph Observation["Interactive observation layer"]
BT["bpftrace<br/>scripts and one-liners<br/>on-call investigation"]
end
subgraph Development["Application development layer"]
LB["libbpf-bootstrap<br/>project scaffold<br/>bespoke tools"]
end
subgraph Education["Learning curriculum layer"]
TUT["eBPF Developer Tutorial<br/>65 lessons, CO-RE<br/>C, Go, Rust"]
end
KERNEL["Linux kernel:<br/>verifier, JIT, hooks, maps"]
BT -->|"LLVM-compiled BPF,<br/>loaded via libbpf"| KERNEL
LB -->|"bpftool skeletons,<br/>libbpf"| KERNEL
TUT -->|"ecc/ecli, libbpf,<br/>cilium/ebpf, libbpf-rs"| KERNEL
TUT -.->|"teaches"| BT
TUT -.->|"lesson 11 derives from"| LB
They compose rather than compete. The tutorial's lesson 0 prescribes a learning plan: read ebpf.io and docs.ebpf.io, try the bpftrace tutorial (about 1 hour), the BCC Python tutorial, a libbpf-bootstrap example (about 2 hours), then lessons 1-10 of the curriculum. The tutorial repository even vendors both worlds: a bpftrace tutorial port under src/bpftrace-tutorial, lesson 11 derived from libbpf-bootstrap, and org starter templates that play the same role as libbpf-bootstrap (libbpf-starter-template, cilium-ebpf-starter-template, libbpf-rs-starter-template).
Profiles¶
bpftrace: The Observation Instrument¶
A high-level tracing language for Linux. LLVM compiles scripts into BPF bytecode, and libbpf loads and attaches them (bcc remains only a build-time dependency for a few residual paths). Probes are addressed by name patterns such as tracepoint:syscalls:sys_enter_open* or kprobe:tcp_*, and discovery is built in (bpftrace -l, -lv). Scripts read like awk with probe triggers:
Strengths: minutes to an answer, no build system, no user-space program to maintain, packaged in every major distro and as static AppImages (x86_64, arm64). Limits: everything stays inside one script. Complex state machines, network data paths and long-lived daemons outgrow a tracing DSL, even with 0.24-0.26 imports, macros and .bpf.c linking. Since 0.26 the minimum kernel is 6.1, and tracepoint args need BTF since 0.25. Scripts compile at run time, so the target host needs the bpftrace binary (static builds avoid a separate LLVM install).
AOT mode
An ahead-of-time path exists in the source tree (--aot plus a bpftrace-aotrt runtime), but it is undocumented (absent from the man page and --help) and each artifact is locked to the exact bpftrace version that produced it. Treat it as experimental; it does not change the "graduate to libbpf" answer for fleets. See bpftrace explanation.
libbpf-bootstrap: The Development Harness¶
The libbpf project's scaffold (BSD-3-Clause). Clone it with submodules and you get a ready *.bpf.c plus user-space loader pair, generated skeleton headers, vmlinux.h for CO-RE relocations and a Makefile that wires clang (v11+), libelf and zlib correctly. It ships about a dozen small demo apps (minimal, bootstrap, uprobe, usdt, fentry, kprobe, xdp, tc, profile, sockfilter, task_iter, lsm), a few in Rust through libbpf-rs, plus an Android build via xmake. The README calls bootstrap "the starting point for your own BPF application". Depth beyond scaffolding (event plumbing architecture, packaging, distribution) is out of scope by design; you bring the systems knowledge.
eBPF Developer Tutorial: The Curriculum¶
Where the other two are instruments, this is instruction. Each of its 65 tutorial directories (lessons 0-54, six features/*, three xpu/* and cgroup) is a finished small tool with a README explaining design choices. It walks the same libbpf machinery that libbpf-bootstrap scaffolds, and covers ground neither of the others does: sched_ext schedulers, HID-BPF, BPF LSM enforcement, BPF qdisc, fsession (Linux 7.0), GPU and NPU tracing. A generated compatibility matrix records minimum kernels (4.8 to 7.0) and CI status: 23 lessons are executed in CI, 31 are build-only, 8 are not in CI and 3 are docs only. Its weakness mirrors its strength: as teaching material it favors legibility over production hardening, and early lessons use the org's ecc/ecli toolchain before moving to plain libbpf.
Dimension Comparison¶
| Dimension | eBPF Developer Tutorial | libbpf-bootstrap | bpftrace |
|---|---|---|---|
| Time to first result | ~1-2 h (lesson 1 end to end, estimate) | ~30 min once clang and libbpf deps are installed (estimate) | Minutes |
| Compile model | Ahead of time (CO-RE objects); some lessons packaged as OCI images or Wasm | Ahead of time (CO-RE) at project build | Compiled at run time; AOT path undocumented and version-locked |
| User-space program | Yes, taught progressively (C, then Go and Rust) | Yes (C or Rust, provided harness) | No |
| Path to production | Teaches toward it (ring buffers, filters, loaders, starter templates) | Is the production starting point | Ad-hoc and on-call use; graduate long-lived tools to libbpf |
| Kernel baseline | Per-lesson matrix, 4.8 to 7.0 | minimal and *_legacy variants run on older kernels; bootstrap uses CO-RE and the BPF ring buffer (5.8+) |
6.1+ with BTF for 0.26+ (older kernels need an older release) |
| Languages | C, Go (cilium/ebpf), Rust (libbpf-rs) | C, Rust (libbpf-rs) | bpftrace DSL only |
Distribution
Only the tutorial teaches how artifacts leave the machine: OCI-image execution (ecli run ghcr.io/...) and Wasm modules appear in its material because the same organization maintains those runtimes (eunomia-bpf, wasm-bpf, bpftime).
Decision Matrix¶
Weighted for "a platform team wants sustainable in-house eBPF capability". Scores (1 to 5) are editorial judgments.
| Criterion (weight) | Tutorial | libbpf-bootstrap | bpftrace |
|---|---|---|---|
| Skill building (35%) | 5 | 2 | 2 |
| Answer speed today (25%) | 1 | 1 | 5 |
| Production trajectory (25%) | 4 | 5 | 2 |
| Maintenance breadth C/Go/Rust (15%) | 5 | 2 | 1 |
| Weighted total | 3.75 | 2.50 | 2.60 |
Primary recommendation: chain them. Use bpftrace for immediate operational questions (and to teach the team probe concepts cheaply), the tutorial as the structured learning track, and libbpf-bootstrap or the org starter templates for the first maintained tool. The three slots complement each other; budget decides how much time each gets, not which one to pick.
Fallback recommendation: if only one can be adopted now, adopt the tutorial. It introduces enough bpftrace (vendored port) and scaffold-shaped builds (lesson 11 onward) to delay the other two without blocking anything.
Related¶
- bpftrace: full research on the observation-layer tool
- eBPF Developer Tutorial: the curriculum and the eunomia-bpf toolchain
- Coroot: a product built on the same kernel hooks (eBPF auto-instrumentation)
- Apache SkyWalking: Rover eBPF profiling agent
- Cilium: eBPF networking in production
- Observability domain and comparisons index
Sources¶
- bpftrace repository, LICENSE (Apache-2.0), CHANGELOG and 0.27 release notes. AOT details from the bpftrace topic's reading of
src/aot/aot.cpp - libbpf-bootstrap repository and README: example list, dependencies, legacy variants;
LICENSEis BSD-3-Clause (both read from raw.githubusercontent.com 2026-09-25) - bpf-developer-tutorial repository and tutorial hub: lesson count, compatibility matrix, lesson 0 learning plan
- Vendored bpftrace tutorial port
Questions¶
- Are the eunomia-bpf starter templates converging functionally with libbpf-bootstrap enough to justify their own comparison note?
- Will bpftrace's undocumented AOT path become supported (which would matter for air-gapped fleets where runtime compilation is unwanted), or be removed along with the remaining bcc build dependency?