Skip to content

bpftrace

Summary

bpftrace is a high-level tracing language for Linux, inspired by awk, C, DTrace and SystemTap. Scripts compile through LLVM to eBPF bytecode and attach through libbpf to kprobes, fentry/fexit, tracepoints, uprobes, USDT and perf events, so a kernel or application question gets answered in minutes without writing a loader program. State lives in per-CPU BPF maps with built-in histograms and aggregations. It is Apache-2.0 licensed, community governed, and ships in every major distro. The latest release is 0.27.0 (2026-09-10). Since 0.22 the language has gained let declarations, block scoping, macros, imports, map declarations and a formatter, and 0.26 raised the minimum kernel to 6.1.

Overview

bpftrace is the interactive instrument of the eBPF stack. Probes are addressed by name (tracepoint:syscalls:sys_enter_openat, kprobe:vfs_read, fentry:tcp_reset) with wildcards and a listing mode (-l, -lv) that shows what the running kernel offers. Aggregations such as count(), hist() and lhist() run in the kernel, and only the reduced result reaches user space. It complements rather than replaces application-level observability stacks and compiled libbpf tools. See the eBPF learning-path comparison.

This compact view shows how a one-liner reaches the kernel. The detailed pipeline is in Explanation.

flowchart LR
    S["bpftrace -e / script.bt"] --> C["bpftrace compiler<br/>parser, BTF types, LLVM IR"]
    C -->|"BPF ELF"| L["libbpf loader"]
    L --> V["kernel verifier + JIT"]
    V --> P["kprobe / fentry / tracepoint /<br/>uprobe / usdt / profile"]
    P --> M[("per-CPU maps")]
    P --> R[("ring buffer")]
    M --> O["bpftrace output<br/>text or NDJSON"]
    R --> O

Key Facts

Attribute Detail
Repository github.com/bpftrace/bpftrace
Website bpftrace.org (tutorials, versioned docs, release notes)
Latest Version 0.27.0 (2026-09-10); previous 0.26.1 (2026-06-02)
Release cadence Two minors per year, about two weeks after each LLVM major (March, September); patch releases as needed; 0.26.0 was an extra off-cycle release
Support window Not published: docs/release_process.md defines one release/0.Y.x branch per minor for post-release fixes, but no LTS line and no end date for older branches (checked 2026-09-28)
License Apache-2.0 (tool). Generated BPF programs load with a GPL-compatible license, "GPL" by default
Governance Community project with a meritocratic, consensus-based model; maintainers are the CODEOWNERS (GOVERNANCE.md). No foundation affiliation stated
Language C++20
Minimum kernel 6.1 (since 0.26.0; 5.15 for 0.24/0.25)
LLVM 18 to 23 for dynamic builds (0.27)
Key dependencies libbpf (vendored submodule), libclang, bcc (build time), optional blazesym
Architectures x86_64, arm64, arm, s390x, loongarch64, mips64, ppc64le, ppc64, riscv64; static AppImages for x86_64 and arm64
Privileges CAP_BPF, CAP_PERFMON, CAP_DAC_READ_SEARCH, CAP_DAC_OVERRIDE (capability check since 0.25)
Distro packages Ubuntu 24.04: 0.20.2 (0.20.2-1ubuntu4.3 in noble-updates, packages.ubuntu.com). Debian 13: 0.23.2 (packages.debian.org). Upstream is 0.27.0 (checked 2026-09-28)
Stars ~10.3k (2026-08-27)

Evaluation

  • Why it is better: Minutes-to-answer for kernel and application questions that otherwise need a C project. It ships in distro repositories and as a static AppImage, so there is no bespoke toolchain on target hosts. Per-CPU aggregation keeps hot-path counting cheap. BTF-typed probes (fentry, tracepoints) make scripts readable without casts.
  • When it fits: On-call investigation, ad-hoc syscall/I/O/scheduler latency analysis, sampling profilers, testing hypotheses before committing to a compiled tool, teaching probe concepts to teams.
  • When it does not: Long-lived daemons shipped to fleets, complex user-space interaction, custom network data paths, or anything needing a maintained binary artifact. Graduate those to libbpf projects (see eBPF Developer Tutorial for the learning path).

Pros and Cons

Pros Cons
One-liners replace hour-long investigations DSL ceiling: complex logic still gets awkward, despite macros and imports
Packaged in all major distros; Apache-2.0 Linux-only; needs privileged capabilities
Built-in histogram, tseries and aggregation stdlib with per-CPU safety Tracing captures sensitive plaintext (args, buffers)
Probe discovery (-l, -lv) makes the kernel explorable Frequent breaking changes across 0.x minors; distro packages lag the docs
BTF-typed fentry and tracepoint args, no headers needed Kernel 6.1+ and BTF needed for the full feature set
Test, bench and format modes for script maintenance Sync map reads are expensive; ring buffer can drop events under load

Common Use Cases

  • Counting syscalls per process: tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }.
  • Latency and size histograms straight off return values: kretprobe:vfs_read { @bytes = hist(retval); }.
  • CPU profiling for flame graphs: profile:hz:99 { @[kstack] = count(); }.
  • Running the 38 bundled tools (opensnoop.bt, biolatency.bt, runqlat.bt, tcpretrans.bt).
  • Exploring probes before anything else: bpftrace -l 'tracepoint:syscalls:sys_enter_*'.

Licensing and Commercial Use

The tool is Apache-2.0 with no commercial restrictions. The BPF programs it generates are loaded into the kernel with a GPL-compatible license string (default "GPL"); since 0.25 only GPL-compatible strings are accepted, because many kernel tracing helpers are GPL-only. This does not affect how you use the output.

Ecosystem and Connections

  • LLVM compiles scripts to BPF bytecode; libbpf loads and attaches them; libclang handles C headers and imported .bpf.c files.
  • Same kernel surface as every CO-RE lesson in the eBPF Developer Tutorial; probe types learned here transfer directly.
  • NDJSON output (-f json) composes with jq, log shippers, and backends such as Grafana via a collector.
  • Community tools live in the separate bpftrace/user-tools repository; bcc remains the recommended step up for complex tools.

Compatibility and Requirements

Minimum kernel 6.1 with BTF, the BPF ring buffer, and the kprobe/uprobe/ftrace config options listed in Reference. LLVM 18 to 23 for source builds. Run bpftrace --info or scripts/check_kernel_features.sh to check a host. Older kernels should use an older bpftrace release.

Latest Versions

Release Date Highlights
0.27.0 2026-09-10 Mostly fixes; str_concat(), container_of(), config builtin; static arm64 AppImage; session probes work on kernel 7.0
0.26.1 2026-06-02 Session-probe and statement-probe fixes
0.26.0 2026-05-26 Imports and & stable, dw_ustack(), source-line uprobes, write_user(), kernel floor 6.1
0.25.0 2026-03-13 Macros and map declarations stable, test/bench probes, --fmt, capability check, tracepoint args need BTF

The full history since 0.20 is in Reference.

Alternatives

  • Hand-written libbpf tools (tutorial path): full control, permanent artifacts.
  • bcc tools (Python front end): richer argument handling for complex tools; upstream calls bcc and bpftrace complementary.
  • Raw ftrace/perf: no BPF needed when kernels cannot load programs.
  • Productized auto-instrumentation: Coroot or Beyla-style agents when you want answers, not queries.

Migration and Lock-in Risks

No vendor lock-in anywhere in the chain. Scripts are small text files. The real migration cost is bpftrace's own 0.x churn: block scoping (0.22), fatal attach errors (0.24), BTF-only tracepoint args (0.25) and signed subtraction (0.26) all broke existing scripts. Pin a version per fleet and use the upstream migration guide when upgrading. See How-to Guides.

Community Health

Active: six minor releases (0.22 to 0.27) between January 2025 and September 2026 on a published LLVM-aligned schedule, monthly open office hours, GitHub Discussions and Discord, and broad distro packaging (Debian, Ubuntu, Fedora, CentOS Stream, Alpine, Arch, Gentoo, nixpkgs, openSUSE). Star count ~10.3k as of 2026-08-27.

Topic Map

  • How-to Guides: install and build, kernel readiness, probe discovery, one-liner cookbook, reusable scripts, upgrades, output pipelines, troubleshooting.
  • Reference: release history, requirements, kernel config, privileges, probe types, stdlib, config and environment variables, CLI flags, bundled tools, packaging.
  • Explanation: compile pipeline, session lifecycle, probe mechanisms, BTF/DWARF, map engine, async events, language evolution, security model.

Sources

Questions

  • Will the undocumented --aot / bpftrace-aotrt path become a supported feature, or be removed with the remaining bcc dependency? Today AOT artifacts are locked to the exact bpftrace version that produced them.
  • When will a 1.0 release freeze the language? The release process already notes that semantic versioning "will matter a lot for >= 1.0.0 releases".
  • What policy linting exists for .bt review (wildcard breadth, sync-read anti-patterns, PII-shaped argument access)? --fmt handles style only.
  • Do hardened distros ship ready-made roles that grant exactly the four capabilities bpftrace 0.25+ checks for?
  • What bridges convert bpftrace NDJSON into OpenTelemetry signals for long-term storage?