BPF updates 09

This is issue 09 of the regular newsletter around BPF written by Alexander Alemayhu. It summarizes ongoing development, presentations, videos and other information related to BPF and XDP. It is released roughly once a week.

The highlights since the previous issue

• New comparison instructions for reducing register pressure, stack usage and potentially smaller programs.
• RFC patchset for BPF socket redirect with a awesome new helper function bpf_sk_redirect_map.
• Verifier fixes, more tests and alignment tracking work got merged.
• The XDP redirect series got merged.
• XDP support for tap got merged

The latest couple of iteration of the alignment tracking are really interesting. It now also comes with some documentation describing the register value tracking and the verifier pruning. Checkout the patches section for all the links.

The Linux 4.13 merge window ended several weeks ago weeks ago and net-next opened up around the same time with a brand new status page. No need to ask for the status anymore for people not being attentive enough. :)

Some more interesting topics

• iproute improvements to error handling reports for tail calls and support for loading map in map.
• LLVM 5.0.0-rc1 is ready for testing.
• MIPS eBPF JIT finally merged.
• Virtio optimizations for XDP.

While a MIPS patch did get applied in June. One file got lost in transmit, but that's now corrected.

Presentations

Videos

Tech Talks @ Kinvolk: Introduction to eBPF Programming by Alban Crequy

Nice introduction to eBPF with demos, code and diagrams.

Jesper Dangaard Brouer - XDP eXpress Data Path

Great technology overview of XDP.

eBPF and IO Visor Project

Mostly high level introductory talk.

Slides

Landlock: programmatic access control

A look at Landlock and how eBPF and some other things are used to reduce security threats.

In case you missed it

Tracing a packet journey using Linux tracepoints, perf and eBPF

Cool tracing tutorial with some perf examples. Also checkout the companion github repository.

The anatomy of “Hello World” python program in bcc

Detailed walkthrough of a BCC example.

XDP Newbies...

Which is a place where people can talk about getting up to speed with setting up an XDP build environment and writing XDP programs.

You can subscribe by sending a email to majordomo@vger.kernel.org, with a message body containing subscribe xdp-newbies. No subject is needed, but you can of course add one if you like.

Projects

A new batch of random projects from Github. Check them out and Remember to give a star, if you like the project ;)

Tracepkt

Trace a ping packet journey across network interfaces and namespace on recent Linux. Supports IPv4 and IPv6.

ocaml-bpf

OCaml embedded eBPF assembler.

bpftrace

BPFtrace is a DTrace-style dynamic tracing tool for linux, based on the extended BPF capabilities available in recent Linux kernels. BPFtrace uses LLVM as a backend to compile scripts to BPF-bytecode and makes use of BCC for interacting with the Linux BPF system.

tcptracer-bpf

tcptracer-bpf is an eBPF program using kprobes to trace TCP events (connect, accept, close). The eBPF program is compiled to an ELF object file.

FlameGraph

Stack trace visualizer http://www.brendangregg.com/flamegraphs.html

BPF userspace tool

The tool allows listing programs and maps on the system as well as simple dumping and modification of the maps.

Random cool note

Netdev 2.2 Call for papers is out! https://www.netdevconf.org/2.2/submit-proposal.html … Dont wait for that last minute rush!

Patches

• Thomas Richter, [PATCHv3] perf bpf: Fix endianness problem when loading parameters in prologue
• Edward Cree, [PATCH net-next] bpf/verifier: track liveness for pruning
• Eric Dumazet, [PATCH net] tcp: fix possible deadlock in TCP stack vs BPF filter
• Jason Wang [PATCH net-next V2 0/3] XDP support for tap
  • [PATCH net-next V2 1/3] tap: use build_skb() for small packet](https://www.spinics.net/lists/kernel/msg2578714.html)
  • [PATCH net-next V2 2/3] net: export some generic xdp helpers](https://www.spinics.net/lists/kernel/msg2578711.html)
  • [PATCH net-next V2 3/3] tap: XDP support](https://www.spinics.net/lists/kernel/msg2578713.html)
• Daniel Borkmann, [PATCH net 0/2] Minor fix in bpf_convert_ctx_access
  • [PATCH net 1/2] net: fix compilation when busy poll is not enabled
  • [PATCH net 2/2] bpf: fix two missing target_size settings in bpf_convert_ctx_access
• Daniel Borkmann, [PATCH] bpf: fix bpf_trace_printk on 32 bit
• William Tu, [PATCHv2 net-next] selftests: bpf: add check for ip XDP redirect
• Wang Nan, [PATCH] perf test llvm: Fix f_mode endianness problem
• Daniel Borkmann, [PATCH iproute2 master] bpf: unbreak libelf linkage for bpf obj loader
• Daniel Borkmann, [PATCH net-next v2 0/9] Add BPF_J{LT,LE,SLT,SLE} instructions
  • [PATCH net-next v2 1/9] bpf: add BPF_J{LT,LE,SLT,SLE} instructions
  • [PATCH net-next v2 2/9] bpf, x86: implement jiting of BPF_J{LT,LE,SLT,SLE}
  • [PATCH net-next v2 3/9] bpf, arm64: implement jiting of BPF_J{LT,LE,SLT,SLE}
  • [PATCH net-next v2 4/9] bpf, sparc64: implement jiting of BPF_J{LT, LE, SLT, SLE}
  • [PATCH net-next v2 5/9] bpf, s390x: implement jiting of BPF_J{LT,LE,SLT,SLE}
  • [PATCH net-next v2 6/9] bpf, ppc64: implement jiting of BPF_J{LT,LE,SLT,SLE}
  • [PATCH net-next v2 7/9] bpf, nfp: implement jiting of BPF_J{LT,LE}
  • [PATCH net-next v2 8/9] bpf: enable BPF_J{LT,LE,SLT,SLE} opcodes in verifier
  • [PATCH net-next v2 9/9] bpf: add test cases for new BPF_J{LT, LE, SLT, SLE} instructions
• Tom Herbert, [PATCH RFC 0/2] stap: Socket tap
  • [PATCH RFC 1/2] bpf: Add a BPF return code to disconnect a connection
  • [PATCH RFC 2/2] stap: Socket tap
• Joel Fernandes, [PATCH RFC v2 0/5] add arm64 cross compilation support to BPF samples
  • [PATCH RFC v2 1/5] samples/bpf: Use getppid instead of getpgrp for array map stress
  • [PATCH RFC v2 2/5] samples/bpf: Enable cross compiler support
  • [PATCH RFC v2 3/5] samples/bpf: Fix inline asm issues building samples on arm64
  • [PATCH RFC v2 4/5] samples/bpf: Fix pt_regs issues when cross-compiling
  • [PATCH RFC v2 5/5] samples/bpf: Add documentation on cross compilation
• James Hogan, [RFC PATCH 0/2] bpf_trace_printk() fixes
  • [RFC PATCH 1/2] bpf: Fix bpf_trace_printk on 32-bit architectures
  • [RFC PATCH 2/2] bpf: Initialise mod[] in bpf_trace_printk
• Edward Cree, [PATCH v5 net-next 00/12] bpf: rewrite value tracking in verifier
  • [PATCH v5 net-next 01/12] bpf/verifier: rework value tracking
  • [PATCH v5 net-next 02/12] bpf/verifier: track signed and unsigned min/max values
  • [PATCH v5 net-next 03/12] bpf/verifier: more concise register state logs for constant var_off
  • [PATCH v5 net-next 04/12] selftests/bpf: change test_verifier expectations
  • [PATCH v5 net-next 05/12] selftests/bpf: rewrite test_align
  • [PATCH v5 net-next 06/12] selftests/bpf: add a test to test_align
  • [PATCH v5 net-next 07/12] selftests/bpf: add test for bogus operations on pointers
  • [PATCH v5 net-next 08/12] selftests/bpf: don't try to access past MAX_PACKET_OFF in test_verifier
  • [PATCH v5 net-next 09/12] selftests/bpf: add tests for subtraction & negative numbers
  • [PATCH v5 net-next 10/12] selftests/bpf: variable offset negative tests
  • [PATCH v5 net-next 11/12] Documentation: describe the new eBPF verifier value tracking behaviour
  • [PATCH v5 net-next 12/12] bpf/verifier: increase complexity limit to 128k
• Yonghong Song, [PATCH net-next v4 0/2] bpf: add support for sys_{enter|exit}_* tracepoints
  • [PATCH net-next v4 1/2] bpf: add support for sys_enter__ and sys_exit__ tracepoints
  • [PATCH net-next v4 2/2] bpf: add a test case for syscalls/sys_{enter|exit}_* tracepoints
• Mickaël Salaün, [PATCH net-next v2 1/2] bpf: Move check_uarg_tail_zero() upward
• Mickaël Salaün, [PATCH net-next v2 2/2] bpf: Extend check_uarg_tail_zero() checks](https://www.spinics.net/lists/netdev/msg448056.html)
• William Tu, [PATCH net-next] selftests: bpf: add a test for XDP redirect
• John Fastabend, [net-next PATCH v2] bpf: devmap fix mutex in rcu critical section
• John Fastabend, [RFC PATCH 0/6] BPF socket redirect
  • [RFC PATCH 1/6] net: early init support for strparser
  • [RFC PATCH 2/6] net: add sendmsg_locked and sendpage_locked to af_inet6
  • [RFC PATCH 3/6] net: fixes for skb_send_sock
  • [RFC PATCH 4/6] net: sockmap with sk redirect support
  • [RFC PATCH 5/6] net: bpf, add skb to sk lookup routines
  • [RFC PATCH 6/6] net: sockmap sample program
• Daniel Borkmann, [PATCH net] bpf: fix byte order test in test_verifier
• Daniel Borkmann, [PATCH net 0/2] Two BPF fixes for s390
  • [PATCH net 1/2] bpf, s390: fix jit branch offset related to ldimm64
  • [PATCH net 2/2] bpf, s390: fix build for libbpf and selftest suite
• David Daney, [PATCH] MIPS: Add missing file for eBPF JIT.
• John Fastabend, [net-next PATCH v2] net: comment fixes against BPF devmap helper calls
• Joel Fernandes, [PATCH] samples/bpf: Fix cross compiler error with bpf sample
• Thomas Richter, [PATCHv2] bpf: fix selftest/bpf/test_pkt_md_access on s390x
• William Tu, [PATCH net-next] bpf: fix the printing of ifindex
• Phil Sutter, [iproute PATCH] bpf: Make bytecode-file reading a little more robust
• William Tu, [PATCH net] samples/bpf: fix bpf tunnel cleanup
• Daniel Borkmann, [PATCH net] bpf: don't indicate success when copy_from_user fails
• John Fastabend, [PATCH] bpf: testing: fix devmap tests
• Luiz Augusto von Dentz, [PATCH BlueZ] monitor: Use BPF to filter packets by index
• Jakub Kicinski, [PATCH net] bpf: don't zero out the info struct in bpf_obj_get_info_by_fd()
• Jakub Kicinski, [PATCH net-next] bpf: install libbpf headers on 'make install'
• Jakub Kicinski, [PATCH net-next] bpf: add helper capable of reading out instructions
• Daniel Borkmann, [PATCH iproute2 master v2 0/2] Minor BPF updates
  • [PATCH iproute2 master v2 1/2] bpf: improve error reporting around tail calls
  • [PATCH iproute2 master v2 2/2] bpf: fix mnt path when from env
• Edward Cree, [PATCH net 0/2] bpf: fix verifier min/max handling in BPF_SUB
  • [PATCH net 1/2] selftests/bpf: subtraction bounds test
  • [PATCH net 2/2] bpf/verifier: fix min/max handling in BPF_SUB
• Dan Carpenter, [PATCH net-next] bpf: dev_map_alloc() shouldn't return NULL
• Douglas Caetano dos Santos, [PATCH] bpf.2: ffix
• Daniel Borkmann, [PATCH net 0/5] BPF map value adjust fix
  • [PATCH net 1/5] bpf: fix mixed signed/unsigned derived min/max value bounds
  • [PATCH net 2/5] bpf: allow to specify log level and reduce it for test_verifier
  • [PATCH net 3/5] bpf: fix up test cases with mixed signed/unsigned bounds
  • [PATCH net 4/5] bpf: add test for mixed signed and unsigned bounds checks
  • [PATCH net 5/5] bpf: more tests for mixed signed and unsigned bounds checks
• Jason Wang [PATCH net-next V2 0/5] Refine virtio-net XDP
  • [PATCH net-next V2 1/5] virtio_ring: allow to store zero as the ctx
  • [PATCH net-next V2 2/5] virtio-net: pack headroom into ctx for mergeable buffers
  • [PATCH net-next V2 3/5] virtio-net: switch to use new ctx API for small buffer
  • [PATCH net-next V2 4/5] virtio-net: do not reset during XDP set
  • [PATCH net-next V2 5/5] virtio-net: switch off offloads on demand if possible on XDP set
• John Fastabend, [net-next PATCH] net: fix build error in devmap helper calls
• John Fastabend, [net-next PATCH 00/12] Implement XDP bpf_redirect
  • [net-next PATCH 01/12] ixgbe: NULL xdp_tx rings on resource cleanup
  • [net-next PATCH 02/12] net: xdp: support xdp generic on virtual devices
  • [net-next PATCH 03/12] xdp: add bpf_redirect helper function
  • [net-next PATCH 04/12] xdp: sample program for new bpf_redirect helper
  • [net-next PATCH 05/12] net: implement XDP_REDIRECT for xdp generic
  • [net-next PATCH 06/12] ixgbe: add initial support for xdp redirect
  • [net-next PATCH 07/12] xdp: add trace event for xdp redirect
  • [net-next PATCH 08/12] bpf: add devmap, a map for storing net device references
  • [net-next PATCH 09/12] bpf: add bpf_redirect_map helper routine
  • [net-next PATCH 10/12] xdp: Add batching support to redirect map
  • [net-next PATCH 11/12] net: add notifier hooks for devmap bpf map
  • [net-next PATCH 12/12] xdp: bpf redirect with map sample program
• Andy Gospodarek, [PATCH net-next] samples/bpf: add option for native and skb mode for redirect apps
• Cong Wang, [Patch net] bpf: check NULL for sk_to_full_sk() return value
• Daniel Borkmann, [PATCH iproute2 -master 0/3] BPF updates
  • [PATCH iproute2 -master 1/3] bpf: remove obsolete samples
  • [PATCH iproute2 -master 2/3] bpf: support loading map in map from obj
  • [PATCH iproute2 -master 3/3] bpf: dump id/jited info for cls/act programs
• Yonghong Song, bpf: generate better lowering code for certain select/setcc instructions
• Yonghong Song, bpf: fix a compilation bug due to unused variable for release build
• Kefeng Wang, [PATCH] bpf: fix return in bpf_skb_adjust_net
• Colin King, [PATCH][bpf-next] bpf: add missing break in for the TCP_BPF_SNDCWND_CLAMP case
• Lawrence Brakmo, [PATCH net-next] bpf: fix return in load_bpf_file
• Yonghong Song, [PATCH net v2] samples/bpf: fix a build issue
• Al Viro, [RFC] get_compat_bpf_fprog(): don't copyin field-by-field
• Rafael Espindola, Fully fix the movw/movt addend.

Please note that netdev receives a lot of patches and the list above is not meant to be comprehensive.

Happy eBPF hacking! ;)