Kernel Recipes 2019 - The ubiquity but also the necessity of eBPF as a technology to keep the kernel relevant

•

3 likes•1,601 views

BPF provides a new way for users to solve problems and get flexibility now through rapid prototyping directly in the kernel. This is surprising because BPF allows freedom of expression within containment and safety. For BPF to succeed, education is essential - those who understand it must teach others so more people can benefit.

Software

Keeping the kernel relevant with
BPF
David S. Miller (Red Hat Inc.)

Kernel APIs Are Hard
You must deﬁne the problem being solved
Also, consider existing interfaces
Look into extensibility and future needs
Are there holes in the design
Can it be exploited
Will it be obsolete soon

Kernel Changes Take Time
Then you have to implement it
You have to write tests for it (!!!)
You must propose it upstream
You must address feedback
And repeat all above steps each time
Enterprise distros get it a year later

Kernel APIs Are Conﬁnement
You deﬁne the interface and boundaries
And then that’s it
No ﬂexibility
This is inherent in syscall interface structure
The limits are always small and precise

Systems Developers Are Arrogant
To design syscalls properly you must be arrogant
You are putting the caller into a box
You “know better” than the user
Users don’t want to be in a box
Users want to solve their problems
And this may change tomorrow

Users Want Arbitrary Policy
… and they want it now
Maximum ﬂexibility is mandatory
This mean rapid prototyping…
...but for kernel things
By the user
For the user

Kernel Development Pace
New features should be well thought out
Implementation and design should be discussed
Bug ﬁxes should attack the problem not the symptoms
Testing should be complete and rigorous
All of these things take time...

Picasso
Born October 25, 1881
Blue period: 1901-1904
Red Period: 1904-1906
Leading to...

Les Demoiselles d'Avignon
Painted by Picasso around 1907
Widely regarded as the ﬁrst cubist work
Hidden in Picasso’s studio for some time
Like BPF, people just weren’t ready for it yet

Why is BPF Surprising?
It fundamentally seems contradictory
It provides freedom of expression…
...yet containment and safety
All at the same time
This can’t possibly work

Education is Essential
We need knowledge transfer
If you understand BPF…
...explain it to other people
If you know someone who understands BPF…
...ask them to teach it to you
Do it for the sake of the kernel

Thank You
Linus Torvalds
Alexei Starovoitov and Daniel Borkmann
Jesper Dangaard Brouer
And all the sad cats...

Similar to Kernel Recipes 2019 - The ubiquity but also the necessity of eBPF as a technology to keep the kernel relevant

From open source labs to ceo methods and advice by sysfera

fOSSa - Free Open Source Software Academia Conference

Class (1)

Haboosh Biboo

“Can you make this?” Our manager slides over a screenshot from Tron/Oblivion/Any Sci-fi movie with a sticky note that says “our data here” and pitches us an idea. We are stuck between a cool idea and tons of unknowns. We know that the availability, accuracy, and nature of our data will directly affect the quality and outcome of our project. So how do we begin developing? We fake it while we make it. This is what the talk is about: the importance of developing with fake data, the types of data we can fake, and some useful strategies for getting up and running with fake data. http://www.okcruby.org/blog/2015/02/05/february-2015-meeting/

Fake It While We Make It (Data-Driven Prototyping)

Ryan LaBouve

A tutorial session on UXD hacks I gave at O'Reilly Etech in 2004. Original context here: http://conferences.oreillynet.com/cs/et2004/view/e_sess/4767 "User-Centered Design and participatory product development are established, proven techniques for making interfaces and information understandable. But how is it possible to use them when your knowledge, the technology, and the possible markets are moving so quickly? Is it possible to create alpha-tech that defines a new market and is a joy to use? UI Design for Alien Cowboys is a three-hour tutorial and workshop that proposes that it is."

UI For Alien Cowboys

Matt Jones

Rethinking Object Orientation - By Kathleen Dollard Decades after object orientation design altered programming, it’s still evolving, and we’re still learning to use it better. Many changes in the tools we use and how we write applications affect the approach we take to OOD. Some of these changes relate to architecture where approaches like SOA and the layering revolution behind Silverlight alter the place of traditional OOD within the bigger picture of architecture. Other changes are language improvements that alter the very meaning of the phrase “object” from a design point of view. Language features that alter our implementation of logical objects include generics, extension methods, delegates/lambda expressions, partial classes/methods, reflection, anonymous types, and declarative programming. We’ll also explore the growing role of interfaces as a contractual base in composable applications and explore differences between traditional applications and ecosystem empowering applications. I’m really excited to give this talk to a group with diverse skillsets! Come ready for multi-way conversations because I want to learn from you.

Rethinking Object Orientation

IASA

This is a fairly rough presentation targeted at helping managers understand various Agile Engineering practices:, CI, Pair Programming, TDD, and the Mikado Method. This consists of a lot of game instructions - some I created (like Test Driven Tinkering and Pair Poetry) to others I lifted and modified some (like Lego CI and Agile Jenga). The details of the Creative Commons license applied to this deck is detailed on my blog: http://paulmboos.com/about/creative-commons-license/

Agile Engineering for Managers Workshop

Paul Boos

Greg Young "Why Event Sourced Systems Fail"

Fwdays

At some point, the code you write today will be deleted and replaced with something new. This talk will discuss the life cycle of a large code base, and how to manage it over time to accommodate rewrites, giving examples from a major rewrite of the Firefox build and release pipeline over the last two years. You'll learn how to replace components of a running distributed system while keeping it operational, the proverbial replacing the wing of an airplane in flight.

From hello world to goodbye code

Kim Moir

Lec 01 introduction

UmairMuzaffar9

A somewhat more verbose version of https://www.slideshare.net/JonathanRoss74/the-art-of-performance-tuning. Presented at JavaOne 2017 [CON4027], this presentation takes a practical, hands-on look at Java performance tuning. It discusses methodology (spoiler: it’s the scientific method) and how to apply it to Java SE systems (on any budget). Exploring concrete examples with tools such as the Oracle Java Mission Control feature of Oracle Java SE Advanced, VisualVM, YourKit, and JMH, the presentation focuses on ways of measuring performance, how to interpret data, ways of eliminating bottlenecks, and even how to avoid future performance regressions.

The Art Of Performance Tuning - with presenter notes!

Jonathan Ross

10 Years of the Cathedral and the Bazaar

Haggen So

(E book pdf) thinking in patterns with java

Raffaella D'angelo

The Psychology of C# Analysis

Coverity

Tech-ED 2011 Survival Guide

Robert MacLean

Flow based-1994

getdownload

Presented at Lean Day West - Portland, OR. Sept. 17, 2013 How do you take a gigantic organization like PayPal and begin to transform the experiences? Engineering is often the key blocker in being able to achieve a high rate of innovation. In this talk, Bill Scott will give specific examples on implemented Lean UX in a 13,000 person company, re-factored the technology stack and changed the way engineers work with design & product partners. In addition, Bill will provide additional examples that go back to his early days writing one of the first Macintosh games to his more recent work at Netflix and the power of treating the user interface layer as the experimentation layer.

6 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action

Bill Scott

Paul Wilford - From a Small Experiment to a Half a Billion... | Agile Deliver...

Kayleigh Tiernan

Architects and design-org

Kinshuk Adhikary

Chaos Engineering Talk at DevOps Days Austin

matthewbrahms

Life & Work of Butler Lampson | Turing100@Persistent

Persistent Systems Ltd.

Similar to Kernel Recipes 2019 - The ubiquity but also the necessity of eBPF as a technology to keep the kernel relevant (20)

From open source labs to ceo methods and advice by sysfera

Class (1)

Fake It While We Make It (Data-Driven Prototyping)

UI For Alien Cowboys

Rethinking Object Orientation

Agile Engineering for Managers Workshop

Greg Young "Why Event Sourced Systems Fail"

From hello world to goodbye code

Lec 01 introduction

The Art Of Performance Tuning - with presenter notes!

10 Years of the Cathedral and the Bazaar

(E book pdf) thinking in patterns with java

The Psychology of C# Analysis

Tech-ED 2011 Survival Guide

Flow based-1994

6 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action

Paul Wilford - From a Small Experiment to a Half a Billion... | Agile Deliver...

Architects and design-org

Chaos Engineering Talk at DevOps Days Austin

Life & Work of Butler Lampson | Turing100@Persistent

More from Anne Nicolas

Wanting to avoid the Android experience, Google developers always aimed to make their Chrome OS Linux kernels as close to mainline as possible. However, when Chromebooks were first created, Google was left with no choice, the mainline kernel, in some subsystems, still did not have all the functionalities needed by Chromebooks. Hence, similarly to Android, Chrome OS had to develop their own out-of-tree code for the kernel and maintain that for a few different kernel versions. Luckily, over the last few years a strong and consistent effort has been happening to bring Chromebook devices closer to mainline. It has led to significant improvements that now make it possible to run mainline on Chrome OS devices. And not only Chromebooks, as these significant strides are also improving Arm-based SOCs and other key components of the rich Chromebook hardware ecosystem. In this talk, we will look at how and why upstream support for Chromebooks improved, the current status of various models, and what we expect in the future. Enric Balletbò i Serra

Kernel Recipes 2019 - Driving the industry toward upstream first

Anne Nicolas

As the name would suggest, a Non-Maskable Interrupt (NMI) is an interrupt-like feature that is unaffected by the disabling of classic interrupts. In Linux, NMIs are involved in some features such as performance event monitoring, hard-lockup detector, on demand state dumping, etc… Their potential to fire when least expected can fill the most seasoned kernel hackers with dread. AArch64 (aka arm64 in the Linux tree) does not provide architected NMIs, a consequence being that features benefiting from NMIs see their use limited on AArch64. However, the Arm Generic Interrupt Controller (GIC) supports interrupt prioritization and masking, which, among other things, provides a way to control whether or not a set of interrupts can be signaled to a CPU. This talk will cover how, using the GIC interrupt priorities, we provide a way to configure some interrupts to behave in an NMI-like manner on AArch64. We’ll discuss the implementation, some of the complications that ensued and also some of the benefits obtained from it. Julien Thierry

Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMI

Anne Nicolas

At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative. Gustavo A.R. Silva

Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernel

Anne Nicolas

Kernel Recipes 2019 - Metrics are money

Anne Nicolas

The Linux kernel project includes a huge amount of documentation, but that information has seen little in the way of care over the years. The amount of care has increased significantly recently, though, and things are improving quickly. Listen as the kernel’s documentation maintainer discusses the current state of the kernel’s docs, how we got here, where we’re trying to go, and how you can help. Jonathan Corbet

Kernel Recipes 2019 - Kernel documentation: past, present, and future

Anne Nicolas

Modern SoC designs incorporate technologies from numerous vendors, each with their own inconsistent, confusing, undocumented and even contradictory terminology. The result is a mess of acronyms and product names which have a surprising impact on the ability to develop reusable, modular code thanks to the nature of the underlying IP being obscured. This presentation will dive into some of the misnomers plaguing the Arm ecosystem, with the aim of explaining why things are like they are, how they fit together under the architectural umbrella and how you, as a developer, can decipher the baffling ingredients list of your next SoC design! Will Deacon

Embedded Recipes 2019 - Knowing your ARM from your ARSE: wading through the t...

Anne Nicolas

GNU poke is a new interactive editor for binary data. Not limited to editing basic ntities such as bits and bytes, it provides a full-fledged procedural, interactive programming language designed to describe data structures and to operate on them. Once a user has defined a structure for binary data (usually matching some file format) she can search, inspect, create, shuffle and modify abstract entities such as ELF relocations, MP3 tags, DWARF expressions, partition table entries, and so on, with primitives resembling simple editing of bits and bytes. The program comes with a library of already written descriptions (or “pickles” in poke parlance) for many binary formats. GNU poke is useful in many domains. It is very well suited to aid in the development of programs that operate on binary files, such as assemblers and linkers. This was in fact the primary inspiration that brought me to write it: easily injecting flaws into ELF files in order to reproduce toolchain bugs. Also, due to its flexibility, poke is also very useful for reverse engineering, where the real structure of the data being edited is discovered by experiment, interactively. It is also good for the fast development of prototypes for programs like linkers, compressors or filters, and it provides a convenient foundation to write other utilities such as diff and patch tools for binary files. This talk (unlike Gaul) is divided into four parts. First I will introduce the program and show what it does: from simple bits/bytes editing to user-defined structures. Then I will show some of the internals, and how poke is implemented. The third block will cover the way of using Poke to describe user data, which is to say the art of writing “pickles”. The presentation ends with a status of the project, a call for hackers, and a hint at future works. Jose E. Marchesi

Kernel Recipes 2019 - GNU poke, an extensible editor for structured binary data

Anne Nicolas

Operating system distributors often face challenges that are somewhat different from that of upstream kernel developers. For instance, some kernel updates often need to stay at least binary compatible with modules that might be “out of tree” for some time. In that context, being able to automatically detect and analyze changes to the binary interface exposed by the kernel to its module does have some noticeable value. The Libabigail framework is capable of analyzing ELF binaries along with their accompanying debug info in the DWARF format, detect and report changes in types, functions, variables and ELF symbols. It has historically supported that for user space shared libraries and application so we worked to make it understand the Linux kernel binaries. In this presentation, we are going to present the current support of ABI analysis for Linux Kernel binaries, the challenges we face, how we address them and the plans we have for the future. Dodji Seketeli, Jessica Yu, Matthias Männich

Kernel Recipes 2019 - Analyzing changes to the binary interface exposed by th...

Anne Nicolas

Different upgrade and update strategies exist when it comes to embedded Linux system. If at development time none of these strategies have been chosen, adding them afterwards can be tedious task. Even harder it gets when the system is already deployed in the field and only accessible via a 3G connection. This talk is a developer experience of putting in place exactly that. Giving a return of experience on one way of doing it on a system running Barebox and a Yocto-based distribution. Patrick Boettcher

Embedded Recipes 2019 - Remote update adventures with RAUC, Yocto and Barebox

Anne Nicolas

Traditionally graphics drivers were one of the last hold-outs of proprietary software in an embedded Linux system. This situation is changing with open-source graphics drivers showing up for almost all of the graphics acceleration peripherals on the market right now. This talk will show how open-source graphics drivers are making embedded systems less special, as well as trying to provide an overview of the Linux graphics stack, de-mystifying what is often seen as black magic GPU stuff from outside observers. Lucas Stach

Embedded Recipes 2019 - Making embedded graphics less special

Anne Nicolas

This talk will explore Open Source Hardware projects relevant to Linux, including boards like BeagleBone, Olimex OLinuXino, Giant board and more. Looking at the benefits and challenges of designing Open Source Hardware for a Linux system, along with BeagleBoard.org’s experience of working with community, manufacturers, and distributors to create an Open Source Hardware platform. In closing also looking at the future, Libre Silicon like RISC-V designs, and where this might take Linux. Drew Fustini

Embedded Recipes 2019 - Linux on Open Source Hardware and Libre Silicon

Anne Nicolas

The I2C subsystem is not the shiniest part of the Linux Kernel. For embedded devices, though, it is one of the many puzzle pieces which just have to work. Wolfram Sang has the experience of maintaining this subsystem for nearly 7 years now. This talk gives a short overview of how maintaining works in general and specifically in this subsystem. But mainly, it will highlight noteworthy points in the timeline and lessons learnt from that. It will present trends, not so much regarding I2C but more the Linux Kernel and the embedded ecosystem in general. And of course, there will be plenty of anecdotes and bits from behind the scenes for your entertainment. Wolfram Sang

Embedded Recipes 2019 - From maintaining I2C to the big (embedded) picture

Anne Nicolas

ITRenew is selling recertified OCP servers under the Sesame brand, those servers come either with their original UEFI BIOS or with LinuxBoot. The LinuxBoot project is pushing the Linux kernel inside bios flash and using userland programs as bootloader. To achieve quality on our software stack, as any project, we need to test it. Traditional BIOS are tested by hand, this is 2019 we need to do it automatically! We already presented the hardware setup behind the LinuxBoot CI, this talk will focus on the software. We use u-root for our userland bootloader; this software is written in Go so we naturally choose to use Go for our testing too. We will present how we are using and extending the Go native test framework `go test` for testing embedded systems (serial console) and improving the report format for integration to a CI. Julien Viard de Galbert

Embedded Recipes 2019 - Testing firmware the devops way

Anne Nicolas

About 60% of the Linux kernel source tree is devoted to drivers for a large variety of supported hardware components. Especially in the embedded world, the number of different SoC families, versions, and revisions, integrating a myriad of “IP cores”, keeps on growing. In this presentation, Geert will explain how to match drivers against hardware, and how to support a wide variety of (dis)similar devices, without turning platform and driver code into an entangled bowl of spaghetti.tra Starting with a brief history of driver matching in Linux, he will fast-forward to device-tree based matching. He will discuss ways to handle slight variations of the same hardware devices, and different SoC revisions, each with their own quirks and bugs. Finally, Geert will show best practices for evolving device drivers in a maintainable way, based on his experiences as an embedded Linux kernel developer and maintainer. Geert Uytterhoeven

Embedded Recipes 2019 - Herd your socs become a matchmaker

Anne Nicolas

Buildroot is a popular and easy to use embedded Linux build system. It generates, in few minutes, lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs. This talk is about the integration of LLVM/clang into Buildroot. In 2018, Valentin Korenblit, supervised by Romain Naour, worked on this topic during his internship at Smile ECS. After a short introduction about llvm/clang and Buildroot, this talk will go through the numerous issues discovered while adding llvm/clang componants and how these issues were fixed. Romain will also detail the work in progress and the work to be done based on llvm/clang libraries (OpenCL, Compiler-rt, BCC. Chromium, ldd). Romain Naour

Embedded Recipes 2019 - LLVM / Clang integration

Anne Nicolas

This talk introduces JTAG debugging capabilities, both for debugging hardware and software. Marek first explains what the JTAG stands for and explains the operation of the JTAG state machine. This is followed by an introduction to free software JTAG tools, OpenOCD and urJTAG. Marek shortly explains how to debug software using those tools and how that ties into the JTAG state machine. However, JTAG was designed for testing hardware. Marek explains what boundary scan testing (BST) is, what are BSDL files and their format, and practically demonstrates how to blink an LED using BST and only free software tools. Marek Vasut

Embedded Recipes 2019 - Introduction to JTAG debugging

Anne Nicolas

PipeWire is an open source project that aims to greatly improve audio and video handling under Linux. Utilising a fresh design, it bridges use cases that have been previously addressed by different tools – or not addressed at all -, providing ground for building complex, yet secure and efficient, multimedia systems. In this talk, Julien is going to present the PipeWire project and the concepts that make up its design. In addition, he is going to give an update of the current and future work going on around PipeWire, both upstream and in Automotive Grade Linux, an early adopter that Julien is actively working on. Julian Bouzas

Embedded Recipes 2019 - Pipewire a new foundation for embedded multimedia

Anne Nicolas

Ftrace’s most powerful feature is the function tracer (and function graph tracer which is built from it). But to have this enabled on production systems, it had to have its overhead be negligible when disabled. As the function tracer uses gcc’s profiling mechanism, which adds a call to “mcount” (or more recently fentry, don’t worry if you don’t know what this is, it will all be explained) at the start of almost all functions, it had to do something about the overhead that causes. The solution was to turn those calls into “nops” (an instruction that the CPU simply ignores). But this was no easy feat. It took a lot to come up with a solution (and also turning a few network cards into bricks). This talk will explain the history of how ftrace came about implementing the function tracer, and brought with it the possibility of static branches and soon static calls! Steven Rostedt

Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started

Anne Nicolas

Suricata is a network threat detection engine using network packets capture to reconstruct the traffic till the application layer and find threats on the network using rules that define behavior to detect. This task is really CPU intensive and discarding non interesting traffic is a solution to enable a scaling of Suricata to 40gbps and other. This talk will present the latest evolution of Suricata that knows uses eBPF and XDP to bypass traffic. Suricata 5.0 is supporting the hardware XDP to provide ypass with network card such as Netronome. It also takes advantage of pinned maps to get persistance of the bypassed flows. This talk will cover the different usage of XDP and eBPF in Suricata and shows how it impact performance and usability. If development time permit, the talk will also cover AF_XDP and the impact on this new capture method on Suricata. Eric Leblond

Kernel Recipes 2019 - Suricata and XDP

Anne Nicolas

System memory configuration is a transparent operation nowadays, something that we all came to expect to just work out of the box. Still, it does happen behind the scenes every single time we boot our computers. This requires the cooperation of hardware components on the mainboard and on memory modules themselves, as well as firmware code to drive these. While it is possible to just let it happen, having a deeper understanding of how it works makes it possible to access valuable information from the operating system at run-time. I will take you through the history of system memory configuration from the mid 70s to now. We will explore the different types of memory modules, how their configuration data is stored and how the firmware can access them. We will see which problems had to be solved along the way and how they were solved. Lastly we will see how Linux supports reading the memory configuration information and what you can do with that information. Jean Delvare

Kernel Recipes 2019 - Marvels of Memory Auto-configuration (SPD)

Anne Nicolas

More from Anne Nicolas (20)