Every modern multimedia-oriented ARM SoC usually has a number of display controllers, to drive a screen or an LCD panel, and a GPU, to provide 3D acceleration. The Linux kernel framework of choice to support these controllers is the DRM subsystem. This talk will walk through the DRM stack, the architecture of a DRM/KMS driver and the interaction between the display and GPU drivers. The presentation is based on the work we have done to develop a DRM driver for the Allwinner SoCs display controller with multiple outputs, such as parallel display interfaces, HDMI or MIPI-DSI. The work done to make the ARM Mali OpenGL driver work on top of a mainline DRM/KMS driver will also be detailed, as well as the more traditional, Mesa-based, solution used in a variety of other platforms. Maxime Ripard, Free Electrons

1. Kernel Recipes 2017
An introduction to the
Linux DRM subsystem
Maxime Ripard
Free Electrons
maxime@free-electrons.com
c Copyright 2004-2017, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!
Embedded Linux
Developers
Free Electrons
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 1/1

2. Maxime Ripard
Embedded Linux engineer and trainer at Free Electrons
Embedded Linux development: kernel and driver development, system integration,
boot time and power consumption optimization, consulting, etc.
Embedded Linux training, Linux driver development training and Android system
development training, with materials freely available under a Creative Commons
license.
http://free-electrons.com
Contributions
Co-maintainer for the sunXi SoCs from Allwinner
Contributor to a couple of other open-source projects, Buildroot, U-Boot, Barebox
Living in Toulouse, south west of France
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 2/1

3. An introduction to the Linux DRM subsystem
Introduction
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 3/1

4. A long long time ago, in a galaxy (not so) far, far away
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 4/1

5. Framebuffers
Display hardware was dead simple...
.. and so was the API to drive it.
Introducing... fbdev!
Allows for three things:
Mode-Setting
Accessing the (only) buffer
Optional 2d acceleration: draw, copy, etc.
And access to the device registers...
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 5/1

6. That 90’s show
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 6/1

7. Back to the future
Two different trends
Embedded devices starting to show up, with their low power needs ⇒ Display
engines need to accelerate more things
Desktop displays getting more and more fancy in order to play Quake in 4k VR ⇒
Bigger and bigger GPUs
Led to two different outcomes:
Interface to drive GPU devices through the kernel: DRM
Hacks piling on in order to fit embedded use-cases: omapdss, pxafb
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 7/1

8. Composition Evolved
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 8/1

9. The droid you’re looking at
Sean Paul and Zach Reizner - Google - XDC2016
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 9/1

10. Can I talk to the manager?
DRM was initially introduced to deal with the GPU’s need
Initialize the card, load its firmware, etc.
Share the GPU command queue between multiple applications
Manage the memory (allocation, access)
But not modesetting!
All the modesetting was in the userspace, especially in X
Race between rendering and modesetting
Only one graphical application that needed to remain there all the time
(Lack of) Abstraction!
Introduction of the Kernel Mode-Setting (KMS) to move the modesetting back
into the kernel
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 10/1

11. Kill it with fire!
Now, fbdev could be implemented on top of KMS...
... Or removed entirely
Call for deprecation in 2012 (Hi Laurent!)
Last fbdev driver merged in 2014
First ARM DRM driver: exynos in 2011
Followed: arm, armada, atmel-hclcdc, fsl-dcu, hisilicon, imx, mediatek, meson,
msm, mxsfb, omapdrm, pl111, rcar-du, rockchip, shmobile, sti, stm, sun4i, tegra,
tve200, etc...
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 11/1

12. Two nodes to go please
Initially, DRM was created for devices that were both displaying and rendering
(your traditionnal PC graphics card).
On embedded devices, it’s never really been like that
the GPU is discrete and comes from a third party
the display engine is usually designed by the SoC vendor
DRM and KMS APIs requiring the same level of privilege, with one master, and
were both exposed on the same device file
Creation of render nodes
Also useful for things like GPGPU, off-screen rendering, more flexible access
control
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 12/1

13. An introduction to the Linux DRM subsystem
DRM/KMS
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 13/1

14. The pixels must flow
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 14/1

15. KMS components
Planes
Image source
Associated with one (or more!) framebuffers
Holds a resized / cropped version of that framebuffer
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 15/1

16. Page flipping
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 16/1

17. KMS components
Planes
Image source
Associated with one (or more!) framebuffers
Holds a resized / cropped version of that framebuffer
CRTCs
Take the planes, and does the composition
Contains the display mode and parameters
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1

18. KMS components
Planes
Image source
Associated with one (or more!) framebuffers
Holds a resized / cropped version of that framebuffer
CRTCs
Take the planes, and does the composition
Contains the display mode and parameters
Encoders
Take the raw data from the CRTC and convert it to a particular format
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1

19. KMS components
Planes
Image source
Associated with one (or more!) framebuffers
Holds a resized / cropped version of that framebuffer
CRTCs
Take the planes, and does the composition
Contains the display mode and parameters
Encoders
Take the raw data from the CRTC and convert it to a particular format
Connectors
Outputs the encoded data to an external display
Handles hotplug events
Reads EDIDs
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1

20. Allwinner display pipeline
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 18/1

21. DRM vs SoC pipeline
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 19/1

22. DRM Stack: KMS
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 20/1

23. DRM Stack: GEM
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 21/1

24. DRM Stack: CMA
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 22/1

25. DRM Stack: PRIME
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 23/1

26. DRM Stack: GPUs
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 24/1

27. An introduction to the Linux DRM subsystem
Vendor solutions...
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 25/1

28. Solutions
The GPU found in most Allwinner SoCs is the Mali-400 from ARM (with a
variable number of cores)
There are two options to support that GPU:
Lima
Reversed engineered proof-of-concept
Triggered the reverse engineering effort of the GPUs (freedreno, etnaviv, etc.)
Development (close to?) stopped three years ago, and then resumed a couple of
monthes ago
ARM-Provided support
Featureful
Two parts: GPL kernel driver and proprietary OpenGL ES implementation
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 26/1

29. DRM Stack: GPU
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 27/1

30. Development
Everything is provided by ARM on their website (if you’re lucky)
On the userspace side, you just need to put the library they provided on your
system
On the driver side, you need to create a platform glue that will deal with:
Memory mapping
Interrupts
Clocks
Reset lines
Power Domains
Basically everything needed for the GPU to operate properly on your SoC
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 28/1

31. X11 integration
We need a DDX (Device Dependent X) driver
xf86-video-modesetting is working on top of KMS and GBM (MESA-defined
user-space API to allocate buffers)
ARM developped xf86-video-armsoc for SoC using a 3rd party GPU (Mali,
PowerVR, Vivante, etc.)
Relies on KMS for the display configuration, driver-specific ioctl for buffer
allocations and vendor-provided OpenGL ES implementation
Just have to write a small glue to use your driver allocator, and give some hints to
X about what your hardware support (hw cursor, vblank, etc.)
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 29/1

32. Questions?
Maxime Ripard
maxime@free-electrons.com
Slides under CC-BY-SA 3.0
http://free-electrons.com/pub/conferences/2017/kr/ripard-drm
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 30/1