1. Unrestricted © Siemens AG 2018
Jan Kiszka | Linaro Connect, March 19, 2018
Partitioning ARM Systems
With the Jailhouse Hypervisor

2. Unrestricted © Siemens AG 2017
Page 2 Corporate Technology
About /me, about this project
• Jan Kiszka <jan.kiszka@siemens.com>
• Member of embedded Linux team at Siemens Corporate Technology
• (In-house) consultant, architect, developer for OSS
• Focus on kernel, real-time, virtualization, embedded build systems
• Upstream contributor
• https://github.com/siemens/jailhouse
• Not a product of Siemens, rather an infrastructure component
• Started as open source project by Siemens
• Published for broader industrial usage and contributions

3. Unrestricted © Siemens AG 2017
Page 3 Corporate Technology
Agenda
Introduction to Jailhouse hypervisor
Current status on ARM
Architectural insights
Future directions
Summary
Discussion

4. Unrestricted © Siemens AG 2017
Page 4 Corporate Technology
Jailhouse: Static Partitioning for Multicore Systems
• Focus on maintaining static partitions
• No scheduling
• 1:1 resource assignment
• (Almost) no device emulation
• Keep runtime code base minimal
• Hard RT properties with minimal overhead
• Enable / simplify safety certification
Design Goals
RTOS /
Bare-
Metal
Hardware
Linux
Core 4Core 3Core 1 Core 2
Jailhouse Hypervisor
Device A Device B Device C Device D
Stahlkocher,
CC BY-SA 3.0
2nd
Linux

5. Unrestricted © Siemens AG 2017
Page 5 Corporate Technology
Boot Process of Jailhouse
Power-On
Boot
Loader
Typical
Hypervisor
Partition 1 OS
Partition n OS
Jailhouse
Boot
Loader
Partition 2 OS
Partition n OS
Partition 2 OS
Power-On Root LinuxLinux
(Yet Another
Boot Loader)
...
...

6. Unrestricted © Siemens AG 2017
Page 6 Corporate Technology
Management Interface via Linux
linux # jailhouse enable system.cell
linux # jailhouse cell create realtime.cell
linux # jailhouse cell load my-cell rtos.bin
linux # jailhouse cell start my-cell
linux # jailhouse cell destroy my-cell
linux # jailhouse cell linux linux.cell kernel -i initrd -d dtb
linux # jailhouse disable

7. Unrestricted © Siemens AG 2017
Page 7 Corporate Technology
Modes of Operation – Trusting Linux?
Linux
Jailhouse
Cell 1
Cell 2 Cell 3
Linux
Jailhouse
Cell 1
Cell 3
Cell 2
Open Model Safety Model
• Linux (root cell) is in control
• Cells not involved in management
decisions
• Sufficient if root cell is trusted
• Linux controls, but...
• Cells can be configured to vote
over management decisions
• Building block for safe operation

8. Unrestricted © Siemens AG 2017
Page 8 Corporate Technology
Jailhouse Status on ARM
ARMv7
• Support for Banana-Pi, Orange-Pi, NVIDIA Jetson TK1,
VExpress, emtrion emCON-RZ/G1x
• Non-upstream: TI Sitara AM572x-EVM
• GICv2 and v3
• SMMU on to-do list
ARMv8
• Support for AMD Seattle, LeMaker HiKey, Xilinx ZynqMP,
NVIDIA Jetson TX1, ESPRESSObin, NXP i.MX8MQ
• Works inside QEMU (via virt machine and GICv3)
It's small
• Currently ~7k lines of code (ARMv8)

9. Unrestricted © Siemens AG 2017
Page 9 Corporate Technology
Architectural Overview
Hypervisor
Hardware
Page MappingPage Allocator
Virtual CPU
IOMMU HW Access Filters
IRQ Controller
Arch. Specifics: Mapping, PCI, Life Cycle, ...
Inter-Cell Communication
PCI Access
Life Cycle Management
MMIO Access
Debug Output
VM, IRQ, Exception Entry UART Output
Minimal libc
Jailhouse Management Tool
/sys/devices/jailhouse /dev/jailhouse
Cell Image Cell
Config
Jailhouse Image
Cell
ConfigCell Image System
Config
Linux Kernel Jailhouse Driver Module

10. Unrestricted © Siemens AG 2017
Page 10 Corporate Technology
Sharing Devices under Jailhouse
Jailhouse
Guest B
Hardware
Core 1 Core 2
Storage
Core 3 Core 4
LAN
Guest A
Shared Memory Device
IRQ
vETH
ivshmem-net
vETH
ivshmem-net
NFS etc.
Open issue: ivshmem (v2.0) vs. vhost-pci (virtio)

11. Unrestricted © Siemens AG 2017
Page 11 Corporate Technology
Secure Boot with Jailhouse – Static Chain
Boot
Loader
Partition n OS
Partition 2 OS
Power-On
Full-featured
Linux
Minimal Linux
(kernel + initrd
with Jailhouse)
...
Jailhouse
• Simple model, feasible with all architectures
• Prevents undesired hardware access of full-featured Linux
• To-do: cell image validation by Jailhouse (if not part of initrd)

12. Unrestricted © Siemens AG 2017
Page 12 Corporate Technology
Ongoing Developments
Generated demo & testing images
• WiP at https://github.com/siemens/jailhouse-images
• Currently generates Debian x86 image for QEMU/KVM
• Allows easy exploration of Jailhouse “look & feel”
• Planned next: ARM64 QEMU image
• Then: reference board images
Speculation barriers
• Already well isolated in static setups
• Further isolate cells inside the hypervisor
→ CPU-local memory views
• Prototype exists for x86, to be extended to ARM now

13. Unrestricted © Siemens AG 2017
Page 13 Corporate Technology
Future Developments
Configuration format
• Binary format optimized for runtime usage → should remain
• Source format currently C structure → should be improved
• Device Tree? Also on x86?
• Custom YAML description?
Non-Linux root cells
• Straightforward with many RTOSes
• Catch: we need stable & versioned hypervisor boot interface
Early partitioning
• Create cells via boot loader or EFI helper
• Cell reload / restart during runtime without root cell?
Clock partitioning
• Provide infrastructure to help with moderating clock access
• Avoid clock driver reimplementations in hypervisor → firmware service?

14. Unrestricted © Siemens AG 2017
Page 14 Corporate Technology
Why Jailhouse?
• Designed for real-time
• Full CPU isolation
• Minimal I/O latencies
• Designed for safety & security
• No emulation, no scheduling, minimal interfaces
• Target code size: <10k LOC/arch (runtime even smaller)
• Safety certification under preparation (waiting for safe hardware)
• Designed as true Open Source
• GPLv2, public for 4.5 years
• Active community, including CPU vendors
• Could eventually make into the kernel

15. Page 15
Thank you!
Jan Kiszka <jan.kiszka@siemens.com>