This presentation will introduce Display Handler, an open source implementation and framework for providing client virtualized display and input handling within Xen. Display Handler provides a modular approach to both graphics virtualization and input multiplexing. Designed from the ground up to be extensible, the base implementation includes a DRM dumb buffer based renderer, though research is being done on integrating Intel’s GVT-g as an alternative renderer. It includes a fully functional input server which can be extended to support different input sources and output formats as well as providing configurable filtering. The base implementation includes a Windows display driver for Windows XP/7 support, a Linux framebuffer driver for basic Linux guest support, and generic QEMU guest support. A Windows 10 guest display driver is in development in addition to a proper Linux DRM guest driver. It was written in C++ and includes a full suite of unit tests. The presentation will cover the motivation behind the development of Display Handler, the overall architecture, and future directions planned for the framework, especially how it can fit in with various other graphics virtualization technologies that are currently under development. Challenges on providing a beneficial user experience in multi-VM workstations will also be discussed.

1. 1
Display Handler
A framework and implementation for client virtualization
interaction
Brendan Kerrigan

2. 2
Outline
• Background
• Goals
• Design
• Features
• Future Goals and Development

3. 3
Background
• Previously we had created a secure
compositing technology on top of
surfman, within XenClientXT
• At the time we were limited to
working within a proprietary solution
• Uncertainty in XCXT’s future
prompted us to find an independent
way forward

4. 4
Background
• Began design in Fall 2013 after a
period of survey and
experimentation with what existed in
open source
• Several prototypes were made:
• Wayland
• Mir
• eGL
• DRM

5. 5
Background
• We settled on a design that would utilize a plugin
architecture
• Minimize damage of lock-in to a specific
technology
• Allow extension by outside parties
• LGPL was the intended license to allow for 3rd
parties to develop proprietary bits that were
interoperable
• Initial support would be based on DRM dumb buffer
rendering

6. 6
Outline
• Background
• Goals
• Design
• Features
• Future Goals and Development

7. 7
Goals
• Modular, testable codebase
• Flexible multi-monitor support
(extended, cloned, pinned,
seamless)
• Intuitive monitor hotplugging
• Lower the surrounding code
necessary to prototype new
rendering technologies

8. 8
Goals
• Up to 6 monitor support
• 4K resolution support
• Minimize the effort in bringing in
new hardware compatibility

9. 9
Outline
• Background
• Goals
• Design
• Features
• Future Goals and Development

10. 10
Architecture
Renderer
Full Screen
Compositor
Input Server
PV Backend
Qemu
Backend
libivc
Guest
Manager
OXT
FB
XPDM
Qemu
DCL
libivc libivc libivc
Dis(play) Man(ager)
Xec XLlibivc

11. 11
Architecture
Renderer
Full Screen
Compositor
Input Server
PV Backend
Qemu
Backend
libivc
Guest
Manager
OXT
FB
XPDM
Qemu
DCL
libivc libivc libivc
Dis(play) Man(ager)
Xec XLlibivc
Interface with the applicable toolstack
module (currently support xec and xl)
Guest
Manager

12. 12
Architecture
Renderer
Full Screen
Compositor
Input Server
PV Backend
Qemu
Backend
libivc
Guest
Manager
OXT
FB
XPDM
Qemu
DCL
libivc libivc libivc
Dis(play) Man(ager)
Xec XLlibivc
Somewhat poorly named, more like a
window manager
Full Screen
Compositor

13. 13
Architecture
Renderer
Full Screen
Compositor
Input Server
PV Backend
Qemu
Backend
libivc
Guest
Manager
OXT
FB
XPDM
Qemu
DCL
libivc libivc libivc
Dis(play) Man(ager)
Xec XLlibivc
Does the final blitting to the scanout buffer
(currently DRM dumb buffer support is
included)
Renderer

14. 14
Architecture
Renderer
Full Screen
Compositor
Input Server
PV Backend
Qemu
Backend
libivc
Guest
Manager
OXT
FB
XPDM
Qemu
DCL
libivc libivc libivc
Dis(play) Man(ager)
Xec XLlibivc
Provides information about currently
available displays’ orientation, resolution,
and layout
Dis(play) Man(ager)

15. 15
Architecture
Renderer
Full Screen
Compositor
Input Server
PV Backend
Qemu
Backend
libivc
Guest
Manager
OXT
FB
XPDM
Qemu
DCL
libivc libivc libivc
Dis(play) Man(ager)
Xec XLlibivc
A new input server module is included, and
this brings the input server back into the
same process space as the rest of the
display code
Input Server

16. 16
Architecture
Input Sources
libinput
vnc
???
Input Server Core
Filter
List
VM
List
QEmu
Input Sinks
Focus
VM
PV KBD
Debug

17. 17
Outline
• Background
• Goals
• Design
• Features
• Future Goals and Development

18. 18
Features
• Three modes of operation
• Extended – in focus VM displays
to all monitors
• Cloned – in focus VM displays to
all monitors, but the image is
replicated on each monitor
• Pinned – VMs can be assigned a
subset of available displays

19. 19
Features
• Extended mode
<VM Switch>

20. 20
Features
• Cloned mode
<VM Switch>

21. 21
Features
• Pinned mode
<VM Switch>

22. 22
Features
• Configure displays from the UIVM

23. 23
Features
• Monitor hotplugging is well
supported
• Optional banner that displays VM
name and administrator
configurable string
• Guest/host resolution independence
(auto scaling if necessary)

24. 24
Features
• Performance impact is small
• With multiple HD videos playing full
screen, CPU utilization sits at ~6%
• Audio processing actually uses
more cycles during ad hoc video
playback testing

25. 25
Features
• Currently works on both OpenXT and
upstream Xen
• Large set of unittests accompany the code
• PV display drivers for Windows 7 and 10
(XPDM and KMDOD)
• LinuxFB PV driver
• Experimental DRM driver for multi-monitor
support in Linux
• Qemu guest support (limited resolution)

26. 26
Outline
• Background
• Goals
• Architecture
• Features
• Future Goals and Development

27. 27
Future Goals and
Development
• Open sourcing effort is beginning shortly
• Building community
• Unit testing coverage will be extended
• Taking suggestions for a name!

28. 28
Future Goals and
Development
• Additional renderers are being investigated:
• Intel GVT-g as a renderer, providing
mediated GPU acceleration using native
Intel drivers in guest
• GL based renderer, providing some
improvement in performance and power
consumption (especially in scaling cases)
• virGL based renderer, providing API
forwarding style 3D graphics to guests

29. 29
Future Goals and
Development
• Extending the input server to support some
more modern input technologies
• Better touch integration and passthrough
• Supporting other sensor data being sent to
guests
• Accelerometer
• GPS
• UI configuration of gesture control