Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - Andrei Semenov - Vates

•

0 j'aime•521 vues

In cloud computing environments, VMs require fast access to resources like storage and networking. The hardware that the VMs access is implemented in software and/or by passing through a dedicated hardware device. Software-based solutions consume extra CPU cycles, thus resulting in poor performance. Also, these require to expose a device-model to the guest, thus increasing the attack surface. Conversely, hardware passthrough provides better performance and security but can be expensive in terms of the number of physical resources, since each device is dedicated to a single VM. This talk focuses on how Vates is working on sharing hardware resources among VMs by relying on dedicated processors named Data Processing Units (DPU). More precisely, Vates work on offloading Xen hypervisor of storage emulation by relying on Kalray K200 DPU PCIe controllers, a hardware accelerator based on MPPA architecture. ----------------------------------------- The CloudStack Collaboration Conference 2023 took place on 23-24th November. The conference, arranged by a group of volunteers from the Apache CloudStack Community, took place in the voco hotel, in Porte de Clichy, Paris. It hosted over 350 attendees, with 47 speakers holding technical talks, user stories, new features and integrations presentations and more.

Technologie

Enabling DPU HW in XCP-ng
cloud platform environment

Agenda
I. XCP-ng cloud platform
II. I/O devices virtualization
III. Kalray K-200 DPU accelerator
IV. Missing bits K-200  XCP-ng
V. Next
2

XCP-ng cloud platform
➢ XEN Hypervisor based solution
➢ Type I (bare metal) Hypervisor
➢ Pionners of ParaVirtualization
➢ CPU/Memory Efficient Virtualization
➢ Enhanced Security (Reduced TCB,…)
➢ Dom0 – (resources dispatch, VM
mangement
➢ Many ways of I/O Virtualization
3
Hardware
Xen Hypervisor
Dom0
VM
DomU
VM
DomU
VM
DomU
VM
DomU
VM
DomU
VM

I/O devices virtualization
I/O Device Software Emulation :
Trapping accesses to and emulating HW
controller registers (MMIO or IO ports)
Pros & Cons:
+ use of native driver in VM
- poor performances
- complex (i.e. error prone)
- difficult to implement in Hypervisor
- bottleneck effect
4
Ex: ATA interface (PIO mode)
~200 ATA commands

I/O devices virtualization
I/O Device Sharing :
The HW controller context is saved and
restored on some event (scheduling the
VM to run,…)
Pros & Cons:
+ use of native driver in VM
- performances issues
- not appliable to complex devices
- security issues (side-channels,….)
5
DomU#1
VM
DomU#2
VM
DomU#3
VM
framebuffer

I/O devices virtualization
I/O Device Paravirtualizing :
The higher abstraction of device is « offered »
to the VM (client/server model)
Pros & Cons:
+ simple implementation
+ correct performances
- need specific driver in the VM
- bottleneck effect
6
DomU VM
Dom0 VM
Back-end
Driver
Front-end
Driver
Xen Hypervisor
HW
ctrl
r/w block
Ex: Virtual Block Device

I/O devices virtualization
I/O Device Passthrough :
Granting the device access to the VM
Pros & Cons:
+ using of native drivers
+ excellent performances
+ full fonctoinnalities
+ no additional code (no that much)
- costly solution
- difficult to scale
7
DomU VM
NVMe
driver
Xen Hypervisor
Ex: NVMe Device
NVMe Ctrl

8
IDEAL WORLD
➢ provide to the VM an access to HW device (passthrough)
➢ the SW flexibility to specify the device capacities
➢ the solutions that scales in Cloud Environnement
Flexibles HW accelerators: SR-IOV, DPU

Kalray K-200 DPU
K200 :
➢ virtual NVMe controllers based on the top of real
NVMe resources
➢ MPPA VLIW proprietary architecture
➢ PCIe P2P transaction (totally off- loading CPU)
➢ IOMMU to « accomodate » 2 distinct address spaces
➢ higly configurable vNVMe devices (size, raid, …)
➢ user mode driver (VFIO based)
➢ SPDK based toostack to manage virtual NVMe
9
NVMe#1 NVMe#2 NVMe#2
vNVMe#1 …. vNVMe#n

Missing bits to enable K200 integration
10
K200
Driver
NVMe
driver
vNVMe
K200
DomU
Dom0
VFIO
framework
IOMMU
framework
PV IOMMU
driver
Xen Hyp
IOMMU framework
XAPI
toolstack
Major steps & developpements:
➢ XEN IOMMU framework modifications
to export IOMMU acces to guest
➢ Linux kernel PV IOMMU driver
➢ XAPI toolstack to issuing SPDK
commands
➢ Tweaking in DomU
IOMMU

12
THANKS!
Any questions?
You can find me at andrei.semenov@vates.tech

Recommandé

Xen and Client Virtualization: the case of XenClient XT

The Linux Foundation

This talk will discuss the challenges of client virtualization and introduce at a technical level XenClient XT, a security-oriented client virtualization product by Citrix. By describing XenClient XT architecture and features, it will be shown how the unique Xen's design and its support for modern x86 platform hardware can increase security and isolation among VMs. Disaggregation of services provided by the platform will be a key of this talk. It will also be shown how third party software components can provide services to VMs in a secure and controlled way.

Xen revisited

Shahbaz Sidhu

This document summarizes the history and development of the Xen virtualization project. It discusses how Xen addressed the issues with server sprawl and lack of isolation in early operating systems. It describes the benefits of server consolidation and manageability that virtualization provided. It also outlines the different approaches Xen took to virtualizing memory management and network interfaces to improve performance.

Rmll Virtualization As Is Tool 20090707 V1.0

guest72e8c1

Virtualization can be used as a tool for consolidating information systems. There are several common issues that come up with virtualization including ensuring sufficient processor architecture support, network capacity, and dealing with legacy physical hardware. It is important to analyze legacy systems and map application relationships before starting virtualization. Popular hypervisors include KVM, Xen, and OpenVZ. KVM is recommended due to its integration with Linux. Libvirt provides an abstraction layer for different hypervisors. Orchestrators like Enomalism can help manage large virtualized environments through a web interface. Tools were also discussed for snapshotting images, configuring networks, and preventing out-of-memory issues.

RMLL / LSM 2009

Franck_Villaume

Virtualization can be used as a tool for consolidating information systems. There are several common issues to address when starting virtualization including ensuring sufficient processor architecture support, network capacity, and dealing with physical legacy systems. Key steps include analyzing legacy systems and mapping application relationships. Popular hypervisors include KVM, Xen, and OpenVZ. KVM is recommended due to its integration with Linux. Libvirt provides an abstraction layer for different hypervisors. Orchestrators like Enomalism can help manage virtual machines. Useful related tools include those for snapshots, configuration, and dealing with out of memory situations.

5. IO virtualization

Hwanju Kim

This document discusses I/O virtualization and GPU virtualization. It covers: - Two approaches to I/O virtualization: hosted and device driver approaches. Hosted has lower engineering cost but lower performance. - Methods to optimize para-virtualized I/O including split-driver models, reducing data copy costs, and hardware supports like IOMMU and SR-IOV. - Challenges of GPU virtualization including whether to take a low-level virtualization or high-level API remoting approach. API remoting is preferred due to closed and evolving GPU hardware. - Hardware pass-through of GPUs for high performance but low scalability. Industry solutions for remote desktop

Handout2o

Shahbaz Sidhu

Virtualization allows multiple virtual machines to run on a single physical machine. It relies on hardware advances like multi-core CPUs and networking improvements. Virtualization works by either emulating hardware, trapping privileged instructions and emulating them, dynamic binary translation, or paravirtualization where the guest OS is aware it is virtualized. I/O virtualization can emulate devices, use paravirtualized drivers, or directly assign devices to VMs. This enables server consolidation and efficient utilization of resources in cloud computing.

Usenix Invited Talk

webhostingguy

The document discusses the history and future of virtual machines. It summarizes that virtual machines were originally developed in the 1960s for mainframe computers but fell out of favor. Modern virtualization technologies like VMware have enabled running multiple operating systems on commodity hardware simultaneously with good performance. The document outlines VMware's virtualization technology and products, and provides examples of how virtual machines can be used for testing, server consolidation, application compatibility, and security.

Ina Pratt Fosdem Feb2008

The Linux Foundation

The document discusses the goals and roadmap of the Xen Project, an open source hypervisor. The Xen Project aims to build the industry standard hypervisor, maintain high performance, and support multiple CPU types from servers to clients to mobile phones. Virtualization provides benefits like server consolidation and increased manageability. Hardware advances like VT-x/AMD-V further reduce overhead allowing near-zero overhead virtualization. The roadmap includes improvements to security, scalability, client/mobile support, and research into areas like VM streaming and hardware fault tolerance.

Recommandé

Xen and Client Virtualization: the case of XenClient XT

The Linux Foundation

Xen revisited

Shahbaz Sidhu

Rmll Virtualization As Is Tool 20090707 V1.0

RMLL / LSM 2009

5. IO virtualization

Handout2o

Usenix Invited Talk

Ina Pratt Fosdem Feb2008

The Linux Foundation

CIF16/Scale14x: The latest from the Xen Project (Lars Kurth, Chairman of Xen ...

The Linux Foundation

An important facilitator of Unikernel development, Xen Project continues to develop new and interesting technologies to support the needs of the next generation datacenter. Potentially game-changing technologies like Unikernels will never reach their full potential unless the hypervisor they rely on can handle a large number of potentially tiny VMs effectively and efficiently. In this talk, Xen Project Advisory Board Chairman Lars Kurth will discuss some of the major advances in the hypervisor produced in last year's releases (4.5 and 4.6). He will also discuss some of the work in development which could appear in upcoming releases.

ELC21: VM-to-VM Communication Mechanisms for Embedded

Stefano Stabellini

Static partitioning is becoming increasingly common in embedded. A static hypervisor, such as Xen dom0less, is employed to split the hardware resources into multiple domains and run a different OS in each domain. For instance, Linux and Zephyr. Only the simplest static partitioning configurations don't involve any data exchanges between the domains. Often, communication and data exchanges between two or more environments are required to complete the data processing pipeline that implements the target application. However, the VM-to-VM communication mechanisms available in static partitioning configurations are typically more limited compared to general-purpose hypervisors. For example, PV drivers are not available to Xen dom0less domains. This presentation will discuss the need for communication in static partitioning setups and it will present the technical challenges involved in getting traditional communication methods to work, including Xen PV drivers and VirtIO. The talk will also provide simpler alternatives based on shared memory and interrupt notifications to set up domain-to-domain data streams: simpler techniques that are easily exploitable both by Linux and by tiny baremetal applications as well.

Graphics virtualization

The Linux Foundation

Graphics virtualization

The Linux Foundation

Xen Euro Par07

congvc

Raritan Dominion KSX II Customer Presentation

santiagotiprigan

The document discusses the Dominion KSX II, an integrated KVM and serial control appliance that provides secure remote access and control of IT assets. It has features like virtual media, absolute mouse synchronization, AES encryption, dual Ethernet ports, and remote power control. The KSX II offers secure remote management of servers located at branch offices to address issues around lack of control, security threats, and limited IT resources.

Introduction to docker

Christophe Muller

This is the notes of a presentation I gave to our IT dept., people who know a lot about VMs! They include a description of differences betwen a VM and a container, why would someone would want to use Docker, how it works (at 30,000 feet), some hints of what are the hub and orchestration, some Dockerfiles examples: jenkins slave, jenkins master, sinopia server, etc. and finally some new features Docker is going to propose in the future and how I intend to mix Configuration tools, such as Ansible, and Docker.

Cloud os launch journey to cloud os with windows server 2012 r2

Susantha Silva

This document discusses new features in Windows Server 2012 R2 that enhance private cloud capabilities. It introduces Generation 2 virtual machines that improve performance and security. Features like live migration and replication now offer faster speeds. Storage can be resized online and virtual machines cloned live. The document also explores extending private clouds to Microsoft Azure for additional scalability and flexibility across hybrid cloud environments.

virtualization and hypervisors

Gaurav Suri

The document discusses a mid-evaluation of a major project comparing several hypervisors. It will compare Xen, KVM, VMware, and VirtualBox based on their technical differences and performance benchmarks. The benchmarks will test CPU speed, network speed, I/O speed, and performance running various server workloads. This comparison will help determine the best hypervisor for a given virtualization situation. Key factors that will be compared include OS support, security, CPU speed, network speed, I/O speed, and response times.

Virtualization Primer for Java Developers

Richard McDougall

This document provides an overview and summary of key concepts around virtualization that will be covered in more depth at a technical deep dive session, including: - Virtualization capabilities for desktops/laptops and servers including workstation virtualization and server consolidation. - How virtual machines work and the overhead associated with virtualization. - Properties of virtualization like partitioning, isolation, and encapsulation. - Benefits of server virtualization like consolidation, simpler management, and automated resource pooling. - Comparison of "hosted" and vSphere virtualization architectures. - Technologies used in virtualization like binary translation, hardware assistance from Intel VT/AMD-V. - Ability to virtualize CPU intensive applications with

XS Boston 2008 Self IO Emulation

The Linux Foundation

This document describes SIOEMU, a self-IO emulation technique that allows non-x86 operating systems like OpenVMS to run on Xen/ia64 virtual machines. It does so by having a firmware within the domain handle all IO emulation instead of relying on Qemu in the control domain. This makes the domains more flexible and improves performance by avoiding domain scheduling for IO operations. The firmware emulates devices like IDE and network interfaces to provide full system emulation. Initial results show it can run Linux and OpenVMS domains, but ongoing work is needed to support SMP, save/restore, and add support for devices like VGA.

Unikernels: the rise of the library hypervisor in MirageOS

Docker, Inc.

Achieving the Ultimate Performance with KVM

DevOps.com

Building and managing a cloud is not an easy task. It needs solid knowledge, proper planning and extensive experience in selecting the proper components and putting them together. Many companies build new-age KVM clouds, only to find out that their applications & workloads do not perform well. Join this webinar to learn how to get the most out of your KVM cloud and how to optimize it for performance. Join this webinar and learn: Why performance matters and how to measure it properly? What are the main components of an efficient new-age cloud? How to select the right hardware? How to optimize CPU and memory for ultimate performance? Which network components work best? How to tune the storage layer for performance?

Unikernels: Rise of the Library Hypervisor

Anil Madhavapeddy

Device virtualization and management in xen

Lingfei Kong

The document discusses device virtualization and management in the Xen virtualization platform. It covers topics such as Xen architecture, paravirtualization of devices using the backend/frontend model, full virtualization of devices using emulation, and direct assignment of physical devices to virtual machines. The backend driver runs in privileged domain 0 and multiplexes access to the actual device, while the frontend driver runs in unprivileged guest domains. Paravirtualized devices use XenBus and XenStore for communication, while fully virtualized devices rely on emulation with QEMU. Direct device assignment grants physical devices directly to guests without virtualization.

LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR

Vanika Kapoor

Xen virtualization allows multiple virtual machines to run simultaneously on a single physical server. This increases hardware utilization and makes provisioning new servers easier. NFS allows files to be accessed remotely over a network, enabling file sharing between systems. NFS uses RPC to perform file operations like reads, writes and attribute retrieval. It has advantages of flexibility but also security risks if not configured properly. Newer NFS versions aim to improve performance and mandate strong authentication.

OffensiveCon2022: Case Studies of Fuzzing with Xen

Tamas K Lengyel

KF/x is an open-source fuzzing framework that leverages virtual machine introspection to fuzz code running inside virtual machines. It was used to discover vulnerabilities in the Virtio driver in Linux and a heap overflow in the 7z parser of Symantec Endpoint Protection. KF/x allows taking full memory snapshots of a VM, forking it to generate new test cases, and monitoring for crashes or desired program states. This approach found issues that may have otherwise remained undetected due to the lack of proper fuzzing tools for systems running complex, isolated software like antivirus programs.

Fuzzing_with_Xen.pdf

distortdistort

KF/x is an open-source fuzzing framework that leverages virtual machine introspection to fuzz code running inside virtual machines. It was used to discover vulnerabilities in the Virtio and 7zip parsers of Symantec Endpoint Protection. By transplanting VM snapshots between KVM and Xen, it was able to fuzz the Virtio driver, finding no issues. Fuzzing the 7zip parser with KF/x uncovered a controlled heap overflow that was likely silently fixed. The talk demonstrates how KF/x provides a stable, reusable, and extensible foundation for full-system fuzzing of complex software like antivirus products.

Erlang on OSv

Zvi Avraham

This document discusses how to port Erlang and OTP to run on OSv without forking or executing external processes. Erlang ports allow communication with external processes but rely on forking and executing the port executable. As OSv does not support forking or execution, an alternative approach for Erlang ports is needed. Suggested approaches include using linked-in drivers written as shared objects, NIFs, or a custom in-process protocol to communicate with external processes without forking.

PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)

iXsystems

CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue

Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - Andrei Semenov - Vates

Recommandé

Recommandé

Contenu connexe

Similaire à Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - Andrei Semenov - Vates

Similaire à Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - Andrei Semenov - Vates (20)

Plus de ShapeBlue

Plus de ShapeBlue (20)

Dernier

Dernier (20)

Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - Andrei Semenov - Vates