"Session ID: HKG18-102 Session Name: HKG18-102 - DPDK on Arm64 Platforms Speaker: Yi He Track: Networking ★ Session Summary ★ DPDK (Data Plane Development Kit) is a set of libraries and drivers for fast packet processing. Till now, most of DPDK performance statistics available in public are based on x86 platforms. This presentation is going to share some latest info & experience about: \* The test methodologies and scenarios setup on different Arm platforms \* The overview and comparison of DPDK performance statistics in different scenarios & platforms \* The identified factors that may impact the DPDK performance \* The fine tuning & code optimization for DPDK performance on Arm platforms --------------------------------------------------- ★ Resources ★ Event Page: http://connect.linaro.org/resource/hkg18/hkg18-102/ Presentation: http://connect.linaro.org.s3.amazonaws.com/hkg18/presentations/hkg18-102.pdf Video: http://connect.linaro.org.s3.amazonaws.com/hkg18/videos/hkg18-102.mp4 --------------------------------------------------- ★ Event Details ★ Linaro Connect Hong Kong 2018 (HKG18) 19-23 March 2018 Regal Airport Hotel Hong Kong --------------------------------------------------- Keyword: Networking 'http://www.linaro.org' 'http://connect.linaro.org' --------------------------------------------------- Follow us on Social Media https://www.facebook.com/LinaroOrg https://www.youtube.com/user/linaroorg?sub_confirmation=1 https://www.linkedin.com/company/1026961"

1. © 2018 Arm Limited
DPDK on Arm64
Status Review & Plan
Song.zhu@arm.com Yi.He@arm.com Herbert.Guan@arm.com
• 19/03/2018

2. 2 © 2018 Arm Limited
DPDK Overview
• Data Plane Development Kit
• A set of libraries and drivers for fast packet processing
• Runs mostly in Linux userland
• The first supported CPU was Intel x86 and it is now extended to IBM POWER and ARM.
• Major members and contributors
• Several active members on Arm platforms
• Arm is one of the golden members

3. 3 © 2018 Arm Limited
DPDK usage modes
• DPDK framework provides components on which Network applications can be built
• Stack below shows application stack and locations where DPDK is used.
Packet IO
Host I/F VM I/F
Network functions &
Terminations
Server Functions/
Applications
NIC HW
DPDK finds home in this
region where we perform
fast handling of incoming or
outgoing packets
OVS/ VPP VM switching
VPP, TCP termination, FW,
Tunnel Termination, Crypto
etc run in this layer
Nginx, Memcached,
Cloud apps etc

4. 4 © 2018 Arm Limited
DPDK on Arm64 Status
• Multiple active members on Arm platforms
• ARM platform porting & optimization
• Many thanks to Cavium
• Functional verification / enabling
• DTS (DPDK Test Suite) test in bare-metal & virtualization
• Several platforms / NICs selected
• DPDK performance on Arm platforms
• Throughput/Latency perf test on multiple ARM64 platforms and NICs
• x86 platform for reference & cross check
• Use cases setup
• DPDK in container, Nginx with DPDK + mTCP, etc.
Linux
PCI-Express
NIC
Physical Function
PMD driver (PF)
IXIA Traffic Generator
or
DTS Tester

5. 5 © 2018 Arm Limited
DTS & Functional Enabling
• DTS test suites verification & investigation
• All test suites were executed (incl. virtualization suites)
• 60+ patches sent upstream for DPDK/DTS fixes
• Our goal – Identify a set of DTS test cases suitable for Arm
• Ensure DPDK is properly supported on Arm
• But NOT fix all DTS issues
• Use the selected test suites for CI setup on Arm64
• 206 of 432 test cases passed, 226 failed
• Notes:
• Test results retrieved by DPDK 17.11, on Platform Y
48%
52%
DTS Cases Pass Rate
PASS
FAIL
20%
11%
21%12%
20%
16%
Failure Analysis
BUG
DTS
NIC
Intel Only
ENV Setup
Other

6. 6 © 2018 Arm Limited
Arm Internal DPDK CI Setup
Purpose
• Get ready for upcoming public DPDK Lab
• Internal Patch review -- Automatic sanity check for every new patches
• Daily build -- to early detect code/functional broken in master
Status
• 2 Selected platforms with 3 type of NICs
• All DTS Unit test suite cases at beginning
Next plan
• Add more test suites (may need to fix some issues first)
• Copy setup to Linaro lab (with proper HWs)
Jenkins
Gerrit
Node
(run DTS)
Config 1
Tester
DUT
Config 2
Tester
DUT
… more
Tester
DUT
New Commit
Daily Build
SSH Connections
Direct NIC Port Connections

7. 7 © 2018 Arm Limited
Performance summary
• RFC-2544 test
• Single core, bare-metal
• Several factors impacting DPDK performance
• TxD/RxD number
• Glibc & GCC version – newer is better
• Some ARM64 platforms shows better performance
than x86 on some sub-system unit test
• Memcpy, OpenSSL AES
• Profiling – CPU cycle hotspots show notable difference
between different ARM64 platforms
0.0
5.0
10.0
15.0
20.0
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
64 128 256 512 1024 1500
Platform Y RFC-2544 testing (2*10Gbps)
Throughtput - x520 Throughput - x710
Latency (us) - x520 Latency (us) - x710

8. 8 © 2018 Arm Limited
Our plan
Performance
CommunityRoadmap
Functional
• Use case & Virtualization scenarios
investigation
• CI setup with selected platforms, NICs
& DTS test suites
• Architectural enhancements
• Continue with the bare-metal
performance investigation &
optimization, e.g. PMD, Crypto drivers,
etc
• Investigate performance in
virtualization & multi-core scenarios
• Co-work with Linaro on DPDK
• Build up connections with DPDK
community and Arm partners
• Define DPDK on Arm roadmap with
partners

9. 9 © 2018 Arm Limited
Community Collaboration
• CI setup (LEG, need proper platforms)
• Regular sync up inside Linaro (LEG weekly meeting, etc.)
• Contact channel/mailer (e.g. dpdk@linaro.org?)

10. 1010
Thank You
Danke
Merci
谢谢
ありがとう
Gracias
Kiitos
감사합니다
धन्यवाद
‫תודה‬
© 2018 Arm Limited