Resource: LCU13 Name: Discussion on ODP – Fastpath networking applications on manycore SoCs Date: 29-10-2013 Speaker: Ola Liljedahl Video: http://www.youtube.com/watch?v=moWTXkbDrCQ

1. Fastpath networking applications
on manycore SoCs
Ola Liljedahl, Oct 29 2013

2. connect.linaro.org
0
20
40
60
80
100
120
140
160
1995 2000 2005 2010 2015 2020 2025 2030
• Super-linear growth in:
o Number of users
o Number of connected
devices
o Number of over the top
(OTT) applications
o Number of protocols (RFC’s)
o Bandwidth usage
o Power consumption of
network infrastructure
What Is Happening
Logos and trademarks are used for illustration only and remain the property of their respective owners.

3. connect.linaro.org
So the Requirements Change
• Need aggregate capacity of multiple processors
• Need hardware accelerations for PPA-efficient execution of
repetitive work
• Need flexibility of software and programmable hardware
• Need familiar programming environments with robust tools
• Need portability between different implementations
• Need high abstraction level
• Need efficient support for virtualisation

4. connect.linaro.org
Example Logical View of Packet Processing Flow
Ingress
interfaces
Scheduler*
Thread
Thread
Thread
Thread
queue
queue
queue
queue
queue
Classifier*
queue
queue
queue
Crypto
Axel
queue
queue
queue
queue
queue
queue
queue
queue
Scheduler/
Shaper*
Egress
interfaces
Wirespeed domain Wirespeed domainNon-wirespeed domain
*May be SW or HW-based

5. connect.linaro.org
The Pillars of OpenDataPlane
" Event Machine
o Event based IO object handling paradigm and application interface
" SoC Abstraction
o Portable API’s for access to HW/SoC resources
" “Bare Metal” Linux
o Minimal overhead and deterministic execution in Linux user space
EM
SoCA
BML
Application

6. connect.linaro.org
Application Interface
• ODP adopts Event Machine, originally developed by Nokia Solutions and
Networks
o Open-ended architecture with reusable concepts
o C-based API’s and implementations
• Event based programming model abstraction for handling IO
o Logical queue based interface
o Supports packet flows, physical and virtual network interfaces, accelerators, SW endpoints
o Events represent different types of data that should be processed: packets, timers, baseband
data, HW notifications, SW messages...
• Supports scheduler based IO model
o Enables scheduling of IO events using different algorithms and knowledge of work in
progress
o Support for both HW and SW based schedulers
• Supports different IO load balancing approaches
o Enables implicit synchronisation and mutual exclusion between threads
• Proven, already half a dozen HW-specific implementations

7. connect.linaro.org
APIs for IO Load Balancing – “Push” model
IO event objects
distributed to queues
Threads
Threads
processing high
bandwidth IO
Threads
sharing latency
critical events
Classifier
Programmable (with
dedicated CPU’s) or
SW Configurable
IO objects pulled
directly from queues
Queues
Thread
handling lower-
frequency
events
O
D
P
A
P
Is

8. connect.linaro.org
APIs for IO Load Balancing – “Pull” model
Queue groups
Scheduler
Threads
Idle/unused thread
Execution objects
associated with queues
IP-fwd
IP-reasm
GTP-U
DPI
RoHC
Exception
Log svc
O
D
P
A
P
Is
IO objects scheduled
on “Pull” requests
Classifier
Programmable
(with dedicated
CPUs) or SW
Configurable
IO event objects
distributed to queues
Name svc

9. connect.linaro.org
SoC Abstraction
• Portable API’s for accelerator access, timer management & delivery...
• Per-operation access functions + queues & events for completion
notification
• Example: crypto (as proposed)
• int odp_crypto_operation(
• struct odp_crypto_session *sesssion,
• odp_packet_buf_desc *pkt_buf_desc, // Data to process
• const struct odp_data_range *cipher_data,
• const struct odp_data_range *hash_data,
• unsigned hash_result_offset,
• caddr_t override_iv_ptr,
• unsigned num_notif, const em_notif_t *notif_tbl // Events and queue-id to use for notification
when operation is complete
• )
o Supports both synchronous and asynchronous operation
o Supports both processor/ISA and loosely coupled accelerator implementations
o Might define an (optional) synchronous blocking function as well as this is a
common usage
• Compression, DPI, timer management potential candidates

10. connect.linaro.org
“Bare Metal” Linux
• Need to dedicate some cores for dataplane applications
" No TLB misses (huge pages for code and data)
" No context switches (one worker thread per HW-thread/core)
" No thread migration (core affinity and isolation)
" No ticks (NOHZ/dynticks)
" No interrupts
" No kernel background tasks
ü Direct HW access from user space
• Can still
ü Launch, supervise and kill applications, dump core
ü Debug and profile applications
ü Perform system calls from the application

11. More about Linaro: http://www.linaro.org/about/
More about Linaro engineering: http://www.linaro.org/engineering/
How to join: http://www.linaro.org/about/how-to-join
Linaro members: www.linaro.org/members
connect.linaro.org