Using Agilio SmartNICs for OpenStack Networking Acceleration

2. © 2016 NETRONOME 2 Agenda • Overview of Traditional OpenStack Networking Options  OVS (with/without connection tracking), Contrail vRouter, SR-IOV, VirtIO… • Issues: Flexibility, Performance, CPU Utilization • Accelerating and Offloading OpenStack Networking using Agilio™ SmartNICs • Results: Features, Throughput, CPU Utilization • Implementation Details: Hardware and Software Architectures • Evolution of OpenStack Networking • Supporting P4, C and eBPF Programmability • Integration Activities • Conclusions

3. © 2016 NETRONOME Traditional OpenStack Networking Options 3 Forwarding / Virtual Switching Technology OpenStack   Control Plane Forwarding Overlays (Tunnels) Security Groups (Microsegmentation, Stateful Firewalling) SR-IOV with regular NIC Vendor specific plugin Limited    E.g. MAC/VLAN  based directing None None

4. © 2016 NETRONOME Traditional OpenStack Networking Options 3 Forwarding / Virtual Switching Technology OpenStack   Control Plane Forwarding Overlays (Tunnels) Security Groups (Microsegmentation, Stateful Firewalling) SR-IOV with regular NIC Vendor specific plugin Limited    E.g. MAC/VLAN  based directing None None OVS ML2 plugin   (optional: SDN controller)   OVN plugin L2 / L3 VXLAN, GRE,   (for OVN) GENEVE  …    Emerging:   NSH (for NFV) OVS 2.5+ can interface to Linux conntrack    (Previously Linux bridge with iptables/nftables)

5. © 2016 NETRONOME Traditional OpenStack Networking Options 3 Forwarding / Virtual Switching Technology OpenStack   Control Plane Forwarding Overlays (Tunnels) Security Groups (Microsegmentation, Stateful Firewalling) SR-IOV with regular NIC Vendor specific plugin Limited    E.g. MAC/VLAN  based directing None None OVS ML2 plugin   (optional: SDN controller)   OVN plugin L2 / L3 VXLAN, GRE,   (for OVN) GENEVE  …    Emerging:   NSH (for NFV) OVS 2.5+ can interface to Linux conntrack    (Previously Linux bridge with iptables/nftables) Contrail vRouter Contrail plugin (via Contrail controller) L2 / L3 MPLS in UDP,   MPLS in GRE,  VXLAN  … Built in, via flow table

6. © 2016 NETRONOME Traditional OpenStack Networking Options 3 Forwarding / Virtual Switching Technology OpenStack   Control Plane Forwarding Overlays (Tunnels) Security Groups (Microsegmentation, Stateful Firewalling) SR-IOV with regular NIC Vendor specific plugin Limited    E.g. MAC/VLAN  based directing None None OVS ML2 plugin   (optional: SDN controller)   OVN plugin L2 / L3 VXLAN, GRE,   (for OVN) GENEVE  …    Emerging:   NSH (for NFV) OVS 2.5+ can interface to Linux conntrack    (Previously Linux bridge with iptables/nftables) Contrail vRouter Contrail plugin (via Contrail controller) L2 / L3 MPLS in UDP,   MPLS in GRE,  VXLAN  … Built in, via flow table Additional features: load balancing, NAT etc.

7. © 2016 NETRONOME OpenStack Networking Options Evaluated 4 Forwarding / Virtual Switching Technology Traditional Approach Agilio™ SmartNIC Accelerated Approach SR-IOV Limited expressiveness to direct traffic to VMs (no support for general match/action rules, tunnel termination, stateful firewalling)   High throughput    No VM migration support Full OVS or vRouter virtual switching incl. tunnel termination, stateless/stateful firewalling   and SR-IOV based data delivery to VMs     High throughput    VirtIO integration, supporting VM migration OVS    and    Contrail vRouter High expressiveness - match/action, tunnels, stateless/stateful firewalling etc.   Limited throughput   High CPU utilization (e.g. 50% of cores) Same expressiveness - match/action, tunnels, stateless/stateful firewalling etc.  Higher throughput (~5x higher)   Lower CPU utilization (~10x lower)

8. © 2016 NETRONOME 5 Agilio™ CX SmartNIC Family • Optimized for standard server based cloud data centers • Low Profile Half Length PCIe form factor, power < 25W • Based on Netronome’s NFP-4xxx silicon (72 C programmable cores, 8 threads each) • 2GB DRAM for lookup tables / state tables (millions of entries) • Dataplane fully implemented in software 1x 40GbE 2x 40GbE2x 10GbE New: 2x 25GbE Also available: Agilio™ LX 2x40G / 1x100G with dual PCIe interfaces, 120 cores, 8GB DRAM…

10. © 2016 NETRONOME 7 OVS Throughput vs. Number of Rules 5 10 15 20 25 30 OVS in Kernel Space OVS in User Space on DPDK 100 Wildcard Rules 1000 Wildcard Rules 10000 Wildcard Rules 64000 Wildcard Rules MillionsofPacketsperSecond 12 CPU Cores 12 CPU Cores OVS Offloaded to Agilio™ CX-4000 1 CPU Core 5X Throughput Improvement + 90% CPU Savings OVS L2/L3 Forwarding to 8 VMs with 64K Flows

12. © 2016 NETRONOME 9 Efficiency: Throughput Per Server CPU Core Throughput with single server CPU core  dedicated to network related processing MillionsofPacketsperSecond • 50x Efficiency vs. Kernel OVS • 20x Efficiency vs. User Mode (DPDK) OVS    => Replace 3-6 racks with 1 rack! Throughput / efficiency similar for Agilio™ Contrail vRouter Benefit for your use case:   search for  “netronome.com roi calculator”

13. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API (Nova, Neutron) Execute Action OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act

14. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act

15. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss

16. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss Miss

17. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API 2 2 OVS userspace agent populates kernel cache (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss Miss

18. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API 2 2 OVS userspace agent populates kernel cache (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) 3 Offload datapath: copy match tables, sync stats 3 OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss Miss

19. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API 2 2 OVS userspace agent populates kernel cache (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) 3 Offload datapath: copy match tables, sync stats 3 4 Flow tracking: per-microflow state learning 4 Self Learning Exact Match  Flow Tracker Miss Hit OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss Miss

20. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API 2 2 OVS userspace agent populates kernel cache (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) 3 Offload datapath: copy match tables, sync stats 3 Conn track FTP SIP 4 Flow tracking: per-microflow state learning 4 Self Learning Exact Match  Flow Tracker Miss Hit OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss Miss

21. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API 2 2 OVS userspace agent populates kernel cache (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) Conn track 3 Offload datapath: copy match tables, sync stats 3 Conn track FTP SIP 4 Flow tracking: per-microflow state learning 4 Self Learning Exact Match  Flow Tracker Miss Hit OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss Miss

22. © 2016 NETRONOME Offload Model: Agilio™ OVS Acceleration 10 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API 2 2 OVS userspace agent populates kernel cache (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) Conn track 3 Offload datapath: copy match tables, sync stats 3 5 Offload connection tracking: synchronize state 5 Conn track FTP SIP 4 Flow tracking: per-microflow state learning 4 Self Learning Exact Match  Flow Tracker Miss Hit OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss Miss

23. © 2016 NETRONOME Agilio™ vRouter Acceleration 11 vRouter Subsystem vRouter Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK vRouter CLI 1 Configuration via controller or CLI: BGP over XMPP (Nova, Neutron) Execute Action vRouter Flow Classify vRouter Forwarding vRouter Forwarding

24. © 2016 NETRONOME vRouter Datapath vRouter Flow Classify Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) vRouter Forwarding vRouter Forwarding Agilio™ vRouter Acceleration 11 vRouter Subsystem vRouter Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK vRouter CLI 1 Configuration via controller or CLI: BGP over XMPP (Nova, Neutron) Execute Action vRouter Flow Classify vRouter Forwarding vRouter Forwarding

25. © 2016 NETRONOME vRouter Datapath vRouter Flow Classify Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) vRouter Forwarding vRouter Forwarding Agilio™ vRouter Acceleration 11 vRouter Subsystem vRouter Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK vRouter CLI 1 Configuration via controller or CLI: BGP over XMPP (Nova, Neutron) Execute Action vRouter Flow Classify vRouter Forwarding vRouter Forwarding 2 2 Offload forwarding: pre-emptively copied

26. © 2016 NETRONOME vRouter Datapath vRouter Flow Classify Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) vRouter Forwarding vRouter Forwarding Agilio™ vRouter Acceleration 11 vRouter Subsystem vRouter Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK vRouter CLI 1 Configuration via controller or CLI: BGP over XMPP (Nova, Neutron) Execute Action vRouter Flow Classify Miss vRouter Forwarding vRouter Forwarding 2 2 Offload forwarding: pre-emptively copied

27. © 2016 NETRONOME vRouter Datapath vRouter Flow Classify Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) vRouter Forwarding vRouter Forwarding Agilio™ vRouter Acceleration 11 vRouter Subsystem vRouter Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK vRouter CLI 1 Configuration via controller or CLI: BGP over XMPP (Nova, Neutron) Execute Action vRouter Flow Classify Miss Miss vRouter Forwarding vRouter Forwarding 2 2 Offload forwarding: pre-emptively copied

28. © 2016 NETRONOME vRouter Datapath vRouter Flow Classify Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) vRouter Forwarding vRouter Forwarding Agilio™ vRouter Acceleration 11 vRouter Subsystem vRouter Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK vRouter CLI 1 Configuration via controller or CLI: BGP over XMPP 3 3 Userspace agent populates kernel tables (Nova, Neutron) Execute Action vRouter Flow Classify Miss Miss vRouter Forwarding vRouter Forwarding 2 2 Offload forwarding: pre-emptively copied

29. © 2016 NETRONOME vRouter Datapath vRouter Flow Classify Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) vRouter Forwarding vRouter Forwarding Agilio™ vRouter Acceleration 11 vRouter Subsystem vRouter Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK vRouter CLI 1 Configuration via controller or CLI: BGP over XMPP 3 3 Userspace agent populates kernel tables (Nova, Neutron) Execute Action 4 Offload flows: copy tables, sync stats 4 vRouter Flow Classify Miss Miss vRouter Forwarding vRouter Forwarding 2 2 Offload forwarding: pre-emptively copied

30. © 2016 NETRONOME vRouter Datapath vRouter Flow Classify Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) vRouter Forwarding vRouter Forwarding Agilio™ vRouter Acceleration 11 vRouter Subsystem vRouter Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK vRouter CLI 1 Configuration via controller or CLI: BGP over XMPP 3 3 Userspace agent populates kernel tables (Nova, Neutron) Execute Action 4 Offload flows: copy tables, sync stats 4 vRouter Flow Classify Miss Miss vRouter Forwarding vRouter Forwarding Notes: - Control protocol: BGP over XMPP - Tunnel protocols: VXLAN, MPLS over UDP / GRE - Distinct policy (flows - ACLs) and forwarding (IP/MPLS) - Flow table is used to implement stateful firewalling 2 2 Offload forwarding: pre-emptively copied

32. © 2016 NETRONOME SmartNIC Firmware: Pre-programmed or Custom • SmartNIC with dynamically downloadable firmware Contrail OVS OpenStack ONOS ODL Linux BSD • OVS / Contrail / Linux eBPF datapath on host can be accelerated by SmartNIC D P D K eBPF

33. © 2016 NETRONOME SmartNIC Firmware: Pre-programmed or Custom Compiler Debugger Run-Time app.P4 app.C Editor • Firmware can be developed in P4 and/or C • TBD how to integrate custom programmed datapaths (implemented in P4 / C etc.) into OpenStack • SmartNIC with dynamically downloadable firmware Contrail OVS OpenStack ONOS ODL Linux BSD • OVS / Contrail / Linux eBPF datapath on host can be accelerated by SmartNIC D P D K eBPF

34. © 2016 NETRONOME 13 SmartNIC Datapath “Worker” Software Architecture • Load balancer distributes each packet to next available thread for optimum throughput • Hardware assisted reordering ensures packet order is maintained • Flow tracker statefully learns / tracks millions of sessions • Matching performed using DRAM-backed tables - capacity > 500k entries • Actions efficiently performed in on-chip memory Parse Match Act Flow Tracker Learn microflows Cache action Datapath Run to completion Load  Balance C Plugin Re-  order Pool of worker threads on flow processing cores net or PCIE net or PCIE = Ring / Work Queue (multi producer / consumer)

35. © 2016 NETRONOME 14 Example: P4 “main” implementing a simple NIC header_type eth_hdr { fields { dst : 48; src : 48; etype : 16; } } header eth_hdr eth; parser start { return eth_parse; } parser eth_parse { extract(eth); return ingress; } action drop_act() { drop(); } action fwd_act(port) { modify_field(standard_metadata.egress_spec, port); } table in_tbl { reads { standard_metadata.ingress_port : exact; } actions { fwd_act; drop_act; } } control ingress { apply(in_tbl); }

36. © 2016 NETRONOME SmartNIC Firmware (P4/C) Example of Fully Customized Datapath (P4 / C) 15 Run-Time Interface Server (x86 - Linux) PCIe Agilio™ SmartNIC Virtual Machine 1 VNF Kernel Mode (C) netdev P4 / C Development Environment Edit - Debug Control App Populate tables, display statistics Security µVNF (C) Timestamp µVNF (C) Latency Stats µVNF (C) Virtual Machine 2 VNF User Mode  (C) DPDK Timestamp µVNF (C) Latency Stats µVNF (C) Match Protocol Meter Other TCP Concepts: • P4 and C running on SmartNIC   implements datapath - e.g.  defines protocols, match / action  behavior • Datapath steers traffic to VNFs  running on x86 server and   on SmartNIC

37. © 2016 NETRONOME SmartNIC Firmware (P4/C) Example of Fully Customized Datapath (P4 / C) 15 Run-Time Interface Server (x86 - Linux) PCIe Agilio™ SmartNIC Virtual Machine 1 VNF Kernel Mode (C) netdev P4 / C Development Environment Edit - Debug Control App Populate tables, display statistics Security µVNF (C) Timestamp µVNF (C) Latency Stats µVNF (C) Virtual Machine 2 VNF User Mode  (C) DPDK Timestamp µVNF (C) Latency Stats µVNF (C) Match Protocol Meter Other TCP Concepts: • P4 and C running on SmartNIC   implements datapath - e.g.  defines protocols, match / action  behavior • Datapath steers traffic to VNFs  running on x86 server and   on SmartNIC To Discuss:   OpenStack  Integration

38. © 2016 NETRONOME Extending OpenStack Networking - P4/C Plugins 16 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API 2 2 OVS userspace agent populates kernel cache (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) 3 Offload datapath: copy match tables, sync stats 3 OVS  Kernel DP Match/Act OVS  Kernel DP Match/Act Miss Miss

39. © 2016 NETRONOME Extending OpenStack Networking - P4/C Plugins 16 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API 1 Configuration via controller, CLI, or Callable API 2 2 OVS userspace agent populates kernel cache (Nova, Neutron) Execute Action Open vSwitch Datapath Execute Action (e.g. Entunnel, Deliver to VM, Send to Port) 3 Offload datapath: copy match tables, sync stats 3 OVS  Kernel DP Match/Act Datapath Extension or Plugin P4 / C in Sandbox DP Ext. 4 Datapath extension software 4 4 OVS  Kernel DP Match/Act Miss Miss

40. © 2016 NETRONOME OVS “on” SmartNIC P4 Datapath 17 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API (Nova, Neutron) Execute Action OVS  Kernel DP Match/Act P4 Matching

41. © 2016 NETRONOME OVS “on” SmartNIC P4 Datapath 17 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API (Nova, Neutron) Execute Action P4 Generated Datapath Execute P4 Action OVS  Kernel DP Match/Act P4 Matching

42. © 2016 NETRONOME OVS “on” SmartNIC P4 Datapath 17 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API (Nova, Neutron) Execute Action P4 Generated Datapath Execute P4 Action OVS  Kernel DP Match/Act P4 Matching Fallback

43. © 2016 NETRONOME OVS “on” SmartNIC P4 Datapath 17 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API (Nova, Neutron) Execute Action P4 Generated Datapath Execute P4 Action OVS  Kernel DP Match/Act P4 Matching Fallback Fallback

44. © 2016 NETRONOME P4 “into” OVS Datapath 18 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API (Nova, Neutron) Execute P4 / OVS Action P4 / OVS Matching P4 / OVS Matching

45. © 2016 NETRONOME P4 “into” OVS Datapath 18 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API (Nova, Neutron) P4 Generated Datapath Execute P4 / OVS Action Execute P4 / OVS Action P4 / OVS Matching P4 / OVS Matching

46. © 2016 NETRONOME P4 “into” OVS Datapath 18 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API (Nova, Neutron) P4 Generated Datapath Execute P4 / OVS Action Execute P4 / OVS Action P4 / OVS Matching P4 / OVS Matching Fallback

47. © 2016 NETRONOME P4 “into” OVS Datapath 18 Open vSwitch Subsystem OVS Agent OpenFlow Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK OVS CLI Callable API (Nova, Neutron) P4 Generated Datapath Execute P4 / OVS Action Execute P4 / OVS Action P4 / OVS Matching P4 / OVS Matching Fallback Fallback

48. © 2016 NETRONOME P4 “instead of” OVS (Datapath and Control) 19 Open vSwitch Subsystem P4 Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK (Nova, Neutron) P4 Generated Datapath Execute P4 Action Execute Action P4 or eBPF or XDP P4 Matching Fallback Fallback 1 Protocol(s) to be discussed (could become callable API)

49. © 2016 NETRONOME P4 “instead of” OVS (Datapath and Control) 19 Open vSwitch Subsystem P4 Agent Virtual Machine Virtual Machine Virtual Machine x86 Kernel x86 Userspace PCIe Virtual Machine SR-IOV / VirtIO VFs SR-IOV / VirtIO VFs Agilio™  SmartNIC Apps Apps 1 netdev or DPDK netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK Apps netdev or DPDK (Nova, Neutron) P4 Generated Datapath Execute P4 Action Execute Action P4 or eBPF or XDP P4 Matching Fallback Fallback 1 Protocol(s) to be discussed (could become callable API) Other open issues: - Downloading programs via  OpenStack or other systems  - Scheduling VMs to run on  nodes with acceleration hardware  (Nova)    Acceleration discussion started at OpenStack Spring 2016 (Austin) 

50. © 2016 NETRONOME 20 Integration / Open Sourcing Activities Area Activities Linux drivers VF netdev (kernel device driver) upstreamed in kernel 4.5 Patches available for older kernels, or use VirtIO Representative netdev (for fallback processing of traffic) proposals imminent FreeBSD drivers Kernel device driver implemented DPDK drivers Poll mode driver upstreamed in DPDK 2.2 Patches available for older DPDK versions Open vSwitch acceleration  integration Hooks for acceleration proposed in October 2014 Iteration in progress OpenStack integration Proposing plugins and agents to support virtual switching acceleration via RFE process Integration for OVS in process - Mirantis, Ericsson CEE etc. Integration for Contrail vRouter in process - Juniper etc.

51. © 2016 NETRONOME 20 Integration / Open Sourcing Activities Area Activities Linux drivers VF netdev (kernel device driver) upstreamed in kernel 4.5 Patches available for older kernels, or use VirtIO Representative netdev (for fallback processing of traffic) proposals imminent FreeBSD drivers Kernel device driver implemented DPDK drivers Poll mode driver upstreamed in DPDK 2.2 Patches available for older DPDK versions Open vSwitch acceleration  integration Hooks for acceleration proposed in October 2014 Iteration in progress OpenStack integration Proposing plugins and agents to support virtual switching acceleration via RFE process Integration for OVS in process - Mirantis, Ericsson CEE etc. Integration for Contrail vRouter in process - Juniper etc. Also: P4 and protocol independent forwarding, switchdev, eBPF / XDP…

52. © 2016 NETRONOME 21 Next Steps • Use Agilio™ SmartNICs with existing dataplanes • Use Agilio™ OVS (with / without Conntrack) • Use Agilio™ vRouter • Future: Agilio™ eBPF/XDP, etc.  • Program Agilio™ SmartNICs • Use APIs (on x86 servers) - with above dataplanes • Program in P4 and/or C (on SmartNIC / on x86)  • Improve performance + free up server resources!

Using Agilio SmartNICs for OpenStack Networking Acceleration

Using Agilio SmartNICs for OpenStack Networking Acceleration

Recommandé

Contenu connexe

Tendances

Tendances(20)

En vedette

En vedette(18)

Similaire à Using Agilio SmartNICs for OpenStack Networking Acceleration

Similaire à Using Agilio SmartNICs for OpenStack Networking Acceleration(20)

Plus de Netronome

Plus de Netronome(20)

Dernier

Dernier(20)

Using Agilio SmartNICs for OpenStack Networking Acceleration