Osdc2014 openstack networking yves_fauser

1. OpenStack Networking Overview of the networking challenges and solu5ons in OpenStack Yves Fauser Network Virtualiza5on Pla>orm System Engineer @ VMware OSDC 2014, Berlin, 08-‐10.04

2. The perfect storm hOp://en.wikipedia.org/wiki/File:Hurricane_Isabel_from_ISS.jpg

3. The perfect storm => •  Very feature rich vSwitch (Tunneling, QoS, monitoring & management, automated control through OpenFlow and OVSDB) •  Part of the Linux Kernel since 3.3 => •  OpenFlow and OVSDB (RFC 7047) is used between OpenVSwitch and external controllers •  Numerous OpenSource and Commercial controllers emerged in the last years •  Examples; NOX, Beacon, Floodlight, OpenDayLight, VMware NSX, Big Controller, NEC, etc. => •  OpenStack drives the need for ﬂexible and fast network deployment models •  The OpenStack Neutron Project oﬀers a network abstrac5on that enables OpenSource Projects and commercial implementa5ons to innovate with and for OpenStack

4. Open vSwitch

5. Open vSwitch Features vs. Linux-‐Bridge Feature Open vSwitch Linux Bridge MAC Learning Bridge X X VLAN support (802.1Q) X (na5ve in OVS) using ‘vlan’ Sta5c Link Aggrega5on (LAG) X (na5ve in OVS) using ‘ifenslave’ Dynamic Link Aggrega5on (LACP) X (na5ve in OVS) using ‘ifenslave’ Support for MAC-‐in-‐IP encapsula5on (GRE, VXLAN, …) X (na5ve in OVS) VXLAN support in 3.7 Kernel + iproute2 Traffic capturing / SPAN (RSPAN with encap. into GRE) X (na5ve in OVS) Using advanced traffic control Flow monitoring (NetFlow, sFlow, IPFIX, …) X (na5ve in OVS) e.g. using ipt_ne>low External management interfaces (OpenFlow & OVSDB) X Mul5ple-‐Table forwarding pipeline with flow-‐caching engine X Performance improvements (e.g. RSS Support) X hOp://openvswitch.org/features/ hOps://github.com/homework/openvswitch/blob/master/WHY-‐OVS

6. br-‐tun (flow tables) Linux IP stack + rouDng table 192.168.10.1 WEB WEB APP APP Config/State DB ovsdb-‐server ovs-‐vswitchd eth0 MGMT eth1 kernel user Tunnel Ports (to Linux IP Stack) br-‐int (flow tables) Open vSwitch (OVS) Configura5on Data Interface (ovsdb, CLI, …) Flow Data Interface (OpenFlow, CLI, … Transport Network Flows

7. Br-‐0 (flow tables) Linux IP stack + rouDng table 192.168.10.1 WEB WEB APP APP Config/State DB ovsdb-‐server ovs-‐vswitchd eth0 MGMT eth1 kernel user Flows & Tunnel Ports (to Linux IP Stack) br-‐int (flow table) Open vSwitch with a controller cluster Transport Network TCP 6633 OpenFlow TCP 6632 OVSDB Controller Cluster

8. Common misconcepEons with regards to controllers §  Misconcep5on 1) Traffic will flow through the controller cluster, un5l a specific flow is installed in the switch through OpenFlow §  It depends! §  Most architectures don’t send any traffic to the controller (e.g. VMware NSX doesn’t do it) §  In some architectures, where address space is limited (e.g. CAM/TCAM in low end ToR Switches), the controller gets the first few data packets, and then installs a flow in the Hardware. This is usually not the case when controlling OVS, as OVS holds the Tables in the Hypervisors Memory (and there is plenty!) §  Misconcep5on 2) The controller is a single point of failure §  Controllers are usually deployed as scale out clusters §  Depending on the chosen architecture, even a complete controller cluster outage doesn’t affect traffic forwarding

9. OpenFlow and Controller based Networks

10. MulEple incarnaEons of SDN So what is SDN? It depends on the were you stand! hOp://upload.wikimedia.org/wikipedia/commons/f/f8/Blind_men_and_elephant3.jpg

11. Data plane •  Hardware speciﬁc •  Bound by ASIC/TCAM limits in physical devices Control plane •  Distributed protocols used • OSPF, STP, etc. •  Populates the data plane with forward. entries Internal API §  The core concept of OpenFlow is control and data plane separa5on §  There are heated debates if the use of “Hybrid” approaches qualify for being “real SDN” §  The purist point of view is; Without the clear separa5on of control and data plane, one should not call his solu5on an “SDN solu5on” SDN deﬁned – Control / Data plane separaEon

12. Data plane •  Hardware speciﬁc •  Bound by ASIC/TCAM limits in physical devices Control plane Internal API Control plane •  Central management of forwarding tables •  Populates the data plane with forwarding entries using OpenFlow as an external “southbound” interface OpenFlow Controller §  The core concept of OpenFlow is control and data plane separa5on §  There are heated debates if the use of “Hybrid” approaches qualify for being “real SDN” §  The purist point of view is; Without the clear separa5on of control and data plane, one should not call his solu5on an “SDN solu5on” SDN deﬁned – Control / Data plane separaEon

13. SDN Controllers “Landscape” (incomplete list) OpenSource Controllers Commercial Controllers •  C++ and Phython controllers open sourced by Nicira •  NOX was the ﬁrst controller in the ‘market’ hOp://www.noxrepo.org •  Commercial con5nua5on of NOX with a focus on “Network virtualiza5on” using Overlays hOp://www.projec>loodlight.org •  Java based controller •  Focused to enable ‘apps’ to evolve independently of the control plane func5on •  Backed by BigSwitch Networks Engineers •  Commercial version of Floodlight controller by BigSwitch Networks with a focus on OpenFlow controlled Switch Fabrics hOps://openﬂow.stanford.edu/display/Beacon/Home •  First Java based controller •  Basis of Floodlight hOp://www.opendaylight.org •  Java based controller •  “community-‐led, open, industry-‐supported framework” hOp://yuba.stanford.edu/~casado/of-‐sw.html And a lot more @: etc …

14. Network VirtualizaEon, an “SDN ApplicaEon”

15. What are the key components of network virtualization?!

16. Network VirtualizaEon – A technical definiEon Network virtualiza5on is: §  A reproduc5on of physical networks: §  Q: Do you have L2 broadcast / mul5cast, so apps do not need to be modified? §  Q: Do you have the same visibility and control over network behavior? §  A fully isolated environment: §  Q: Could two tenants decide to use the same RFC 1918 private IP space? §  Q: Could you clone a network (IPs, MACs, and all) and deploy a second copy? §  Physical network loca5on independent: §  Q: Can two VMs be on the same L2 logical network, while in different physical L2 networks? §  Q: Can a VM migrate without disrup5ng its security policies, packet counters, or flow state? §  Physical network state independent: §  Q: Do physical devices need to be updated when a new network/workloads is provisioned? §  Q: Does the applica5on depend on a feature in the physical switch specific to a vendor? §  Q: If a physical device died and was replaced, would applica5on details need to be known? §  Network virtualiza5on is NOT: §  Running network func5onality in a VM (e.g., Router or Load-‐balancer VM)

17. OpenStack Projects & Networking

18. Some of the Integrated (aka ‘Core’) projects Image repo (glance) Object Storage (Swix) Network (Neutron) Block Storage (cinder) Iden5ty (keystone) Dashboard (horizon) Provides UI for other projects Provides AuthenDcaDon and Service Catalog for other Projects Compute (nova) Provides Images Stores Images as Objects Provides volumes Provides network connecDvity

19. OpenStack Networking before Neutron nova-api (OS,EC2,Admin) nova-console (vnc/vmrc) nova-compute Nova DB nova-scheduler nova- consoleauth Hypervisor (KVM, Xen, etc.) Queue nova-cert Libvirt, XenAPI, etc. nova-metadata §  Nova has its own networking service – nova-‐network. It was used before Neutron §  Nova-‐network is s5ll present today, and can be used instead of Neutron nova-network nova-volume Network-Providers (Linux-Bridge or OVS with brcompat, dnsmasq, IPTables) Volume-Provider (iSCSI, LVM, etc.) §  Nova-‐network does -‐ §  base L2 network provisioning through Linux Bridge (brctl) §  IP Address management for Tenants (in SQL DB) §  conﬁgure DHCP and DNS entries in dnsmasq §  conﬁgure fw-‐policies and NAT in IPTables (nova-‐compute) §  Nova-‐network only knows 3 basic Network-‐Models; §  Flat & Flat DHCP – direct bridging of Instance to external eth. Interface with and w/o DHCP §  VLAN based – Every tenant gets a VLAN, DHCP enabled Inspired by

20. Nova-‐Networking – Drawbacks that lead to develop Neutron §  Nova-‐Networking is missing an well defined API for consuming networking services (tenant API for defined topologies and addresses) §  Nova-‐Networking only allows for the 3 simple models; Flat, Flat/DHCP and VLAN/DHCP, all of those are limited in scale and flexibility – e.g. max. 4094 VLAN ID limit §  Closed solu5on; No ability to use network services from 3rd par5es and/or to integrate with Network vendors or overcome the limita5ons of Nova-‐Network §  No support for: §  Advanced Open vSwitch features like Network Virtualiza5on (IP-‐Tunnels instead of VLANs) §  Mul5ple user configurable networks per project §  User configurable routers (L3 Devices)

21. OpenStack Neutron – Plugin Concept Neutron   Core API" Neutron Service (Server)" " •  L2 network abstrac5on deﬁni5on and management, IP address management •  Device and service aOachment framework •  Does NOT do any actual implementa5on of abstrac5on " Plugin API" " Vendor/User Plugin" •  Maps abstrac5on to implementa5on on the Network (Overlay e.g. NSX or physical Network) •  Makes all decisions about *how* a network is to be implemented •  Can provide addi5onal features through API extensions. •  Extensions can either be generic (e.g. L3 Router / NAT), or Vendor Speciﬁc " Neutron  API Extension" Extension API implementa5on is op5onal

22. Core and service plugins §  Core plugin implement the “core” Neutron API func5ons (l2 Networking, IPAM, …) §  Service plugins implements addi5onal network services (l3 rou5ng, Load Balancing, Firewall, VPN) §  Implementa5ons might choose to implement relevant extensions in the Core plugin itself Neutron   Core API" Function" Core " L3 " FW " Core " L3 " FW " Core " L3 " FW " Plugin" Core Plugin " Core Plugin " FW plugin " Core Plugin " FW plugin " L3 plugin "

23. OpenStack Neutron – Plugin locaEons ! # cat /etc/neutron/neutron.conf | grep "core_plugin"! core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin! ! # cat /etc/neutron/neutron.conf | grep "service_plugins”! service_plugins = neutron.services.l3_router.l3_router_plugin.L3RouterPlugin! ! ! # ls /usr/share/pyshared/neutron/plugins/! bigswitch cisco embrane __init__.py metaplugin ml2 nec openvswitch ryu! brocade common hyperv linuxbridge midonet mlnx nicira plumgrid! ! # ls /usr/share/pyshared/neutron/services/! firewall __init__.py l3_router loadbalancer metering provider_configuration.py service_base.py vpn" "

24. OpenStack Neutron – Modular Plugins §  Before the modular plugin (ML2), every team or vendor had to implement a complete plugin including IPAM, DB Access, etc. §  The ML2 Plugin separates core func5ons like IPAM, virtual network id management, etc. from vendor/implementa5on speciﬁc func5ons, and therefore makes it easier for vendors not to reinvent to wheel with regards to ID Management, DB access … §  Exis5ng and future non-‐modular plugins are called “monolithic” plugins §  ML2 calls the management of network types “type drivers”, and the implementa5on speciﬁc part “mechanism drivers” Arista Cisco Linux Bridge OVS etc. Mechanism Drivers" GRE VLAN VXLAN etc. Type Drivers" Type Manager" Mechanism Manager " ML2 Plugin & API Extensions"

25. OpenStack Neutron ML2 – locaEons ! # cat /etc/neutron/plugins/ml2/ml2_conf.ini | grep type_drivers! # the neutron.ml2.type_drivers namespace.! # Example: type_drivers = flat,vlan,gre,vxlan! type_drivers = gre! ! # cat /etc/neutron/plugins/ml2/ml2_conf.ini | grep mechanism_drivers! # to be loaded from the neutron.ml2.mechanism_drivers namespace.! # Example: mechanism_drivers = arista! # Example: mechanism_drivers = cisco,logger! mechanism_drivers = openvswitch,linuxbridge! ! ! # ls /usr/share/pyshared/neutron/plugins/ml2/drivers/! cisco l2pop mechanism_ncs.py mech_hyperv.py mech_openvswitch.py type_gre.py type_tunnel.py type_vxlan.py __init__.py mech_agent.py mech_arista mech_linuxbridge.py type_flat.py type_local.py type_vlan.py! "

26. Some of the Plugins available in the market (1/2) §  ML2 modular Plugin §  With support for the type drivers: local, ﬂat, VLAN, GRE, VXLAN §  And the following mechanism drivers: Arista, Cisco Nexus, Hyper-‐V Agent, L2 Popula5on, Linuxbridge, Open vSwitch Agent, Tail-‐f NCS §  Open vSwitch Plugin – The most used (Open Source) plugin today §  Supports GRE based Overlays, NAT/Security groups, etc. §  Depreca5on planned for Icehouse release in favor of ML2 §  Linuxbridge Plugin §  Limited to L2 func5onality, L3, ﬂoa5ng IPs and provider networks. No support for Overlays §  Depreca5on planned for Icehouse release in favor of ML2

27. Some of the Plugins available in the market (2/2) §  VMware NSX (aka Nicira NVP) Plugin §  Network Virtualiza5on solu5on with centralized controller + OpenVSwitch §  Cisco UCS / Nexus 5000 Plugin §  Provisions VLANs on Nexus 5000 switches and on UCS Fabric-‐Interconnect as well as UCS B-‐Series Servers network card (palo adapter) §  Can use GRE and only conﬁgure OVS, but then there’s no VLAN provisioning §  NEC and Ryu Plugin §  Openﬂow Hop-‐by-‐Hop implementa5ons with NEC or Ryu controller §  Other plugins include Midokura, Juniper (Contrail), Big Switch, Brocade, Plumgrid, Embrane, Melanox §  LBaaS Service Plugins from; A10 and Citrix §  This List can only be incomplete, please check the latest informa5on on: §  hOps://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers §  hOp://www.sdncentral.com/openstack-‐neutron-‐quantum-‐plug-‐ins-‐comprehensive-‐list/

28. New Plugins / ML2 Drivers in Icehouse Release §  New ML2 Mechanism Drivers: §  Mechanism Driver for OpenDaylight Controller §  Brocade ML2 Mechanism Driver for VDX Switch Cluster §  New Neutron Plugins §  IBM SDN-‐VE Controller Plugin §  Nuage Networks Controller Plugin §  Service Plugins §  Embrane and Radware LBaaS driver §  Cisco VPNaaS driver §  Various §  VMware NSX -‐ DHCP and Metadata Service §  This list is incomplete, please see here for more details: hOps://blueprints.launchpad.net/neutron/icehouse

29. Neutron –OVS Agent Architecture §  The following components play a role in OVS Agent Architecture §  Neutron-‐OVS-‐Agent: Receives tunnel & flow setup informa5on from OVS-‐Plugin and programs OVS to build tunnels and to steers traffic into those tunnels §  Neutron-‐DHCP-‐Agent: Sets up dnsmasq in a namespace per configured network/subnet, and enters mac/ip combina5on in dnsmasq dhcp lease file §  Neutron-‐L3-‐Agent: Sets up iptables/rou5ng/NAT Tables (routers) as directed by OVS Plugin or ML2 OVS mech_driver §  In most cases GRE or VXLAN overlay tunnels are used, but flat and vlan modes are also possible IP Stack Neutron-‐ Network-‐Node nova-‐compute hypervisor VM VM IP Stack Compute Node nova-‐compute hypervisor VM VM Compute Node External Network (or VLAN) WAN/ Internet iptables/ rouDng Layer 3 Transport Network dnsmasq NAT & floaDng -‐IPs iptables/ rouDng N.-‐L3-‐Agent N.-‐DHCP-‐Agent N.-‐OVS-‐Agent ovsdb/ ovsvsd Neutron-‐Server + OVS-‐Plugin N.-‐OVS-‐Agent N.-‐OVS-‐Agent ovsdb/ ovsvsd ovsdb/ ovsvsd Layer 3 Transport Net. IP Stack br-‐int br-‐int br-‐tun br-‐int br-‐tun br-‐tun L2 in L3 (GRE) Tunnel dnsmasq br-‐ex

30. §  Centralized scale-‐out controller cluster controls all Open vSwitches in all Compute-‐ and Network Nodes. It configures the tunnel interfaces and programs the flow tables of OVS §  NSX L3 Gateway Service (scale-‐out) is taking over the L3 rou5ng and NAT func5ons §  NSX Service-‐Node relieves the Compute Nodes from the task of replica5ng broadcast, unknown unicast and mul5cast traffic sourced by VMs §  Security-‐Groups are implemented na5vely in OVS, instead of iptables/ebtables IP Stack Neutron-‐ Network-‐Node nova-‐compute hypervisor VM VM IP Stack Compute Node nova-‐compute hypervisor VM VM Compute Node Management Network WAN/ Internet dnsmasq N.-‐DHCP-‐Agent ovsdb/ ovsvsd Neutron-‐Server + NVP-‐Plugin ovsdb/ ovsvsd ovsdb/ ovsvsd Layer 3 Transport Net. IP Stack br-‐int br-‐int br-‐0 br-‐int br-‐0 br-‐0 L2 in L3 (STT) Tunnel dnsmasq Using “SDN controllers” VMware NSX Plugin example NSX L3GW + NAT Layer 3 Transport Network NSX Controller Cluster NSX Service-‐Node

31. hOps://www.ﬂickr.com/photos/17258892@N05/2588347668/lightbox/

32. Thank You! And have a great Conference OSDC 2014, Berlin, 08-‐10.04

33. Backup Slides

35. Neutron – OVS – Tunnel Structure §  This output shows the OVS conﬁg on the OpenStack Network-‐Node before any logical network has been conﬁgured # ovs-vsctl show! 09d5b89a-600d-4da3-b761-11206456385a! Bridge br-ex! Port br-ex! Interface br-ex! type: internal! Port "eth2"! Interface "eth2"! Bridge br-tun! Port br-tun! Interface br-tun! type: internal! Port patch-int! Interface patch-int! type: patch! options: {peer=patch-tun}! Port "gre-172.16.0.11"! Interface "gre-172.16.0.11"! type: gre! options: {in_key=flow, local_ip="172.16.0.10", out_key=flow, remote_ip="172.16.0.11"}! Port "gre-172.16.0.12"! Interface "gre-172.16.0.12"! type: gre! options: {in_key=flow, local_ip="172.16.0.10", out_key=flow, remote_ip="172.16.0.12"}! Bridge br-int! Port patch-tun! Interface patch-tun! type: patch! options: {peer=patch-int}! Port br-int! Interface br-int! type: internal! ovs_version: "1.10.2"!

36. Neutron – OVS – Tunnel Structure §  This output shows the OVS conﬁg on the OpenStack Network-‐Node before any logical network has been conﬁgured # ovs-vsctl show! 09d5b89a-600d-4da3-b761-11206456385a! Bridge br-ex! Port br-ex! Interface br-ex! type: internal! Port "eth2"! Interface "eth2"! Bridge br-tun! Port br-tun! Interface br-tun! type: internal! Port patch-int! Interface patch-int! type: patch! options: {peer=patch-tun}! Port "gre-172.16.0.11"! Interface "gre-172.16.0.11"! type: gre! options: {in_key=flow, local_ip="172.16.0.10", out_key=flow, remote_ip="172.16.0.11"}! Port "gre-172.16.0.12"! Interface "gre-172.16.0.12"! type: gre! options: {in_key=flow, local_ip="172.16.0.10", out_key=flow, remote_ip="172.16.0.12"}! Bridge br-int! Port patch-tun! Interface patch-tun! type: patch! options: {peer=patch-int}! Port br-int! Interface br-int! type: internal! ovs_version: "1.10.2"! ! # Interface to first compute node! ! Port "gre-172.16.0.11"! Interface "gre-172.16.0.11"! type: gre! options: ! {in_key=flow, local_ip="172.16.0.10", out_key=flow, remote_ip="172.16.0.11"}! ! # Interface to second compute node! ! Port "gre-172.16.0.12"! Interface "gre-172.16.0.12"! type: gre! options: ! {in_key=flow, local_ip="172.16.0.10", out_key=flow, remote_ip="172.16.0.12"}!

37. Neutron – OVS – Tunnel Structure §  This output shows the OVS conﬁg on the OpenStack Network-‐Node before any logical network has been conﬁgured # ovs-vsctl show! 09d5b89a-600d-4da3-b761-11206456385a! Bridge br-ex! Port br-ex! Interface br-ex! type: internal! Port "eth2"! Interface "eth2"! Bridge br-tun! Port br-tun! Interface br-tun! type: internal! Port patch-int! Interface patch-int! type: patch! options: {peer=patch-tun}! Port "gre-172.16.0.11"! Interface "gre-172.16.0.11"! type: gre! options: {in_key=flow, local_ip="172.16.0.10", out_key=flow, remote_ip="172.16.0.11"}! Port "gre-172.16.0.12"! Interface "gre-172.16.0.12"! type: gre! options: {in_key=flow, local_ip="172.16.0.10", out_key=flow, remote_ip="172.16.0.12"}! Bridge br-int! Port patch-tun! Interface patch-tun! type: patch! options: {peer=patch-int}! Port br-int! Interface br-int! type: internal! ovs_version: "1.10.2"! ! # Patch from br-tun table to br-int table! ! Port patch-int! Interface patch-int! type: patch! options: {peer=patch-tun}! ! # patch from br-int table to br-tun table! ! Port patch-tun! Interface patch-tun! type: patch! options: {peer=patch-int}!

38. Neutron – Internal Network CreaEon §  Now we will create a logical L2 network, without any subnet assigned to it # neutron net-create Internal-Network! ! Created a new network:! +---------------------------+--------------------------------------+! | Field | Value |! +---------------------------+--------------------------------------+! | admin_state_up | True |! | id | 56a76117-8910-4d85-b91d-8e6842e0a510 |! | name | Internal-Network |! | provider:network_type | gre |! | provider:physical_network | |! | provider:segmentation_id | 1 |! | shared | False |! | status | ACTIVE |! | subnets | |! | tenant_id | b1178a03969b4f638937f5a632fb547a |! +---------------------------+--------------------------------------+! ! # neutron net-list! +--------------------------------------+------------------+---------+! | id | name | subnets |! +--------------------------------------+------------------+---------+! | 56a76117-8910-4d85-b91d-8e6842e0a510 | Internal-Network | |! +--------------------------------------+------------------+---------+!

39. Neutron – Internal Subnet CreaEon §  Now we will create and aOach a new Subnet to the L2 network, without any subnet assigned to it # neutron subnet-create Internal-Network --name Internal-Subnet 10.12.13.0/24! Created a new subnet:! +------------------+------------------------------------------------+! | Field | Value |! +------------------+------------------------------------------------+! | allocation_pools | {"start": "10.12.13.2", "end": "10.12.13.254"} |! | cidr | 10.12.13.0/24 |! | dns_nameservers | |! | enable_dhcp | True |! | gateway_ip | 10.12.13.1 |! | host_routes | |! | id | b4c95b8b-65a4-402e-8359-69b55d6c9bf1 |! | ip_version | 4 |! | name | Internal-Subnet |! | network_id | 56a76117-8910-4d85-b91d-8e6842e0a510 |! | tenant_id | b1178a03969b4f638937f5a632fb547a |! +------------------+------------------------------------------------+! ! # neutron subnet-list -c id -c cidr -c name! +--------------------------------------+----------------+-----------------+! | id | cidr | name |! +--------------------------------------+----------------+-----------------+! | b4c95b8b-65a4-402e-8359-69b55d6c9bf1 | 10.12.13.0/24 | Internal-Subnet |! +--------------------------------------+----------------+-----------------+! ! # ip netns show! #! §  Note: The dhcp namespace will be created when the ﬁrst instance boots

40. Neutron – external Network CreaEon 1/2 §  Now we will create a external network deﬁni5on, and add an IP subnet and pool to it # neutron net-create External-Net --router:external=True! ! Created a new network:! +---------------------------+--------------------------------------+! | Field | Value |! +---------------------------+--------------------------------------+! | admin_state_up | True |! | id | 8998c547-ff7c-45f8-884a-a6d4bcaa5de7 |! | name | External-Net |! | provider:network_type | gre |! | provider:physical_network | |! | provider:segmentation_id | 2 |! | router:external | True |! | shared | False |! | status | ACTIVE |! | subnets | |! | tenant_id | b1178a03969b4f638937f5a632fb547a |! +---------------------------+--------------------------------------+! !

41. Neutron – external Network CreaEon 2/2 §  Now we will create a external network deﬁni5on, and add an IP subnet and pool to it # neutron subnet-create External-Net 172.16.65.0/24 ! --allocation-pool start=172.16.65.100,end=172.16.65.150! ! Created a new subnet:! +------------------+----------------------------------------------------+! | Field | Value |! +------------------+----------------------------------------------------+! | allocation_pools | {"start": "172.16.65.100", "end": "172.16.65.150"} |! | cidr | 172.16.65.0/24 |! | dns_nameservers | |! | enable_dhcp | True |! | gateway_ip | 172.16.65.1 |! | host_routes | |! | id | 16eb9d34-819f-4525-99ab-ec9358ea132f |! | ip_version | 4 |! | name | |! | network_id | 8998c547-ff7c-45f8-884a-a6d4bcaa5de7 |! | tenant_id | b1178a03969b4f638937f5a632fb547a |! +------------------+----------------------------------------------------+! ! !

43. Neutron – Router CreaEon 2/4 §  Now we will create a router, and connect it to the “Uplink” (external network) we created earlier # neutron router-show MyRouter! +-----------------------+-----------------------------------------------------------------------------+! | Field | Value |! +-----------------------+-----------------------------------------------------------------------------+! | admin_state_up | True |! | external_gateway_info | {"network_id": "8998c547-ff7c-45f8-884a-a6d4bcaa5de7", "enable_snat": true} |! | id | bda86e19-4831-4bfb-b3f4-bb79113ceab1 |! | name | MyRouter |! | routes | |! | status | ACTIVE |! | tenant_id | b1178a03969b4f638937f5a632fb547a |! +-----------------------+-----------------------------------------------------------------------------+! ! # neutron router-port-list MyRouter -c fixed_ips! +--------------------------------------------------------------------------------------+! | fixed_ips |! +--------------------------------------------------------------------------------------+! | {"subnet_id": "b4c95b8b-65a4-402e-8359-69b55d6c9bf1", "ip_address": "10.12.13.1"} |! | {"subnet_id": "16eb9d34-819f-4525-99ab-ec9358ea132f", "ip_address": "172.16.65.100"} |! +--------------------------------------------------------------------------------------+! !

44. Neutron – Router CreaEon 3/4 §  Now that the router is created, and interfaces are assigned to it, we will see a new namespace # ip netns show! qrouter-bda86e19-4831-4bfb-b3f4-bb79113ceab1! ! # ip netns exec qrouter-bda86e19-4831-4bfb-b3f4-bb79113ceab1 /bin/bash! ! # ip addr! 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN ! link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00! inet 127.0.0.1/8 scope host lo! inet6 ::1/128 scope host ! valid_lft forever preferred_lft forever! 10: qg-f9d1f494-7f: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN ! link/ether fa:16:3e:02:9a:1c brd ff:ff:ff:ff:ff:ff! inet 172.16.65.100/24 brd 172.16.65.255 scope global qg-f9d1f494-7f! inet6 fe80::f816:3eff:fe02:9a1c/64 scope link ! valid_lft forever preferred_lft forever! 11: qr-a86dfa2b-9c: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN ! link/ether fa:16:3e:7b:1a:92 brd ff:ff:ff:ff:ff:ff! inet 10.12.13.1/24 brd 10.12.13.255 scope global qr-a86dfa2b-9c! inet6 fe80::f816:3eff:fe7b:1a92/64 scope link ! valid_lft forever preferred_lft forever! ! # netstat -rn! Kernel IP routing table! Destination Gateway Genmask Flags MSS Window irtt Iface! 0.0.0.0 172.16.65.1 0.0.0.0 UG 0 0 0 qg-f9d1f494-7f! 10.12.13.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-a86dfa2b-9c! 172.16.65.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-f9d1f494-7f!

45. Neutron – Router CreaEon 4/4 – OVS View §  The OVS show will now show the tap interfaces to the router Namespace, and to the external interface root@os-controller:/home/localadmin# ovs-vsctl show! 09d5b89a-600d-4da3-b761-11206456385a! Bridge br-ex! Port "qg-f9d1f494-7f"! Interface "qg-f9d1f494-7f"! type: internal! Port br-ex! Interface br-ex! type: internal! Port "eth2"! Interface "eth2”! ! .... SNIP ....! ! Bridge br-int! Port patch-tun! Interface patch-tun! type: patch! options: {peer=patch-int}! Port "qr-a86dfa2b-9c"! tag: 1! Interface "qr-a86dfa2b-9c"! type: internal! Port br-int! Interface br-int! type: internal! ovs_version: "1.10.2"! ! # external router interface is patched to br-ex, and therefore bridged out to interface eth2! ! # Internal router interface is patched to br-int, and therefore connected to the ‘br-int’ flow table!

46. Neutron – Horizon Dashboard View

48. Neutron – Horizon Dashboard View

49. Neutron – DHCP Namespace / dnsmasq process §  Axer the ﬁrst Instances was started, Neutron created the dhcp namespace and started a dnsmasq process in it # ip netns show! qdhcp-56a76117-8910-4d85-b91d-8e6842e0a510! qrouter-bda86e19-4831-4bfb-b3f4-bb79113ceab1! ! # ip netns exec qdhcp-56a76117-8910-4d85-b91d-8e6842e0a510 /bin/bash! ! # ip addr! ... SNIP ...! 12: tap383cd579-5e: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN ! link/ether fa:16:3e:de:5f:bf brd ff:ff:ff:ff:ff:ff! inet 10.12.13.3/24 brd 10.12.13.255 scope global tap383cd579-5e! inet 169.254.169.254/16 brd 169.254.255.255 scope global tap383cd579-5e! inet6 fe80::f816:3eff:fede:5fbf/64 scope link ! valid_lft forever preferred_lft forever! ! # ps -ef | grep dnsmasq! nobody 16209 1 0 22:29 ? 00:00:00 dnsmasq --no-hosts --no-resolv --strict-order --bind- interfaces --interface=tap383cd579-5e --except-interface=lo --pid-file=/var/lib/neutron/dhcp/ 56a76117-8910-4d85-b91d-8e6842e0a510/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/56a76117-8910-4d85- b91d-8e6842e0a510/host --dhcp-optsfile=/var/lib/neutron/dhcp/56a76117-8910-4d85-b91d-8e6842e0a510/opts --leasefile-ro --dhcp-range=set:tag0,10.12.13.0,static,86400s --dhcp-lease-max=256 --conf-file= -- domain=openstacklocal! root 22102 15608 0 22:58 pts/0 00:00:00 grep --color=auto dnsmasq! ! # cat /var/lib/neutron/dhcp/56a76117-8910-4d85-b91d-8e6842e0a510/host! fa:16:3e:ee:1e:2f,host-10-12-13-2.openstacklocal,10.12.13.2! fa:16:3e:7b:1a:92,host-10-12-13-1.openstacklocal,10.12.13.1! fa:16:3e:17:75:f6,host-10-12-13-4.openstacklocal,10.12.13.4! !

50. Neutron – Instance config file §  Here’s what the network part of the Instance configura5on for KVM looks like -- COMPUTE NODE 1 ---! ! # virsh list! Id Name State! ----------------------------------------------------! 6 instance-0000000b running! ! # virsh dumpxml 6! <domain type='kvm' id='6'>! <name>instance-0000000b</name>! ! ... SNIP ...! ! <interface type='bridge'>! <mac address='fa:16:3e:64:20:31'/>! <source bridge='br-int'/>! <virtualport type='openvswitch'>! <parameters interfaceid='32141443-073a-4be9-993b-51f3e131b037'/>! </virtualport>! <target dev='tap32141443-07'/>! <model type='virtio'/>! <alias name='net0'/>! <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>! </interface>! ! ... SNIP ...! ! # Instance Port id ‘tap32141443-07’!

51. Neutron – OVS view aber Instances are connected §  Now let’s examine what the patches and ﬂow tables look like on OVS axer the Instances were started -- COMPUTE NODE 1 ---! ! root@os-compute-1:/home/localadmin# ovs-vsctl show! Bridge br-int! ... SNIP ...! Port patch-tun! Interface patch-tun! type: patch! options: {peer=patch-int}! Port "tap32141443-07"! tag: 6! Interface "tap32141443-07"! Bridge br-tun! ... SNIP ...! Port "gre-172.16.0.12"! Interface "gre-172.16.0.12"! type: gre! options: {in_key=flow, local_ip="172.16.0.11", out_key=flow, remote_ip="172.16.0.12"}! Port patch-int! Interface patch-int! type: patch! options: {peer=patch-tun}! Port "gre-172.16.0.10"! Interface "gre-172.16.0.10"! type: gre! options: {in_key=flow, local_ip="172.16.0.11", out_key=flow, remote_ip="172.16.0.10"}! ovs_version: "1.10.2"! ! # Instance Port id ‘tap32141443-07’! ! # Instance Port mapping into br-int flow-table!

52. Neutron – OVS ﬂows created through rootwrap by OVS-‐Agent §  OVS ﬂows and interfaces get created through rootwrapper by the OVS Agent -- COMPUTE NODE 1 ---! ! # tail -f /var/log/syslog ! ! Apr 6 23:51:34 os-compute-1 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may- exist add-port br-int tap60b3782b-80 -- set Interface tap60b3782b-80 "external-ids:attached-mac="fa: 16:3e:64:20:31"" -- set Interface tap60b3782b-80 "external-ids:iface-id="60b3782b-8096-497d-96a4- f3a8dc187eb6"" -- set Interface tap60b3782b-80 "external-ids:vm-id="17f0fdee-3ecd-440f-8e77- c43d2fcda9de"" -- set Interface tap60b3782b-80 external-ids:iface-status=active! ! Apr 6 23:51:37 os-compute-1 neutron-rootwrap: (root > root) Executing ['/usr/bin/ovs-ofctl', 'mod- flows’,'br_tun’,'hard_timeout=0,idle_timeout=0,priority=1,table=21,dl_vlan=6,actions=strip_vlan,set_tu nnel:1, output 3,2'] (filter match = ovs-ofctl)! ! Apr 6 23:51:37 os-compute-1 neutron-rootwrap: (root > root) Executing ['/usr/bin/ovs-ofctl', 'add- flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=1,actions=mod_vlan_vid: 6,resubmit(,10)'] (filter match = ovs-ofctl)! ! Apr 6 23:51:37 os-compute-1 neutron-rootwrap: (root > root) Executing ['/usr/bin/ovs-vsctl', '-- timeout=2', 'set', 'Port', 'tap60b3782b-80', 'tag=6'] (filter match = ovs-vsctl)! ! Apr 6 23:51:37 os-compute-1 ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port tap60b3782b-80 tag=6! ! Apr 6 23:51:37 os-compute-1 neutron-rootwrap: (root > root) Executing ['/usr/bin/ovs-ofctl', 'del- flows', 'br-int', 'in_port=7'] (filter match = ovs-ofctl)! !

53. Neutron – OVS – MAC learning §  OVS with OVS Agent s5ll uses classic MAC Learning to understand were which MAC Address is in the Network -- COMPUTE NODE 1 ---! ! # ovs-appctl fdb/show br-int! port VLAN MAC Age! 4 6 fa:16:3e:64:20:31 4! -1 6 fa:16:3e:de:5f:bf 4! ! # ovs-appctl dpif/show br-int! br-int (system@ovs-system):! !lookups: hit:1461 missed:343! !flows: cur: 0, avg: 8.634, max: 39, life span: 7746(ms)! ! !hourly avg: add rate: 0.654/min, del rate: 0.658/min! ! !overall avg: add rate: 0.775/min, del rate: 0.775/min! !br-int 65534/1: (internal)! !patch-tun 1/none: (patch: peer=patch-int)! !tap60b3782b-80 7/4:! ! # ovs-appctl dpif/show br-tun! br-tun (system@ovs-system):! !lookups: hit:568 missed:364! !flows: cur: 0, avg: 9.707, max: 39, life span: 5976(ms)! ! !hourly avg: add rate: 0.730/min, del rate: 0.731/min! ! !overall avg: add rate: 0.817/min, del rate: 0.817/min! !br-tun 65534/2: (internal)! !gre-172.16.0.10 2/3: (gre: key=flow, local_ip=172.16.0.11, remote_ip=172.16.0.10)! !gre-172.16.0.12 3/3: (gre: key=flow, local_ip=172.16.0.11, remote_ip=172.16.0.12)! !patch-int 1/none: (patch: peer=patch-tun)!

54. Neutron – OVS – Table Structure §  OVS Agent programs a complex Table structure into OVS hOps://wiki.openstack.org/wiki/Ovs-‐ﬂow-‐logic

55. Neutron – IPTable Rules – Compute Nodes – Security G. §  The following output shows what Neutron conﬁgures into IPTables on the compute node to implement security groups -- COMPUTE NODE 1 ---! ! # iptables –L! … SNIP …! Chain neutron-openvswi-i7fff0812-9 (1 references)! target prot opt source destination ! DROP all -- anywhere anywhere state INVALID! RETURN all -- anywhere anywhere state RELATED,ESTABLISHED! RETURN tcp -- anywhere anywhere tcp multiport dports tcpmux:65535! RETURN icmp -- anywhere anywhere ! RETURN udp -- anywhere anywhere udp multiport dports 1:65535! RETURN udp -- 10.12.13.3 anywhere udp spt:bootps dpt:bootpc! … SNIP …! Chain neutron-openvswi-o7fff0812-9 (2 references)! target prot opt source destination ! RETURN udp -- anywhere anywhere udp spt:bootpc dpt:bootps! ! neutron-openvswi-s7fff0812-9 all -- anywhere anywhere ! DROP udp -- anywhere anywhere udp spt:bootps dpt:bootpc! DROP all -- anywhere anywhere state INVALID! RETURN all -- anywhere anywhere state RELATED,ESTABLISHED! RETURN all -- anywhere anywhere ! ! Chain neutron-openvswi-s7fff0812-9 (1 references)! target prot opt source destination ! RETURN all -- 10.12.13.2 anywhere MAC FA:16:3E:43:C6:20! DROP all -- anywhere anywhere !

56. Neutron – IPTable Rules – Compute Nodes – Security G. §  The following output shows what Neutron conﬁgures into IPTables on the compute node to implement security groups -- COMPUTE NODE 1 ---! ! # iptables –L! … SNIP …! Chain neutron-openvswi-i7fff0812-9 (1 references)! target prot opt source destination ! DROP all -- anywhere anywhere state INVALID! RETURN all -- anywhere anywhere state RELATED,ESTABLISHED! RETURN tcp -- anywhere anywhere tcp multiport dports tcpmux:65535! RETURN icmp -- anywhere anywhere ! RETURN udp -- anywhere anywhere udp multiport dports 1:65535! RETURN udp -- 10.12.13.3 anywhere udp spt:bootps dpt:bootpc! … SNIP …! Chain neutron-openvswi-o7fff0812-9 (2 references)! target prot opt source destination ! RETURN udp -- anywhere anywhere udp spt:bootpc dpt:bootps! ! neutron-openvswi-s7fff0812-9 all -- anywhere anywhere ! DROP udp -- anywhere anywhere udp spt:bootps dpt:bootpc! DROP all -- anywhere anywhere state INVALID! RETURN all -- anywhere anywhere state RELATED,ESTABLISHED! RETURN all -- anywhere anywhere! ! Chain neutron-openvswi-s7fff0812-9 (1 references)! target prot opt source destination ! RETURN all -- 10.12.13.2 anywhere MAC FA:16:3E:43:C6:20! DROP all -- anywhere anywhere ! ! # Inbound rule to Instances! ! # Default outbound allow dhcp! ! # ‘Port Security Rule’ only allow Instance MAC outbound!

58. Neutron – add ﬂoaEng-‐ip to instance §  We will now add a ﬂoa5ng-‐ip to an instance # nova show Instance1! +--------------------------------------+----------------------------------------------------------+! | Property | Value |! +--------------------------------------+----------------------------------------------------------+! | status | ACTIVE |! | updated | 2014-04-08T00:08:23Z |! | OS-EXT-STS:task_state | None |! | OS-EXT-SRV-ATTR:host | os-compute-1 |! | key_name | None |! | image | CirrOS 0.3.1 (55438187-bc0e-4245-b4a7-edb338cf47bd) |! ! ... SNIP ...|! ! | accessIPv4 | |! | accessIPv6 | |! | Internal-Network network | 10.12.13.2, 172.16.65.101 |! | progress | 0 |! | OS-EXT-STS:power_state | 1 |! | OS-EXT-AZ:availability_zone | nova |! | config_drive | |! +--------------------------------------+----------------------------------------------------------+! !

59. Neutron –ﬂoaEng-‐ip, router namespace §  This is what a ﬂoa5ng IP looks like in the router Namespace and in IPTables # ip netns exec qrouter-c6687e7c-ab1c-4336-ab1e-8021f9c59925 /bin/bash! ! # ip addr! ! 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN ! link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00! inet 127.0.0.1/8 scope host lo! inet6 ::1/128 scope host ! valid_lft forever preferred_lft forever! 13: qg-92d91e4c-2d: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN ! link/ether fa:16:3e:58:f6:2c brd ff:ff:ff:ff:ff:ff! inet 172.16.65.100/24 brd 172.16.65.255 scope global qg-92d91e4c-2d! inet 172.16.65.101/32 brd 172.16.65.101 scope global qg-92d91e4c-2d! inet6 fe80::f816:3eff:fe58:f62c/64 scope link ! valid_lft forever preferred_lft forever! 14: qr-8abeb2b0-a6: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN ! link/ether fa:16:3e:18:6e:93 brd ff:ff:ff:ff:ff:ff! inet 10.12.13.1/24 brd 10.12.13.255 scope global qr-8abeb2b0-a6! inet6 fe80::f816:3eff:fe18:6e93/64 scope link ! valid_lft forever preferred_lft forever! ! ! # Router IP! # configured floating-ip!

60. Neutron –ﬂoaEng-‐ip, IPTables NAT §  This is what a ﬂoa5ng IP looks like in the router Namespace and in IPTables # iptables -t nat -L! ! ...SNIP ...! ! Chain neutron-l3-agent-OUTPUT (1 references)! target prot opt source destination ! DNAT all -- anywhere 172.16.65.101 to:10.12.13.2! ! Chain neutron-l3-agent-POSTROUTING (1 references)! target prot opt source destination ! ACCEPT all -- anywhere anywhere ! ctstate DNAT! ! Chain neutron-l3-agent-PREROUTING (1 references)! target prot opt source destination ! REDIRECT tcp -- anywhere 169.254.169.254 tcp dpt:http redir ports 9697! DNAT all -- anywhere 172.16.65.101 to:10.12.13.2! ! Chain neutron-l3-agent-float-snat (1 references)! target prot opt source destination ! SNAT all -- 10.12.13.2 anywhere to:172.16.65.101! ! Chain neutron-l3-agent-snat (1 references)! target prot opt source destination ! neutron-l3-agent-float-snat all -- anywhere anywhere ! SNAT all -- 10.12.13.0/24 anywhere to:172.16.65.100! ! Chain neutron-postrouting-bottom (1 references)! target prot opt source destination ! neutron-l3-agent-snat all -- anywhere anywhere !

61. Neutron –ﬂoaEng-‐ip, IPTables NAT §  This is what a ﬂoa5ng IP looks like in the router Namespace and in IPTables # iptables -t nat -L! ! ...SNIP ...! ! Chain neutron-l3-agent-OUTPUT (1 references)! target prot opt source destination ! DNAT all -- anywhere 172.16.65.101 to:10.12.13.2! ! Chain neutron-l3-agent-POSTROUTING (1 references)! target prot opt source destination ! ACCEPT all -- anywhere anywhere ! ctstate DNAT! ! Chain neutron-l3-agent-PREROUTING (1 references)! target prot opt source destination ! REDIRECT tcp -- anywhere 169.254.169.254 tcp dpt:http redir ports 9697! DNAT all -- anywhere 172.16.65.101 to:10.12.13.2! ! Chain neutron-l3-agent-float-snat (1 references)! target prot opt source destination ! SNAT all -- 10.12.13.2 anywhere to:172.16.65.101! ! Chain neutron-l3-agent-snat (1 references)! target prot opt source destination ! neutron-l3-agent-float-snat all -- anywhere anywhere ! SNAT all -- 10.12.13.0/24 anywhere to:172.16.65.100! ! Chain neutron-postrouting-bottom (1 references)! target prot opt source destination ! neutron-l3-agent-snat all -- anywhere anywhere ! ! # floating-ip DNAT! ! # floating-ip SNAT! ! # default SNAT for all instances!

Osdc2014 openstack networking yves_fauser

Osdc2014 openstack networking yves_fauser

Recommandé

Contenu connexe

Tendances

Tendances(20)

En vedette

En vedette(15)

Similaire à Osdc2014 openstack networking yves_fauser

Similaire à Osdc2014 openstack networking yves_fauser(20)

Dernier

Dernier(20)

Osdc2014 openstack networking yves_fauser