As more businesses explore the benefits of cloud computing, network managers will have to increasingly meet the challenges of redesigning their networks for the cloud. This talk will introduce the basic concepts of Open vSwitch & Openflow and show how these technologies can help satisfy these needs. We will also explain how Open vSwitch fits into XCP and XenServer and routing of dataflows. First part of talk discussing the networking challenges that cloud implementers face. - Networking challeges - Data isolation Introducing Open vSwitch - What it is, and its features - Why it is important in a virtualized environment Openflow - Basics of Openflow - How flows are routed in XenServer & XCP

1. CloudStack Collaboration
Conference 2012
Under the Hood: Open vSwitch & OpenFlow in XCP &
XenServer
Dominic Curran
Citrix XenServer Software Engineer
Nov 2012
Slides available under CC BY-SA 3.0

2. 2
Contents
●
Linux Bridge
●
What does Open vSwitch Provide ?
●
Open vSwitch Project
●
How Open vSwitch works
●
How Openflow works
Slides available under CC BY-SA 3.0

3. 3
What's wrong with Linux Bridge ?
●
Linux Bridge: ●
Open vSwitch:
– Nothing ! – Full L2-L4 matching capability
– L2 switch. Fast & reliable – Decision in userspace
– Simple forwarding – ACLs, QoS & Bonding
– Mobility of state
Userspace Module
Userspace Module
VM
VM VM
VM
Bridge Kernel Module
Bridge Kernel Module OVS Kernel Module
OVS Kernel Module
VM
VM VM
VM
packet packet
Slides available under CC BY-SA 3.0

4. 4
What does Open vSwitch Provide ?
●
Provide greater visibility into the XenServer networking layer
●
Provide distributed fine grained networking configuration and
control policies
●
Allows XenServer to be part of switching fabric
●
Makes Network Admins happy.
●
A Little History:
– Research project from Standford University
– Bay Area startup – Nicira
– Partnered with Nicira 2009
Slides available under CC BY-SA 3.0

5. 5
Open vSwitch Project
●
http://openvswitch.org
●
Developed under GPL & Apache 2 License
●
Mailing Lists
– Announcements: announce@openvswitch.org
– User-level discussion: discuss@openvswitch.org
– Dev (code review, etc): dev@openvswitch.org
– Mail Archives available
●
As of Linux 3.3 patches started to be accepted for Open
vSwitch.
Slides available under CC BY-SA 3.0

6. master
git clone git://openvswitch.org/openvswitch
branch-1.8
branch-1.7
branch-1.6
branch-1.5 branch-1.4.3
XS 6.1
branch-1.4.2
LTS
branch-1.4.1
Open vSwitch Git Tree
Source repository:
branch-1.4
branch-1.3
Slides available under CC BY-SA 3.0
branch-1.2
XS 6.0.2
branch-1.1 XS 6.0
LTS
$
vlan-maint
●
(Open vSwitch 1.0)
6

7. 7
Open vSwitch OpenSource Contributors
$ git log | grep Author | cut -d “:” -f 2 | sort | uniq -c | sort | -nr
In no particular order:
●
Redhat
●
IBM
●
NEC
●
HP
●
Nicira
●
Fujitsu
●
Google
●
Broadcom
●
Cisco
●
Intel
●
Citrix
Slides available under CC BY-SA 3.0

8. 8
Main Components
●
Open vSwitch
– Open source: www.openvswitch.org
– Kernel mode module for packet switching (openvswitch_mod.ko)
– User space daemon for rules evaluation (ovsdb-server & ovs-switchd)
– Configured via OpenFlow protocol: www.openflow.org
XenServer
XenServer XenServer
XenServer
●
OpenFlow Controller Host Host
Host
vSwitch HostvSwitch
vSwitch vSwitch
– Open vSwitch has a number of nice standalone features
– BUT remote management is probably its most powerful
– Nearly every aspect can be controlled remotely
OpenFlow
OpenFlow
– Typically a deployment uses a centralized Controller Controller
Controller
– DVSC (Distributed Virtual Switch Controller)
●
One DVS Controller can manage multiple XenServer pools
XenServer
XenServer
●
Can be hosted in the same pool as it is managing Host
Host vSwitch
●
Provides Flexible way to group hosts for network management vSwitch
Slides available under CC BY-SA 3.0

9. 9
Architecture of Open vSwitch
●
Each host runs a vSwitch userspace daemon plus kernel module
– ovs-vswitchd & ovsdb-server
– Drop-in replacement for the Linux bridge based stack.
– Daemon evaluates flow rules to determine destination of each frame.
– Pushes rules down into the kernel as necessary.
●
In isolation the vSwitch acts much as the Linux bridge does:
– Layer-2 MAC learning on physical and virtual interfaces.
– From the XenServers toolstack level there is very little observable difference.
– Under the hood the management tools, configuration of the stack etc are very
different.
●
Optionally may connect to central Controller for distributed coordination and control
Slides available under CC BY-SA 3.0

10. 1
0
Main Components on one XenServer Host
JSON/RPC
ovsdb-server ovs-vswitchd
Packet
Classifier FlowTable
FlowTable
FlowTable
User
Kernel
Netlink openvswitch_mod.ko
Management Protocol (Unix Socket)
Hash lookup
Table
Slides available under CC BY-SA 3.0

11. 1
1
ovsdb-server & ovs-vswitchd
●
ovsdb-server
– Database that holds switch-level configuration
– Custom database with properties of:
●
Value contraints
●
Weak references
●
Garbage collection
– Log-based (helpful from debugging)
– Speaks Management protocol to ovs-vswitchd & Controller
●
ovs-vswitchd
– Core component in the system
– Supports multiple independent datapaths (bridges)
– Communicates
●
With controller using OpenFlow
●
With ovsdb-server using management protocol
●
With kernel module using netlink
●
Packet classifier supports flow lookup with wildcard rules
Slides available under CC BY-SA 3.0

12. 1
2
openvswitch_mod.ko
●
Fast path
●
Kernel module handles packet switching
●
Exact-match cache of flows
– Lookup & forwarding
●
Designed to be fast and simple
– Knows nothing of Openflow
●
Implements Tunnel encapsulation/decapsulation
Slides available under CC BY-SA 3.0

13. 1
3
How the Controller fits with the other Components
Controller
ssl Off-box
# ovs-vsctl … xenbr1
xenbr1 xenbr2
xenbr2
JSON/RPC
# ovs-appctl ...
# ovs-ofctl ... ovsdb-server ovs-vswitchd
Packet
Classifier FlowTable
FlowTable
User
Kernel
Netlink openvswitch_mod.ko
Management Protocol (6632/TCP)
Hash lookup
OpenFlow Channel (6633/TCP)
Table
Command Line (ovs-vsctl, ovs-appctl, ..)
Slides available under CC BY-SA 3.0

14. 1
4 XenServer Packet Flow – Packet Miss
DVS Controller Accessible via Browser
OFPT_PACKET_IN
DomU ssl
Dom0
xenbr1
xenbr1 xenbr2
xenbr2
JSON/RPC
XAPI
ovsdb-server ovs-vswitchd
MISS #2
Toolstack
Packet
Classifier FlowTable
FlowTable
User
Kernel
Incoming openvswitch_mod.ko
Packet MISS #1
eth1 Hash lookup
Table
Slides available under CC BY-SA 3.0

15. XenServer Host
VM- Domain2
Flow Table Entries 10.80.228.31
1
5
xenbr0
xenbr0
(Charlie)
●
VM Charlie sends a ping to Bob.
Port 1 Port 2
# ovs-dpctl show xenbr0
system@xapi0: PIF
lookups: hit:103033 missed:77944 lost:0
eth0 vif2.0
flows: 30
port 0: xenbr0 (internal)
port 1: eth0
port 2: vif2.0
●
Then dump flows: 10.80.2.144
(Bob)
# ovs-dpctl dump-flows xenbr0 | grep "10.80.2.144"
in_port(2),eth(src=72:41:36:a2:8c:d9,dst=00:21:1b:f3:63:45),eth_type(0x0800),i
Flow 1
pv4(src=10.80.228.31,dst=10.80.2.144,proto=1,tos=0,ttl=64,frag=no),icmp(type=8
,code=0), packets:5013, bytes:491274, used:0.760s, actions:1
in_port(1),eth(src=00:21:1b:f3:63:45,dst=72:41:36:a2:8c:d9),eth_type(0x0800),i
Flow 2
pv4(src=10.80.2.144,dst=10.80.228.31,proto=1,tos=0,ttl=62,frag=no),icmp(type=0
,code=0), packets:5013, bytes:491274, used:0.760s, actions:2
●
L2-L4 Exact Match
●
Total Number of packet matches of this type
●
Total Number of bytes for this flow match
●
Time flow was last updated
●
Actions. In this case switch packet to Port 2.
Slides available under CC BY-SA 3.0

16. 1
6
What about SYN attacks ?
●
SYN attack form of Denial-of-Service attacks
Client SYN Server Attacker SYN Server
SYN-ACK
Wait.. SYN-ACK Wait..
And wait...
And wait...
ACK
OK Timeout!
TCP – Three Way Handshake
Legitimate
Client
SYN
● For Open vSwitch this is particular problem !
??
– Each SYN is a new flow, thus a user-kernel interaction
● Recent Open vSwitch introduced a 'Flow Governor'
– Detects large quantities of short-lived flows
– Stops flow setup in kernel module, thus missing a user to kernel interaction
Slides available under CC BY-SA 3.0

17. 1
7
Open vSwitch Features
●
Supports OpenFlow 1.0
●
Bonding
●
802.1Q VLAN
●
Tunneling – Supports GRE mode
●
Ingress policing
●
NetFlow v5
●
Port mirroring (SPAN and RSPAN)
●
ACLs (through OpenFlow)
●
Bridge compatibility
●
Documented in ovs-vswitchd.conf man page:
– http://openvswitch.org/cgi-bin/ovsman.cgi?page=vswitchd%2Fovs-vswitchd.conf.5
Slides available under CC BY-SA 3.0

18. 1
8
What is OpenFlow ?
●
OpenFlow allows the path of network packets to be setup by
software.
●
It is the enabler of 'Software Defined Networking'.
●
Standard way to control flow-tables in commercial switches
and routers.
●
Gives Controller high level access to the datapath routing
decisions of a switch.
●
Currently, support is limited to 1.0 plus a number of extensions
●
Now managed by Open Networking Foundation (ONF)
Slides available under CC BY-SA 3.0

19. 1
9
How does the Protocol work ?
●
It uses a binary protocol for faster processing
●
Header: 8-bit 8-bit 16-bit 32-bit
version Type Length Transaction id
●
Simple protocol for running on embedded
CPUs in hardware switches
●
vSwitch connects over SSL to port 6633 on the
Controller.
Slides available under CC BY-SA 3.0

20. 2
0
OpenFlow Initial Setup Protocol
Message Types
vSwitch Controller
● Controller-to-Switch OFPT_HELLO
- Version
● Asynchronous OFPT_HELLO of vSwitch
- Version of
● Symmetric Controller
OFPT_ECHO_REQUES
T
EPLY
OFPT_ECHO_R
Connection Established
S
OFPT_FEATURE
- Datapath id
- Capabilities OFPT_FEATURES_RE
PLY
- Actions
- Ports
- Name
- HW Address
- Port number
Slides available under CC BY-SA 3.0

21. 2
1
OpenFlow Protocol
vSwitch Controller
OFPFC_ADD
Change entry OFPFC_MODIFY
in FlowTable E
OFPFC_DELET
Reason to send: OFPT_PACKET_IN
- Packet miss - In Port
OR - Reason
- Explicit rule - Buffer id
- Data
EQUEST
REPLY with: OFPT_STATS_R
- Switch description OFPT_STATS_REPLY
- Flow statistics
- Table statistics
- Queue statistics
- Port statistics
_REQUEST
OFPT_BARRIER
Synchronize OFPT_BARRIER_REP
LY
Slides available under CC BY-SA 3.0

22. 2
2
What happens if Controller connections is lost ?
vSwitch OFPT_ECHO_REQUES
T Controller
?
ECHO_REQUEST
timeout ???
●
Immediately enter one of two modes:
– Fail secure mode
– Fail standalone mode
Slides available under CC BY-SA 3.0

23. 2
3
WireShark Plugin for OpenFlow Protocol
●
http://www.openflow.org/wk/index.php/OpenFlow_Wireshark_Dissector
●
Command Line Alternative:
# ovs-ofctl snoop <bridge>
Slides available under CC BY-SA 3.0

24. 2
4
Questions ?
●
My Email: dominic.curran@citrix.com
This work is licensed under a
Creative Commons Attribution-ShareAlike 3.0 Unported License.