pycon apac 2013 presentation http://apac-2013.pycon.jp/ja/program/sessions.html#session-14-1110-rooma0762-en2-ja videos are available at http://www.youtube.com/watch?v=Ow-aXpMO8-o

1. Ryu: SDN framework and Python
experience
Isaku Yamahata <yamahata@private.email.ne.jp>
<isaku.yamahata@gmail.com>
Pycon APAC 2013, September 14

2. Agenda
● Introduction
● Ryu: SDN framework
● Ryu Internals
– Openstack support
● Ryu development
● Python experience through Ryu
This presentation represents my personal view/opinion. Not Ryu project nor any companies.

3. Who am I?
● My background is OS/virtualization/cloud
– Not network guy
● Programming language
– C/C++/assembler
● Projects I've contributed
– Linux
– Virtualization
● Qemu, KVM, Xen
– OpenStack
● Nova, neutron(formarly quantum) Ryu plugin
– Open vSwitch
● My python experience had begun with OpenStack and Ryu

4. Introduction

5. What's SDN? And why?
●
Software Defined Networking
– Making network programmable
– http://www.opennetsummit.org/why-sdn.html
●
SDN is a disruptive technology that is making
networks programmable by ordinary programmers
using ordinary software running on ordinary
operating systems in ordinary servers. With SDN,
the introduction of new features becomes less
manual, less prone to error, and faster to
implement.
– [Paraphrased from the HotSDN ‘13 Solicitaion]
●
Software Defined Networking (SDN) is a refactoring
of the relationship between network devices and the
software that controls them.
● Motivations behind SDN
– Academic research
● Allow researchers to program/modify
switches freely
– Industry technology trends
● Virtualization/cloud technology
● Network is behind those technology
progress
● Networkig virtualization/automation

6. From http://opennetsummit.org/why.html
Separating data/control plan

7. From http://www.opennetsummit.org/why-sdn.html
SDN architecture

8. Openflow
datapath(hardware)
controlpath(software)
datapath(hardware)
controlpath(software) openflow
Openflow controller
Openflow protocol(tcp/ssl)
Ethernet switch Openflow ethernet switch
● protocol to control switches

9. Flow table and match/action
Openflow controller
Flow table
MAC
src
MAC
dst
IP
src
IP
dst
TCP
src
TCP
dst ... action
Packet in event
When entry miss
* * * * * 80
output
port N
port PacketportPacket
Packet match Action

10. Ryu, a component-based
software-defined networking
framework

11. What's Ryu?
流 龍
Flow
Oriental Dragion,
A god of water
Manages flow control to
enable intelligent networking
http://ja.wikipedia.org/wiki/%E8%B5%A4%E7%9B%AE%E5%9B%9B%E5%8D%81%E5%85%AB%E6%BB%9D

12. What's Ryu?
● a component-based software-defined
networking framework
● License: Apache 2.0
● Fully written in Python
● Supports various protocols for managing
network devices
– OpenFlow, Netconf, OF-config, SNMP etc.
●
Official site http://osrg.github.io/ryu/
●
MLhttps://lists.sourceforge.net/lists/listinfo/ryu-devel
●
Download https://github.com/osrg/ryu
●
Documentation http://ryu.readthedocs.org/en/latest/
● Wiki https://github.com/osrg/ryu/wiki

13. Supported features/protocols
● Openflow protoocol
– OF-wire: 1.0,1.2, 1.3, Nicira extension
– OF-config 1.1
● Non-openflow protocols
– Netconf, OVSDB, netflow, sflow,
VRRP, SNMP
● Snmp: Enterprise OID: 41786
– Ryu can configure Open vSwitch
directly without ovs-vsctl, ovsdb-client
Some features are under development. The patches can be found on ML archive.
● RyuApp, library
– Packet library
– STP, LACP
– Sample apps, etc...
– Conversion from/to JSON
representation from/to OF
– RPC to communicate/control Ryu
● Integration with other project
– OpenStack
– HA with Zookeeper
– IDS(Intrusion Detection System)
with snort

14. OF/firewall/router REST API
● OF REST (ofctl_reset)
– Insert/delete openflow rule
● Firewall (rest_firewall)
– Configure firewall
– https://github.com/osrg/ryu/wiki/Third-Party-Tools,-Etc.
● Router(rest_router)
– Configure router
Ryu
REST
OF REST API
add a flow entry
POST http://example.org/stats/flownetry/add
delete flow entries
DELETE http://example.org/stats/flowentry/delete
get flow stats
GET http://example.org/stats/flow/{dpid}
allow
drop
firewall
OF switch
Ryu
REST
OF switch
router

15. Topology Viewer
● Discover topology
by LLDP and
show
topology/flows
dynamically

16. HA support
● Centralized controller is single pointer of
failure(SPOF)
● Ryu + Zookeeper can be used to address
SPOF
Ryu Ryu
zookeeper
OF switch
master slave
failover

17. IDS support
● Snort integration
https://github.com/osrg/ryu/wiki/Snort-Integration
Ryu
OF switch
Sort
Control app IDS(snort)
1. L1-L4 matching
2. send patcket to IDS
3. alert4. take actions
e.g. loggingt

18. Ryu Internals

19. Ryu implementation
● Quite normal python program from the point of implementation
view
– It doesn't use any special tricks
● Event driven
– Event source/dispatcher/sink
– Core(= Event dispatcher) is very small
– It is so generic that Ryu can be used without OpenFlow
● Component based
– Event source/sink are created as components
● Even OpenFlow related codes are so
– Message passing via events, not directly communite.

20. Ryu architecture
● Follows standard SDN architecture
OpenFlow switch OpenFlow switch Network device
SDN apps
Well defined API
(REST, RPC...)
Open protocols
(OpenFlow, OF-config,
NETConfig, OVSDB...)
SDN apps SDN apps
Ryu SDN framework
OpenFlow
Parser/serializer
Event dispatcher
Ryu built-in app
(tenant isolation,
Topology discovery, firewall )
Ryu App
libraries
Protocol support
(OVSDB, VRRP, ...)
Ryu App...
operator openstack User app
Control layer
Application layer

21. Aio/thread
● Uses eventlet
– Like OpenStack
– gevent was used before
– switched to eventlet for pypy
● twisted was not adopted for simplicity
● eventlet(or gevent) is cooperative threading, so
some cautions are needed
– This is different from preemptive threading like pthread

22. Event Dispatcher
● class AppManager and class RyuApp
● The guts of Ryu
● Decouples event sources/sinks
– Event sources generate whatever events
– Event sinks register handlers dynamically
● Dispatches events based on class of events
– To event sinks that want class of events
– Class is a first class object in Python
● knows which methods are interested in which event by
methods attributes
– Methods are also first class object in Python
RyuAppRyuApp
queue
BRICKS
Event
Determin which RyuApp to deliver
Based on class of event
dispatch
Events are read only because
It is shared with many RyuApps
Event sink
Event dispatcher
RyuAppRyuApp
queue
RyuAppRyuApp
queueEvent source
EventEvent source

23. Event source/sink
● source
– Call methods of the event dispatcher to generate events
● sink
– Subclass of class RyuApp
● Event dispatcher knows which methods are interested in which
events
– Event handlers are invoked in its own thread context of each
RyuApp
– To avoid race condition
– Direct queuing is also possible
RyuApp
queue
Event thread
Consuming events
Event

24. Event request/reply
● request/reply messaging between RyuApps for
easy programming
RyuApp
Event thread
RyuAppevent
queue
RyuApp
Event thread
RyuApp
event
queue
reply
queue
reply
queue
request
event
reply
event
1. queue request event
3. process request
4. queue back result
5. wake up
waiting event thread
If necessary
2. wait for reply if
synchronous

25. OpenFlow parser and its event
● Only controller part is supported
● OF events are created automatically on startup
– Introspection is used
● “Where EventOFPxxx is defined?” is FAQ
ofproto_v1_N_parser
OFPxxx EventOFPxxx
ofp_event

26. Connection to OpenFlow switch
● class OpenFlowController, class Datapath
● Receiving loop and sending loop
OpenFlow switch
Receiving thread
Generates OFPEvents
Sending thread
Serialize and send
OF packets
Send queue
EventOFP
message
Ryu
Datapath
RyuAppRyuApp
queue
Event sink

27. OpenStack support

28. OpenStack Component
● Composed of Many
component
● Neutron
– Plugin architecture
– Able to support many
network technology
service Openstack project
compute nova
storage swift(object)
glance(image)
cinder(block)
identity keystone
network neutron
... ...

29. Ryu Plugin for Neutron
● L2 isolation
● Multi tenant w/o or w/ VLAN
– Mac address based
– VLAN
– GRE tunnel

30. Overview of Ryu plugin
Compute-node
Vif driver
Create
OVS port
Ryu
agent OVS
OVS
initialization
OVS
Ryu
agent
OVS
initialization
L3 agent
Neutron Node
Neutron DB
(Network ID, key)
Ryu server
(Network ID, key)
Neutron API
Ryu node
r
Ryu REST
OpenFlow & OVSDB
Neutron server
Ryu
plugin
Network node

31. OpenStack L2 isolation: logical view
VM VM VM VM VM VM
Tenant X Tenant Y

32. OpenStack L2 isolation: physical view
Compute
Node
Compute
Node
Compute
/network
OVS
OVS
Tenant X
VM
Tenant Y
VM
Tenant X
VM
Tenant Y
VM
Tenant X
VM
Tenant Y
VM
Tenant => GRE key
GRE tunnel
OpenFlow
Tenant X
GRE key = M
Tenant Y
GRE key = N
L2 over L3 with GRE tunnel
- Able to span over network
segments
(l2 segment can over multi
data centers)
- can coexists with
Conventional network
technology
Set GRE key
Deliver packets
based on GRE key
En/de-cupsel packet
Into/from GRE packet
Ryu
nw-gw
nw-gw OVS

33. Table 0 Table 1 Table 2
Src table Tunnel out Local out
VM port
match action
in_port
src mac
set_tunnel
goto table 1
in_port drop
match action
tunnel_id
dst mac
output(tunnel)
goto table 2
match action
tunnel_id
dst mac
output(vm)
tunnel_id goto table 2
tunnel_id drop
Tunnel
port in_port
tunnel_id
goto table 2
in_port drop
OVS
VM1
VM2
GRE tunnel
tunnel
port
VM port
In port
Flow Table Usage
Nicira extension is used for GRE tunnel

34. GRE tunneling with OpenStack
● Composed of several RyuApps
● Network tenant creation
– Assign GRE key
– Create gateway
● Guest VM instance creation
– Create port
● Tenant ↔ key ↔ port relationship
– Set flow to the VM port
● Tunnel port management
– Create/delete tunnel port
● Track physical compute node
– Set flow to the tunnel port
rest_quantum
gre_tunnel
tunnel_port_updater
quantum_adapter
REST
OVS
ovsdbOpenFlow
Neutron
Ryu
quantum: former name of neutron project

35. Ryu development

36. Development process
● Open development
● Linux style
● Discuss on Mailing List openly
● Send/review patches on Mailing List
– git format-patch
– git send-patch
– No pull request on github
● Evolution
– Ryu has evoleved from
very small program
http://dir.gmane.org/gmane.network.ryu.devel

37. Python experience through Ryu

38. Python
●
Good things
– Easy/fast to learn/use
– Many useful features
●
Dynamic language, first class everything, decorator, introspection...
– Especially introspection is very useful
– Decorator is handy
– Many useful libraries
● Bad things
– Hard to debug
● debugger(pdb) is unstable
● Debugger isn't compatible with eventlet
– Magic attributes(__xxx___)
– Many similar libraries: which to use?

39. AIO libraries
● Gevent → Eventlet
● In general, monky-patching is ugly hack and
very fragile
● Monkey patching of gevent/eventlet works
stably
● Hit some issues and patches are proposed.
● epoll is removed by monkey patching

40. Threading
● eventlet(or gevent) is cooperative threading
– Needs special care for protection
● Starvation
● Thread scheduling
– Different from native threading like pthread
● Synchronization primitives
● Hard to debug
– When debugger(pdb) tries to stop, the thread is switched to other thread
● Need to consider
– Native vs green
– Giant Interpreter Lock
– What context to deliver events?

41. Performance
● Gevent performs slightly better than eventlet
– But it's very slight.
– Needs other approach for more performance boost
● Pypy
– Needs patch for eventlet
● Mulit process?

42. Network programming
● For IGMP with VRRP
● Needs to read Cpython code or C-module code
– Much better than unsupported, though

43. Thank you
Questions?
http://osrg.github.com/ryu/