6. kakao
eth1
Compute node1
linux bridge
vm
IP:10.10.100.2/32
192.1.1.201
Routing Table
Default GW 192.168.1.1 eth1
Host Route dest 10.10.100.2/32
to 10.10.100.1
connected dest 192.168.100.2
Routing Table
1 10.10.100.2/32 via 192.1.1.201
2 10.10.100.3/32 via 192.168.1.202
3 192.168.100.2/32 via 192.168.1.201, 192.1
68.1.202
192.1.1.202
Switch
Namespace
global name
space
IPTable
DNAT Dest 192.168.100.2 is for
warded to 10.10.100.2
Compute Node Router
Veth pair
Gateway 10.10.100.1
neutron-
linuxbridge-agent
neutron-dhcp-agent
neutron-l3-agent
Host Rout
e
dest 10.10.100.2/32
to 10.10.100.1
New IP 192.168.100.2
connected dest 192.168.100.2
Neutron Floating IP
eth1
Compute node1
linux bridge
vm
IP:10.10.100.3/32
Routing Table
Default GW 192.168.1.1 eth1
Host Route dest 10.10.100.3/32
to 10.10.100.1
connected dest 192.168.100.2
Switch
Namespace
global name
space
IPTable
DNAT Dest 192.168.100.2 is for
warded to 10.10.100.2
Compute Node Router
Veth pair
Gateway 10.10.100.1
neutron-
linuxbridge-agent
neutron-dhcp-agent
neutron-l3-agent
Host Rout
e
dest 10.10.100.3/32
to 10.10.100.1
New IP 192.168.100.2
connected dest 192.168.100.2
Neutron Floating IP
192.1.1.202
7. kakao
•
•
• process
• dhcp agent, ml2 agent, l3 agent, metadata agent and nova compute
• state
• linux bridge connection stage
• routing table list
• iptable
• bgp state
kakao
eth0
Compute node
nova-compute
neutron-
linuxbridge-
agent
neutron-dhcp-
agent
Gateway
10.10.100.1
linux bridge
vm
IP:10.10.100.2/32
Routing Table
1 10.10.100.2/32 via 192.1.1.201
BGP
192.1.1.202 BGP
Virtual Switch block
Process block
vlan.bgp
vlan.0
Virtual Router
Service Route Table
1
192.1.1.201
8. kakao
•
•
• Consul is hashcorp’s product
• Designed for Datacenter level coordination and service discovery
consul
agent
consul server
hostname
•nova process
•neutron process
•routing state
•etc...
process check
state check
fail?
yes
Alrams
-kakaotalk
-URL
12. kakao
It solves a problem
• Tenant network link layer(East-West) separation issue.
It creates problems
• Tennant Network Performance
• Latency due to the geographical separation
• full mesh between the nodes
• Retransmission due to fat(+50bytes) L2 frames
• Jumbo frame is not the right solutions if you try to cover datacenter,
actually you can’t
• Vxlan offloading is also do-able, but needs money
• NAT bottleneck
• e.g. openstack neutron network node bottle neck issue
• DVR can distribute the NAT network but it needs router
13. kakao
It solves a problem
• Tenant network link layer(East-West) separation issue.
It creates problems
• Tennant Network Performance
• Latency due to the geographical separation
• Retransmission due to fat(+50bytes) L2 frames
• Jumbo frame is not the right solutions if you try to cover datacenter,
actually you can’t
• Vxlan offloading is also do-able, but needs money
• NAT bottleneck (North-South)
• e.g. openstack neutron network node bottle neck issue
• DVR can distribute the NAT network but it needs routers
18. kakao
•
• IP routing lookup
• Attaches labels
• Forwards based on label
•
• Use label to route ( This determined by RR)
•
• Removes label
• Packet is delivered using normal routing
Pic. by Qumulus
20. kakao
•
• The important thing is that Doesn’t touch Packet in L2
• no fat packet
• it add label at L3
•
• Just thinks about that I didn’t use overlay network IP, only I add label. So, don’’t
need NAT at all
• Use the legacy IP , So the existing routing and network modeling is just working
fine.