Go Daddy architecture show & tell presentation from the OpenStack Liberty summit. Includes details on network IP usage API extensions, and Glance and Nova metadata services.
3. 3
Update from PHL
• API to query network IP usage
• Neutron spec:
https://review.openstack.org/180803
Please comment!
• Patches:
https://github.com/godaddy/openstack-neutron/tree/network-ip-usage
https://github.com/godaddy/openstack-
neutron/commit/fcf325f9f9f7a9f87ba6bc1c53f9212d0e2decee
Network usage API extension
4. 4 4
Architecture Overview
Network and Physical
api servers
(control plane)
L3 Network Spine
L3 accs1 L3 accs2 L3 accsN
pod servers
compute 1
compute N
pod servers
compute 1
compute N
pod servers
compute 1
compute N
LB
5. 5
Glance/DHCP Agents/Metadata
api servers
(control plane)
haproxy
Glance API
Endpoint (SSL)
Control Plane Compute Pods
pod servers
glance-api
glance-registry
nova-metadata-api
neutron-dhcp-agent
compute
servers
Pod 2
pod servers
glance-api
glance-registry
nova-metadata-api
neutron-dhcp-agent
compute
servers
Pod 1
pod servers
glance-api
glance-registry
nova-metadata-api
neutron-dhcp-agent
compute
servers
Pod N
6. 6
Glance/DHCP Agents/Metadata
api servers
(control plane)
haproxy
Glance API
Endpoint (SSL)
Control Plane Compute Pods
pod servers
glance-api
glance-registry
nova-metadata-api
neutron-dhcp-agent
compute
servers
Pod 2
pod servers
glance-api
glance-registry
nova-metadata-api
neutron-dhcp-agent
compute
servers
Pod 1
pod servers
glance-api
glance-registry
nova-metadata-api
neutron-dhcp-agent
compute
servers
Pod N
WASTEFUL
7. 7
Improvement Ideas
• Run glance, neutron-dhcp-agent locally on
all compute nodes
• One leader to cache from Ceph?
• Centralized nova-metadata-api
• All servers in rack now the same type
• Simplifies provisioning & builds
Explain where and how we run metadata service (don’t use neutron metadata proxy.)
https://etherpad.openstack.org/p/PHL-ops-arch-show-tell
disparate public and private cloud in each geographical region
US west regions in place today
US east coast coming in Q2
EU/AP later this year
Mostly used for dev/test, but starting to on board production applications and VPS-like hosting
Saved >$2.5M in 2014 by using OpenStack
We use Anvil to build packages (Kris is a core), and Puppet with stackforge modules for config management
Keystone, Nova, Neutron, Glance core services. Also Heat and Ceilometer, but those aren’t really used.
Use HAProxy to SSL terminate a lot of things
Glance runs on pod servers (close to compute) to optimize network transfer
Keystone with AD backend for users/groups identity
No Swift or Ceph yet
VMs get a single root volume on local disk, and config drive
Using Nova cells
No live migration