OpenStack Boston Summit session about bringing Neutron networking to your baremetal or nova instance supported Kubernetes clusters

1. Kuryr-Kubernetes
Adding Pods to your Datacenter Networking
Irena Berezovsky @irenab
Antoni Segura Puimedon @celebdor

2. Kuryr-Kubernetes Project motivation
● Hard to connect VMs, bare metal and nested containers
○ No unified networking infrastructure
● Overlay
2
for Pods running in VMs
○ Performance, latency, SLA, management penalties
● Need for a smooth transition to the Cloud Native
Applications
○ Ability to transition workloads to microservices at your own pace
○ VMs and Pods sharing networking infrastructure
● Bare Metal OpenStack Storage support with
Fuxi-Kubernetes

3. Kuryr-Kubernetes Project Mission
● Neutron, unified, community sourced networking
for Pods & VMs
● OpenStack vendor support experience in the
Container space
● Get Neutron users faster into container workloads
○ VMs and Pods on the same Neutron network
○ Enable both L2 and L3 connectivity between OS VMs and
K8s Pods
● Easier transition to microservices
○ Connect to VM layer in the same infrastructure

4. Bare Metal Use
Case
● Centralized Kuryr
Controller
● Kuryr Controller maps
○ K8s Pods into
Neutron ports
○ K8s Services into
Neutron Load
Balancers
● Kuryr CNI on each
Worker node performs
Pod binding

5. Pod in VM Use
Case
● Security
● Easier node allocation
● Single overlay
● VM and Pods as
targetable network
resources
● Can use either Neutron
trunk ports or macvlan
based VM port
allocation

6. Mixed Use Case
● Connect to existing
services in VMs
● Legacy applications
alongside
microservices
● VM NFVs
● Use existing cloud for
Kubernetes workloads

7. Supported functionality
● Pods networking
○ Kubernetes native networking
○ Pods as Neutron ports on the cluster
Neutron network
○ Single tenant
○ Full connectivity enabled by default
● Kubernetes ClusterIP Services
○ Implemented by Neutron LBaaSv2
● Bare Metal and Pod in VM support

8. Kuryr-Kubernetes Architecture

9. Kubernetes
Services
● Cluster IP translates to
Neutron VIP
● Service endpoints
translate to Pool
Members
● Uses Neutron Lbaas v2
● Planned addition of
Octavia driver

11. Kuryr Controller
● Secure connection to the Neutron API Server
○ Keystone as Authorization service
● Watches Kubernetes API resources with a service account
● Stevedore Plugin based Network resources translation
○ Handlers: Receive Kubernetes resource events and patch them
○ Drivers: Used by handlers to allocate Neutron resources, allowing multiple
implementations and vendors.
● Os-vif for interface plugging

12. Kuryr Controller ServiceAccount
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kuryrctl
namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: kuryrctl-global
subjects:
- kind: User
name: kuryrctl
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: kuryrctl
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion:
rbac.authorization.k8s.io/v1beta1
metadata:
name: kuryrctl
rules: - apiGroups:
- ""
verbs:
- get
- list
- watch
resources:
- deployments
- endpoints
- ingress
- nodes
- pods
- policies
- services
- apiGroups:
- ""
verbs:
- update
- patch
resources:
- endpoints
- ingress
- pods
- policies
- nodes
- services
- services/status

13. Kuryr CNI Driver
● Kuryr CNI driver only
communicates with
Kubernetes API
○ Kubelet already has
connection to K8s API
○ Performs local binding
of the neutron port
● Supports CNI versioned
output (0.3.0)
● Watches Pod resources
for Controller-driven vif
annotations
"annotations": {
"openstack.org/kuryr-vif": {
"active": true,
"address": "fa:16:3e:6c:1f:ff",
"bridge_name": "br-int",
"has_traffic_filtering": true,
"id": "ba8f8d4b-1dfb-4aaf-8ab2-80c25711da3f",
"network": {
"bridge": "br-int",
"id": "a10c5bf4-99b2-4b0d-82b1-2a2639dda4de",
"label": "private",
"mtu": 1450,
"multi_host": false,
"should_provide_bridge": false,
"should_provide_vlan": false,
"subnets": {[{
"cidr": "10.0.0.0/26",
"dns": [],
"gateway": "10.0.0.1",
"ips": [{
"address": "10.0.0.8"}],
"routes": []
}]}
},
"plugin": "ovs",
"port_profile": {
"interface_id": "ba8f8d4b-1dfb-4aaf-8ab2-80c25711da3f"
},
"preserve_on_delete": false,
"vif_name": "tapba8f8d4b-1d"
}
}

14. Kuryr CNI ServiceAccount
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kuryrcni
namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: kuryrcni-global
subjects:
- kind: User
name: kuryrcni
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: kuryrcni
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion:
rbac.authorization.k8s.io/v1beta1
metadata:
# "namespace" omitted since ClusterRoles
are not namespaced
name: kuryrcni
rules:
- apiGroups:
- ""
verbs:
- get
- list
- watch
resources:
- pods

15. Controller - CNI baremetal pod creation

16. Trunk ports - Neutron extension

17. Trunk ports - Neutron extension

18. Controller - CNI pod-in-VM creation

19. Cluster service creation flow

20. Scaling Kuryr
● Generic resource
Pooling
○ VIF
○ Load Balancers*
● Stevedore pluggability
to choose pooling
behavior
● Pre-allocates Neutron
resources in batch
operations
● Burst tolerant

21. Demo: Guestbook
● 2-tier
● 3 services
● PHP frontend, Redis backend

22. ● Features
○ LoadBalancer Kubernetes Service Type
○ Resource Management
○ Ingress support
○ Policy support
○ Multi-Tenancy, Multiple Networks support
○ Management CLI
● Improvements
○ CNI split into exec and daemon
○ Handler/Driver Profiles
○ Active-Passive HA
What’s Next

23. Join us
● Project launchpad
○ https://launchpad.net/kuryr-kubernetes
● Repository
○ https://github.com/openstack/kuryr-kubernetes
● IRC
○ Weekly meeting #openstack-meeting-4 Mondays 14:00 UTC
○ #openstack-kuryr at Freenode
● Mailing list
○ [kuryr] in openstack-dev@lists.openstack.org

24. Resources
● Documentation
○ https://docs.openstack.org/developer/kuryr-kubernetes
● Getting started
○ https://ltomasbo.wordpress.com/2017/01/29/side-by-side-and-nested-kubernetes-and-op
enstack-deployment-with-kuryr/

25. Demo: services
Demo recording