From AWS Community Day 2019! Learn how to use Kubernetes native constructs to build Virtual Clusters, so that your teams can focus on delivering business value.

1. Virtual Kubernetes Clusters
On Amazon EKS
Jim Bugwadia | September 13, 2019

2. 2
Kubernetes is the new Cloud OS…
Use Containers
& Microservices
Leverage Open
Source Innovation
Adopt a
Dev-Ops Culture

3. 3
…but remains complex to manage at scale!
Source: Market Pulse Survey of Kubernetes Adoption
Management complexity is
the biggest hurdle overall
to wider adoption

4. 4
Agenda
• What Amazon EKS provides
• Why Virtualize K8s
• How to Virtualize K8s
• Q&A

5. 5
About me
• @JimBugwadia
• Founder and CEO at Nirmata
• Working on large-scale distributed
systems (C++, Java, JS, Go) since 1994
CKA-1700-0169-0100

6. Amazon EKS
6

7. 7
EKS Core Features
1. Managed Kubernetes Control Plane
2. Integrated with IAM
3. Logging (AWS CloudTrail)
4. Storage
o GP2 (AWS EBS); CSI drivers for EFS and FSx (Lustre) in alpha
5. Networking
o VPC networking support
o Elastic Load Balancing
o Service Mesh (AWS AppMesh)
o Service Registry (AWS CloudMap)

8. 8
Amazon EKS
https://eksworkshop.com/

9. 9
Nirmata EKS Provisioning

10. 10
Nirmata EKS Provisioning

11. 11
Nirmata EKS Provisioning

12. 12
Nirmata EKS Provisioning

13. 13

14. 14
Nirmata provides K8s lifecycle management

15. 15
Nirmata provides K8s lifecycle management

16. 16
Nirmata provides K8s lifecycle management

17. Virtual Clusters
Why?
17

18. 18
Virtualization
1. Software-defined form of a physical resource
2. No significant difference than actual resource
3. Some level of isolation and restrictions

19. 19
Why Virtualize
1. Virtualization makes things easier to use
Virtual Machine Physical Machine
Virtual Network Physical Network
Virtual Storage Physical Storage
HardEasy

20. 20
Why Virtualize
1. Virtualization makes things easier to use
2. Virtualization makes things easier to share
3. Virtualization optimizes resource usage
Physical
Virtual Virtual Virtual

21. 21
Why Virtualize
1. Virtualization makes things easier to use
2. Virtualization makes things easier to share
3. Virtualization optimizes resource usage
4. Virtualization enable new solutions and usage models
o Business Continuity (vMotion)
o Cloud Computing

22. 22
Why Virtualize Kubernetes
1. Easier to use (for developers and product teams)
2. Proper separation of concerns between Dev and Ops
3. Optimize resource utilization
4. Reduce number of clusters (and add-ons) to manage
5. Enable new usage patterns (placement, continuity, etc.)

23. Virtual Clusters
How?
23

24. 24
Kubernetes Architecture
kube-apiserver
kube-controller
kube-scheduler
etcd
kubelet
kube-proxy
Kubernetes Control Plane Worker Nodes

25. 25
Layered Architecture View
Data Plane
Control Plane
Management Plane Nirmata and Others
Kubernetes Components
Your Applications

26. 26
Control Plane Virtualization
• Each tenant gets their own (virtual) control plane
• Allows each tenants to use different resource versions,
namespaces, etc.
• Virtual Cluster Proposal from Alibaba:
o Uses a “tenant” master and “super” master. Scheduling is done by
the “super” control plane.
o Being reviewed in Kubernetes multi-tenancy WG
(github.com/kubernetes-sigs/multi-tenancy)
o Allows isolation of cluster-wide resources
o Workloads can share worker nodes

27. 27
Data Plane Virtualization
• Namespaces
Kubernetes supports multiple virtual clusters backed by
the same physical cluster. These virtual clusters are called
namespaces.
https://kubernetes.io/docs/concepts/over
view/working-with-objects/namespaces/

28. Pod Security Policies
28
Kubernetes Virtual Clusters
Limits & Quotas
Network Policies
Access Controls
Namespace
Workloads
(deployments, etc.)

29. 29
Kubernetes Virtual Clusters
Security
Backups / DR
Logging
Monitoring
Change Mgmt.
Ingress
Service MeshSecrets
Pod Security Policies
Limits & Quotas
Network Policies
Access Controls
Namespace
Workloads
(deployments, etc.)

30. 30
Kubernetes Virtual Clusters (a real-world example)
Pod Security Policies
Limits & Quotas
Network Policies
Access Controls
Namespace
Workloads
(deployments, etc.)

31. 31
WG Solutions and Proposals
• Tenant Custom Resource Definition
o Use a CRD to manage tenant lifecycle and namespaces
• Hierarchical Namespaces
o Create a “parent” namespace and allow “child” namespaces
o Hierarchical Namespace Controller - Design Doc
• Policy Management
o Use a Kubernetes policy management solution to generate tenant
configurations
• Some combination of above

32. 32
Kyverno – Kubernetes Native Policy Management
• Policies can provide common
configurations and enforce best
practices
• Kyverno is a open source policy
management tool designed for
Kubernetes
• Kyverno allows you to validate,
generate, and enforce
configurations per namespace /
workload
kyverno.io

33. 33
nctl – Nirmata CLI

34. 34
Virtual Clusters – Self Service
Check available
clusters

35. 35
Virtual Clusters – Self Service
Create
Environment

36. 36
Virtual Clusters – Self Service
Use kubectl

37. 37
Virtual Clusters – Self Service
Run workload

38. 38
Virtual Clusters – Self Service
Access is
restricted

39. 39
Nirmata provides complete visibility & management

40. 40
Nirmata audits workloads and reports violations
Confidential

41. Summary
41

42. 42

43. 43
Summary
1. Amazon EKS provide a reliable way of
provisioning and managing the
Kubernetes control plane
2. For production enterprise Kubernetes
managing cluster add-ons, shared
services, and Kubernetes
configurations is essential
3. Nirmata provide an easy way to build
self-service secure virtual clusters on
EKS (or any other managed K8s
service).

44. Nirmata – The Kubernetes Management Plane
Proprietary & Confidential44
Kubernetes Components, Services, and Workloads
K8s
Data Center
K8s
Clouds
K8s
Edge
Service Mgmt VisibilityGovernance Compliance Optimization
The Nirmata Platform
Change Mgmt. Audit Logs Health Capacity Isolation Policies TuningDiscovery
Any
Infrastructure
Nirmata Cloud
or
Private Edition
Any App

45. 45
Resources
• Kubernetes Multi-tenancy WG
o Kubernetes slack #wg-multitenancy
o https://github.com/kubernetes-sigs/multi-tenancy
• Nirmata - Explore for free at nirmata.io
• @JimBugwadia

46. Thank-You!
https://try.nirmata.io