[Global logic] container runtimes and kubernetes
•
0 j'aime•314 vues
GL DevOps Experts are committed to sharing with our community as much knowledge about Docker and Kubernetes as possible. Thinking about Kubernetes? Join Vadym Fabiianskiy and Andrii Mandubyra, GlobalLogic Lviv DevOps Experts and learn: Container Runtime specifics What are the building blocks of K8S? How does Kubernetes work? Deployment and release strategies
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à [Global logic] container runtimes and kubernetes
Similaire à [Global logic] container runtimes and kubernetes (20)
Plus de GlobalLogic Ukraine
Plus de GlobalLogic Ukraine (20)
Dernier
Dernier (20)
[Global logic] container runtimes and kubernetes
- 1. 1 Confidential Container Runtimes and Kubernetes ● Andriy Mandybura - Senior DevOps Engineer ● Vadym Fabiianskyi - Lead DevOps Engineer September 2020
- 2. 2 Confidential Description GL DevOps Experts are committed to sharing with our community as much knowledge about Docker and Kubernetes as possible. Thinking about Kubernetes? Join Vadym Fabiianskiy and Andrii Mandubyra, GlobalLogic Lviv DevOps Experts and learn: ● Container Runtime specifics ● What are the building blocks of K8S? ● How does Kubernetes work? ● Deployment and release strategies DevOps фахівці у GlobalLogic прагнуть поділитись із нашою спільнотою знаннями про Docker та Kubernetes. Думаєш про Kubernetes? Приєднайся до наших львівських DevOps фахівців Вадима Фабіянського та Андрія Мандибури та дізнайся більше про: ● Особливості Container Runtime ● Які складові K8S? ● Як працює Kubernetes? ● Стратегії розгортання та випуску програмного забезпечення
- 3. 3 Confidential Agenda 1. What is Container Runtime? 2. What is Container Runtime Interface? 3. Why is Container Runtime Interface used? 4. Q&A 5. How does Kubernetes work? 6. Deployment and release strategy 7. Q&A
- 4. Confidential ● Over 11 years of professional experience in IT industry. ● Cloud Infrastructure expert, including experience in building production-grade ecosystems in Telecom domain. ● Software engineering background in multiple domains. Expertise starting from cloud back-end solutions to OLAP/OLTP/Warehouse data processing. ● Participated in different projects development including back-end, web, cloud, real-time media processing, data analysis, payment processing projects for SaaS and OSS/BSS solutions. ● Expert in AWS/Azure cloud stacks. ● Infrastructure security expert. ● Experience in cutting edge technology DevOps stacks and solutions, such as Terraform, Kubernetes, GitLab, and Python. ● Experience of work in Scrum environment. ● A Master’s degree in Computer Sciences at Chisinau State University, Moldova. Vadym Fabiianskyi - Lead DevOps Engineer ● Більше 11 років досвіду в ІТ індустрії. ● Експерт у хмарній інфраструктурі, включаючи досвід розробки екосистем виробничого рівня у галузі телекомунікацій. ● Досвід програмного забезпечення у різних галузях. Навички в імплементації хмарних back-end рішень та обробці даних OLAP/OLTP/Warehouse. ● Брав участь у розробці різноманітних проектів, включаючи back-end, веб, хмарні рішення, обробку медіа в режимі реального часу, аналіз даних та проекти з обробки платежів для SaaS та OSS/BSS. ● Фахівець у хмарних стеках AWS/Azure. ● Фахівець з інфраструктурної безпеки. ● Досвід роботи із DevOps стеками та рішеннями передових технологій, таких як Terraform, Kubernetes, GitLab та Python. ● Досвід роботи зі Scrum. ● Ступінь магістра з комп’ютерних наук у Молдовському державному університеті.
- 5. Confidential ● 6 years of professional experience in IT industry. ● Cloud Infrastructure expert, including experience building production grade ecosystems in Big Data domain. ● Expert in AWS, Azure technologies stacks. ● Infrastructure security expert. ● A Master’s degree in Computer Sciences at Lviv Polytechnic University, Ukraine. Andrii Mandybura - Senior DevOps Engineer ● 6 років досвіду в ІТ індустрії. ● Фахівець у хмарній інфраструктурі, включаючи досвід розробки екосистем виробничого рівня у Big Data. ● Фахівець у стеках технологій AWS, Azure. ● Фахівець з інфраструктурної безпеки. ● Ступінь магістра з комп’ютерних наук у Національному університеті “Львівська політехніка”.
- 6. 6 Confidential 6 What is Container Runtime?
- 7. 7 Confidential Content 1. Containerization overview 2. Container Runtime specifics (Low/High) 3. Container Runtime Interface (CRI) 4. Docker and Dockerization 5. Docker in Kubernetes
- 8. 8 Confidential The way to the definition App 1 App 2 App 3 Bins/Lib Bins/Lib Bins/Lib Guest OS Guest OS Guest OS Hypervisor Infrastructure Virtual Machines App 1 App 2 App 3 Bins/Lib Bins/Lib Bins/Lib Operating System Infrastructure Container Engine Containers
- 9. 9 Confidential Types of Container Runtimes runc cri-o containerd Low Level High Level
- 10. 10 Confidential Overview of low-level Container Runtime Networking Other OS subsystems ... Linux namespaces Linux cgroups RUNC
- 11. 11 Confidential Pod A Cgroups Containerd overview Kubelet cri-containerd image service runtime service ocicni Pod A Namespaces Pod B sandbox container container A containerd shim containerd shim Client CRI gRPC
- 12. 12 Confidential CRI-O overview kubelet pod 1 pod 2 image service CNI OCI generate runtime service github.com/containers/image github.com/containers/storage common common common common commoninfra container infra container container A container B container C gRPC library library
- 13. 13 Confidential 13 What is Container Runtime Interface?
- 14. 14 Confidential Container Runtime Interface (CRI) ● CRI is the interface between the client and Container Runtime. ● With CRI, Kubernetes can communicate with multiple Container Runtimes. ● With a CRI client, like kubelet, Kubernetes can communicate with runtime (e.g., kata runc) to create and generate a container in the OS layer.
- 16. 16 Confidential Docker Engine overview Docker Engine Docker CLI Docker REST API Docker Server NetworkingContainerd Persistent Storage Tmpfs mounts Bind Mounts runC SnapshotterVolumes Network Driver EndpointSandbox IPAM Driver Linux Security Filesystem access CgroupsNamespace Storage Driver NoneOverlayBridgeHost macvlan
- 17. 17 Confidential Docker in Kubernetes ecosystem ● Docker is more than enough for Kubernetes. ● Mismatch of release sync between Docker and Kubernetes. ● Extra memory/CPU use due to extra layer with Docker.
- 18. 18 Confidential
- 19. 19 Confidential 19 How does Kubernetes work?
- 20. 20 Confidential Contents 1. Orchestration a. Ways of applying b. When do we need an orchestrator c. Why Kubernetes (advantages) 2. What is Kubernetes a. history and some facts b. components c. objects d. on-premises setup e. cloud setup: kubernetes managed services 3. Use cases a. setup b. configuration c. objects d. special objects e. scalability f. monitoring 4. Deployment strategies 5. Materials to learn
- 21. 21 Confidential Orchestration - ways of applying ● Docker ● Docker-compose ● Docker Swarm (Docker, Inc.) ● Amazon ECS (AWS managed service) ● Hashicorp Nomad (exotic, but simple) ● Kubernetes ● RedHat Openshift (Kubernetes+) ● Marathon (Apache Mesos) one host/localhost Development Environment several hosts/cluster Production Environment
- 22. 22 Confidential Orchestration When do we need an orchestrator ● Frequent releases ● Microservices + lots of containers ● DevOps, SRE, IaC way ● Custom and flexible infrastructure management logic + auto scalability ● High level of fault tolerance (SLA 99.9) ● You would like to have sweet dreams :)
- 23. 23 Confidential Orchestration Why Kubernetes (advantages) ● Production ready ● Huge ecosystem ● Active development ● Extendable ● Scalability ● Features ● Cloud native/friendly
- 24. 24 Confidential What is Kubernetes History and some facts ● Was founded by Joe Beda, Brendan Burns, and Craig McLuckie + other Google engineers ● Heavily influenced by Google's Borg system ● First release in 2014 ● Implemented in Go ● Google partnered with the Linux Foundation to form the Cloud Native Computing Foundation (CNCF) ● On March 6, 2018, Kubernetes Project reached ninth place in commits at GitHub
- 25. 25 Confidential Kubernetes Control Plane Kubernetes Nodes kube-apiserver kubelet kube-proxy kubelet kube-proxy kubelet kube-proxy etcd kube-controller manager cloud-controller manager kube-scheduler What is Kubernetes - components
- 26. 26 Confidential Host A Pod Container Container Network Namespace 10.10.16.3 Pod Network External Volume What is Kubernetes - Kubernetes Pod architecture
- 27. 27 Confidential What is Kubernetes - Kubernetes Pod yaml example
- 28. 28 Confidential What is Kubernetes - Kubernetes components Pod startup flow DockerAPI Server etcd Scheduler Kubelet create Pod write watch (new Pod) bind Pod write watch (bound Pod) Docker run update Pod status write
- 29. 29 Confidential What is Kubernetes - Kubernetes objects ReplicaSet DaemonSet Deployment StatefulSet Job CronJob CRD Service Ingress NamespaceConfigMap PV Secrets Pod
- 30. 30 Confidential What is Kubernetes - on-premise setup etcd controller manager scheduler kube- apiserver Control Plane nodes - 1, 2, … n kubectl Node 1 Pods System Services kubelet Container Runtime End users LB Node 1 Load Balancer Storage Node 2 Pods System Services kubelet Container Runtime Pods System Services kubelet Container Runtime
- 31. 31 Confidential What is Kubernetes - cloud setup etcd controller manager scheduler kube- apiserver Control Plane kubectl Node 1 Pods System Services kubelet Container Runtime End users Cloud Provider Network Edge Load Balancer Storage Node 2 Pods System Services kubelet Container Runtime
- 32. 32 Confidential Use case On-premise setup ● VM (for control plane, nodes) ● Setup tools: Kubespray, Kubeadm, Kops Cloud setup ● Cloud account ● VM (web console or infrastructure management tools, for nodes only) ● Setup tools: Kubespray, Kops, eksctl (AWS)
- 33. 33 Confidential Use case Configuration ● yaml files and kubectl ● Configuration management tools (Ansible, etc.) ● Helm Objects ● Deployments (applications) ● Services (applications) ● PVs/PVCs (applications) ● StatefulSets (databases) ● DaemonSets (monitoring) ● Ingresses Special objects ● Ingress Controllers (Nginx, ALB) ● Controllers, Operators, CRD (cert-manager, core-dns, external-dns)
- 34. 34 Confidential Use case Scalability ● HPA ● Cluster autoscaler (AWS, GKE, Azure, Alibaba, DO, etc.) ● Cloud native (AWS ASG) Monitoring ● Visualization: Kubernetes Dashboard (+ out-of-the-box cloud native solutions) ● Logging: fluentD + Elasticsearch + Kibana (+ out-of-the-box cloud native solutions) ● Metrics: Metrics-server, Prometheus (+ out-of-the-box cloud native solutions)
- 35. 35 Confidential 35 Deployment and release strategy
- 36. 36 Confidential Deployment strategies Rolling Deployment time time instances instances Blue-Green Release time Canary Release 2x capacity Recreate Deployment time0 … 1 capacity Load Balancer V1 V1 V1 Load Balancer V1 V1 V1 V2 V2 V2 Load Balancer V2 V2 V2 V1 V1 V1 instances instances
- 37. 37 Confidential Materials to learn ● Security ● Service Discovery ● Networking ● Scalability ● Storage ● Logging ● Monitoring
- 38. 38 Confidential
- 40. 40 Confidential Core Concept KBs Object Store Kubelet Kube controller etcd Kube scheduler ContainerContainer ImageImage Container Runtime Kube- apiserver Master Worker
- 41. 41 Confidential Components Flow 1/2 User creates Deployment Deployment Create Event Reconcile Deployment Create ReplicaSet Reconcile ReplicaSet Reconcile Pods Reconcile ReplicaSet Reconcile Deployment Reconcile Pods Start Container API API Deployment controller Deployment controller Replication controller Replication controller Scheduler Scheduler Node(Kubelet) Node(Kubelet) ReplicaSet Create Event Create Pods Pod Create Event Schedule Podes to Nodes Pod Update Event Update Pod Status Pod Update Events Pod Update Events ReplicaSet Update Events Update Deployment Status
- 42. 42 Confidential Components Flow 2/2 Controller-managerClient API Server etcd Scheduler Kubelet Docker 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
- 43. 43 Confidential Understanding Kubernetes API Server Event Queues (immutable Logs) ConsumerProducer Kubernetes Metric Server Horizontal Pod Auroscaler ReplicaSet Controller Scheduler Kubelet “X” CPU Usage is 20 % CPU Usage is 30 % CPU Usage is 80 % CPU Usage is 30 % Scaled to 2 Scaled to 3 Scaled to 8 Added 2 Pods Added 1 Pod Bound Pod to Kubelet “X” Bound Pod to Kubelet “X” Bound Pod to Kubelet “X” Horizontal Pod Auroscaler ReplicaSet Controller Scheduler Metrics Pod ReplicaSet Pod
- 44. 44 Confidential Abstractions and primitives Main Container Sidecar node.js Disk git Pod Sidecar Pattern Container Container python memcached Pod Sidecar Pattern localhost app containers init containers Pod Initializer Pattern Main Sidecar Container 2 Container 1 ExecutionSequence Main Container Sidecar java monitoring Pod Adapter Pattern Container A Container B ENV_A1 ENV_A2 /etc/annotations /etc/labels volume A Pod Self Awareness Pattern ENV_B1 ENV_B2 Pod Manifest + Runtime Information API Server Kubernetes client inject query mount Application Pod Application Pod Work Execution Container Custom Work Handler Work Execution Container Custom Work Handler Work Queue Persistence Work Coordinator Container Work Queue Pattern Disk
- 45. What is the best Container Runtime solution ? 45
- 46. Containerd (CRI runtime solution) ● Graduated by CNCF on February 28, 2019. ● Containerd is an OCI compliant core container runtime designed to be embedded into larger systems ● Comparing by Docker containerd provides the minimum set of functionality to execute containers and manage images on node. 46
- 49. Integration of Containerd in kubernetes 49
- 50. Containerd vs. Docker Compared with the current Docker CRI implementation (dockershim), cri-containerd eliminates an extra hop in the stack, making the stack more stable and efficient. 50
- 51. CRI-O ● CRI-O is a Cloud Native Computing Foundation incubating project. ● CRI-O implements the Kubelet Container Runtime Interface (CRI) using OCI ● Comparing by Docker CRI-O provides the minimum set of functionality to execute containers and manage images on node. 51
- 52. Cri-Containerd vs Cri-o (Bucketbench) 52 Solution 1 thrd 2 thrd 3 thrd 4 thrd 5 thrd 6 thrd 7 thrd 8 thrd 9 thrd 10 thrd Cri-o 0.08 0.13 0.18 CRIContaine rd 0.14 0.25 0.35 0.44 0.49 For Bucketbench project, you can find the code at https://github.com/estesp/bucketbench
- 54. Container runtime interface And Container network interface 54
- 55. Cri & Cni ● Container runtime provides the right configuration to the Container network interface plugin. ● Container runtime invokes Network Plugin (bridge) when container is ADDeD. ● Container runtime will create network namespaces with the help of the network plugin. ● Container network interface Identify the network the container must attach to 55
- 56. Demo 56
- 57. 57 Confidential Conclusion In Kubernetes, there are a master node and multiple worker nodes. Each worker node can handle multiple pods. Pods are just a bunch of containers clustered together as a working unit. You can start designing your applications using pods. Once your pods are ready, you can specify pod definitions to the master node, and how many you want to deploy. From this point, Kubernetes is in control. It takes the pods and deploys them to the worker nods.