SlideShare une entreprise Scribd logo
1  sur  19
Télécharger pour lire hors ligne
Version 1.0
Cassandra on Kubernetes
An Anant Corporation Story.
Docker/Kubernetes/Helm Part 1
Containers : “Get with the program”
● Docker - One “Dockerfile” becomes an Image which runs as one container on a node / host.
● Docker-Compose - Can run multiple containers on a node / host in as a composition. (Kompose)
● Docker Swarm - Can run docker-compose at scale on many node / hosts.
● Mesosphere/DCOS - Mesos, grand daddy of containerization adopted Docker, and Kubernetes
● Kubernetes - Basically everyone who does anything uses Kubernetes to deploy big applications.
Docker vs VM
Docker Architecture
Docker Images Process
Docker Commands and Processes
Kubernetes General Architecture
Nodes (minimum of 3 physical nodes)
■ Master Nodes / Control Plane (at least 1)
● A master node is a node which controls and
manages a set of worker nodes (workloads
runtime) and resembles a cluster in Kubernetes. ...
All external communication to the cluster is via the
API-Server. Kube-Controller-Manager, which runs a
set of controllers for the running cluster.
● Whenever master node under kubernetes fails, the
cluster still remains in an operational mode. It
doesn't affect pod creation or service member
changes. If worker node fails, master stops
receiving updates from worker node
● High availability / multi-master nodes use a load
balancer to manage the API server which serves
both the clients and the worker nodes
Kubernetes General Architecture
Worker Nodes (at least 2)
● The worker nodes run the workloads. The worker node
contains kubelet and kube-proxy that both connects to the
pods within the docker. ... A kubelet takes information
from the master node and ensures that any pods assigned
to it are running and configured in the desired state. All
Kubernetes nodes must have a kubelet.
Kubernetes General Architecture
Containers
■ By themselves, don’t do much in Kubernetes.
■ Need to make it part of a Pod.
Kubernetes General Architecture
Pods
■ Pods are the smallest, most basic deployable objects in
Kubernetes. A Pod represents a single instance of a
running process in your cluster. Pods contain one or more
containers, such as Docker containers. When a Pod runs
multiple containers, the containers are managed as a
single entity and share the Pod's resources.
Kubernetes General Architecture
Workloads
■ Deployments / ReplicaSet (Stateless)
● Node Express
● Python Flask
● Java Springboot
■ StatefulSets (State)
● MySQL
● Cassandra
● Leverages “PersistentVolume” that is managed as a resource.
■ DaemonSet
● “Sidecar” local to that node
● Monitoring / Management
■ Job / CronJob
● Tasks that run and stop.
● Job - one-off tasks
● CronJob - recurring tasks
Helm Charts
○ Components
■ The client (CLI), which lives on your local
workstation.
■ The server (Tiller), which lives on the Kubernetes
cluster to execute what’s needed.
○ Concepts
■ Chart: A package of pre-configured Kubernetes
resources.
■ Release: A specific instance of a chart which has
been deployed to the cluster using Helm.
■ Repository: A group of published charts which can
be made available to others.
○ Why?
■ Boosts productivity
■ Reduces duplication & complexity
■ Smoothens the learning curve
■ Simplifies deployments
Helm Charts
Helm Charts
Operators
○ The Operator pattern aims to capture the key aim of a human operator who is managing a service or set of services.
Human operators who look after specific applications and services have deep knowledge of how the system ought to
behave, how to deploy it, and how to react if there are problems.
○ Tasks
■ deploying an application on demand
■ taking and restoring backups of that application's state
■ handling upgrades of the application code alongside related changes such as database schemas or extra
configuration settings
■ publishing a Service to applications that don't support Kubernetes APIs to discover them
■ simulating failure in all or part of your cluster to test its resilience
■ choosing a leader for a distributed application without an internal member election process
○ Why?
■ Robots are better at doing repetitive tasks
■ Simplifies the overall complexity of an operation into a few configurable components
■ Makes pattern reuse possible
Helm Charts
○ How to make Operators? (via Operator SDK)
■ Helm
■ Go
■ Java
■ Ansible
Cassandra on Kubernetes
Cassandra on Kubernetes
Resources
● https://kubernetes.io/docs/tutorials/kubernetes-basics/explore/explore-intro/
● https://kubernetes.io/docs/concepts/workloads/
● https://medium.com/velotio-perspectives/demystifying-high-availability-in-kubernetes-using-
kubeadm-3d83ed8c458b#:~:text=Kubernetes High-Availability is about,access to same
worker nodes.
● https://enterprisersproject.com/article/2020/9/pod-cluster-container-what-is-difference
● https://medium.com/developerworld/pod-vs-node-in-kubernetes-26c858988f94
● https://www.bmc.com/blogs/kubernetes-helm-charts/
● https://www.vamsitalkstech.com/architecture/big-data-kubernetes-a-reference-architecture-
for-spark-with-kubernetes-2-4/
● banzaicloud/kafka-operator
● https://www.vembu.com/blog/docker-vs-kubernetes-differences-architecture-and-use-cases/
● https://tanzu.vmware.com/developer/guides/kubernetes/create-helm-chart/
Strategy: Scalable Fast Data
Architecture: Cassandra, Spark, Kafka
Engineering: Node, Python, JVM,CLR
Operations: Cloud, Container
Rescue: Downtime!! I need help.
 www.anant.us | solutions@anant.us | (855) 262-6826
3 Washington Circle, NW | Suite 301 | Washington, DC 20037

Contenu connexe

Tendances

The (mutable) config management showdown
The (mutable) config management showdownThe (mutable) config management showdown
The (mutable) config management showdownBob Killen
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetesRishabh Indoria
 
KubeCon EU 2016: Bringing an open source Containerized Container Platform to ...
KubeCon EU 2016: Bringing an open source Containerized Container Platform to ...KubeCon EU 2016: Bringing an open source Containerized Container Platform to ...
KubeCon EU 2016: Bringing an open source Containerized Container Platform to ...KubeAcademy
 
Building CLI Applications with Golang
Building CLI Applications with GolangBuilding CLI Applications with Golang
Building CLI Applications with GolangAnshul Patel
 
Kubernetes presentation
Kubernetes presentationKubernetes presentation
Kubernetes presentationGauranG Bajpai
 
Deploying your first application with Kubernetes
Deploying your first application with KubernetesDeploying your first application with Kubernetes
Deploying your first application with KubernetesOVHcloud
 
Kubernetes - introduction
Kubernetes - introductionKubernetes - introduction
Kubernetes - introductionSparkbit
 
Leverage LXC/LXD with Kubernetes
Leverage LXC/LXD with KubernetesLeverage LXC/LXD with Kubernetes
Leverage LXC/LXD with KubernetesLin Sun
 
Ansible, integration testing, and you.
Ansible, integration testing, and you.Ansible, integration testing, and you.
Ansible, integration testing, and you.Bob Killen
 
Kubernetes: An Introduction to the Open Source Container Orchestration Platform
Kubernetes: An Introduction to the Open Source Container Orchestration PlatformKubernetes: An Introduction to the Open Source Container Orchestration Platform
Kubernetes: An Introduction to the Open Source Container Orchestration PlatformMichael O'Sullivan
 
Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes IntroductionEric Gustafson
 
The Rise of the Container: The Dev/Ops Technology That Accelerates Ops/Dev
The Rise of the Container:  The Dev/Ops Technology That Accelerates Ops/DevThe Rise of the Container:  The Dev/Ops Technology That Accelerates Ops/Dev
The Rise of the Container: The Dev/Ops Technology That Accelerates Ops/DevRobert Starmer
 
Platform Orchestration with Kubernetes and Docker
Platform Orchestration with Kubernetes and DockerPlatform Orchestration with Kubernetes and Docker
Platform Orchestration with Kubernetes and DockerJulian Strobl
 
Fission: Serverless Functions for Kubernetes
Fission: Serverless Functions for KubernetesFission: Serverless Functions for Kubernetes
Fission: Serverless Functions for KubernetesSoam Vasani
 
A practical introduction to Kubernetes (IPC 2018)
A practical introduction to Kubernetes (IPC 2018)A practical introduction to Kubernetes (IPC 2018)
A practical introduction to Kubernetes (IPC 2018)Robert Lemke
 

Tendances (20)

The (mutable) config management showdown
The (mutable) config management showdownThe (mutable) config management showdown
The (mutable) config management showdown
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
KubeCon EU 2016: Bringing an open source Containerized Container Platform to ...
KubeCon EU 2016: Bringing an open source Containerized Container Platform to ...KubeCon EU 2016: Bringing an open source Containerized Container Platform to ...
KubeCon EU 2016: Bringing an open source Containerized Container Platform to ...
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
 
Building CLI Applications with Golang
Building CLI Applications with GolangBuilding CLI Applications with Golang
Building CLI Applications with Golang
 
Kubernetes presentation
Kubernetes presentationKubernetes presentation
Kubernetes presentation
 
Deploying your first application with Kubernetes
Deploying your first application with KubernetesDeploying your first application with Kubernetes
Deploying your first application with Kubernetes
 
Kubernetes - introduction
Kubernetes - introductionKubernetes - introduction
Kubernetes - introduction
 
Leverage LXC/LXD with Kubernetes
Leverage LXC/LXD with KubernetesLeverage LXC/LXD with Kubernetes
Leverage LXC/LXD with Kubernetes
 
Ansible, integration testing, and you.
Ansible, integration testing, and you.Ansible, integration testing, and you.
Ansible, integration testing, and you.
 
Kubernetes: An Introduction to the Open Source Container Orchestration Platform
Kubernetes: An Introduction to the Open Source Container Orchestration PlatformKubernetes: An Introduction to the Open Source Container Orchestration Platform
Kubernetes: An Introduction to the Open Source Container Orchestration Platform
 
Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes Introduction
 
The Rise of the Container: The Dev/Ops Technology That Accelerates Ops/Dev
The Rise of the Container:  The Dev/Ops Technology That Accelerates Ops/DevThe Rise of the Container:  The Dev/Ops Technology That Accelerates Ops/Dev
The Rise of the Container: The Dev/Ops Technology That Accelerates Ops/Dev
 
Platform Orchestration with Kubernetes and Docker
Platform Orchestration with Kubernetes and DockerPlatform Orchestration with Kubernetes and Docker
Platform Orchestration with Kubernetes and Docker
 
Swarm migration
Swarm migrationSwarm migration
Swarm migration
 
Fission: Serverless Functions for Kubernetes
Fission: Serverless Functions for KubernetesFission: Serverless Functions for Kubernetes
Fission: Serverless Functions for Kubernetes
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
 
Containers and HPC
Containers and HPCContainers and HPC
Containers and HPC
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
A practical introduction to Kubernetes (IPC 2018)
A practical introduction to Kubernetes (IPC 2018)A practical introduction to Kubernetes (IPC 2018)
A practical introduction to Kubernetes (IPC 2018)
 

Similaire à Apache Cassandra Lunch #41: Cassandra on Kubernetes - Docker/Kubernetes/Helm Part 1

Scalable Spark deployment using Kubernetes
Scalable Spark deployment using KubernetesScalable Spark deployment using Kubernetes
Scalable Spark deployment using Kubernetesdatamantra
 
Kubernetes #1 intro
Kubernetes #1   introKubernetes #1   intro
Kubernetes #1 introTerry Cho
 
Kubernetes workshop
Kubernetes workshopKubernetes workshop
Kubernetes workshopKumar Gaurav
 
Google Cloud Platform Kubernetes Workshop IYTE
Google Cloud Platform Kubernetes Workshop IYTEGoogle Cloud Platform Kubernetes Workshop IYTE
Google Cloud Platform Kubernetes Workshop IYTEGokhan Boranalp
 
Spark on Kubernetes
Spark on KubernetesSpark on Kubernetes
Spark on Kubernetesdatamantra
 
CD in kubernetes using helm and ksonnet. Stas Kolenkin
CD in kubernetes using helm and ksonnet. Stas KolenkinCD in kubernetes using helm and ksonnet. Stas Kolenkin
CD in kubernetes using helm and ksonnet. Stas KolenkinDataArt
 
Kubernetes 101 for Beginners
Kubernetes 101 for BeginnersKubernetes 101 for Beginners
Kubernetes 101 for BeginnersOktay Esgul
 
08 - kubernetes.pptx
08 - kubernetes.pptx08 - kubernetes.pptx
08 - kubernetes.pptxRanjithM61
 
Containers kuberenetes
Containers kuberenetesContainers kuberenetes
Containers kuberenetescsegayan
 
Kubernetes a comprehensive overview
Kubernetes   a comprehensive overviewKubernetes   a comprehensive overview
Kubernetes a comprehensive overviewGabriel Carro
 
Kubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverviewKubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverviewAnkit Shukla
 
Kubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewKubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewBob Killen
 
Kubernetes and CoreOS @ Athens Docker meetup
Kubernetes and CoreOS @ Athens Docker meetupKubernetes and CoreOS @ Athens Docker meetup
Kubernetes and CoreOS @ Athens Docker meetupMist.io
 
Academy PRO: Docker. Part 1
Academy PRO: Docker. Part 1Academy PRO: Docker. Part 1
Academy PRO: Docker. Part 1Binary Studio
 

Similaire à Apache Cassandra Lunch #41: Cassandra on Kubernetes - Docker/Kubernetes/Helm Part 1 (20)

Scalable Spark deployment using Kubernetes
Scalable Spark deployment using KubernetesScalable Spark deployment using Kubernetes
Scalable Spark deployment using Kubernetes
 
Kubernetes #1 intro
Kubernetes #1   introKubernetes #1   intro
Kubernetes #1 intro
 
Kubernetes workshop
Kubernetes workshopKubernetes workshop
Kubernetes workshop
 
Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
 
Google Cloud Platform Kubernetes Workshop IYTE
Google Cloud Platform Kubernetes Workshop IYTEGoogle Cloud Platform Kubernetes Workshop IYTE
Google Cloud Platform Kubernetes Workshop IYTE
 
Gdg izmir kubernetes
Gdg izmir kubernetesGdg izmir kubernetes
Gdg izmir kubernetes
 
Spark on Kubernetes
Spark on KubernetesSpark on Kubernetes
Spark on Kubernetes
 
CD in kubernetes using helm and ksonnet. Stas Kolenkin
CD in kubernetes using helm and ksonnet. Stas KolenkinCD in kubernetes using helm and ksonnet. Stas Kolenkin
CD in kubernetes using helm and ksonnet. Stas Kolenkin
 
Kubernetes 101 for Beginners
Kubernetes 101 for BeginnersKubernetes 101 for Beginners
Kubernetes 101 for Beginners
 
JOSA TechTalks - Docker in Production
JOSA TechTalks - Docker in ProductionJOSA TechTalks - Docker in Production
JOSA TechTalks - Docker in Production
 
08 - kubernetes.pptx
08 - kubernetes.pptx08 - kubernetes.pptx
08 - kubernetes.pptx
 
Containers kuberenetes
Containers kuberenetesContainers kuberenetes
Containers kuberenetes
 
Containers kuberenetes
Containers kuberenetesContainers kuberenetes
Containers kuberenetes
 
Containers kuberenetes
Containers kuberenetesContainers kuberenetes
Containers kuberenetes
 
Kubernetes PPT.pptx
Kubernetes PPT.pptxKubernetes PPT.pptx
Kubernetes PPT.pptx
 
Kubernetes a comprehensive overview
Kubernetes   a comprehensive overviewKubernetes   a comprehensive overview
Kubernetes a comprehensive overview
 
Kubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverviewKubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverview
 
Kubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewKubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive Overview
 
Kubernetes and CoreOS @ Athens Docker meetup
Kubernetes and CoreOS @ Athens Docker meetupKubernetes and CoreOS @ Athens Docker meetup
Kubernetes and CoreOS @ Athens Docker meetup
 
Academy PRO: Docker. Part 1
Academy PRO: Docker. Part 1Academy PRO: Docker. Part 1
Academy PRO: Docker. Part 1
 

Plus de Anant Corporation

QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137Anant Corporation
 
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdfKono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdfAnant Corporation
 
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache PinotData Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache PinotAnant Corporation
 
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...Anant Corporation
 
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPTAutomate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPTAnant Corporation
 
Episode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
Episode 2: The LLM / GPT / AI Prompt / Data Engineer RoadmapEpisode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
Episode 2: The LLM / GPT / AI Prompt / Data Engineer RoadmapAnant Corporation
 
Machine Learning Orchestration with Airflow
Machine Learning Orchestration with AirflowMachine Learning Orchestration with Airflow
Machine Learning Orchestration with AirflowAnant Corporation
 
Cassandra Lunch 130: Recap of Cassandra Forward Talks
Cassandra Lunch 130: Recap of Cassandra Forward TalksCassandra Lunch 130: Recap of Cassandra Forward Talks
Cassandra Lunch 130: Recap of Cassandra Forward TalksAnant Corporation
 
Data Engineer's Lunch 90: Migrating SQL Data with Arcion
Data Engineer's Lunch 90: Migrating SQL Data with ArcionData Engineer's Lunch 90: Migrating SQL Data with Arcion
Data Engineer's Lunch 90: Migrating SQL Data with ArcionAnant Corporation
 
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...Anant Corporation
 
Cassandra Lunch 129: What’s New: Apache Cassandra 4.1+ Features & Future
Cassandra Lunch 129: What’s New:  Apache Cassandra 4.1+ Features & FutureCassandra Lunch 129: What’s New:  Apache Cassandra 4.1+ Features & Future
Cassandra Lunch 129: What’s New: Apache Cassandra 4.1+ Features & FutureAnant Corporation
 
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...Anant Corporation
 
Data Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackData Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackAnant Corporation
 
Data Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
Data Engineer's Lunch #83: Strategies for Migration to Apache IcebergData Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
Data Engineer's Lunch #83: Strategies for Migration to Apache IcebergAnant Corporation
 
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOpsApache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOpsAnant Corporation
 
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache CassandraApache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache CassandraAnant Corporation
 
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...Anant Corporation
 
Data Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessData Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessAnant Corporation
 

Plus de Anant Corporation (20)

QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
 
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdfKono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
 
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache PinotData Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
 
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
 
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPTAutomate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
 
YugabyteDB Developer Tools
YugabyteDB Developer ToolsYugabyteDB Developer Tools
YugabyteDB Developer Tools
 
Episode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
Episode 2: The LLM / GPT / AI Prompt / Data Engineer RoadmapEpisode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
Episode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
 
Machine Learning Orchestration with Airflow
Machine Learning Orchestration with AirflowMachine Learning Orchestration with Airflow
Machine Learning Orchestration with Airflow
 
Cassandra Lunch 130: Recap of Cassandra Forward Talks
Cassandra Lunch 130: Recap of Cassandra Forward TalksCassandra Lunch 130: Recap of Cassandra Forward Talks
Cassandra Lunch 130: Recap of Cassandra Forward Talks
 
Data Engineer's Lunch 90: Migrating SQL Data with Arcion
Data Engineer's Lunch 90: Migrating SQL Data with ArcionData Engineer's Lunch 90: Migrating SQL Data with Arcion
Data Engineer's Lunch 90: Migrating SQL Data with Arcion
 
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
 
Cassandra Lunch 129: What’s New: Apache Cassandra 4.1+ Features & Future
Cassandra Lunch 129: What’s New:  Apache Cassandra 4.1+ Features & FutureCassandra Lunch 129: What’s New:  Apache Cassandra 4.1+ Features & Future
Cassandra Lunch 129: What’s New: Apache Cassandra 4.1+ Features & Future
 
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
 
Data Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackData Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data Stack
 
CL 121
CL 121CL 121
CL 121
 
Data Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
Data Engineer's Lunch #83: Strategies for Migration to Apache IcebergData Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
Data Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
 
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOpsApache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
 
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache CassandraApache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
 
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
 
Data Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessData Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise Consciousness
 

Dernier

Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshareraiaryan448
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabiaahmedjiabur940
 
Ranking and Scoring Exercises for Research
Ranking and Scoring Exercises for ResearchRanking and Scoring Exercises for Research
Ranking and Scoring Exercises for ResearchRajesh Mondal
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesBoston Institute of Analytics
 
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarjSCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarjadimosmejiaslendon
 
Pentesting_AI and security challenges of AI
Pentesting_AI and security challenges of AIPentesting_AI and security challenges of AI
Pentesting_AI and security challenges of AIf6x4zqzk86
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样wsppdmt
 
sourabh vyas1222222222222222222244444444
sourabh vyas1222222222222222222244444444sourabh vyas1222222222222222222244444444
sourabh vyas1222222222222222222244444444saurabvyas476
 
Case Study 4 Where the cry of rebellion happen?
Case Study 4 Where the cry of rebellion happen?Case Study 4 Where the cry of rebellion happen?
Case Study 4 Where the cry of rebellion happen?RemarkSemacio
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证pwgnohujw
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...yulianti213969
 
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...LuisMiguelPaz5
 
jll-asia-pacific-capital-tracker-1q24.pdf
jll-asia-pacific-capital-tracker-1q24.pdfjll-asia-pacific-capital-tracker-1q24.pdf
jll-asia-pacific-capital-tracker-1q24.pdfjaytendertech
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格q6pzkpark
 
DAA Assignment Solution.pdf is the best1
DAA Assignment Solution.pdf is the best1DAA Assignment Solution.pdf is the best1
DAA Assignment Solution.pdf is the best1sinhaabhiyanshu
 
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Klinik Aborsi
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteedamy56318795
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNKTimothy Spann
 
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...Voces Mineras
 

Dernier (20)

Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshare
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
 
Ranking and Scoring Exercises for Research
Ranking and Scoring Exercises for ResearchRanking and Scoring Exercises for Research
Ranking and Scoring Exercises for Research
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting Techniques
 
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarjSCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
 
Pentesting_AI and security challenges of AI
Pentesting_AI and security challenges of AIPentesting_AI and security challenges of AI
Pentesting_AI and security challenges of AI
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
 
sourabh vyas1222222222222222222244444444
sourabh vyas1222222222222222222244444444sourabh vyas1222222222222222222244444444
sourabh vyas1222222222222222222244444444
 
Case Study 4 Where the cry of rebellion happen?
Case Study 4 Where the cry of rebellion happen?Case Study 4 Where the cry of rebellion happen?
Case Study 4 Where the cry of rebellion happen?
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
 
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
 
jll-asia-pacific-capital-tracker-1q24.pdf
jll-asia-pacific-capital-tracker-1q24.pdfjll-asia-pacific-capital-tracker-1q24.pdf
jll-asia-pacific-capital-tracker-1q24.pdf
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
 
DAA Assignment Solution.pdf is the best1
DAA Assignment Solution.pdf is the best1DAA Assignment Solution.pdf is the best1
DAA Assignment Solution.pdf is the best1
 
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
 
Abortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
Abortion pills in Doha {{ QATAR }} +966572737505) Get CytotecAbortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
Abortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
 
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
 

Apache Cassandra Lunch #41: Cassandra on Kubernetes - Docker/Kubernetes/Helm Part 1

  • 1. Version 1.0 Cassandra on Kubernetes An Anant Corporation Story. Docker/Kubernetes/Helm Part 1
  • 2. Containers : “Get with the program” ● Docker - One “Dockerfile” becomes an Image which runs as one container on a node / host. ● Docker-Compose - Can run multiple containers on a node / host in as a composition. (Kompose) ● Docker Swarm - Can run docker-compose at scale on many node / hosts. ● Mesosphere/DCOS - Mesos, grand daddy of containerization adopted Docker, and Kubernetes ● Kubernetes - Basically everyone who does anything uses Kubernetes to deploy big applications.
  • 7. Kubernetes General Architecture Nodes (minimum of 3 physical nodes) ■ Master Nodes / Control Plane (at least 1) ● A master node is a node which controls and manages a set of worker nodes (workloads runtime) and resembles a cluster in Kubernetes. ... All external communication to the cluster is via the API-Server. Kube-Controller-Manager, which runs a set of controllers for the running cluster. ● Whenever master node under kubernetes fails, the cluster still remains in an operational mode. It doesn't affect pod creation or service member changes. If worker node fails, master stops receiving updates from worker node ● High availability / multi-master nodes use a load balancer to manage the API server which serves both the clients and the worker nodes
  • 8. Kubernetes General Architecture Worker Nodes (at least 2) ● The worker nodes run the workloads. The worker node contains kubelet and kube-proxy that both connects to the pods within the docker. ... A kubelet takes information from the master node and ensures that any pods assigned to it are running and configured in the desired state. All Kubernetes nodes must have a kubelet.
  • 9. Kubernetes General Architecture Containers ■ By themselves, don’t do much in Kubernetes. ■ Need to make it part of a Pod.
  • 10. Kubernetes General Architecture Pods ■ Pods are the smallest, most basic deployable objects in Kubernetes. A Pod represents a single instance of a running process in your cluster. Pods contain one or more containers, such as Docker containers. When a Pod runs multiple containers, the containers are managed as a single entity and share the Pod's resources.
  • 11. Kubernetes General Architecture Workloads ■ Deployments / ReplicaSet (Stateless) ● Node Express ● Python Flask ● Java Springboot ■ StatefulSets (State) ● MySQL ● Cassandra ● Leverages “PersistentVolume” that is managed as a resource. ■ DaemonSet ● “Sidecar” local to that node ● Monitoring / Management ■ Job / CronJob ● Tasks that run and stop. ● Job - one-off tasks ● CronJob - recurring tasks
  • 12. Helm Charts ○ Components ■ The client (CLI), which lives on your local workstation. ■ The server (Tiller), which lives on the Kubernetes cluster to execute what’s needed. ○ Concepts ■ Chart: A package of pre-configured Kubernetes resources. ■ Release: A specific instance of a chart which has been deployed to the cluster using Helm. ■ Repository: A group of published charts which can be made available to others. ○ Why? ■ Boosts productivity ■ Reduces duplication & complexity ■ Smoothens the learning curve ■ Simplifies deployments
  • 14. Helm Charts Operators ○ The Operator pattern aims to capture the key aim of a human operator who is managing a service or set of services. Human operators who look after specific applications and services have deep knowledge of how the system ought to behave, how to deploy it, and how to react if there are problems. ○ Tasks ■ deploying an application on demand ■ taking and restoring backups of that application's state ■ handling upgrades of the application code alongside related changes such as database schemas or extra configuration settings ■ publishing a Service to applications that don't support Kubernetes APIs to discover them ■ simulating failure in all or part of your cluster to test its resilience ■ choosing a leader for a distributed application without an internal member election process ○ Why? ■ Robots are better at doing repetitive tasks ■ Simplifies the overall complexity of an operation into a few configurable components ■ Makes pattern reuse possible
  • 15. Helm Charts ○ How to make Operators? (via Operator SDK) ■ Helm ■ Go ■ Java ■ Ansible
  • 18. Resources ● https://kubernetes.io/docs/tutorials/kubernetes-basics/explore/explore-intro/ ● https://kubernetes.io/docs/concepts/workloads/ ● https://medium.com/velotio-perspectives/demystifying-high-availability-in-kubernetes-using- kubeadm-3d83ed8c458b#:~:text=Kubernetes High-Availability is about,access to same worker nodes. ● https://enterprisersproject.com/article/2020/9/pod-cluster-container-what-is-difference ● https://medium.com/developerworld/pod-vs-node-in-kubernetes-26c858988f94 ● https://www.bmc.com/blogs/kubernetes-helm-charts/ ● https://www.vamsitalkstech.com/architecture/big-data-kubernetes-a-reference-architecture- for-spark-with-kubernetes-2-4/ ● banzaicloud/kafka-operator ● https://www.vembu.com/blog/docker-vs-kubernetes-differences-architecture-and-use-cases/ ● https://tanzu.vmware.com/developer/guides/kubernetes/create-helm-chart/
  • 19. Strategy: Scalable Fast Data Architecture: Cassandra, Spark, Kafka Engineering: Node, Python, JVM,CLR Operations: Cloud, Container Rescue: Downtime!! I need help.  www.anant.us | solutions@anant.us | (855) 262-6826 3 Washington Circle, NW | Suite 301 | Washington, DC 20037