SlideShare une entreprise Scribd logo
Platform Engineering:
Manage your infrastructure
using Kubernetes and Crossplane
Ahmed AbouZaid
November 2023
About
Ahmed AbouZaid
With 10+ years of hands-on experience and an M.Sc. in Data Engineering, I’m
passionate about DevOps and Cloud-Native, specializing in Cloud and Kubernetes.
I’m also a Free/Open source geek and book author.
I enjoy facilitating the growth of both businesses and individuals.
My favorite topics are DevOps transformation, automation, data, and metrics.
Find me at:
DevOpsHive.net | Tech.aabouzaid.com | LinkedIn.com/in/aabouzaid
Overview
● Scenario
● What is Crossplane?
● How it look like?
● Crossplane Concepts
● How Crossplane Works
● Pros and Cons
● Conclusion
● Resources
● Questions
Let’s have a look at a real-world scenario where the
DevOps engineers decided to build an abstraction
layer to be used by all dev teams (aka Platform
Engineering), and they found the solution!
Crossplane, the open-source control plane!
Scenario
Scenario (Cont.)
03
The dev teams follow the
principle of "you build it, you
manage it", but just for the
application, not the
infrastructure, because they
are not experienced with
Terraform.
01
A mid-size company has 10
development teams. Its tech
stack is mainly Kubernetes
for the applications and
Terraform for the
infrastructure.
02
Each team has a 2-week
sprint; at the end of the
sprint, they need to spin a
fresh env for the final
acceptance tests. That env
should have the Kubernetes
deployment for the
application, Cloud Database,
and Cloud Cache.
Scenario (Cont.)
06
With more coming custom
changes, it was hard for the
DevOps engineers to handle
all those requests even with
automation. Here the time to
build an abstraction layer to
be used by all dev teams
(aka Platform Engineering).
04
Each time a team needs that
new env, they ask the DevOps
engineers to push the magic
button to spin those new
Cloud env.
05
The DevOps engineers
decided to automate that
process, however, they
needed to implement 2
different systems for that:
the infrastructure
automation for Terraform
and the application
automation for Kubernetes.
What is Crossplane?
Crossplane is a Cloud Native Computing Foundation (CNCF) project
built on the foundation of Kubernetes to orchestrate anything.
At the core, it is an open-source Kubernetes add-on that extends
Kubernetes to enable the deployment and management of
infrastructure and services of various Cloud providers and
on-premises environments.
In other words, Crossplane is not just a tool but an extensible
Kubernetes operator and framework.
How it look like?
This is an example of Crossplane Claim, namely
PostgreSQLInstance from Google Cloud
Platform Configuration.
It’s same like any manifest of Kubernetes and
could be even be part of GitOps flow, Helm
chart, or Kustomize object.
You can also created your own composition
according to you needs.
apiVersion: gcp.platformref.upbound.io/v1alpha1
kind: PostgreSQLInstance
metadata:
name: super-app-db
namespace: super-app
spec:
parameters:
storageGB: 10
passwordSecretRef:
namespace: default
name: psql-secret
key: password
clusterRef:
id: platform-ref-gcp-cluster
writeConnectionSecretToRef:
name: platform-ref-gcp-db-con
How it look like? (Cont.)
Here is an example of a Crossplane claim based
on a Composition I’ve created that spins a new
GKE cluster. Full example at:
github.com/DevOpsHiveCloud/crossplane-composite
-resource-demo
apiVersion: cluster.devopshive.net/v1alpha1
kind: GKE
metadata:
name: ci
namespace: ci
spec:
name: ci-cluster
region: europe-west1
parameters:
cluster:
minVersion: 1.27.3-gke.100
nodes:
autoscaling:
minNodeCount: 0
maxNodeCount: 64
writeConnectionSecretToRef:
name: cluster-conn-ci-cluster
Crossplane Concepts
Component Abbr. Scope Functionality
Provider - Cluster
Creates new Kubernetes Custom Resource Definitions
for an external service.
Provider Config PC Cluster Applies settings for a Provider.
Managed Resource MR Cluster
A Provider resource created and managed by
Crossplane inside the Kubernetes cluster.
Composition - Cluster
A template for creating multiple managed resources
at once (think as a Terraform module).
Crossplane Concepts (Cont.)
Component Abbr. Scope Functionality
Composite
Resources
XR Cluster
Uses a Composition template to create multiple
managed resources as a single Kubernetes object
(think as a Terraform tfvars file).
CompositeResource
Definitions
XRD Cluster
Defines the API schema for Composite Resources
and Claims (think as Kubernetes Custom Resource
Definition).
Claims XC Namespace
Like a Composite Resource, but namespace scoped
(think as Kubernetes Custom Resource).
How Crossplane Works
Component Functionality
Crossplane Configuration Compose Custom API
Crossplane Providers Connect to Cloud API
Crossplane Runtime Manage External APIs (Create/Update/Delete)
Kubernetes Runtime
(API Machinery, Controller Runtime)
Events, Watch, Reconciliation, CRDs, OpenAPI,
Persistence (etcd), Run Workloads, Ingress, RBAC
Components of Crossplane powered Control plane
How Crossplane Works (Cont.)
Connect Compose Consume
Managed Resources
XRD
Composite Resource Definition
App
A Cloud API Composition Claim
Provider Configuration
Platform Team Platform Team Development Team
Claims are namespaced objects consumed by platform users (e.g., development teams)
Pros Cons
● Cloud Native, extensible, declarative,
API-driven using Kubernetes-style
● Could be used with the same Kubernetes
ecosystem tools (e.g. ArgoCD for GitOps)
● Stateless (unlike Terraform, no need to
manage the state)
● Already using Terraform providers and
support most of the Cloud services
● Bootstrap is challenging (it needs an
already running Kubernetes cluster)
● Relatively new compared to other solutions
(missing utils and helpers)
● Hard to debug and troubleshoot (you will
use “kubectl get events” a lot)
● It requires some time to get used to
Composition concepts (module-like config)
Pros and Cons
Crossplane is a great framework for managing infrastructure using the Kubernetes style
and benefits from the that ecosystem (ArgoCD, Helm, Kustomize, etc.).
There are many use cases where it can perfectly fit in already. And at the time of writing
these words (November 2023), the Marketplace has numerous enterprise and community
providers configurations. Also Composition Functions graduated to beta.
However, it’s a relatively new ecosystem and still evolving, so it might not be the optimal
solution for every workload. But it’s probably a matter of time to grow more. So, if it’s not
your fit now, consider revisiting in the future.
Conclusion
Resources
● docs.crossplane.io
● blog.crossplane.io/crossplane-vs-terraform
● blog.upbound.io/composing-a-platform-by-patching-crossplane-resources
● innoq.com/en/articles/2022/07/infrastructure-self-service-with-crossplane
● thenewstack.io/crossplane-what-most-people-get-wrong-and-how-to-get-it-right
● masterpoint.io/updates/passing-on-crossplane
For more resources:
github.com/DevOpsHiveCloud/awesome-crossplane
Thank You :-)
LinkedIn.com/in/aabouzaid
Tech.aabouzaid.com
Questions?
DevOpsHive.net

Contenu connexe

Tendances

Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
Rishabh Indoria
 
Meetup 23 - 03 - Application Delivery on K8S with GitOps
Meetup 23 - 03 - Application Delivery on K8S with GitOpsMeetup 23 - 03 - Application Delivery on K8S with GitOps
Meetup 23 - 03 - Application Delivery on K8S with GitOps
Vietnam Open Infrastructure User Group
 
KCD Italy 2022 - Application driven infrastructure with Crossplane
KCD Italy 2022 - Application driven infrastructure with CrossplaneKCD Italy 2022 - Application driven infrastructure with Crossplane
KCD Italy 2022 - Application driven infrastructure with Crossplane
sparkfabrik
 
GitOps with ArgoCD
GitOps with ArgoCDGitOps with ArgoCD
GitOps with ArgoCD
CloudOps2005
 
Kubernetes GitOps featuring GitHub, Kustomize and ArgoCD
Kubernetes GitOps featuring GitHub, Kustomize and ArgoCDKubernetes GitOps featuring GitHub, Kustomize and ArgoCD
Kubernetes GitOps featuring GitHub, Kustomize and ArgoCD
Sunnyvale
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
Michal Cwienczek
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
Raffaele Di Fazio
 
Container Orchestration using Kubernetes
Container Orchestration using KubernetesContainer Orchestration using Kubernetes
Container Orchestration using Kubernetes
Hesham Amin
 
Gitops Hands On
Gitops Hands OnGitops Hands On
Gitops Hands On
Brice Fernandes
 
DevOps - A Gentle Introduction
DevOps - A Gentle IntroductionDevOps - A Gentle Introduction
DevOps - A Gentle Introduction
CodeOps Technologies LLP
 
Introduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopIntroduction to Kubernetes Workshop
Introduction to Kubernetes Workshop
Bob Killen
 
ArgoCD Meetup PPT final.pdf
ArgoCD Meetup PPT final.pdfArgoCD Meetup PPT final.pdf
ArgoCD Meetup PPT final.pdf
amanmakwana3
 
01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx
TamalBanerjee16
 
CD using ArgoCD(KnolX).pdf
CD using ArgoCD(KnolX).pdfCD using ArgoCD(KnolX).pdf
CD using ArgoCD(KnolX).pdf
Knoldus Inc.
 
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Edureka!
 
Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)
Megan O'Keefe
 
Gitops: a new paradigm for software defined operations
Gitops: a new paradigm for software defined operationsGitops: a new paradigm for software defined operations
Gitops: a new paradigm for software defined operations
Mariano Cunietti
 
Deploy Application on Kubernetes
Deploy Application on KubernetesDeploy Application on Kubernetes
Deploy Application on Kubernetes
Opsta
 
Kubernetes - introduction
Kubernetes - introductionKubernetes - introduction
Kubernetes - introduction
Sparkbit
 
Free GitOps Workshop + Intro to Kubernetes & GitOps
Free GitOps Workshop + Intro to Kubernetes & GitOpsFree GitOps Workshop + Intro to Kubernetes & GitOps
Free GitOps Workshop + Intro to Kubernetes & GitOps
Weaveworks
 

Tendances (20)

Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Meetup 23 - 03 - Application Delivery on K8S with GitOps
Meetup 23 - 03 - Application Delivery on K8S with GitOpsMeetup 23 - 03 - Application Delivery on K8S with GitOps
Meetup 23 - 03 - Application Delivery on K8S with GitOps
 
KCD Italy 2022 - Application driven infrastructure with Crossplane
KCD Italy 2022 - Application driven infrastructure with CrossplaneKCD Italy 2022 - Application driven infrastructure with Crossplane
KCD Italy 2022 - Application driven infrastructure with Crossplane
 
GitOps with ArgoCD
GitOps with ArgoCDGitOps with ArgoCD
GitOps with ArgoCD
 
Kubernetes GitOps featuring GitHub, Kustomize and ArgoCD
Kubernetes GitOps featuring GitHub, Kustomize and ArgoCDKubernetes GitOps featuring GitHub, Kustomize and ArgoCD
Kubernetes GitOps featuring GitHub, Kustomize and ArgoCD
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Container Orchestration using Kubernetes
Container Orchestration using KubernetesContainer Orchestration using Kubernetes
Container Orchestration using Kubernetes
 
Gitops Hands On
Gitops Hands OnGitops Hands On
Gitops Hands On
 
DevOps - A Gentle Introduction
DevOps - A Gentle IntroductionDevOps - A Gentle Introduction
DevOps - A Gentle Introduction
 
Introduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopIntroduction to Kubernetes Workshop
Introduction to Kubernetes Workshop
 
ArgoCD Meetup PPT final.pdf
ArgoCD Meetup PPT final.pdfArgoCD Meetup PPT final.pdf
ArgoCD Meetup PPT final.pdf
 
01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx
 
CD using ArgoCD(KnolX).pdf
CD using ArgoCD(KnolX).pdfCD using ArgoCD(KnolX).pdf
CD using ArgoCD(KnolX).pdf
 
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
 
Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)
 
Gitops: a new paradigm for software defined operations
Gitops: a new paradigm for software defined operationsGitops: a new paradigm for software defined operations
Gitops: a new paradigm for software defined operations
 
Deploy Application on Kubernetes
Deploy Application on KubernetesDeploy Application on Kubernetes
Deploy Application on Kubernetes
 
Kubernetes - introduction
Kubernetes - introductionKubernetes - introduction
Kubernetes - introduction
 
Free GitOps Workshop + Intro to Kubernetes & GitOps
Free GitOps Workshop + Intro to Kubernetes & GitOpsFree GitOps Workshop + Intro to Kubernetes & GitOps
Free GitOps Workshop + Intro to Kubernetes & GitOps
 

Similaire à Platform Engineering: Manage your infrastructure using Kubernetes and Crossplane

Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
QAware GmbH
 
[Global logic] container runtimes and kubernetes
[Global logic] container runtimes and kubernetes[Global logic] container runtimes and kubernetes
[Global logic] container runtimes and kubernetes
GlobalLogic Ukraine
 
Architecting multi-cloud ready applications
Architecting multi-cloud ready applicationsArchitecting multi-cloud ready applications
Architecting multi-cloud ready applications
Swaminathan Vetri
 
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
sparkfabrik
 
Kubernetes: The Next Research Platform
Kubernetes: The Next Research PlatformKubernetes: The Next Research Platform
Kubernetes: The Next Research Platform
Bob Killen
 
The path to a serverless-native era with Kubernetes
The path to a serverless-native era with KubernetesThe path to a serverless-native era with Kubernetes
The path to a serverless-native era with Kubernetes
sparkfabrik
 
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
Srijan Technologies
 
DevOpsDaysRiga 2018: Anton Babenko - What you see is what you get… for AWS in...
DevOpsDaysRiga 2018: Anton Babenko - What you see is what you get… for AWS in...DevOpsDaysRiga 2018: Anton Babenko - What you see is what you get… for AWS in...
DevOpsDaysRiga 2018: Anton Babenko - What you see is what you get… for AWS in...
DevOpsDays Riga
 
Webinar: Dealing with automation tool overload!
Webinar: Dealing with automation tool overload!Webinar: Dealing with automation tool overload!
Webinar: Dealing with automation tool overload!
Cloudify Community
 
Introduction to Kubernetes - Docker Global Mentor Week 2016
Introduction to Kubernetes - Docker Global Mentor Week 2016Introduction to Kubernetes - Docker Global Mentor Week 2016
Introduction to Kubernetes - Docker Global Mentor Week 2016
Opsta
 
DevEx | there’s no place like k3s
DevEx | there’s no place like k3sDevEx | there’s no place like k3s
DevEx | there’s no place like k3s
Haggai Philip Zagury
 
DevOps for Databricks
DevOps for DatabricksDevOps for Databricks
DevOps for Databricks
Databricks
 
Gluecon Preso: Hybrid Container Infrastructure
Gluecon Preso: Hybrid Container InfrastructureGluecon Preso: Hybrid Container Infrastructure
Gluecon Preso: Hybrid Container Infrastructure
rhirschfeld
 
Cloud Deployment Toolkit
Cloud Deployment ToolkitCloud Deployment Toolkit
Cloud Deployment Toolkit
Bret Piatt
 
KubeCon NA - 2021 Tools That I Wish Existed 3 Years Ago To Build a SaaS Offering
KubeCon NA - 2021 Tools That I Wish Existed 3 Years Ago To Build a SaaS OfferingKubeCon NA - 2021 Tools That I Wish Existed 3 Years Ago To Build a SaaS Offering
KubeCon NA - 2021 Tools That I Wish Existed 3 Years Ago To Build a SaaS Offering
Mauricio (Salaboy) Salatino
 
Kubernetes #1 intro
Kubernetes #1   introKubernetes #1   intro
Kubernetes #1 intro
Terry Cho
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architectures
nine
 
Kubernetes or OpenShift - choosing your container platform for Dev and Ops
Kubernetes or OpenShift - choosing your container platform for Dev and OpsKubernetes or OpenShift - choosing your container platform for Dev and Ops
Kubernetes or OpenShift - choosing your container platform for Dev and Ops
Tomasz Cholewa
 
DevOps Spain 2019. David Cañadillas -Cloudbees
DevOps Spain 2019. David Cañadillas -CloudbeesDevOps Spain 2019. David Cañadillas -Cloudbees
DevOps Spain 2019. David Cañadillas -Cloudbees
atSistemas
 
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring BootSpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
VMware Tanzu
 

Similaire à Platform Engineering: Manage your infrastructure using Kubernetes and Crossplane (20)

Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
 
[Global logic] container runtimes and kubernetes
[Global logic] container runtimes and kubernetes[Global logic] container runtimes and kubernetes
[Global logic] container runtimes and kubernetes
 
Architecting multi-cloud ready applications
Architecting multi-cloud ready applicationsArchitecting multi-cloud ready applications
Architecting multi-cloud ready applications
 
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
 
Kubernetes: The Next Research Platform
Kubernetes: The Next Research PlatformKubernetes: The Next Research Platform
Kubernetes: The Next Research Platform
 
The path to a serverless-native era with Kubernetes
The path to a serverless-native era with KubernetesThe path to a serverless-native era with Kubernetes
The path to a serverless-native era with Kubernetes
 
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
 
DevOpsDaysRiga 2018: Anton Babenko - What you see is what you get… for AWS in...
DevOpsDaysRiga 2018: Anton Babenko - What you see is what you get… for AWS in...DevOpsDaysRiga 2018: Anton Babenko - What you see is what you get… for AWS in...
DevOpsDaysRiga 2018: Anton Babenko - What you see is what you get… for AWS in...
 
Webinar: Dealing with automation tool overload!
Webinar: Dealing with automation tool overload!Webinar: Dealing with automation tool overload!
Webinar: Dealing with automation tool overload!
 
Introduction to Kubernetes - Docker Global Mentor Week 2016
Introduction to Kubernetes - Docker Global Mentor Week 2016Introduction to Kubernetes - Docker Global Mentor Week 2016
Introduction to Kubernetes - Docker Global Mentor Week 2016
 
DevEx | there’s no place like k3s
DevEx | there’s no place like k3sDevEx | there’s no place like k3s
DevEx | there’s no place like k3s
 
DevOps for Databricks
DevOps for DatabricksDevOps for Databricks
DevOps for Databricks
 
Gluecon Preso: Hybrid Container Infrastructure
Gluecon Preso: Hybrid Container InfrastructureGluecon Preso: Hybrid Container Infrastructure
Gluecon Preso: Hybrid Container Infrastructure
 
Cloud Deployment Toolkit
Cloud Deployment ToolkitCloud Deployment Toolkit
Cloud Deployment Toolkit
 
KubeCon NA - 2021 Tools That I Wish Existed 3 Years Ago To Build a SaaS Offering
KubeCon NA - 2021 Tools That I Wish Existed 3 Years Ago To Build a SaaS OfferingKubeCon NA - 2021 Tools That I Wish Existed 3 Years Ago To Build a SaaS Offering
KubeCon NA - 2021 Tools That I Wish Existed 3 Years Ago To Build a SaaS Offering
 
Kubernetes #1 intro
Kubernetes #1   introKubernetes #1   intro
Kubernetes #1 intro
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architectures
 
Kubernetes or OpenShift - choosing your container platform for Dev and Ops
Kubernetes or OpenShift - choosing your container platform for Dev and OpsKubernetes or OpenShift - choosing your container platform for Dev and Ops
Kubernetes or OpenShift - choosing your container platform for Dev and Ops
 
DevOps Spain 2019. David Cañadillas -Cloudbees
DevOps Spain 2019. David Cañadillas -CloudbeesDevOps Spain 2019. David Cañadillas -Cloudbees
DevOps Spain 2019. David Cañadillas -Cloudbees
 
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring BootSpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
 

Plus de Ahmed AbouZaid

Kubernetes Security Best Practices - With tips for the CKS exam
Kubernetes Security Best Practices - With tips for the CKS examKubernetes Security Best Practices - With tips for the CKS exam
Kubernetes Security Best Practices - With tips for the CKS exam
Ahmed AbouZaid
 
Docker Best Practices Workshop
Docker Best Practices WorkshopDocker Best Practices Workshop
Docker Best Practices Workshop
Ahmed AbouZaid
 
Kubernetes Requests and Limits
Kubernetes Requests and LimitsKubernetes Requests and Limits
Kubernetes Requests and Limits
Ahmed AbouZaid
 
DevOps for Engineers
DevOps for EngineersDevOps for Engineers
DevOps for Engineers
Ahmed AbouZaid
 
How contributing to Open-source made me a better DevOps
How contributing to Open-source made me a better DevOpsHow contributing to Open-source made me a better DevOps
How contributing to Open-source made me a better DevOps
Ahmed AbouZaid
 
Developing Ansible Dynamic Inventory Script - Nov 2017
Developing Ansible Dynamic Inventory Script - Nov 2017Developing Ansible Dynamic Inventory Script - Nov 2017
Developing Ansible Dynamic Inventory Script - Nov 2017
Ahmed AbouZaid
 
Introduction to InfluxDB and TICK Stack
Introduction to InfluxDB and TICK StackIntroduction to InfluxDB and TICK Stack
Introduction to InfluxDB and TICK Stack
Ahmed AbouZaid
 
Getting Started with Ansible
Getting Started with AnsibleGetting Started with Ansible
Getting Started with Ansible
Ahmed AbouZaid
 
Why Ubuntu? - Arabic
Why Ubuntu? - ArabicWhy Ubuntu? - Arabic
Why Ubuntu? - Arabic
Ahmed AbouZaid
 

Plus de Ahmed AbouZaid (9)

Kubernetes Security Best Practices - With tips for the CKS exam
Kubernetes Security Best Practices - With tips for the CKS examKubernetes Security Best Practices - With tips for the CKS exam
Kubernetes Security Best Practices - With tips for the CKS exam
 
Docker Best Practices Workshop
Docker Best Practices WorkshopDocker Best Practices Workshop
Docker Best Practices Workshop
 
Kubernetes Requests and Limits
Kubernetes Requests and LimitsKubernetes Requests and Limits
Kubernetes Requests and Limits
 
DevOps for Engineers
DevOps for EngineersDevOps for Engineers
DevOps for Engineers
 
How contributing to Open-source made me a better DevOps
How contributing to Open-source made me a better DevOpsHow contributing to Open-source made me a better DevOps
How contributing to Open-source made me a better DevOps
 
Developing Ansible Dynamic Inventory Script - Nov 2017
Developing Ansible Dynamic Inventory Script - Nov 2017Developing Ansible Dynamic Inventory Script - Nov 2017
Developing Ansible Dynamic Inventory Script - Nov 2017
 
Introduction to InfluxDB and TICK Stack
Introduction to InfluxDB and TICK StackIntroduction to InfluxDB and TICK Stack
Introduction to InfluxDB and TICK Stack
 
Getting Started with Ansible
Getting Started with AnsibleGetting Started with Ansible
Getting Started with Ansible
 
Why Ubuntu? - Arabic
Why Ubuntu? - ArabicWhy Ubuntu? - Arabic
Why Ubuntu? - Arabic
 

Dernier

AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUUThings to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
FODUU
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 

Dernier (20)

AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUUThings to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 

Platform Engineering: Manage your infrastructure using Kubernetes and Crossplane

  • 1. Platform Engineering: Manage your infrastructure using Kubernetes and Crossplane Ahmed AbouZaid November 2023
  • 2. About Ahmed AbouZaid With 10+ years of hands-on experience and an M.Sc. in Data Engineering, I’m passionate about DevOps and Cloud-Native, specializing in Cloud and Kubernetes. I’m also a Free/Open source geek and book author. I enjoy facilitating the growth of both businesses and individuals. My favorite topics are DevOps transformation, automation, data, and metrics. Find me at: DevOpsHive.net | Tech.aabouzaid.com | LinkedIn.com/in/aabouzaid
  • 3. Overview ● Scenario ● What is Crossplane? ● How it look like? ● Crossplane Concepts ● How Crossplane Works ● Pros and Cons ● Conclusion ● Resources ● Questions
  • 4. Let’s have a look at a real-world scenario where the DevOps engineers decided to build an abstraction layer to be used by all dev teams (aka Platform Engineering), and they found the solution! Crossplane, the open-source control plane! Scenario
  • 5. Scenario (Cont.) 03 The dev teams follow the principle of "you build it, you manage it", but just for the application, not the infrastructure, because they are not experienced with Terraform. 01 A mid-size company has 10 development teams. Its tech stack is mainly Kubernetes for the applications and Terraform for the infrastructure. 02 Each team has a 2-week sprint; at the end of the sprint, they need to spin a fresh env for the final acceptance tests. That env should have the Kubernetes deployment for the application, Cloud Database, and Cloud Cache.
  • 6. Scenario (Cont.) 06 With more coming custom changes, it was hard for the DevOps engineers to handle all those requests even with automation. Here the time to build an abstraction layer to be used by all dev teams (aka Platform Engineering). 04 Each time a team needs that new env, they ask the DevOps engineers to push the magic button to spin those new Cloud env. 05 The DevOps engineers decided to automate that process, however, they needed to implement 2 different systems for that: the infrastructure automation for Terraform and the application automation for Kubernetes.
  • 7.
  • 8. What is Crossplane? Crossplane is a Cloud Native Computing Foundation (CNCF) project built on the foundation of Kubernetes to orchestrate anything. At the core, it is an open-source Kubernetes add-on that extends Kubernetes to enable the deployment and management of infrastructure and services of various Cloud providers and on-premises environments. In other words, Crossplane is not just a tool but an extensible Kubernetes operator and framework.
  • 9. How it look like? This is an example of Crossplane Claim, namely PostgreSQLInstance from Google Cloud Platform Configuration. It’s same like any manifest of Kubernetes and could be even be part of GitOps flow, Helm chart, or Kustomize object. You can also created your own composition according to you needs. apiVersion: gcp.platformref.upbound.io/v1alpha1 kind: PostgreSQLInstance metadata: name: super-app-db namespace: super-app spec: parameters: storageGB: 10 passwordSecretRef: namespace: default name: psql-secret key: password clusterRef: id: platform-ref-gcp-cluster writeConnectionSecretToRef: name: platform-ref-gcp-db-con
  • 10. How it look like? (Cont.) Here is an example of a Crossplane claim based on a Composition I’ve created that spins a new GKE cluster. Full example at: github.com/DevOpsHiveCloud/crossplane-composite -resource-demo apiVersion: cluster.devopshive.net/v1alpha1 kind: GKE metadata: name: ci namespace: ci spec: name: ci-cluster region: europe-west1 parameters: cluster: minVersion: 1.27.3-gke.100 nodes: autoscaling: minNodeCount: 0 maxNodeCount: 64 writeConnectionSecretToRef: name: cluster-conn-ci-cluster
  • 11. Crossplane Concepts Component Abbr. Scope Functionality Provider - Cluster Creates new Kubernetes Custom Resource Definitions for an external service. Provider Config PC Cluster Applies settings for a Provider. Managed Resource MR Cluster A Provider resource created and managed by Crossplane inside the Kubernetes cluster. Composition - Cluster A template for creating multiple managed resources at once (think as a Terraform module).
  • 12. Crossplane Concepts (Cont.) Component Abbr. Scope Functionality Composite Resources XR Cluster Uses a Composition template to create multiple managed resources as a single Kubernetes object (think as a Terraform tfvars file). CompositeResource Definitions XRD Cluster Defines the API schema for Composite Resources and Claims (think as Kubernetes Custom Resource Definition). Claims XC Namespace Like a Composite Resource, but namespace scoped (think as Kubernetes Custom Resource).
  • 13. How Crossplane Works Component Functionality Crossplane Configuration Compose Custom API Crossplane Providers Connect to Cloud API Crossplane Runtime Manage External APIs (Create/Update/Delete) Kubernetes Runtime (API Machinery, Controller Runtime) Events, Watch, Reconciliation, CRDs, OpenAPI, Persistence (etcd), Run Workloads, Ingress, RBAC Components of Crossplane powered Control plane
  • 14. How Crossplane Works (Cont.) Connect Compose Consume Managed Resources XRD Composite Resource Definition App A Cloud API Composition Claim Provider Configuration Platform Team Platform Team Development Team
  • 15. Claims are namespaced objects consumed by platform users (e.g., development teams)
  • 16. Pros Cons ● Cloud Native, extensible, declarative, API-driven using Kubernetes-style ● Could be used with the same Kubernetes ecosystem tools (e.g. ArgoCD for GitOps) ● Stateless (unlike Terraform, no need to manage the state) ● Already using Terraform providers and support most of the Cloud services ● Bootstrap is challenging (it needs an already running Kubernetes cluster) ● Relatively new compared to other solutions (missing utils and helpers) ● Hard to debug and troubleshoot (you will use “kubectl get events” a lot) ● It requires some time to get used to Composition concepts (module-like config) Pros and Cons
  • 17. Crossplane is a great framework for managing infrastructure using the Kubernetes style and benefits from the that ecosystem (ArgoCD, Helm, Kustomize, etc.). There are many use cases where it can perfectly fit in already. And at the time of writing these words (November 2023), the Marketplace has numerous enterprise and community providers configurations. Also Composition Functions graduated to beta. However, it’s a relatively new ecosystem and still evolving, so it might not be the optimal solution for every workload. But it’s probably a matter of time to grow more. So, if it’s not your fit now, consider revisiting in the future. Conclusion
  • 18. Resources ● docs.crossplane.io ● blog.crossplane.io/crossplane-vs-terraform ● blog.upbound.io/composing-a-platform-by-patching-crossplane-resources ● innoq.com/en/articles/2022/07/infrastructure-self-service-with-crossplane ● thenewstack.io/crossplane-what-most-people-get-wrong-and-how-to-get-it-right ● masterpoint.io/updates/passing-on-crossplane For more resources: github.com/DevOpsHiveCloud/awesome-crossplane