SlideShare une entreprise Scribd logo
1  sur  54
Gianluca Arbezzano
Site Reliability Engineer @InfluxData
● https://gianarb.it
● @gianarb
What I like:
● I make dirty hacks that look awesome
● I grow my vegetables 🍅🌻🍆
● Travel for fun and work
@gianarb - gianluca@influxdb.com
@gianarb - gianluca@influxdb.com
@gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.6 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.7 @gianarb - gianluca@influxdb.com
DevOps likes automation.
Automation likes code.
YAML is not code.
Inspired by a true events
Kubernetes
1. You Know!
Your team knows and
use Docker for local
development and
testing
2. Kubernetes!
Everyone speaks
about kubernetes.
3. Hire!
You don’t know why
but you hired a
DevOps that kind of
know k8s.
3. Excitement!
You are moving
everything and
everyone to
kubernetes
We need to make our
hands dirty
Spin up a cluster that you
can break
Bring developers in the loop
Deploy CI on Kubernetes
Bring developers in the loop
Run your code in prod
Bring developers in the loop
K8s as code: From YAML to code (golang)
1. You have the ability to use Golang autocomplete as documentation, reference for every
kubernetes resources
2. You feel less a YAML engineer (great feeling btw)
3. Code is better than YAML! You can reuse it, compile it, embed it in other projects.
K8s as code: From YAML to code (golang)
Tiny cli
to make
the
migration
to golang
Some
manual
refactoring
K8s as code: From YAML to code (golang)
Tiny cli
to make
the
migration
to golang
Some
manual
refactoring
● Continue to improve our CI to validate that YAML and Go file are the same,
and the resources in Kubernetes are like the Go file.
● Maybe we will be able to remove the YAML at some point.
GitOps
Your Git repository is the entrypoint for all your code changes.
Infrastructure is ‘as code’, so the place where you make it happen should be Git.
Read More on weave.com
https://www.weave.works/technologies/gitops/
The secret of
success
Don’t be scared and write your
own tools!
Why Kubernetes
is so powerful, complex
and widely adopted?
Why AWS
is so expensive?
What do you do
to justify these costs?
© 2018 InfluxData. All rights reserved.24 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.25 @gianarb - gianluca@influxdb.com
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: {{ template "drone.fullname" . }}-agent
labels:
app: {{ template "drone.name" . }}
chart: "{{ .Chart.Name }}-{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
component: agent
spec:
replicas: {{ .Values.agent.replicas }}
template:
metadata:
annotations:
checksum/secrets: {{ include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }}
{{- if .Values.agent.annotations }
{{ toYaml .Values.agent.annotations | indent 8 }
{{- end }}
labels:
app: {{ template "drone.name" . }}
release: "{{ .Release.Name }}"
component: agent
API are
the keys for
your success!
Image credit: Pixabay
© 2018 InfluxData. All rights reserved.27 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.28 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.29 @gianarb - gianluca@influxdb.com
containerd.io
© 2018 InfluxData. All rights reserved.30 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.31 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.32 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.33 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.34 @gianarb - gianluca@influxdb.com
© 2018 InfluxData. All rights reserved.35 @gianarb - gianluca@influxdb.com
We use docker as
replacement for systemd
for process management
© 2018 InfluxData. All rights reserved.36 @gianarb - gianluca@influxdb.com
DIND - Docker in Docker
$ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker sh
$ docker info
Containers: 48
Running: 1
Paused: 0
Stopped: 47
containerd version: 9f2e07b1fc1342d1c48fe4d7bbb94cb6d1bf278b.m
runc version: 871ba2e58e24314d1fab4517a80410191ba5ad01
init version: fec3683
Kernel Version: 4.20.13-arch1-1-ARCH
Operating System: Arch Linux
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.42GiB
Name: gianarb
UX for OPS.
Because everyone needs to feel
like at home...
Instrumentation, observability
and monitoring
~ @gianarb - https://gianarb.it ~
The secret is all about how do you
combine things together
~ @gianarb - https://gianarb.it ~
Metric
s
~ @gianarb - https://gianarb.it ~
Logs
~ @gianarb - https://gianarb.it ~
Traces
~ @gianarb - https://gianarb.it ~
Often our
aggregations looks
a bit twisted...
@gianarb - gianluca@influxdb.com
Distributed Tracing
Tracing is a way to correlate
logs using a set of IDs
@gianarb - gianluca@influxdb.com
Normal state vs Current state
Instrumentation code is a first citizen in your
codebase: OpenCensus
● Open Source project sponsored by Google
● It is a SPEC plus a set of libraries in different languages to instrument your
application
● To collect metrics, traces and events.
OpenCensus
Common
Interface to
collect stats
and traces
from your app
Different
exporters to
persist your
data
gianarb.it ~ @gianarb
# HELP http_requests_total The total number of HTTP requests.
# TYPE http_requests_total counter
http_requests_total{method="post",code="200"} 1027 1395066363000
http_requests_total{method="post",code="400"} 3 1395066363000
# Escaping in label values:
msdos_file_access_time_seconds{path="C:DIRFILE.TXT",error="Cannot find file:n"FILE.TXT""}
1.458255915e9
# Minimalistic line:
metric_without_timestamp_and_labels 12.47
# A weird metric from before the epoch:
something_weird{problem="division by zero"} +Inf -3982045
# A histogram, which has a pretty complex representation in the text format:
# HELP http_request_duration_seconds A histogram of the request duration.
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_bucket{le="0.05"} 24054
http_request_duration_seconds_bucket{le="0.1"} 33444
http_request_duration_seconds_bucket{le="0.2"} 100392
http_request_duration_seconds_bucket{le="0.5"} 129389
http_request_duration_seconds_bucket{le="1"} 133988
http_request_duration_seconds_bucket{le="+Inf"} 144320
http_request_duration_seconds_sum 53423
http_request_duration_seconds_count 144320
OpenMetrics
v2 Prometheus exposition format
gianarb.it ~ @gianarb
func FetchMetricFamilies(url string, ch chan<- *dto.MetricFamily, certificate string, key string,
skipServerCertCheck bool) error {
defer close(ch)
var transport *http.Transport
if certificate != "" && key != "" {
cert, err := tls.LoadX509KeyPair(certificate, key)
if err != nil {
return err
}
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{cert},
InsecureSkipVerify: skipServerCertCheck,
}
tlsConfig.BuildNameToCertificate()
transport = &http.Transport{TLSClientConfig: tlsConfig}
} else {
transport = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: skipServerCertCheck},
}
}
https://github.com/prometheus/prom2json/blob/master/prom2json.go#L123
Summary:
★ Do not be scared to write your code!
★ Use the API
★ Instrumentation code is a first class
citizen
★ Keep calm and observe all together!
© 2018 InfluxData. All rights reserved.53 @gianarb - gianluca@influxdb.com
Credits and Links
¨ https://www.weave.works/technologies/gitops/
¨ http://gianarb.it
¨ https://thenewstack.io/why-you-cant-afford-to-ignore-distributed-tracing-for-observability/
¨ https://www.honeycomb.io/blog/
¨ https://gianarb.it/blog/infra-as-code-short-long-ttl-resource
¨ https://gianarb.it/blog/kubernetes-shared-informer
¨ https://github.com/OpenObservability/OpenMetrics
¨ https://promcon.io/2018-munich/slides/openmetrics-transforming-the-prometheus-exposition-format-into-a
-global-standard.pdf
¨ https://medium.com/opentracing/merging-opentracing-and-opencensus-f0fe9c7ca6f0
~ @gianarb - https://gianarb.it ~
Thanks
@gianarb

Contenu connexe

Tendances

Infrasturcure-as-code with Kubernetes and Rancher - September 2019 Online Meetup
Infrasturcure-as-code with Kubernetes and Rancher - September 2019 Online MeetupInfrasturcure-as-code with Kubernetes and Rancher - September 2019 Online Meetup
Infrasturcure-as-code with Kubernetes and Rancher - September 2019 Online Meetup
Shannon Williams
 

Tendances (20)

JCConf 2020 Observing in Microservices
JCConf 2020 Observing in MicroservicesJCConf 2020 Observing in Microservices
JCConf 2020 Observing in Microservices
 
Developing Skills for Amazon Echo
Developing Skills for Amazon EchoDeveloping Skills for Amazon Echo
Developing Skills for Amazon Echo
 
Apache big-data-2017-spark-profiling
Apache big-data-2017-spark-profilingApache big-data-2017-spark-profiling
Apache big-data-2017-spark-profiling
 
An Introduction to Rancher
An Introduction to RancherAn Introduction to Rancher
An Introduction to Rancher
 
Infinum Android Talks #04 - How to publish an Android archive (.aar) to Maven...
Infinum Android Talks #04 - How to publish an Android archive (.aar) to Maven...Infinum Android Talks #04 - How to publish an Android archive (.aar) to Maven...
Infinum Android Talks #04 - How to publish an Android archive (.aar) to Maven...
 
Real-Time Data Processing Pipeline & Visualization with Docker, Spark, Kafka ...
Real-Time Data Processing Pipeline & Visualization with Docker, Spark, Kafka ...Real-Time Data Processing Pipeline & Visualization with Docker, Spark, Kafka ...
Real-Time Data Processing Pipeline & Visualization with Docker, Spark, Kafka ...
 
More tips and tricks for running containers like a pro - Rancher Online MEetu...
More tips and tricks for running containers like a pro - Rancher Online MEetu...More tips and tricks for running containers like a pro - Rancher Online MEetu...
More tips and tricks for running containers like a pro - Rancher Online MEetu...
 
Infrasturcure-as-code with Kubernetes and Rancher - September 2019 Online Meetup
Infrasturcure-as-code with Kubernetes and Rancher - September 2019 Online MeetupInfrasturcure-as-code with Kubernetes and Rancher - September 2019 Online Meetup
Infrasturcure-as-code with Kubernetes and Rancher - September 2019 Online Meetup
 
Seven perilous pitfalls to avoid with Java | DevNation Tech Talk
Seven perilous pitfalls to avoid with Java | DevNation Tech TalkSeven perilous pitfalls to avoid with Java | DevNation Tech Talk
Seven perilous pitfalls to avoid with Java | DevNation Tech Talk
 
Hands-on with Rancher 2.0 and Kubernetes - October 2017 Rancher Online Meetup
Hands-on with Rancher 2.0 and Kubernetes - October 2017 Rancher Online MeetupHands-on with Rancher 2.0 and Kubernetes - October 2017 Rancher Online Meetup
Hands-on with Rancher 2.0 and Kubernetes - October 2017 Rancher Online Meetup
 
Supercharging CI/CD with GitLab and Rancher - June 2017 Online Meetup
Supercharging CI/CD with GitLab and Rancher - June 2017 Online MeetupSupercharging CI/CD with GitLab and Rancher - June 2017 Online Meetup
Supercharging CI/CD with GitLab and Rancher - June 2017 Online Meetup
 
What's new with Apache Camel 3? | DevNation Tech Talk
What's new with Apache Camel 3? | DevNation Tech TalkWhat's new with Apache Camel 3? | DevNation Tech Talk
What's new with Apache Camel 3? | DevNation Tech Talk
 
Deploying Containers with Rancher
Deploying Containers with RancherDeploying Containers with Rancher
Deploying Containers with Rancher
 
Everything-as-code. A polyglot adventure. #DevoxxPL
Everything-as-code. A polyglot adventure. #DevoxxPLEverything-as-code. A polyglot adventure. #DevoxxPL
Everything-as-code. A polyglot adventure. #DevoxxPL
 
Nordic APIs - Automatic Testing of (RESTful) API Documentation
Nordic APIs - Automatic Testing of (RESTful) API DocumentationNordic APIs - Automatic Testing of (RESTful) API Documentation
Nordic APIs - Automatic Testing of (RESTful) API Documentation
 
Alfresco Development Framework Basic
Alfresco Development Framework BasicAlfresco Development Framework Basic
Alfresco Development Framework Basic
 
Contributing to open source as a non developer - #wclax
Contributing to open source as a non developer - #wclaxContributing to open source as a non developer - #wclax
Contributing to open source as a non developer - #wclax
 
Gitlab ci, cncf.sk
Gitlab ci, cncf.skGitlab ci, cncf.sk
Gitlab ci, cncf.sk
 
Automate CI/CD with Rancher
Automate CI/CD with RancherAutomate CI/CD with Rancher
Automate CI/CD with Rancher
 
2020.02.15 DelEx - CI/CD in AWS Cloud
2020.02.15 DelEx - CI/CD in AWS Cloud2020.02.15 DelEx - CI/CD in AWS Cloud
2020.02.15 DelEx - CI/CD in AWS Cloud
 

Similaire à DevOps Fest 2019. Gianluca Arbezzano. DevOps never sleeps. What we learned from InfluxDB v1 to v2

Similaire à DevOps Fest 2019. Gianluca Arbezzano. DevOps never sleeps. What we learned from InfluxDB v1 to v2 (20)

OSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoringOSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoring
 
OSDC 2018 | Distributed Monitoring by Gianluca Arbezzano
OSDC 2018 | Distributed Monitoring by Gianluca ArbezzanoOSDC 2018 | Distributed Monitoring by Gianluca Arbezzano
OSDC 2018 | Distributed Monitoring by Gianluca Arbezzano
 
Kubernetes debug like a pro
Kubernetes debug like a proKubernetes debug like a pro
Kubernetes debug like a pro
 
Streamlined Geek Talk
Streamlined Geek TalkStreamlined Geek Talk
Streamlined Geek Talk
 
How to Create the API Document from Real API and Localization
How to Create the API Document from Real API and Localization How to Create the API Document from Real API and Localization
How to Create the API Document from Real API and Localization
 
Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...
Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...
Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
 
Spark + AI Summit 2019: Apache Spark Listeners: A Crash Course in Fast, Easy ...
Spark + AI Summit 2019: Apache Spark Listeners: A Crash Course in Fast, Easy ...Spark + AI Summit 2019: Apache Spark Listeners: A Crash Course in Fast, Easy ...
Spark + AI Summit 2019: Apache Spark Listeners: A Crash Course in Fast, Easy ...
 
Apache Spark Listeners: A Crash Course in Fast, Easy Monitoring
Apache Spark Listeners: A Crash Course in Fast, Easy MonitoringApache Spark Listeners: A Crash Course in Fast, Easy Monitoring
Apache Spark Listeners: A Crash Course in Fast, Easy Monitoring
 
Getting More Out of the Node.js, PHP, and Python Agents - AppSphere16
Getting More Out of the Node.js, PHP, and Python Agents - AppSphere16Getting More Out of the Node.js, PHP, and Python Agents - AppSphere16
Getting More Out of the Node.js, PHP, and Python Agents - AppSphere16
 
Deploying Rails Apps with Chef and Capistrano
 Deploying Rails Apps with Chef and Capistrano Deploying Rails Apps with Chef and Capistrano
Deploying Rails Apps with Chef and Capistrano
 
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...
 
Fandogh Cloud workshop slides
Fandogh Cloud workshop slides Fandogh Cloud workshop slides
Fandogh Cloud workshop slides
 
FOXX - a Javascript application framework on top of ArangoDB
FOXX - a Javascript application framework on top of ArangoDBFOXX - a Javascript application framework on top of ArangoDB
FOXX - a Javascript application framework on top of ArangoDB
 
Vinay Kumar [InfluxData] | InfluxDB API Overview | InfluxDays 2022
Vinay Kumar [InfluxData] | InfluxDB API Overview  | InfluxDays 2022Vinay Kumar [InfluxData] | InfluxDB API Overview  | InfluxDays 2022
Vinay Kumar [InfluxData] | InfluxDB API Overview | InfluxDays 2022
 
At the helm of kubernetes
At the helm of kubernetesAt the helm of kubernetes
At the helm of kubernetes
 
Spring Boot & Spring Cloud on PAS- Nate Schutta (1/2)
Spring Boot & Spring Cloud on PAS- Nate Schutta (1/2)Spring Boot & Spring Cloud on PAS- Nate Schutta (1/2)
Spring Boot & Spring Cloud on PAS- Nate Schutta (1/2)
 
DevOps Pipelines and Metrics Driven Feedback Loops
DevOps Pipelines and Metrics Driven Feedback LoopsDevOps Pipelines and Metrics Driven Feedback Loops
DevOps Pipelines and Metrics Driven Feedback Loops
 
Cloud-native Patterns
Cloud-native PatternsCloud-native Patterns
Cloud-native Patterns
 
Cloud-native Patterns (July 4th, 2019)
Cloud-native Patterns (July 4th, 2019)Cloud-native Patterns (July 4th, 2019)
Cloud-native Patterns (July 4th, 2019)
 

Plus de DevOps_Fest

DevOps Fest 2020. Барух Садогурский и Леонид Игольник. Устраиваем DevOps без ...
DevOps Fest 2020. Барух Садогурский и Леонид Игольник. Устраиваем DevOps без ...DevOps Fest 2020. Барух Садогурский и Леонид Игольник. Устраиваем DevOps без ...
DevOps Fest 2020. Барух Садогурский и Леонид Игольник. Устраиваем DevOps без ...
DevOps_Fest
 
DevOps Fest 2020. Станислав Коленкин. How to connect non-connectible: tips, t...
DevOps Fest 2020. Станислав Коленкин. How to connect non-connectible: tips, t...DevOps Fest 2020. Станислав Коленкин. How to connect non-connectible: tips, t...
DevOps Fest 2020. Станислав Коленкин. How to connect non-connectible: tips, t...
DevOps_Fest
 
DevOps Fest 2020. Андрій Шабалін. Distributed Tracing for microservices with ...
DevOps Fest 2020. Андрій Шабалін. Distributed Tracing for microservices with ...DevOps Fest 2020. Андрій Шабалін. Distributed Tracing for microservices with ...
DevOps Fest 2020. Андрій Шабалін. Distributed Tracing for microservices with ...
DevOps_Fest
 

Plus de DevOps_Fest (20)

DevOps Fest 2020. Сергій Калінець. Building Data Streaming Platform with Apac...
DevOps Fest 2020. Сергій Калінець. Building Data Streaming Platform with Apac...DevOps Fest 2020. Сергій Калінець. Building Data Streaming Platform with Apac...
DevOps Fest 2020. Сергій Калінець. Building Data Streaming Platform with Apac...
 
DevOps Fest 2020. Kohsuke Kawaguchi. GitOps, Jenkins X & the Future of CI/CD
DevOps Fest 2020. Kohsuke Kawaguchi. GitOps, Jenkins X & the Future of CI/CDDevOps Fest 2020. Kohsuke Kawaguchi. GitOps, Jenkins X & the Future of CI/CD
DevOps Fest 2020. Kohsuke Kawaguchi. GitOps, Jenkins X & the Future of CI/CD
 
DevOps Fest 2020. Барух Садогурский и Леонид Игольник. Устраиваем DevOps без ...
DevOps Fest 2020. Барух Садогурский и Леонид Игольник. Устраиваем DevOps без ...DevOps Fest 2020. Барух Садогурский и Леонид Игольник. Устраиваем DevOps без ...
DevOps Fest 2020. Барух Садогурский и Леонид Игольник. Устраиваем DevOps без ...
 
DevOps Fest 2020. James Spiteri. Advanced Security Operations with Elastic Se...
DevOps Fest 2020. James Spiteri. Advanced Security Operations with Elastic Se...DevOps Fest 2020. James Spiteri. Advanced Security Operations with Elastic Se...
DevOps Fest 2020. James Spiteri. Advanced Security Operations with Elastic Se...
 
DevOps Fest 2020. Pavlo Repalo. Edge Computing: Appliance and Challanges
DevOps Fest 2020. Pavlo Repalo. Edge Computing: Appliance and ChallangesDevOps Fest 2020. Pavlo Repalo. Edge Computing: Appliance and Challanges
DevOps Fest 2020. Pavlo Repalo. Edge Computing: Appliance and Challanges
 
DevOps Fest 2020. Максим Безуглый. DevOps - как архитектура в процессе. Две к...
DevOps Fest 2020. Максим Безуглый. DevOps - как архитектура в процессе. Две к...DevOps Fest 2020. Максим Безуглый. DevOps - как архитектура в процессе. Две к...
DevOps Fest 2020. Максим Безуглый. DevOps - как архитектура в процессе. Две к...
 
DevOps Fest 2020. Павел Жданов та Никора Никита. Построение процесса CI\CD дл...
DevOps Fest 2020. Павел Жданов та Никора Никита. Построение процесса CI\CD дл...DevOps Fest 2020. Павел Жданов та Никора Никита. Построение процесса CI\CD дл...
DevOps Fest 2020. Павел Жданов та Никора Никита. Построение процесса CI\CD дл...
 
DevOps Fest 2020. Станислав Коленкин. How to connect non-connectible: tips, t...
DevOps Fest 2020. Станислав Коленкин. How to connect non-connectible: tips, t...DevOps Fest 2020. Станислав Коленкин. How to connect non-connectible: tips, t...
DevOps Fest 2020. Станислав Коленкин. How to connect non-connectible: tips, t...
 
DevOps Fest 2020. Андрій Шабалін. Distributed Tracing for microservices with ...
DevOps Fest 2020. Андрій Шабалін. Distributed Tracing for microservices with ...DevOps Fest 2020. Андрій Шабалін. Distributed Tracing for microservices with ...
DevOps Fest 2020. Андрій Шабалін. Distributed Tracing for microservices with ...
 
DevOps Fest 2020. Дмитрий Кудрявцев. Реализация GitOps на Kubernetes. ArgoCD
DevOps Fest 2020. Дмитрий Кудрявцев. Реализация GitOps на Kubernetes. ArgoCDDevOps Fest 2020. Дмитрий Кудрявцев. Реализация GitOps на Kubernetes. ArgoCD
DevOps Fest 2020. Дмитрий Кудрявцев. Реализация GitOps на Kubernetes. ArgoCD
 
DevOps Fest 2020. Роман Орлов. Инфраструктура тестирования в Kubernetes
DevOps Fest 2020. Роман Орлов. Инфраструктура тестирования в KubernetesDevOps Fest 2020. Роман Орлов. Инфраструктура тестирования в Kubernetes
DevOps Fest 2020. Роман Орлов. Инфраструктура тестирования в Kubernetes
 
DevOps Fest 2020. Андрей Шишенко. CI/CD for AWS Lambdas with Serverless frame...
DevOps Fest 2020. Андрей Шишенко. CI/CD for AWS Lambdas with Serverless frame...DevOps Fest 2020. Андрей Шишенко. CI/CD for AWS Lambdas with Serverless frame...
DevOps Fest 2020. Андрей Шишенко. CI/CD for AWS Lambdas with Serverless frame...
 
DevOps Fest 2020. Александр Глущенко. Modern Enterprise Network Architecture ...
DevOps Fest 2020. Александр Глущенко. Modern Enterprise Network Architecture ...DevOps Fest 2020. Александр Глущенко. Modern Enterprise Network Architecture ...
DevOps Fest 2020. Александр Глущенко. Modern Enterprise Network Architecture ...
 
DevOps Fest 2020. Виталий Складчиков. Сквозь монолитный enterprise к микросер...
DevOps Fest 2020. Виталий Складчиков. Сквозь монолитный enterprise к микросер...DevOps Fest 2020. Виталий Складчиков. Сквозь монолитный enterprise к микросер...
DevOps Fest 2020. Виталий Складчиков. Сквозь монолитный enterprise к микросер...
 
DevOps Fest 2020. Денис Медведенко. Управление сложными многокомпонентными ин...
DevOps Fest 2020. Денис Медведенко. Управление сложными многокомпонентными ин...DevOps Fest 2020. Денис Медведенко. Управление сложными многокомпонентными ин...
DevOps Fest 2020. Денис Медведенко. Управление сложными многокомпонентными ин...
 
DevOps Fest 2020. Павел Галушко. Что делать devops'у если у вас захотели mach...
DevOps Fest 2020. Павел Галушко. Что делать devops'у если у вас захотели mach...DevOps Fest 2020. Павел Галушко. Что делать devops'у если у вас захотели mach...
DevOps Fest 2020. Павел Галушко. Что делать devops'у если у вас захотели mach...
 
DevOps Fest 2020. Сергей Абаничев. Modern CI\CD pipeline with Azure DevOps
DevOps Fest 2020. Сергей Абаничев. Modern CI\CD pipeline with Azure DevOpsDevOps Fest 2020. Сергей Абаничев. Modern CI\CD pipeline with Azure DevOps
DevOps Fest 2020. Сергей Абаничев. Modern CI\CD pipeline with Azure DevOps
 
DevOps Fest 2020. Philipp Krenn. Scale Your Auditing Events
DevOps Fest 2020. Philipp Krenn. Scale Your Auditing EventsDevOps Fest 2020. Philipp Krenn. Scale Your Auditing Events
DevOps Fest 2020. Philipp Krenn. Scale Your Auditing Events
 
DevOps Fest 2020. Володимир Мельник. TuchaKube - перша українська DevOps/Host...
DevOps Fest 2020. Володимир Мельник. TuchaKube - перша українська DevOps/Host...DevOps Fest 2020. Володимир Мельник. TuchaKube - перша українська DevOps/Host...
DevOps Fest 2020. Володимир Мельник. TuchaKube - перша українська DevOps/Host...
 
DevOps Fest 2020. Денис Васильев. Let's make it KUL! Kubernetes Ultra Light
DevOps Fest 2020. Денис Васильев. Let's make it KUL! Kubernetes Ultra LightDevOps Fest 2020. Денис Васильев. Let's make it KUL! Kubernetes Ultra Light
DevOps Fest 2020. Денис Васильев. Let's make it KUL! Kubernetes Ultra Light
 

Dernier

Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 

Dernier (20)

Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17
 
Towards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxTowards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptx
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
 
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
 
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptxHMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17
 
Sociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning ExhibitSociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning Exhibit
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
Micro-Scholarship, What it is, How can it help me.pdf
Micro-Scholarship, What it is, How can it help me.pdfMicro-Scholarship, What it is, How can it help me.pdf
Micro-Scholarship, What it is, How can it help me.pdf
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.ppt
 
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxBasic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
 
REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptx
 
Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdf
 
Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)
 
Fostering Friendships - Enhancing Social Bonds in the Classroom
Fostering Friendships - Enhancing Social Bonds  in the ClassroomFostering Friendships - Enhancing Social Bonds  in the Classroom
Fostering Friendships - Enhancing Social Bonds in the Classroom
 
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdfUGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
 
Making communications land - Are they received and understood as intended? we...
Making communications land - Are they received and understood as intended? we...Making communications land - Are they received and understood as intended? we...
Making communications land - Are they received and understood as intended? we...
 
ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.
 

DevOps Fest 2019. Gianluca Arbezzano. DevOps never sleeps. What we learned from InfluxDB v1 to v2

  • 1. Gianluca Arbezzano Site Reliability Engineer @InfluxData ● https://gianarb.it ● @gianarb What I like: ● I make dirty hacks that look awesome ● I grow my vegetables 🍅🌻🍆 ● Travel for fun and work
  • 5.
  • 6. © 2018 InfluxData. All rights reserved.6 @gianarb - gianluca@influxdb.com
  • 7. © 2018 InfluxData. All rights reserved.7 @gianarb - gianluca@influxdb.com DevOps likes automation. Automation likes code. YAML is not code.
  • 8. Inspired by a true events
  • 10. 1. You Know! Your team knows and use Docker for local development and testing 2. Kubernetes! Everyone speaks about kubernetes. 3. Hire! You don’t know why but you hired a DevOps that kind of know k8s. 3. Excitement! You are moving everything and everyone to kubernetes
  • 11. We need to make our hands dirty
  • 12. Spin up a cluster that you can break Bring developers in the loop
  • 13. Deploy CI on Kubernetes Bring developers in the loop
  • 14. Run your code in prod Bring developers in the loop
  • 15. K8s as code: From YAML to code (golang) 1. You have the ability to use Golang autocomplete as documentation, reference for every kubernetes resources 2. You feel less a YAML engineer (great feeling btw) 3. Code is better than YAML! You can reuse it, compile it, embed it in other projects.
  • 16. K8s as code: From YAML to code (golang) Tiny cli to make the migration to golang Some manual refactoring
  • 17. K8s as code: From YAML to code (golang) Tiny cli to make the migration to golang Some manual refactoring ● Continue to improve our CI to validate that YAML and Go file are the same, and the resources in Kubernetes are like the Go file. ● Maybe we will be able to remove the YAML at some point.
  • 18. GitOps Your Git repository is the entrypoint for all your code changes. Infrastructure is ‘as code’, so the place where you make it happen should be Git. Read More on weave.com https://www.weave.works/technologies/gitops/
  • 20. Don’t be scared and write your own tools!
  • 21. Why Kubernetes is so powerful, complex and widely adopted?
  • 22. Why AWS is so expensive?
  • 23. What do you do to justify these costs?
  • 24. © 2018 InfluxData. All rights reserved.24 @gianarb - gianluca@influxdb.com
  • 25. © 2018 InfluxData. All rights reserved.25 @gianarb - gianluca@influxdb.com apiVersion: extensions/v1beta1 kind: Deployment metadata: name: {{ template "drone.fullname" . }}-agent labels: app: {{ template "drone.name" . }} chart: "{{ .Chart.Name }}-{ .Chart.Version }}" release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" component: agent spec: replicas: {{ .Values.agent.replicas }} template: metadata: annotations: checksum/secrets: {{ include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }} {{- if .Values.agent.annotations } {{ toYaml .Values.agent.annotations | indent 8 } {{- end }} labels: app: {{ template "drone.name" . }} release: "{{ .Release.Name }}" component: agent
  • 26. API are the keys for your success! Image credit: Pixabay
  • 27. © 2018 InfluxData. All rights reserved.27 @gianarb - gianluca@influxdb.com
  • 28. © 2018 InfluxData. All rights reserved.28 @gianarb - gianluca@influxdb.com
  • 29. © 2018 InfluxData. All rights reserved.29 @gianarb - gianluca@influxdb.com containerd.io
  • 30. © 2018 InfluxData. All rights reserved.30 @gianarb - gianluca@influxdb.com
  • 31. © 2018 InfluxData. All rights reserved.31 @gianarb - gianluca@influxdb.com
  • 32. © 2018 InfluxData. All rights reserved.32 @gianarb - gianluca@influxdb.com
  • 33. © 2018 InfluxData. All rights reserved.33 @gianarb - gianluca@influxdb.com
  • 34. © 2018 InfluxData. All rights reserved.34 @gianarb - gianluca@influxdb.com
  • 35. © 2018 InfluxData. All rights reserved.35 @gianarb - gianluca@influxdb.com We use docker as replacement for systemd for process management
  • 36. © 2018 InfluxData. All rights reserved.36 @gianarb - gianluca@influxdb.com DIND - Docker in Docker $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker sh $ docker info Containers: 48 Running: 1 Paused: 0 Stopped: 47 containerd version: 9f2e07b1fc1342d1c48fe4d7bbb94cb6d1bf278b.m runc version: 871ba2e58e24314d1fab4517a80410191ba5ad01 init version: fec3683 Kernel Version: 4.20.13-arch1-1-ARCH Operating System: Arch Linux OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 15.42GiB Name: gianarb
  • 37. UX for OPS. Because everyone needs to feel like at home...
  • 39. ~ @gianarb - https://gianarb.it ~ The secret is all about how do you combine things together
  • 40. ~ @gianarb - https://gianarb.it ~ Metric s
  • 41. ~ @gianarb - https://gianarb.it ~ Logs
  • 42. ~ @gianarb - https://gianarb.it ~ Traces
  • 43. ~ @gianarb - https://gianarb.it ~ Often our aggregations looks a bit twisted...
  • 44. @gianarb - gianluca@influxdb.com Distributed Tracing Tracing is a way to correlate logs using a set of IDs
  • 46. Normal state vs Current state
  • 47. Instrumentation code is a first citizen in your codebase: OpenCensus ● Open Source project sponsored by Google ● It is a SPEC plus a set of libraries in different languages to instrument your application ● To collect metrics, traces and events.
  • 48. OpenCensus Common Interface to collect stats and traces from your app Different exporters to persist your data
  • 49. gianarb.it ~ @gianarb # HELP http_requests_total The total number of HTTP requests. # TYPE http_requests_total counter http_requests_total{method="post",code="200"} 1027 1395066363000 http_requests_total{method="post",code="400"} 3 1395066363000 # Escaping in label values: msdos_file_access_time_seconds{path="C:DIRFILE.TXT",error="Cannot find file:n"FILE.TXT""} 1.458255915e9 # Minimalistic line: metric_without_timestamp_and_labels 12.47 # A weird metric from before the epoch: something_weird{problem="division by zero"} +Inf -3982045 # A histogram, which has a pretty complex representation in the text format: # HELP http_request_duration_seconds A histogram of the request duration. # TYPE http_request_duration_seconds histogram http_request_duration_seconds_bucket{le="0.05"} 24054 http_request_duration_seconds_bucket{le="0.1"} 33444 http_request_duration_seconds_bucket{le="0.2"} 100392 http_request_duration_seconds_bucket{le="0.5"} 129389 http_request_duration_seconds_bucket{le="1"} 133988 http_request_duration_seconds_bucket{le="+Inf"} 144320 http_request_duration_seconds_sum 53423 http_request_duration_seconds_count 144320
  • 51. gianarb.it ~ @gianarb func FetchMetricFamilies(url string, ch chan<- *dto.MetricFamily, certificate string, key string, skipServerCertCheck bool) error { defer close(ch) var transport *http.Transport if certificate != "" && key != "" { cert, err := tls.LoadX509KeyPair(certificate, key) if err != nil { return err } tlsConfig := &tls.Config{ Certificates: []tls.Certificate{cert}, InsecureSkipVerify: skipServerCertCheck, } tlsConfig.BuildNameToCertificate() transport = &http.Transport{TLSClientConfig: tlsConfig} } else { transport = &http.Transport{ TLSClientConfig: &tls.Config{InsecureSkipVerify: skipServerCertCheck}, } } https://github.com/prometheus/prom2json/blob/master/prom2json.go#L123
  • 52. Summary: ★ Do not be scared to write your code! ★ Use the API ★ Instrumentation code is a first class citizen ★ Keep calm and observe all together!
  • 53. © 2018 InfluxData. All rights reserved.53 @gianarb - gianluca@influxdb.com Credits and Links ¨ https://www.weave.works/technologies/gitops/ ¨ http://gianarb.it ¨ https://thenewstack.io/why-you-cant-afford-to-ignore-distributed-tracing-for-observability/ ¨ https://www.honeycomb.io/blog/ ¨ https://gianarb.it/blog/infra-as-code-short-long-ttl-resource ¨ https://gianarb.it/blog/kubernetes-shared-informer ¨ https://github.com/OpenObservability/OpenMetrics ¨ https://promcon.io/2018-munich/slides/openmetrics-transforming-the-prometheus-exposition-format-into-a -global-standard.pdf ¨ https://medium.com/opentracing/merging-opentracing-and-opencensus-f0fe9c7ca6f0
  • 54. ~ @gianarb - https://gianarb.it ~ Thanks @gianarb