Chaos engineering tools offer a great way to test an application’s resiliency in a Kubernetes deployment. But chaos experiments can induce failure modes that have never been seen before, causing incidents to slip through existing alert rules. Detecting these failures can be tricky, and Zebrium solves this with unassisted machine learning. Our interactive panel discusses, in the context of a demo, a set of Litmus chaos engineering experiments against a distributed Kubernetes app and will use Zebrium Autonomous Log Monitoring to auto-detect incidents and provide an indication of the root cause.

1. The webinar will start
in a brief moment.
We are setting up the things for you :)
Thank You
for joining

2. Using Litmus Chaos
Engineering and AI for
Automatic Incident Detection
TOPIC
04 | Mar |
2020

3. @uma_mukkara
Zebrium 1
Chief Architect
Co-Founder
MayaData Inc
Zebrium 2
Developer Advocate
MayaData Inc
Brian Matheson
Developer Advocate
MayaData Inc
Your Presenters:
@@dgildeh
David Gildeh
Product & Strategy
Zebrium
@brian_matheson

4. Agenda
● Introduction
● Litmus Chaos Engineering overview
● Zebrium Overview
● Demo of Litmus + Zebrium
● Update on OpenEBS 1.7 release
● Q&A
● How to get started

5. Introduction to MayaData
Cloud Native
Management for
Cloud Native Data
True Data Agility
Turns Kubernetes itself into a
data layer via OpenEBS for
data & Litmus for ongoing
chaos engineering.

6. What is OpenEBS
What is OpenEBS

7. Litmus
Chaos
Engineering

8. Cloud Native Chaos Engineering
(Introduction)
Chaos engineering for cloud native environments
Chaos engineering done cloud-native way
Chaos engineering done Kubernetes native way

9. Cloud-Native environments
● Credits: Taken from GitLab commit conference slides; Author is Dan Kohn, CNCF
10

10. Linux
Kubernetes
Nodejs
Libraries
Cloud-Native environment
Your code vs entire stack
11
17M SLOC
35M SLOC
12M SLOC
2.5M SLOC
Mycode (50K)
● Kubernetes upgrades happen
frequently
● Associated applications and
libraries will have updates as
often as your code

11. Linux
Kubernetes
Nodejs
Libraries
Cloud-Native environment
Your software environment
- Is very Dynamic
- Needs continuous validation
Your hardware environment
- Is also Dynamic
- Is managed by someone else
12
17M SLOC
35M SLOC
12M SLOC
2.5M SLOC
Mycode (50K)

12. Linux
Kubernetes
Nodejs
Libraries
Cloud-Native environment
You need
- Chaos Engineering
13
17M SLOC
35M SLOC
12M SLOC
2.5M SLOC
Mycode (50K)

13. Cloud-Native environment
You need
- Cloud-Native Chaos
Engineering
14
The other big differences in
Cloud-Native environment
are:
- YAML manifests for
intent (kubectl apply)
- GitOps
Chaos
Engineering
Cloud Native
Chaos Engineering
=

14. Cloud-Native Chaos Engineering
Cloud Native
APIs
POD Deployment
PVC Statefulset
SVC CRDs
For
Development
For Chaos Testing
Cloud Native
APIs
?
Cloud-native
Application
15

15. Cloud Native
APIs
POD Deployment
PVC Statefulset
SVC CRDs
For Chaos Testing
Cloud Native
APIs
Chaos
Resources
New CRDs
Cloud-native
Application
For
Development
Cloud-Native Chaos Engineering
16

16. Chaos
Resources
Cloud-Native Chaos Engineering
17
Chaos Operator
Chaos CRDs
Chaos Metrics

17. Cloud Native Chaos Engineering
Principles

18. Open Source
Principles of Cloud-Native Chaos Engineering
19
Chaos API/CRDs
Pluggable chaos
Community
driven

19. Principles of Cloud-Native Chaos Engineering
20
CNCF blog: http://bit.ly/cncf-chaos

20. Cloud Native Chaos Engineering
Litmus project

21. Litmus project - Introduction
22
● Leading open source project for Chaos Engineering on
Kubernetes
● Apache2 License
● https://github.com/litmuschaos
● 50+ contributors
● 630+ stars
● CNCF Landscape - https://landscape.cncf.io/selected=litmus
● Chaos Hub https://hub.litmuschaos.io/
● CNCF Blog https://bit.ly/cncf-chaos

22. Litmus - use cases
23
Chaos in CI pipelines Chaos in staging
Certify Kubernetes
across upgrades
Chaos in Production

23. Litmus - Cloud Native
24
Open Source Chaos API/CRDs
Pluggable chaos Community driven

24. Cloud Native Chaos Engineering
Getting Started

25. How Litmus works
26
Chaos
Libraries
hub.litmuschaos.io
Chaos
Operator
kubectl apply -f <litmus.yaml>
Chaos
Charts
App
container
Chaos
container
Install Litmus
Install Charts
Inject Chaos
Chaos
Result

26. Cloud-Native Chaos Engineering - Example
apiVersion: v1
kind: Pod
metadata:
name: percona-pod
labels:
app: percona
spec:
containers:
- name: percona
image: percona:2.4
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
Name: demo-vol1-claim
spec:
storageClassName: openebs-
jiva-default
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
Create POD
Create PV
Inject Chaos
Cloud Native
Developer
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
name: engine-percona
spec:
appinfo:
appns: default
applabel: "app=percona"
experiments:
- name: replica-kill
spec:
components:
- name: target-kill
spec:
components:
27

27. ChaosHub
28

28. ChaosHub
29
Charts
Generic
Application
Specific

29. ChaosHub - Generic experiments
30
Generic
Experiments
Pod-Delete Container-kill
pod-network-losspod-network-latency
pod-cpu-hog
node-cpu-hog
disk-filldisk-loss
node-drain
pod-network-corruption

30. Getting involved with Litmus
31
Community - #litmus channel on kubernetes slack
Github - https://github.com/litmuschaos

31. Zebrium

32. Demo

33. What’s new with
OpenEBS 1.7
release

34. OpenEBS 1.7
● Stability fixes for Jiva, cStor and Upgrades.
● Automating ARM builds (NDM)
● cStor CSI Driver (alpha) - API Schema towards v1
● Jiva CSI Driver (alpha) - Volume Resize
● ZFS Local PV (alpha) - Snapshot and Clone
● MayaStor (alpha) - Refactor control plane to have MayaStor
Volume
● Resiliency tests around target and replica failures.
● Automated upgrade test
● cStor CSI volume provisioning/resize/remount tests
● ZFS Local PV added to e2e pipelines
● Jiva CSI Driver tests added to pipeline
Release updates

35. QUESTIONS
& ANSWERS

36. GET
STARTED

37. A complete Data Agility Solution
STORE AND PROTECT VALIDATE AND HARDEN UNDERSTAND AND CONTROL
CONTAINER ATTACHED STORAGE CHAOS ENGINEERING DATA MIGRATION, HA, DR

38. Get Started - Go to mayadata.io

39. Thank You