SlideShare une entreprise Scribd logo
1  sur  29
Télécharger pour lire hors ligne
DevOps tutorial:
How to setup intelligent
machine learning alerts
Sébastien Léger, founder, Loud ML,
loudml.io
16 January 2019
DevOps tutorial:
How to setup intelligent
machine learning alerts
Sébastien Léger
founder and CEO, Loud ML
loudml.io
In this session, you will learn about:
• the TICK stack;
• Loud ML, a popular extension for live
anomaly detection in time-series data;
• how to setup alerting via Kapacitor and
integrate with PagerDuty, and other
alerting tools;
• Donut: the neural net architecture in Loud ML;
• applicable use-cases.
10K+ cumulated downloads in 2018,
lots of feedback, and more!
Joined the Rockstart AI accelerator program
in November 2018.
Thank You!
Agenda
• User stories, DevOps and
IoT
• Typical requirements for
alerting and ML
• Data pipeline
• Live streaming data demo
• Low false positives
(ie, low noise)
• Neural nets, deep-learning,
and Donut
• Loud ML 1.5 – Join the beta
• FAQ
User stories
User story 1: uptime
• My e-commerce site is global.
• Different users from different countries connect every day and
make purchases.
• Planned updates, or DevOps (CICD).
• Will it break anything, or cause downtime?
• How to spot if the # of transactions are correct, or how much
time is spent in the conversion funnel.
User story 2: security
• My e-commerce site is global.
• Different users from different countries connect every day and
make purchases.
• Different volumes from different sources.
User story 3: utilization
• The load changes during the day, during the night, and
during the weekend.
• Cloud or private DC resource utilization versus costs.
User story 4: Internet transit
• Running data center operations across multiple regions.
• Dynamic changes in traffic volume at the network edges.
• Get the right capacity, for the right cost (price per Mb/s).
User story 5: IoT
• PV: voltages, internal temperature, charge cycles, quantity
of electricity produced.
• Remote maintenance: spot damaged batteries.
• Physical infrastructure.
• Patterns in structural frequencies.
• Remote maintenance: spot when significant changes occur.
• Digital clone, industrial IoT (IIoT).
• Normal versus abnormal.
Typical requirements for
alerting and ML
Outliers, and then...
Typical requirements for alerting and ML
Performance
• Near real time, low alert delay.
• Running at scale, 24/7,
10,000+ users, hosts,
applications, devices.
• Low false positives; ie, low noise.
• Developer friendly: fast to
validate and deploy.
Functionality
• Can understand seasonality in the
data; eg, weekend vs daily
patterns, or across regions.
• Can learn and reinforce
continuously using live data.
• Can understand business rules.
• Works with third-party integrations.
Applicable to logs, metrics, events: page views, clicks, online users, orders,
response times, active IPs, syslogs, temperature, acceleration data & more.
Live TICK-L demo
Telegraf + InfluxDb + Chronograf + Kapacitor + Loud ML (AutoML)
Useful resources
• Website: loudml.io
• Blog: medium.com/loud-ml
• Github: github.com/regel/loudml
Data pipeline
Data pipeline
Metrics
and logs
collection
Pre-
processing,
feature
engineering
Database
storage
Machine
Learning
Automation
and alerting
DataViz
T (K)
(C)
(I) (L) (K)
Low false positives ie, low noise
How to evaluate ML fitness in a given application
Low false positives ie, low noise
How to evaluate ML fitness in a given application
Credits: dataschool.io
Precision P=TP/(TP+FP)
Recall R=TP/(TP+FN)
F1-score 2/(1/P+1/R)
Recall
Precision
Donut
arXiv:1802.03903
April 23-27, 2018, Lyon
Neural nets
Encode Decode
Donut neural nets
Baseline
Reconstruction
probability
Encode Sampling Decode
Donut is cool
• Donut has interesting properties.
• Low false positives (ie, low noise).
• It is as good as it gets.
• F1-score = 0.7 to 0.9, from arXiv:1802.03903.
• It can understand seasonality in the data.
• It can learn from labels.
Why Loud ML
Loud ML 1.5 beta
• Donut, plus more:
• Near real time, low alert delay.
• Running at scale, 24/7, 10,000+ users, hosts, applications.
• Can learn and reinforce continuously using live data.
• Developer friendly: fast to deploy, runs on CPUs or GPUs.
• Why Loud ML
• Fast ML deployment for time series data:
• the goal is to remove all the hurdles in AI.
• Explainable: gives % to observe specific values, easy to interpret!
• Agnostic of the underlying database.
• Accessible: the best ML, at a fraction of the cost.
Thank You
Interested in joining the beta?
loudml.io/contact@loud_ml
FAQ
How well does it run?
Will the model continue to learn?
What are the options for
feature engineering?

Contenu connexe

Tendances

Using AIOps to reduce incidents volume
Using AIOps to reduce incidents volumeUsing AIOps to reduce incidents volume
Using AIOps to reduce incidents volumeAmazon Web Services
 
JRI 2021 AIOps for Preventive& Automated Incident Management
JRI 2021 AIOps for Preventive& Automated Incident ManagementJRI 2021 AIOps for Preventive& Automated Incident Management
JRI 2021 AIOps for Preventive& Automated Incident ManagementAlexis KOALLA
 
Data Science Powered Apps for Internet of Things
Data Science Powered Apps for Internet of ThingsData Science Powered Apps for Internet of Things
Data Science Powered Apps for Internet of ThingsVMware Tanzu
 
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...Impetus Technologies
 
What Does Artificial Intelligence Have to Do with IT Operations?
What Does Artificial Intelligence Have to Do with IT Operations?What Does Artificial Intelligence Have to Do with IT Operations?
What Does Artificial Intelligence Have to Do with IT Operations?Precisely
 
AIOps: Your DevOps Co-Pilot
AIOps: Your DevOps Co-PilotAIOps: Your DevOps Co-Pilot
AIOps: Your DevOps Co-PilotDevOps.com
 
Real time machine learning
Real time machine learningReal time machine learning
Real time machine learningVinoth Kannan
 
Pivotal Digital Transformation Forum: Becoming a Data Driven Enterprise
Pivotal Digital Transformation Forum: Becoming a Data Driven EnterprisePivotal Digital Transformation Forum: Becoming a Data Driven Enterprise
Pivotal Digital Transformation Forum: Becoming a Data Driven EnterpriseVMware Tanzu
 
Gartner market guide ai ops platforms
Gartner market guide ai ops platformsGartner market guide ai ops platforms
Gartner market guide ai ops platformsRajeev Mohal
 
Before You Deploy An AIOps System, Do this
Before You Deploy An AIOps System, Do thisBefore You Deploy An AIOps System, Do this
Before You Deploy An AIOps System, Do thisOpsRamp
 
AIOps Roundtable Munich 2018
AIOps Roundtable Munich 2018AIOps Roundtable Munich 2018
AIOps Roundtable Munich 2018Splunk
 
Get your Service Intelligence off to a Flying Start
Get your Service Intelligence off to a Flying StartGet your Service Intelligence off to a Flying Start
Get your Service Intelligence off to a Flying StartSplunk
 
AIOps-Driven Network Performance Management: The First Step Toward Self-Heali...
AIOps-Driven Network Performance Management: The First Step Toward Self-Heali...AIOps-Driven Network Performance Management: The First Step Toward Self-Heali...
AIOps-Driven Network Performance Management: The First Step Toward Self-Heali...Enterprise Management Associates
 
Splunk for AIOps: Reduce IT outages through prediction with machine learning
Splunk for AIOps: Reduce IT outages through prediction with machine learningSplunk for AIOps: Reduce IT outages through prediction with machine learning
Splunk for AIOps: Reduce IT outages through prediction with machine learningDigital Transformation EXPO Event Series
 
Building Real Time Targeting Capabilities - Ryan Zotti, Subbu Thiruppathy - C...
Building Real Time Targeting Capabilities - Ryan Zotti, Subbu Thiruppathy - C...Building Real Time Targeting Capabilities - Ryan Zotti, Subbu Thiruppathy - C...
Building Real Time Targeting Capabilities - Ryan Zotti, Subbu Thiruppathy - C...Sri Ambati
 
AIOps: Anomalous Span Detection in Distributed Traces Using Deep Learning
AIOps: Anomalous Span Detection in Distributed Traces Using Deep LearningAIOps: Anomalous Span Detection in Distributed Traces Using Deep Learning
AIOps: Anomalous Span Detection in Distributed Traces Using Deep LearningJorge Cardoso
 
AIOps Is How We Will Survive DevOps
AIOps Is How We Will Survive DevOpsAIOps Is How We Will Survive DevOps
AIOps Is How We Will Survive DevOpsDEVOPS LIVE Meetup
 
AIOps - The next 5 years
AIOps - The next 5 yearsAIOps - The next 5 years
AIOps - The next 5 yearsMoogsoft
 
Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases
Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use CasesGlobal Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases
Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use CasesSanjay Sharma
 

Tendances (20)

Using AIOps to reduce incidents volume
Using AIOps to reduce incidents volumeUsing AIOps to reduce incidents volume
Using AIOps to reduce incidents volume
 
JRI 2021 AIOps for Preventive& Automated Incident Management
JRI 2021 AIOps for Preventive& Automated Incident ManagementJRI 2021 AIOps for Preventive& Automated Incident Management
JRI 2021 AIOps for Preventive& Automated Incident Management
 
Data Science Powered Apps for Internet of Things
Data Science Powered Apps for Internet of ThingsData Science Powered Apps for Internet of Things
Data Science Powered Apps for Internet of Things
 
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
 
What Does Artificial Intelligence Have to Do with IT Operations?
What Does Artificial Intelligence Have to Do with IT Operations?What Does Artificial Intelligence Have to Do with IT Operations?
What Does Artificial Intelligence Have to Do with IT Operations?
 
AIOps: Your DevOps Co-Pilot
AIOps: Your DevOps Co-PilotAIOps: Your DevOps Co-Pilot
AIOps: Your DevOps Co-Pilot
 
Real time machine learning
Real time machine learningReal time machine learning
Real time machine learning
 
Pivotal Digital Transformation Forum: Becoming a Data Driven Enterprise
Pivotal Digital Transformation Forum: Becoming a Data Driven EnterprisePivotal Digital Transformation Forum: Becoming a Data Driven Enterprise
Pivotal Digital Transformation Forum: Becoming a Data Driven Enterprise
 
Gartner market guide ai ops platforms
Gartner market guide ai ops platformsGartner market guide ai ops platforms
Gartner market guide ai ops platforms
 
Smart App@Pivotal by Dat Tran
Smart App@Pivotal by Dat TranSmart App@Pivotal by Dat Tran
Smart App@Pivotal by Dat Tran
 
Before You Deploy An AIOps System, Do this
Before You Deploy An AIOps System, Do thisBefore You Deploy An AIOps System, Do this
Before You Deploy An AIOps System, Do this
 
AIOps Roundtable Munich 2018
AIOps Roundtable Munich 2018AIOps Roundtable Munich 2018
AIOps Roundtable Munich 2018
 
Get your Service Intelligence off to a Flying Start
Get your Service Intelligence off to a Flying StartGet your Service Intelligence off to a Flying Start
Get your Service Intelligence off to a Flying Start
 
AIOps-Driven Network Performance Management: The First Step Toward Self-Heali...
AIOps-Driven Network Performance Management: The First Step Toward Self-Heali...AIOps-Driven Network Performance Management: The First Step Toward Self-Heali...
AIOps-Driven Network Performance Management: The First Step Toward Self-Heali...
 
Splunk for AIOps: Reduce IT outages through prediction with machine learning
Splunk for AIOps: Reduce IT outages through prediction with machine learningSplunk for AIOps: Reduce IT outages through prediction with machine learning
Splunk for AIOps: Reduce IT outages through prediction with machine learning
 
Building Real Time Targeting Capabilities - Ryan Zotti, Subbu Thiruppathy - C...
Building Real Time Targeting Capabilities - Ryan Zotti, Subbu Thiruppathy - C...Building Real Time Targeting Capabilities - Ryan Zotti, Subbu Thiruppathy - C...
Building Real Time Targeting Capabilities - Ryan Zotti, Subbu Thiruppathy - C...
 
AIOps: Anomalous Span Detection in Distributed Traces Using Deep Learning
AIOps: Anomalous Span Detection in Distributed Traces Using Deep LearningAIOps: Anomalous Span Detection in Distributed Traces Using Deep Learning
AIOps: Anomalous Span Detection in Distributed Traces Using Deep Learning
 
AIOps Is How We Will Survive DevOps
AIOps Is How We Will Survive DevOpsAIOps Is How We Will Survive DevOps
AIOps Is How We Will Survive DevOps
 
AIOps - The next 5 years
AIOps - The next 5 yearsAIOps - The next 5 years
AIOps - The next 5 years
 
Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases
Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use CasesGlobal Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases
Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases
 

Similaire à A DevOps Tutorial to Set-up Intelligent Machine Learning Driven Alerts

IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing a...
IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing a...IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing a...
IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing a...Ryft
 
Devising a practical approach to the Internet of Things
Devising a practical approach to the Internet of ThingsDevising a practical approach to the Internet of Things
Devising a practical approach to the Internet of ThingsGordon Haff
 
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017StampedeCon
 
Data in Motion - tech-intro-for-paris-hackathon
Data in Motion - tech-intro-for-paris-hackathonData in Motion - tech-intro-for-paris-hackathon
Data in Motion - tech-intro-for-paris-hackathonCisco DevNet
 
Data collection and enhancement
Data collection and enhancementData collection and enhancement
Data collection and enhancementankit_ppt
 
A Pragmatic Reference Architecture for The Internet of Things
A Pragmatic Reference Architecture for The Internet of ThingsA Pragmatic Reference Architecture for The Internet of Things
A Pragmatic Reference Architecture for The Internet of ThingsRick G. Garibay
 
Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...
Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...
Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...Spark Summit
 
[Feb 2020] Cours IoT - CentraleSupelec - Master SIO
[Feb 2020] Cours IoT - CentraleSupelec - Master SIO[Feb 2020] Cours IoT - CentraleSupelec - Master SIO
[Feb 2020] Cours IoT - CentraleSupelec - Master SIONicolas Lesconnec
 
Big data session five ( a )f
Big data session five ( a )fBig data session five ( a )f
Big data session five ( a )fmarukanda
 
Real-Time With AI – The Convergence Of Big Data And AI by Colin MacNaughton
Real-Time With AI – The Convergence Of Big Data And AI by Colin MacNaughtonReal-Time With AI – The Convergence Of Big Data And AI by Colin MacNaughton
Real-Time With AI – The Convergence Of Big Data And AI by Colin MacNaughtonSynerzip
 
IoT - Life at the Edge
IoT - Life at the EdgeIoT - Life at the Edge
IoT - Life at the EdgeNick Hunn
 
Real-time, Sensor-based Monitoring of Shipping Containers
Real-time, Sensor-based Monitoring of Shipping ContainersReal-time, Sensor-based Monitoring of Shipping Containers
Real-time, Sensor-based Monitoring of Shipping Containersbenaam
 
2016 iot summit_guy_bieber9
2016 iot summit_guy_bieber92016 iot summit_guy_bieber9
2016 iot summit_guy_bieber9Guy Bieber
 
When Computers are Everywhere, Will we have superpowers.
When Computers are Everywhere, Will we have superpowers.When Computers are Everywhere, Will we have superpowers.
When Computers are Everywhere, Will we have superpowers.Guy Bieber
 
Meetup 4/2/2016 - Functionele en technische architectuur IoT
Meetup  4/2/2016 - Functionele en technische architectuur IoTMeetup  4/2/2016 - Functionele en technische architectuur IoT
Meetup 4/2/2016 - Functionele en technische architectuur IoTDigipolis Antwerpen
 
Big data in Private Banking
Big data in Private BankingBig data in Private Banking
Big data in Private BankingJérôme Kehrli
 
How Tencent Applies Apache Pulsar to Apache InLong —— A Streaming Data Integr...
How Tencent Applies Apache Pulsar to Apache InLong —— A Streaming Data Integr...How Tencent Applies Apache Pulsar to Apache InLong —— A Streaming Data Integr...
How Tencent Applies Apache Pulsar to Apache InLong —— A Streaming Data Integr...StreamNative
 
The Big Data Is A Significant Subject Of Modern Times With...
The Big Data Is A Significant Subject Of Modern Times With...The Big Data Is A Significant Subject Of Modern Times With...
The Big Data Is A Significant Subject Of Modern Times With...Sarah Gordon
 
High five iot #iotconfua
High five iot #iotconfuaHigh five iot #iotconfua
High five iot #iotconfuaAndy Shutka
 

Similaire à A DevOps Tutorial to Set-up Intelligent Machine Learning Driven Alerts (20)

I learning lot
I learning lotI learning lot
I learning lot
 
IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing a...
IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing a...IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing a...
IoT Slam Keynote: Harnessing the Flood of Data with Heterogeneous Computing a...
 
Devising a practical approach to the Internet of Things
Devising a practical approach to the Internet of ThingsDevising a practical approach to the Internet of Things
Devising a practical approach to the Internet of Things
 
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
 
Data in Motion - tech-intro-for-paris-hackathon
Data in Motion - tech-intro-for-paris-hackathonData in Motion - tech-intro-for-paris-hackathon
Data in Motion - tech-intro-for-paris-hackathon
 
Data collection and enhancement
Data collection and enhancementData collection and enhancement
Data collection and enhancement
 
A Pragmatic Reference Architecture for The Internet of Things
A Pragmatic Reference Architecture for The Internet of ThingsA Pragmatic Reference Architecture for The Internet of Things
A Pragmatic Reference Architecture for The Internet of Things
 
Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...
Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...
Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...
 
[Feb 2020] Cours IoT - CentraleSupelec - Master SIO
[Feb 2020] Cours IoT - CentraleSupelec - Master SIO[Feb 2020] Cours IoT - CentraleSupelec - Master SIO
[Feb 2020] Cours IoT - CentraleSupelec - Master SIO
 
Big data session five ( a )f
Big data session five ( a )fBig data session five ( a )f
Big data session five ( a )f
 
Real-Time With AI – The Convergence Of Big Data And AI by Colin MacNaughton
Real-Time With AI – The Convergence Of Big Data And AI by Colin MacNaughtonReal-Time With AI – The Convergence Of Big Data And AI by Colin MacNaughton
Real-Time With AI – The Convergence Of Big Data And AI by Colin MacNaughton
 
IoT - Life at the Edge
IoT - Life at the EdgeIoT - Life at the Edge
IoT - Life at the Edge
 
Real-time, Sensor-based Monitoring of Shipping Containers
Real-time, Sensor-based Monitoring of Shipping ContainersReal-time, Sensor-based Monitoring of Shipping Containers
Real-time, Sensor-based Monitoring of Shipping Containers
 
2016 iot summit_guy_bieber9
2016 iot summit_guy_bieber92016 iot summit_guy_bieber9
2016 iot summit_guy_bieber9
 
When Computers are Everywhere, Will we have superpowers.
When Computers are Everywhere, Will we have superpowers.When Computers are Everywhere, Will we have superpowers.
When Computers are Everywhere, Will we have superpowers.
 
Meetup 4/2/2016 - Functionele en technische architectuur IoT
Meetup  4/2/2016 - Functionele en technische architectuur IoTMeetup  4/2/2016 - Functionele en technische architectuur IoT
Meetup 4/2/2016 - Functionele en technische architectuur IoT
 
Big data in Private Banking
Big data in Private BankingBig data in Private Banking
Big data in Private Banking
 
How Tencent Applies Apache Pulsar to Apache InLong —— A Streaming Data Integr...
How Tencent Applies Apache Pulsar to Apache InLong —— A Streaming Data Integr...How Tencent Applies Apache Pulsar to Apache InLong —— A Streaming Data Integr...
How Tencent Applies Apache Pulsar to Apache InLong —— A Streaming Data Integr...
 
The Big Data Is A Significant Subject Of Modern Times With...
The Big Data Is A Significant Subject Of Modern Times With...The Big Data Is A Significant Subject Of Modern Times With...
The Big Data Is A Significant Subject Of Modern Times With...
 
High five iot #iotconfua
High five iot #iotconfuaHigh five iot #iotconfua
High five iot #iotconfua
 

Plus de DevOps.com

Modernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source SoftwareModernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source SoftwareDevOps.com
 
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...DevOps.com
 
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...DevOps.com
 
Next Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and SnykNext Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and SnykDevOps.com
 
Vulnerability Discovery in the Cloud
Vulnerability Discovery in the CloudVulnerability Discovery in the Cloud
Vulnerability Discovery in the CloudDevOps.com
 
2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and Predictions2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and PredictionsDevOps.com
 
A New Year’s Ransomware Resolution
A New Year’s Ransomware ResolutionA New Year’s Ransomware Resolution
A New Year’s Ransomware ResolutionDevOps.com
 
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)DevOps.com
 
Don't Panic! Effective Incident Response
Don't Panic! Effective Incident ResponseDon't Panic! Effective Incident Response
Don't Panic! Effective Incident ResponseDevOps.com
 
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's CultureCreating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's CultureDevOps.com
 
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with TeleportRole Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with TeleportDevOps.com
 
Monitoring Serverless Applications with Datadog
Monitoring Serverless Applications with DatadogMonitoring Serverless Applications with Datadog
Monitoring Serverless Applications with DatadogDevOps.com
 
Deliver your App Anywhere … Publicly or Privately
Deliver your App Anywhere … Publicly or PrivatelyDeliver your App Anywhere … Publicly or Privately
Deliver your App Anywhere … Publicly or PrivatelyDevOps.com
 
Securing medical apps in the age of covid final
Securing medical apps in the age of covid finalSecuring medical apps in the age of covid final
Securing medical apps in the age of covid finalDevOps.com
 
How to Build a Healthy On-Call Culture
How to Build a Healthy On-Call CultureHow to Build a Healthy On-Call Culture
How to Build a Healthy On-Call CultureDevOps.com
 
The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021DevOps.com
 
Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?DevOps.com
 
Secure Data Sharing in OpenShift Environments
Secure Data Sharing in OpenShift EnvironmentsSecure Data Sharing in OpenShift Environments
Secure Data Sharing in OpenShift EnvironmentsDevOps.com
 
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...DevOps.com
 
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...DevOps.com
 

Plus de DevOps.com (20)

Modernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source SoftwareModernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source Software
 
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
 
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
 
Next Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and SnykNext Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and Snyk
 
Vulnerability Discovery in the Cloud
Vulnerability Discovery in the CloudVulnerability Discovery in the Cloud
Vulnerability Discovery in the Cloud
 
2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and Predictions2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and Predictions
 
A New Year’s Ransomware Resolution
A New Year’s Ransomware ResolutionA New Year’s Ransomware Resolution
A New Year’s Ransomware Resolution
 
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
 
Don't Panic! Effective Incident Response
Don't Panic! Effective Incident ResponseDon't Panic! Effective Incident Response
Don't Panic! Effective Incident Response
 
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's CultureCreating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
 
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with TeleportRole Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
 
Monitoring Serverless Applications with Datadog
Monitoring Serverless Applications with DatadogMonitoring Serverless Applications with Datadog
Monitoring Serverless Applications with Datadog
 
Deliver your App Anywhere … Publicly or Privately
Deliver your App Anywhere … Publicly or PrivatelyDeliver your App Anywhere … Publicly or Privately
Deliver your App Anywhere … Publicly or Privately
 
Securing medical apps in the age of covid final
Securing medical apps in the age of covid finalSecuring medical apps in the age of covid final
Securing medical apps in the age of covid final
 
How to Build a Healthy On-Call Culture
How to Build a Healthy On-Call CultureHow to Build a Healthy On-Call Culture
How to Build a Healthy On-Call Culture
 
The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021
 
Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?
 
Secure Data Sharing in OpenShift Environments
Secure Data Sharing in OpenShift EnvironmentsSecure Data Sharing in OpenShift Environments
Secure Data Sharing in OpenShift Environments
 
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
 
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
 

Dernier

Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????blackmambaettijean
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 

Dernier (20)

Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 

A DevOps Tutorial to Set-up Intelligent Machine Learning Driven Alerts

  • 1. DevOps tutorial: How to setup intelligent machine learning alerts Sébastien Léger, founder, Loud ML, loudml.io 16 January 2019
  • 2. DevOps tutorial: How to setup intelligent machine learning alerts Sébastien Léger founder and CEO, Loud ML loudml.io In this session, you will learn about: • the TICK stack; • Loud ML, a popular extension for live anomaly detection in time-series data; • how to setup alerting via Kapacitor and integrate with PagerDuty, and other alerting tools; • Donut: the neural net architecture in Loud ML; • applicable use-cases.
  • 3. 10K+ cumulated downloads in 2018, lots of feedback, and more! Joined the Rockstart AI accelerator program in November 2018. Thank You!
  • 4. Agenda • User stories, DevOps and IoT • Typical requirements for alerting and ML • Data pipeline • Live streaming data demo • Low false positives (ie, low noise) • Neural nets, deep-learning, and Donut • Loud ML 1.5 – Join the beta • FAQ
  • 6. User story 1: uptime • My e-commerce site is global. • Different users from different countries connect every day and make purchases. • Planned updates, or DevOps (CICD). • Will it break anything, or cause downtime? • How to spot if the # of transactions are correct, or how much time is spent in the conversion funnel.
  • 7. User story 2: security • My e-commerce site is global. • Different users from different countries connect every day and make purchases. • Different volumes from different sources.
  • 8. User story 3: utilization • The load changes during the day, during the night, and during the weekend. • Cloud or private DC resource utilization versus costs.
  • 9. User story 4: Internet transit • Running data center operations across multiple regions. • Dynamic changes in traffic volume at the network edges. • Get the right capacity, for the right cost (price per Mb/s).
  • 10. User story 5: IoT • PV: voltages, internal temperature, charge cycles, quantity of electricity produced. • Remote maintenance: spot damaged batteries. • Physical infrastructure. • Patterns in structural frequencies. • Remote maintenance: spot when significant changes occur. • Digital clone, industrial IoT (IIoT). • Normal versus abnormal.
  • 11. Typical requirements for alerting and ML Outliers, and then...
  • 12. Typical requirements for alerting and ML Performance • Near real time, low alert delay. • Running at scale, 24/7, 10,000+ users, hosts, applications, devices. • Low false positives; ie, low noise. • Developer friendly: fast to validate and deploy. Functionality • Can understand seasonality in the data; eg, weekend vs daily patterns, or across regions. • Can learn and reinforce continuously using live data. • Can understand business rules. • Works with third-party integrations. Applicable to logs, metrics, events: page views, clicks, online users, orders, response times, active IPs, syslogs, temperature, acceleration data & more.
  • 13. Live TICK-L demo Telegraf + InfluxDb + Chronograf + Kapacitor + Loud ML (AutoML)
  • 14. Useful resources • Website: loudml.io • Blog: medium.com/loud-ml • Github: github.com/regel/loudml
  • 17. Low false positives ie, low noise How to evaluate ML fitness in a given application
  • 18. Low false positives ie, low noise How to evaluate ML fitness in a given application Credits: dataschool.io Precision P=TP/(TP+FP) Recall R=TP/(TP+FN) F1-score 2/(1/P+1/R) Recall Precision
  • 22. Donut is cool • Donut has interesting properties. • Low false positives (ie, low noise). • It is as good as it gets. • F1-score = 0.7 to 0.9, from arXiv:1802.03903. • It can understand seasonality in the data. • It can learn from labels.
  • 24. Loud ML 1.5 beta • Donut, plus more: • Near real time, low alert delay. • Running at scale, 24/7, 10,000+ users, hosts, applications. • Can learn and reinforce continuously using live data. • Developer friendly: fast to deploy, runs on CPUs or GPUs. • Why Loud ML • Fast ML deployment for time series data: • the goal is to remove all the hurdles in AI. • Explainable: gives % to observe specific values, easy to interpret! • Agnostic of the underlying database. • Accessible: the best ML, at a fraction of the cost.
  • 25. Thank You Interested in joining the beta? loudml.io/contact@loud_ml
  • 26. FAQ
  • 27. How well does it run?
  • 28. Will the model continue to learn?
  • 29. What are the options for feature engineering?