SlideShare une entreprise Scribd logo
1  sur  23
Télécharger pour lire hors ligne
1
MONITORING AT
SERVICE PROVIDER
SCALE
June 23, 2014
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
INTRODUCTIONS
CHRIS JACKSON
Rackspace CTO (DevOps Practice)
chris.jackson@rackspace.co.uk
@chriswiggy
+44 (0) 7712 854462
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
3
Image courtesy of http://www.toastmasters.krakow.pl/
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
4
THE DEVOPS RACK APPLIANCE…
ALL OF THE THINGS IN 3-CLICKS!
Image courtesy of Axiomtek
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
5
HOW DO WE APPLY DEVOPS
TO SCALE SERVICE?
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
THE DEVOPS PRACTICE AT RACKSPACE
6
THE FIRST WAY – CREATE A DELIVERY SYSTEM
THE SECOND WAY – AMPLIFY FEEDBACK LOOPS
THE THIRD WAY – LEARN & EXPERIMENT CONTINUOUSLY
This is PRODUCT & SUPPORT delivering great service powered by automation
!
Our DevOps practice is a way to accelerate feedback by removing complexity
!
The goal is to never hold our customers up, we must always be faster…
!
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
SCALING TRANSFORMATION
7
SCALING TRANSFORMATIONS
Image courtesy of Wordpress
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
8
SO WHAT ABOUT MONITORING?
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
9
WHY #MONITORINGSUCKS
FOR RACKSPACE
We generate over 40,000 events per second on just our Cloud Monitoring platform.
If a customer is Managed, we troubleshoot ANY alarm that is raised.
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
10
WHY #MONITORINGSUCKS FOR RACKSPACE
Image courtesy of Spot On Lists
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
11
WHY #MONITORINGSUCKS FOR RACKSPACE
Image courtesy of Cepolina
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
12
WHY #MONITORINGSUCKS FOR RACKSPACE
Image courtesy of New Line Cinema
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
13
WHAT ARE WE DOING
TO FIX THIS?
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
SCALE MONITORING TUNING
14
WHEN TUNING IS MANUAL:
EFFORT=tN
Where t = time to configure a single node and N = number of nodes
0 100 200 300 400 500
Effort
# Nodes
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
SCALE MONITORING TUNING
15
WHEN TUNING IS INTELLIGENT:
EFFORT=t1-N
Where t = time to configure a single node and N = number of nodesEffort
# Nodes
Shrink this!
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
TOOLS FOR INTELLIGENT CONFIGURATION
16
Configuration Management
If you know what you want to measure!
If your constraint is just the number of nodes!
Gets complicated when you’re thresholding all of the things…!
Configuration Discovery
We call it Waldo, you know it as Sartori in Openstack!
Uses ohai-solo to find information about your configuration!
Compare to best practice and make changes accordingly…!
Image courtesy of Walker Books Ltd
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
MONITORING DATA ANALYTICS
17
EVENT GENERATION EVENT PROCESSING
EVENT HANDLING
EVENT ANALYTICS
TUNING INSIGHT
CREATE FEEDBACK LOOPS INSIDE MONITORING
REMOVE WORK BY TUNING ALARMS
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
INTELLIGENT ALERTS
18
ANOMALY MONITORING PATTERN RECOGNITION
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
WHAT ARE WE USING?
19
statsd
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
TOOLCHAIN INTEGRATION
20
EVENT GENERATION EVENT PROCESSING
EVENT HANDLING
EVENT ANALYTICS
TUNING INSIGHT
statsd
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
HELP US TO DO THE HARD WORK
21
YOU
US
Things We Want to Know™!
How do you use monitoring?!
How intelligent is the tuning?!
What use of anomaly or threshold?!
Approach for creating tool chains?!
What do you dislike doing?!
We’ll build it and open
source it!!
Image courtesy of Dave Hogan/Getty Images
Twitter - @chriswiggy
http://www.rackspace.com/devops
http://devops.rackspace.com
22
THANK YOU
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
MORE THAN A CLOUD COMPANY
WE PROVIDE
SUPPORTFOR EVERY STEP OF YOUR JOURNEY
JUST STARTING?
RACKSPACE ADVISORY
SERVICES
NEED SERVICES?
RACKSPACE HYBRID
CLOUD
NEED SUPPORT?
RACKSPACE DEVOPS
AUTOMATION SERVICE

Contenu connexe

En vedette

Казка про деревце
Казка про деревцеКазка про деревце
Казка про деревце
ryabich1976
 
η ...συνταγή (1)
η ...συνταγή (1)η ...συνταγή (1)
η ...συνταγή (1)
Athina Kakarikou
 

En vedette (15)

Production testing through monitoring
Production testing through monitoringProduction testing through monitoring
Production testing through monitoring
 
SREcon 2016 Performance Checklists for SREs
SREcon 2016 Performance Checklists for SREsSREcon 2016 Performance Checklists for SREs
SREcon 2016 Performance Checklists for SREs
 
Microservices Workshop All Topics Deck 2016
Microservices Workshop All Topics Deck 2016Microservices Workshop All Topics Deck 2016
Microservices Workshop All Topics Deck 2016
 
Scaling Pinterest's Monitoring
Scaling Pinterest's MonitoringScaling Pinterest's Monitoring
Scaling Pinterest's Monitoring
 
Bruce Lawson: Progressive Web Apps: the future of Apps
Bruce Lawson: Progressive Web Apps: the future of AppsBruce Lawson: Progressive Web Apps: the future of Apps
Bruce Lawson: Progressive Web Apps: the future of Apps
 
How to Speak "Manager"
How to Speak "Manager"How to Speak "Manager"
How to Speak "Manager"
 
Continuous Delivery: Making DevOps Awesome
Continuous Delivery: Making DevOps AwesomeContinuous Delivery: Making DevOps Awesome
Continuous Delivery: Making DevOps Awesome
 
Казка про деревце
Казка про деревцеКазка про деревце
Казка про деревце
 
It's me!
It's me!It's me!
It's me!
 
η ...συνταγή (1)
η ...συνταγή (1)η ...συνταγή (1)
η ...συνταγή (1)
 
Balancing Act
Balancing ActBalancing Act
Balancing Act
 
folklor
folklorfolklor
folklor
 
The History of the Horror Genre
The History of the Horror GenreThe History of the Horror Genre
The History of the Horror Genre
 
BpSM 2014.09. - Sebestyén Endre: Alternatív splicing változások elemzése rákg...
BpSM 2014.09. - Sebestyén Endre: Alternatív splicing változások elemzése rákg...BpSM 2014.09. - Sebestyén Endre: Alternatív splicing változások elemzése rákg...
BpSM 2014.09. - Sebestyén Endre: Alternatív splicing változások elemzése rákg...
 
Vs
VsVs
Vs
 

Similaire à DevOpsDays Amsterdam - Monitoring at Service Provider Scale

Leveraging the Power of the Cloud for Your Business to Grow: Nate Taylor at S...
Leveraging the Power of the Cloud for Your Business to Grow: Nate Taylor at S...Leveraging the Power of the Cloud for Your Business to Grow: Nate Taylor at S...
Leveraging the Power of the Cloud for Your Business to Grow: Nate Taylor at S...
smecchk
 
jclouds Support Training
jclouds Support Trainingjclouds Support Training
jclouds Support Training
Everett Toews
 
Solve the Cross-Cloud Conundrum with jclouds
Solve the Cross-Cloud Conundrum with jcloudsSolve the Cross-Cloud Conundrum with jclouds
Solve the Cross-Cloud Conundrum with jclouds
Everett Toews
 
Consideration for Building a Private Cloud
Consideration for Building a Private CloudConsideration for Building a Private Cloud
Consideration for Building a Private Cloud
OpenStack Foundation
 
Software Development kits
Software Development kitsSoftware Development kits
Software Development kits
Everett Toews
 
The Next Generation IT Department MUST HAVE CLOUD
The Next Generation IT Department MUST HAVE CLOUDThe Next Generation IT Department MUST HAVE CLOUD
The Next Generation IT Department MUST HAVE CLOUD
Rackspace
 
Building cloudy apps
Building cloudy appsBuilding cloudy apps
Building cloudy apps
tonytcampbell
 
Behind the Curtain: Operating an OpenStack Powered Private Cloud
Behind the Curtain: Operating an OpenStack Powered Private CloudBehind the Curtain: Operating an OpenStack Powered Private Cloud
Behind the Curtain: Operating an OpenStack Powered Private Cloud
Niki Acosta
 

Similaire à DevOpsDays Amsterdam - Monitoring at Service Provider Scale (20)

Be a Cloud Native
Be a Cloud NativeBe a Cloud Native
Be a Cloud Native
 
What Does a Cloud Enabled Business Look Like?
What Does a Cloud Enabled Business Look Like?What Does a Cloud Enabled Business Look Like?
What Does a Cloud Enabled Business Look Like?
 
PyCon Russia 2014 - Auto Scale in the Cloud
PyCon Russia 2014 - Auto Scale in the CloudPyCon Russia 2014 - Auto Scale in the Cloud
PyCon Russia 2014 - Auto Scale in the Cloud
 
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
 
Autoscaling Best Practices
Autoscaling Best PracticesAutoscaling Best Practices
Autoscaling Best Practices
 
Leveraging the Power of the Cloud for Your Business to Grow: Nate Taylor at S...
Leveraging the Power of the Cloud for Your Business to Grow: Nate Taylor at S...Leveraging the Power of the Cloud for Your Business to Grow: Nate Taylor at S...
Leveraging the Power of the Cloud for Your Business to Grow: Nate Taylor at S...
 
jclouds Support Training
jclouds Support Trainingjclouds Support Training
jclouds Support Training
 
Solve the Cross-Cloud Conundrum with jclouds
Solve the Cross-Cloud Conundrum with jcloudsSolve the Cross-Cloud Conundrum with jclouds
Solve the Cross-Cloud Conundrum with jclouds
 
Why the Cloud is Important for Non-Profit Orgs
Why the Cloud is Important for Non-Profit OrgsWhy the Cloud is Important for Non-Profit Orgs
Why the Cloud is Important for Non-Profit Orgs
 
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops EngineTearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
 
Consideration for Building a Private Cloud
Consideration for Building a Private CloudConsideration for Building a Private Cloud
Consideration for Building a Private Cloud
 
Software Development kits
Software Development kitsSoftware Development kits
Software Development kits
 
The Next Generation IT Department MUST HAVE CLOUD
The Next Generation IT Department MUST HAVE CLOUDThe Next Generation IT Department MUST HAVE CLOUD
The Next Generation IT Department MUST HAVE CLOUD
 
The Search for Cloud's "God Particle"
The Search for Cloud's "God Particle"The Search for Cloud's "God Particle"
The Search for Cloud's "God Particle"
 
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid CloudMigrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
 
Building cloudy apps
Building cloudy appsBuilding cloudy apps
Building cloudy apps
 
Build Your Custom Performance Testing Framework
Build Your Custom Performance Testing FrameworkBuild Your Custom Performance Testing Framework
Build Your Custom Performance Testing Framework
 
Behind The Scenes: New Rackspace Cloud Control Panel
Behind The Scenes:  New Rackspace Cloud Control PanelBehind The Scenes:  New Rackspace Cloud Control Panel
Behind The Scenes: New Rackspace Cloud Control Panel
 
Behind the Curtain: Operating an OpenStack Powered Private Cloud
Behind the Curtain: Operating an OpenStack Powered Private CloudBehind the Curtain: Operating an OpenStack Powered Private Cloud
Behind the Curtain: Operating an OpenStack Powered Private Cloud
 
Introduction to DevOps - Rackspace tech night
Introduction to DevOps - Rackspace tech nightIntroduction to DevOps - Rackspace tech night
Introduction to DevOps - Rackspace tech night
 

Dernier

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Dernier (20)

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 

DevOpsDays Amsterdam - Monitoring at Service Provider Scale

  • 2. RACKSPACE® HOSTING | WWW.RACKSPACE.COM INTRODUCTIONS CHRIS JACKSON Rackspace CTO (DevOps Practice) chris.jackson@rackspace.co.uk @chriswiggy +44 (0) 7712 854462
  • 3. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 3 Image courtesy of http://www.toastmasters.krakow.pl/
  • 4. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 4 THE DEVOPS RACK APPLIANCE… ALL OF THE THINGS IN 3-CLICKS! Image courtesy of Axiomtek
  • 5. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 5 HOW DO WE APPLY DEVOPS TO SCALE SERVICE?
  • 6. RACKSPACE® HOSTING | WWW.RACKSPACE.COM THE DEVOPS PRACTICE AT RACKSPACE 6 THE FIRST WAY – CREATE A DELIVERY SYSTEM THE SECOND WAY – AMPLIFY FEEDBACK LOOPS THE THIRD WAY – LEARN & EXPERIMENT CONTINUOUSLY This is PRODUCT & SUPPORT delivering great service powered by automation ! Our DevOps practice is a way to accelerate feedback by removing complexity ! The goal is to never hold our customers up, we must always be faster… !
  • 7. RACKSPACE® HOSTING | WWW.RACKSPACE.COM SCALING TRANSFORMATION 7 SCALING TRANSFORMATIONS Image courtesy of Wordpress
  • 8. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 8 SO WHAT ABOUT MONITORING?
  • 9. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 9 WHY #MONITORINGSUCKS FOR RACKSPACE We generate over 40,000 events per second on just our Cloud Monitoring platform. If a customer is Managed, we troubleshoot ANY alarm that is raised.
  • 10. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 10 WHY #MONITORINGSUCKS FOR RACKSPACE Image courtesy of Spot On Lists
  • 11. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 11 WHY #MONITORINGSUCKS FOR RACKSPACE Image courtesy of Cepolina
  • 12. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 12 WHY #MONITORINGSUCKS FOR RACKSPACE Image courtesy of New Line Cinema
  • 13. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 13 WHAT ARE WE DOING TO FIX THIS?
  • 14. RACKSPACE® HOSTING | WWW.RACKSPACE.COM SCALE MONITORING TUNING 14 WHEN TUNING IS MANUAL: EFFORT=tN Where t = time to configure a single node and N = number of nodes 0 100 200 300 400 500 Effort # Nodes
  • 15. RACKSPACE® HOSTING | WWW.RACKSPACE.COM SCALE MONITORING TUNING 15 WHEN TUNING IS INTELLIGENT: EFFORT=t1-N Where t = time to configure a single node and N = number of nodesEffort # Nodes Shrink this!
  • 16. RACKSPACE® HOSTING | WWW.RACKSPACE.COM TOOLS FOR INTELLIGENT CONFIGURATION 16 Configuration Management If you know what you want to measure! If your constraint is just the number of nodes! Gets complicated when you’re thresholding all of the things…! Configuration Discovery We call it Waldo, you know it as Sartori in Openstack! Uses ohai-solo to find information about your configuration! Compare to best practice and make changes accordingly…! Image courtesy of Walker Books Ltd
  • 17. RACKSPACE® HOSTING | WWW.RACKSPACE.COM MONITORING DATA ANALYTICS 17 EVENT GENERATION EVENT PROCESSING EVENT HANDLING EVENT ANALYTICS TUNING INSIGHT CREATE FEEDBACK LOOPS INSIDE MONITORING REMOVE WORK BY TUNING ALARMS
  • 18. RACKSPACE® HOSTING | WWW.RACKSPACE.COM INTELLIGENT ALERTS 18 ANOMALY MONITORING PATTERN RECOGNITION
  • 19. RACKSPACE® HOSTING | WWW.RACKSPACE.COM WHAT ARE WE USING? 19 statsd
  • 20. RACKSPACE® HOSTING | WWW.RACKSPACE.COM TOOLCHAIN INTEGRATION 20 EVENT GENERATION EVENT PROCESSING EVENT HANDLING EVENT ANALYTICS TUNING INSIGHT statsd
  • 21. RACKSPACE® HOSTING | WWW.RACKSPACE.COM HELP US TO DO THE HARD WORK 21 YOU US Things We Want to Know™! How do you use monitoring?! How intelligent is the tuning?! What use of anomaly or threshold?! Approach for creating tool chains?! What do you dislike doing?! We’ll build it and open source it!! Image courtesy of Dave Hogan/Getty Images
  • 23. RACKSPACE® HOSTING | WWW.RACKSPACE.COM MORE THAN A CLOUD COMPANY WE PROVIDE SUPPORTFOR EVERY STEP OF YOUR JOURNEY JUST STARTING? RACKSPACE ADVISORY SERVICES NEED SERVICES? RACKSPACE HYBRID CLOUD NEED SUPPORT? RACKSPACE DEVOPS AUTOMATION SERVICE