Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Monitoring Modern Architectures with Data Science

331 vues

Publié le

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2A6lx09.

Dave Casper talks about how modern data science and algorithms are being applied to "fight machines with machines". Filmed at qconsf.com.

Dave Casper is CTO at Moogsoft. Previously, he served as Senior Vice President and Executive Director at several of the world's largest financial institutions, including UBS, Merrill Lynch, Bank of America, and Deutsche Bank and also co-chaired the Open Data Center Alliance Infrastructure Group.

Publié dans : Technologie
  • Soyez le premier à commenter

Monitoring Modern Architectures with Data Science

  1. 1. Monitoring Modern Architectures with Data Science QCon 2017 Dave Casper, CTO
  2. 2. InfoQ.com: News & Community Site • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ modern-architectures-data-science
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
  4. 4. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 Much has changed since simple distributed client/server architectures and so-too have the technologies and industry practices around monitoring. Cloud-Native, DevOps, blue/green deployments, server-less, edge/fog, IoT all fit into a world much better handled by the emerging Artificial Intelligence for IT Operations domain more-so than traditional ITIL/SDLC approaches. Abstract
  5. 5. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 Software continues to eat the world. Software automates, defines. The world is "going digital" and it's quite exciting -- but this always-connected from-everything-to-everywhere world adds complexity to software systems and this talk will dive in to some of that complexity and how modern data science and algorithms are being applied to "fight machines with machines," so to speak. Abstract
  6. 6. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 -25822282 623992118 1343963318
  7. 7. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017
  8. 8. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 moogsoft
  9. 9. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017
  10. 10. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 discovery monitoring (observing) analytics
  11. 11. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 fluid infrastructure containers dc/os server-less software defined/dynamic
  12. 12. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 anything anywhere anytime data/txfrom mobile IoT bots/RUM
  13. 13. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017
  14. 14. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 "if/else" rules algorithms ML millions millions noise filt. clustering prc deja vu
  15. 15. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 AIOps AI for IT Ops
  16. 16. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 customer/ business perspective
  17. 17. COURAGE INSIGHT CONTEXT VELOCITY ARE YOU READY TO GO DIGITAL ? This slide courtesy Andy Brown, Sandhill East https://www.linkedin.com/in/andybrown63/
  18. 18. “Silicon Valley is coming. There are hundreds of startups with a lot of brains and money working on various alternatives to traditional banking. They are very good at reducing the ‘pain points’ …” JAMIE DIMON JPMorgan Chase & Co. Chairman & Chief Executive Officer April 2015 This slide courtesy Andy Brown, Sandhill East https://www.linkedin.com/in/andybrown63/
  19. 19. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 go digital or die trying
  20. 20. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 gs wants to become "google of wall st."
  21. 21. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 stanford PhD CIO CFO marquee data analytics data api api monitor observe analyze
  22. 22. THE REALLY BIG PICTURE 2020 2021 2022 2023 2024 In 5 – 10 years, every company will be a Digital Software Business Security, service assurance and consumer centricity become THE BOARD LEVEL PRIORITY Enterprises going DIGITAL ADOPT HYBRID IT This slide courtesy Andy Brown, Sandhill East https://www.linkedin.com/in/andybrown63/
  23. 23. 40% Change 60% Run Infrastructure Led Owns Facilities, Data Centers, Hardware, Networks et al Has Refresh Cycles caused by Capital Depreciation Still using Waterfall for App Dev Thinking led by Inf Technologists (hardware, DB, OS et al) Traditional Procurement Less Agile, Change resistant 60% Change 40% Run AppDev starting to lead Owns less Facilities, Data Centers, Hardware, Networks, et al Still Has Refresh Cycles caused by CapitalDepreciation Combination Waterfall & Agilefor App Dev Thinking led by CIO “Move to Cloud” Traditional Procurement weakening More Agile, Less Change resistant 80% Change 20% Run AppDev leads decisioning Doesn’t own hardware Refresh doesn’t exist All Agile for App Dev Thinking led by CIO “Move to Cloud” Cloud Centric “Marketplace” Procurement Embraces Change, Very Agile traditional hybrid digital This slide courtesy Andy Brown, Sandhill East https://www.linkedin.com/in/andybrown63/
  24. 24. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 2045 ?
  25. 25. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 SNMP / traps or Daylight Savings
  26. 26. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 AMRS EMEA APAC EPS every ip interface globally AIOps EdgeOps EdgeOps EdgeOps
  27. 27. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 algorithms we use
  28. 28. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 This slide courtesy our Chief Scientist Dr. Rob Harper -- Do check out his great 3-part blog on Machine Learning in Moogsoft AIOps: https://www.moogsoft.com/author/robharper/
  29. 29. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 By Hui Li on Subconscious Musings April 12, 2017
  30. 30. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 This slide courtesy our Chief Scientist Dr. Rob Harper -- Do check out his great 3-part blog on Machine Learning in Moogsoft AIOps: https://www.moogsoft.com/author/robharper/
  31. 31. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 This slide courtesy our Chief Scientist Dr. Rob Harper -- Do check out his great 3-part blog on Machine Learning in Moogsoft AIOps: https://www.moogsoft.com/author/robharper/
  32. 32. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 regression classification clustering This slide courtesy our Chief Scientist Dr. Rob Harper -- Do check out his great 3-part blog on Machine Learning in Moogsoft AIOps: https://www.moogsoft.com/author/robharper/
  33. 33. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 classification supervised “learn by example” approach. Supervised learning systems need to be given examples of what is “good” and what is “bad” This slide courtesy our Chief Scientist Dr. Rob Harper -- Do check out his great 3-part blog on Machine Learning in Moogsoft AIOps: https://www.moogsoft.com/author/robharper/
  34. 34. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 classification
  35. 35. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 clustering unsupervised Patterns that you didn’t know existed prior. Recommender systems rely heavily on these techniques.
  36. 36. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 supervised machine learning "hot dog?" "not hot dog?"
  37. 37. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017
  38. 38. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 algorithms we use
  39. 39. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 lua code: https://pastebin.com/ZZmSNaHX SethBlingmari/o neural nets https://www.youtube.com/watch?v=qv6UVOQ0F44
  40. 40. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 k-means clustering
  41. 41. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 matrix factorization
  42. 42. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 shannon entropy
  43. 43. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 typical entropy distribution
  44. 44. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 algorithmic workflow
  45. 45. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 millions events de-duplication cluster analysis algorithms situation room teams-centric thousands of alerts algorithmic noise filtering [shannon entropy] tens of alert clusters (situations) "today's warnings are tomorrows outages" "all about the MTTR" algorithmic probable root cause AIOps Algorithmic IT Operations knowledge capture auto-recurrance detect entropy_threshold non-noisy alerts what you're likely doing today L1 "Catch & Dispatch" (automated) ignore situation next steps
  46. 46. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 ...speaking of classification
  47. 47. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 fault vs audit fix → optimize
  48. 48. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 monitoring fail-around analytics analytics fail-around fail-around monitor
  49. 49. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 weld the datacenter doors shut
  50. 50. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 <lofty_tangent> non-technical
  51. 51. THE INDUSTRIAL REVOLUTION 1.0
  52. 52. WE FACE EXISTENTIAL THREATS TO OUR PROGRESS
  53. 53. THE INDUSTRIAL REVOLUTION 2.0 CAN HELP SAVE US
  54. 54. v1.0 v2.0
  55. 55. COMPLEXITY IS THE PRINCIPAL THREAT TO THE REVOLUTION
  56. 56. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 </lofty_tangent> non-technical
  57. 57. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017
  58. 58. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017
  59. 59. NOT FOR GENERAL DISTRIBUTION ONLY INTENDED FOR REGISTERED ATTENDEES OF QCON SF 2017 theory applied sharing | giving back
  60. 60. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ modern-architectures-data-science

×