Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
1
Logging and machine data at Scale
James Governor @monkchips co-founder.
2
“The lumber industry sells what used to be waste —
sawdust, chips, and shredded wood — for a pretty profit.
Today you’ll...
3
4
5
6
7
8
9
Zero Users Care What The System Health
is
All Users Care What Their Experience Is
Nines Don’t Matter if Users Aren’t Hap...
10
Observability bridges tracing, logging, and monitoring
Logs are no longer about history
Real time, streams, and adhoc q...
11
https://simplicable.com/new/machine-data
12
Use Cases
Compliance
DevOps, IT Service management/root cause analysis
ecommerce, digital marketing optimisation
Fraud ...
13
Always Compliance
Anti-money laundering
PCI
GDPR is coming – fines potentially 4% of global turnover
14
Splunk
Dubai Airport instruments literally everything
– Even hand washing in the bathroom from electronic taps and soap...
15
ELK as a disruptive entrant
Started as text search, now as a default for many orgs
Sprint – 3bn events per day
– Digita...
16
17
18
Edge Filtering
IoT
Sampling and polling
19
Tracing at Facebook
2014 Mystery Machine took 2 hours to compute a model from 1.3M traces
2017 Canopy generates and pro...
20
Roll your own kind of sucks
21
Using machine data for business process optimization
Data isn’t on the balance sheet
But data as product is
Data optimi...
22
Cloud Native Tools
Grafana – visualization graph and dashboards,
Prometheus – monitoring and alerting, time series
Flue...
Prochain SlideShare
Chargement dans…5
×

Logging and machine data at Scale. re:Invent 2017

530 vues

Publié le

Talk about the tapping into the unrealised value of log and machine data in order to create new products and services and improved customer experiences. The value of Splunk and rise of ELK for logging and monitoring. Scale at Facebook. Observability as the new hotness, with thanks to Charity Majors. Use cases and customer examples (bosch etc)

Publié dans : Technologie
  • Soyez le premier à commenter

Logging and machine data at Scale. re:Invent 2017

  1. 1. 1 Logging and machine data at Scale James Governor @monkchips co-founder.
  2. 2. 2 “The lumber industry sells what used to be waste — sawdust, chips, and shredded wood — for a pretty profit. Today you’ll find these by-products in synthetic fireplace logs, concrete, ice strengtheners, mulch, particle board, fuel, livestock and pet bedding, winter road traction, weed killing and more” – Jason Fried, CEO Basecamp
  3. 3. 3
  4. 4. 4
  5. 5. 5
  6. 6. 6
  7. 7. 7
  8. 8. 8
  9. 9. 9 Zero Users Care What The System Health is All Users Care What Their Experience Is Nines Don’t Matter if Users Aren’t Happy Charity Majors, Honeycomb.io
  10. 10. 10 Observability bridges tracing, logging, and monitoring Logs are no longer about history Real time, streams, and adhoc queries
  11. 11. 11 https://simplicable.com/new/machine-data
  12. 12. 12 Use Cases Compliance DevOps, IT Service management/root cause analysis ecommerce, digital marketing optimisation Fraud detection IoT and integration with data from physical assets Healthcare Real time security
  13. 13. 13 Always Compliance Anti-money laundering PCI GDPR is coming – fines potentially 4% of global turnover
  14. 14. 14 Splunk Dubai Airport instruments literally everything – Even hand washing in the bathroom from electronic taps and soap Rackspace ingests 3TB of data every day – Security monitoring and management Tactical Assault Light Operator Suit (TALOS) – Monitoring including vital signs
  15. 15. 15 ELK as a disruptive entrant Started as text search, now as a default for many orgs Sprint – 3bn events per day – Digital Transformation 4.0 Project – real-time data from Retail Management and Store Ops – 200 dashboards represent events from logs, databases, emails, syslogs, test messages, and internal and vendor application APIs. Uber, Facebook, Netflix
  16. 16. 16
  17. 17. 17
  18. 18. 18 Edge Filtering IoT Sampling and polling
  19. 19. 19 Tracing at Facebook 2014 Mystery Machine took 2 hours to compute a model from 1.3M traces 2017 Canopy generates and processes 1.3 billion traces per day (spanning end-user devices, web servers, and backend services, backs 129 performance datasets from high-level end-to-end metrics to specific custom use cases) Source https://blog.acolyer.org/2017/11/22/canopy-an-end-to-end-performance-tracing-and-analysis-system/
  20. 20. 20 Roll your own kind of sucks
  21. 21. 21 Using machine data for business process optimization Data isn’t on the balance sheet But data as product is Data optimized services and customer experiences Data culture Observability is the new hotness
  22. 22. 22 Cloud Native Tools Grafana – visualization graph and dashboards, Prometheus – monitoring and alerting, time series Fluentd – data collector and log aggregation Graphite – time series and graphs

×