Learn how the Elastic Stack helped Procter & Gamble achieve a greater understanding of their data, as well as introducing observability to their toolkit to help them be more proactive and provide better services.
2. 2
Agenda
What are we talking about?
Who we are1
Challenges3
The road so far4
What’s next?5
Problem statement2
3. 3
Who we are
• Founded in 1837 (180+ years)
• Superior quality products
• More than 180 countries
• http://www.pg.com/
Making every day more than ordinary
4. 4
Who we are (cont’d)
• Andrea Fuggetta
• Cincinnati (OH)
• Sr. Software Engineer
• Network Automation
Making every day more than ordinary
5. 5
Problem statement
• Insourcing
• Where is our data?
• What data do we have?
• What are my processes and use cases?
• How do we aggregate the data?
• How do we leverage the data?
• Where is our data!?
Where did we start?
6. 6
I have not failed. I've just found
10,000 ways that won't work
Thomas Edison
7. 7
Problem statement
• So many choices
• Trial and error
• What is Elasticsearch?
• Demo sessions
• First visualizations
• Interest from customers
Where to start?
9. 9
Challenges
• Find the processes that govern your data
• Find the people responsible for them
• Find the technology to support the business and use cases
Where to start?
10. 10
Challenges
• How much data do we expect?
‒ Do you know all our data?
‒ Throughput
• Infrastructure
‒ Cloud or DC
‒ Compute power
‒ Memory
‒ Storage
• What product?
‒ Single server
‒ Local cluster
‒ Elastic Cloud
‒ ECE
Scaling – Hosting – Product
11. 11
Challenges
• Define the data sources
‒ Network metrics
• Data flow
‒ 500MB/day
• Infrastructure
‒ Cloud
• What product?
‒ 3 node cluster
‒ Installed manually on VMs
Solutions?
13. 13
The road so far
• Define the data sources
‒ Network devices’ syslogs and metrics
• Data flow
‒ 500MB/day ~6TB/day
• Infrastructure
‒ Cloud
• What product?
‒ Elastic Cloud Enterprise (ECE)
• How?
Current state
17. 17
Results
• Prevented downtime and potential issues
• Increased knowledge of our data
• One destination for logs and metrics
• Easier troubleshooting and forensics
• Increased scalability and mobility
• All in less than 1 year
Long road ahead
18. 18
Results
• Move infrastructure from Azure to AWS
‒ Load balancers
‒ Kafka-like queue (Kinesis)
‒ Virtual Machines
‒ Storage
‒ Monitoring
‒ Installing and configuring software (ECE, Logstash)
• Half day
Examples
19. 19
What’s next?
• More customers – more data
‒ Information Security (SIEM)
‒ Data Science (Search, aggregation, analysis)
• ML
‒ Anomalies detection
• Cloud data and logs
‒ Function beats
‒ Custom ingestion pipelines
• Alerts and actions
‒ Anomalies trigger alerts and scripts to self-heal
• Canvas
‒ Executive views
‒ Hallway monitors
Near future