With the growth of machine and agronomic telematics applications, the teams at John Deere have been on a journey to scale up their Elasticsearch use cases. Learn how they handled the transition and see how they plan to expand to log management, metrics analytics, and other new and exciting initiatives with ECE.

1. Elastic @ John Deere
Tim Arp
Senior Operations Engineer
Intelligent Solutions Group
John Deere

2. John Deere | Elastic @ John Deere | Elastic{on} 20182

3. John Deere | Elastic @ John Deere | Elastic{on} 20183
Gen4 CommandCenter™ Display
Control machine functions and precision ag applications

4. John Deere | Elastic @ John Deere | Elastic{on} 20184
Monitoring with Connect Mobile
Bringing Mobile into the Cab

5. John Deere | Elastic @ John Deere | Elastic{on} 20185

6. John Deere | Elastic @ John Deere | Elastic{on} 20186
Rewind 5 years
• John Deere had 1 mobile app, JDLink
• 6 core components made up JDLink
• 25 servers involved
Monitoring
• Up/down monitoring of URLs
• APM application installed in application’s JVM
• Server level monitoring
• Application logs on the servers exposed via samba shares

7. Then we started to grow…

8. John Deere | Elastic @ John Deere | Elastic{on} 20188
Along comes the Elk stack…
Scaling applications horizontally
Built our first cluster with Elasticsearch 1.4
Log Consolidation
Logstash Fowarder to Logstash
• Web Server logs
• Application logs
• Syslogs
• Load Balancer logs
Log Analytics

9. John Deere | Elastic @ John Deere | Elastic{on} 20189

10. John Deere | Elastic @ John Deere | Elastic{on} 201810
Current Elastic environment
Logcentral @ Deere
25 Node Elastic stack 6.3
• Index rate 20,000 events/s,
• 18 Billion documents
• 11 TB storage
• 14 Applications Logging

11. John Deere | Elastic @ John Deere | Elastic{on} 201811
Current Elastic environment
Logcentral @ Cloud
8 Elasticsearch clusters, 5.1 and 6.2
• Largest cluster has 51 Nodes
• 50 Billion documents
• 200 TB storage
• 100+ Applications logging

12. John Deere | Elastic @ John Deere | Elastic{on} 201812
Elastic Architecture

13. John Deere | Elastic @ John Deere | Elastic{on} 201813
Elasticsearch
for metrics

14. John Deere | Elastic @ John Deere | Elastic{on} 201814
ELK for Metrics?
.

15. John Deere | Elastic @ John Deere | Elastic{on} 201815
Work Tracker @ https://github.com/JohnDeere/work-tracker
Advanced Logging
• Contextualized logs
• URI Template variables
• Structured metadata
Flood Sensor
• Denial of Service (DoS) protection
Zombie Detection
• Long running requests
Contextualized background tasks
Root Cause Detection

16. John Deere | Elastic @ John Deere | Elastic{on} 201816
Advanced Logging
Contextualized Logging
• Request ID
• Elapsed Time (in milliseconds)
• Thread Name
• Zombie status
• Start and End info
• Session ID
• User
• Path

17. John Deere | Elastic @ John Deere | Elastic{on} 201817
Advanced Logging
URI Template Variables
• By default, automatically added to logs
• PathMetadataCleanser
Customization
• Standardize keys
• org_id can become organization_id
• Restrict keys
• Keys that are restricted can be banned
• Transform keys
• Free form conversion

18. John Deere | Elastic @ John Deere | Elastic{on} 201818
Advanced Logging
Structured metadata
• Some work tracker keys are
structured, i.e. elapsed_ms
• Useful for making Kibana
dashboards
• Get statistics from those
structured metadata as well

19. John Deere | Elastic @ John Deere | Elastic{on} 201819

20. John Deere | Elastic @ John Deere | Elastic{on} 201820

21. John Deere | Elastic @ John Deere | Elastic{on} 201821

22. John Deere | Elastic @ John Deere | Elastic{on} 201822
Work Tracker
OSS @ https://github.com/JohnDeere/work-tracker

23. John Deere | Elastic @ John Deere | Elastic{on} 201823
ISG Application Environment
Key Features
• Support from Elastic
• Authentication and Authorization integrated with our SSO provider
• Monitoring and Alerting (Watcher)
• Cross cluster searching
• Seamless upgrades to the latest Elastic stack
• X-Pack – Everything else
Data Management
Remote Management
Guidance
Variable Rate Application
Field and Water Management

24. John Deere | Elastic @ John Deere | Elastic{on} 201824
Elastic Cloud Enterprise
Key Features
• Support from Elastic
• Authentication and Authorization integrated with our SSO provider
• Monitoring and Alerting (Watcher)
• Cross cluster searching
• Seamless upgrades to the latest Elastic stack
• Machine Learning

25. John Deere | Elastic @ John Deere | Elastic{on} 201825
Elastic Cloud Enterprise
Project
• Deploying in the cloud
• 3 groupings of Servers
• Coordinators/Directors
• Proxies
• Allocators
• Role based scaling
• Load Balancers used throughout
• Instrumented with Terraform

26. John Deere | Elastic @ John Deere | Elastic{on} 201826
Logcentral Goals and Challenges
How do you manage the Chaos?
Ingestion
• How do control/monitor/manage each input stream?
• Provide metrics for input stream?
• How should input streams deal with inaccessible Elastic cluster?
• Buffering input streams?
Data Management
• Lifetime of data, hot/warm/cold configuration in Elasticsearch?
• Rollups and Data aggregations