The increasingly real-time requirements of today’s applications are changing how users expect services and products to be delivered and consumed. Enterprises are responding to this by embracing Reactive system architectures coupled with best-in-class data processing tools to create a new class of programs called Fast Data applications. These applications are sparking the emergence of new business models and new services that take advantage of real-time insights to drive user retention, growth, and profitability. While streaming and Fast Data applications are powerful and create significant competitive advantages, they also impose challenges for monitoring and managing the health of the overall system, which ingest constant streams of data from tens or even hundreds of individual, distributed microservices, data sources, and external endpoints. Businesses must therefore rethink their approach if they wish to take full advantage of the Fast Data revolution. In this webinar by Lightbend’s Paul Jasek, Sr. Director of Global Solution Architects, we review: * Why traditional monitoring solutions, built for legacy monolithic applications, are unable to effectively manage these intricately interconnected, distributed, and clustered systems. * What to look for in an effective monitoring solution for streaming and Fast Data applications. * How Lightbend Monitoring’s deep telemetry, automated discovery, configuration, topology visualization, and data-science-driven anomaly detection capabilities help ensure the health, availability and performance of your applications. * How Lightbend Monitoring helps businesses not just in production but also during development, so they can optimize their applications for performance from Day 1. * A live demo that includes a product walkthrough and sample scenarios so you can understand how your team can use Lightbend Monitoring to quickly troubleshoot problems and issues, and reduce MTTR.

1. WEBINAR
How To Get Monitoring Right For Streaming
And Fast Data Systems Built With Spark,
Mesos, Akka, Cassandra and Kafka
Paul Jasek, Senior Director of Global Solution Architects

2. Agenda
1. Fast Data & Streaming Applications
2. The Challenges of Monitoring Fast Data Applications
3. What To Look For In a Fast Data Application
4. Intelligent End-To-End Monitoring from Lightbend
5. Live Demo
6. Questions

5. reactivemanifesto.org
Reactive Underpinnings: Fast Data and streaming applications
often incorporate, or are based on, Reactive principles

6. • Real-time personalization
• Real-time decision-making
• IoT data processing
• Legacy batch processing
modernization
Growing Number Of Use Cases Across Industries
• Serve existing customers better
and reduce churn
• Attract new ones and drive
growth
• Launch new products more
easily
• Enter new markets more quickly.

7. • Rapidly Evolving Ecosystem
• Understanding the Data Pipeline
• Dynamic Architectures
• Intricately Interconnected
• Distributed And Clustered
The Challenges of Monitoring Fast Data Applications

8. Apache Spark, As An Illustrative Example
The Challenges of Monitoring Fast Data Applications
Concern Questions To Ask
Data Health (for a
given application)
• Throughput: is data processing occurring at the expected rate?
• Latency: is data processing occurring within the expected timeframe?
• Error/quality: are there problems with the data being produced?
• Input data: are input data streams flowing into Spark behaving normally? For instance, what are the
throughput rates for Kafka topics feeding into the Spark job?
Dependency Health • Are the systems feeding input into the storm job (such as Kafka) healthy?
• Are the systems that the application is dependent on, such as Memcache or other API endpoints,
healthy?
Service Health • Is the Spark master operating normally? If not, engineering will be unable to re-balance workloads or
restart jobs.
Application Health • Are the application KPIs within normal operating parameters?
Topology Health • Are there resources assigned to the given Spark topology?
• • Are the Spark tasks and executors well-distributed amongst the Spark cluster?
• • Are the performance counters (emitted, failed, latency, etc.) for the given Spark topology normal?
Node System Health • Are the key system metrics (load, CPU, memory, net-i/o, disk-i/o, disk free) operating normally?

9. Can traditional monitoring tools help?

10. Why traditional monitoring tools won’t help you
• Built to monitor monolithic
applications
• Can only be used to extract
metrics and trace information
based on a synchronous flow
• Not built for asynchronous
flows (i.e. in Fast Data and
streaming applications)
• Cannot easily handle streaming
systems running on distributed
clusters

11. • Deep Telemetry
• Domain Expertise
• Automated Discovery
• Real-Time Topology Visualization
• Intelligent, Rapid Troubleshooting
What users need to effectively monitor Fast Data and
streaming applications

12. • Lightbend Monitoring takes a modern approach to instrumenting and
visualizing distributed streaming systems
• Helps users not just in production but also in development (so they can
build their applications right from Day 1)
• Shows the end-to-end status of applications, data frameworks, and the
associated infrastructure in a single view.
Intelligent, End-To-End Monitoring

13. • Deep Telemetry
• Domain Expertise
• Intelligent Anomaly
Detection
• Fine-Grained
Visibility, with Drill-
Down Capabilities
Data-Science Driven Anomaly Detection

14. • Automated Topology
Discovery
• Automatic Metric
Collection
• Real-Time Topology
Visualization
Automated Discovery, Configuration & Topology
Visualization

15. • Single Pane of Glass
Visibility
• Rapid Root Cause
Analysis
• Reduced Mean-Time-
To-Repair (MTTR)
Intelligent, Rapid Troubleshooting

16. • Dramatically reduce the time and cost to identify and remediate issues across
application life-cycle.
• Create happier, more satisfied customers – and lower churn
• Lower HW/infrastructure costs and reduce concerns about chargebacks & SLA
penalties
• Deliver rapid time to value because everything you need for monitoring is packaged
into an easy-to-use solution
Benefits for your business

17. On to the demo…

18. Upgrade your grey matter!
Get the free O’Reilly book by Dr. Dean Wampler,
VP of Fast Data Engineering at Lightbend
bit.ly/lightbend-fast-data

19. End-To-End Monitoring For Your Fast Data
And Streaming Applications From Lightbend
SET UP A 20-MIN DEMO