An AI-Powered Chatbot to Simplify Apache Spark Performance Management

WIFI SSID:SparkAISummit | Password: UnifiedAnalytics

Shivnath Babu
Cofounder/CTO, Unravel
Adjunct Professor, Duke University
An AI-powered Chatbot to
Simplify Spark Performance
Management
#UnifiedAnalytics #SparkAISummit

Meet the speaker
• Cofounder/CTO at Unravel
• Adjunct Professor of Computer Science at
Duke University
• Focusing on ease-of-use and manageability of
data-intensive systems
• Recipient of US National Science Foundation
CAREER Award, three IBM Faculty Awards,
HP Labs Innovation Research Award
3#UnifiedAnalytics #SparkAISummit

What is a Chatbot?

A program which conducts a
conversation via text or voice

Chatbots are making a
real difference

Source: https://chatbottle.co/awards/2018

TOBi
generates
2x more
ecommerce
conversions
in ½ the time
for Vodafone

Zara
provides fast
services to
20% of Zurich
Insurance
customers

Woebot, the
therapist chatbot,
talks to more
people in a day
than a human
therapist does in
a lifetime

Chatbots ó Spark Performance
What is the connection?

The happy Spark user
• Spark is fast
• Spark has easy-to-use and
comprehensive APIs
• Wow, I can do SQL, Streaming,
AI/ML, and Graphs in one system!
• Spark has a rich ecosystem

“I have no clue
which cloud
instance type to
pick for my
workload”
“My cloud
costs are
getting out of
control. Help!”
“I have no
idea why
my app is
slow”
“My app
failed and I
don’t know
why!”
The frustrated Spark user

• Many levels of correlated stack traces
• Identifying the root cause is hard and time consuming
14
Typical app failure in Spark

“My app
failed and I
don’t know
why!”
Spark User Spark Chatbot
“I know that sucks! Let me take
a look here …”
“I see the problem. Executors
are running out of memory”
“Setting
spark.executor.memory to 12g
fixes the problem. I have
verified it. See this run here”
“Wow.
Thanks.
You are
awesome!”

I will show you a Chatbot that
• Makes you more productive
• Saves you time and money
• Becomes your AI-driven Spark Expert in a Bot!

My app is too slow…
17
DATA ENGINEER

I need to make it faster…
18
DATA ENGINEER

Current approach
19
1. Review Spark/YARN UI to find the app
2. Review metrics in the UI
3. Review jobs and stages associated with the app
4. Identify all containers associated with the app
6. Identify “problematic” jobs, stages, or containers
7. Guess which parameters to tune for performance
5. Review and debug container logs
9. Rinse & repeat
8. Do trial-and-error by changing a parameter setting

There has to be a better way

What is going on here?

Messaging
Platform
Bot’s NLP
Layer
Bot’s Backend
Layer
Chatbot Architecture from 30000 ft

Monitoring
Data
Historic Data
&
Probe Data
Recommendation
Algorithm
Cluster Services On-premises and Cloud
App,Goal
Orchestrator
Algorithm running in bot’s backend
Xnext
Probe Algorithm

spark.driver.cores 2
spark.executor.cores
…
10
spark.sql.shuffle.partitions 300
spark.sql.autoBroadcastJoinThres
hold
20MB
…
SKEW('orders', 'o_custId') true
spark.catalog.cacheTable(“orders") true
…
We represent this setting as vector X X
PERFORMANCE
24
Spark tuning parameters

• Find the setting of X that
best meets the goal
• Challenge: Response
surface y = ƒ(X) is
unknown
X
PERFORMANCE
Given: App + Goal

Model the response surface as
The Gaussian Process model captures the
uncertainty in our current knowledge of the
response surface
)()()(ˆ XZXfXy t
+= b
!!
b
!!
)(Xf t
)(XZ
X
PERFORMANCE
Challenge: Response surface
y = ƒ(X) is unknown
Here:
is a regression model
is the residual captured as a
Gaussian Process
#AI7SAIS 26

ò
=
-¥=
-=
)(
)(ˆ
*
*
)())(()(
Xyp
p
Xy dpppdfpXyXEIP
We can now estimate the expected improvement EIP(X) from
doing a probe at any setting X
Gaussian Process model helps estimate EIP(X)
Improvement at any
setting X over the best
performance seen so far
Probability density
function (uncertainty
estimate)
X
Opportunity
27
PERFORMANCE

Get initial set of
monitoring data from
history or via
probes: <X1,y1>,
<X2,y2>, …, <Xn,yn>
1
Select next probe
Xnext based on all
history and probe data
available so far to
calculate the setting
with maximum expected
improvement EIP(X)
2
Bootstrap
Probe Algorithm
Until the
stopping
condition
is
reached
#AI7SAIS 28
PERFORMANCE
X

4 6 8 10 12
02468
x1
y
4 6 8 10 12
02468
x1
y
4 6 8 10 12
02468
x1
y
4 6 8 10 12
02468
x1
y
X
Performance
U
EIP(X)
U
Xnext: Do next
probe here
This approach
balances
Exploration Vs.
Exploitation
U
Exploration
U
Exploitation

Credit: https://discovery.rsm.nl/articles/detail/130-how-to-balance-exploration-and-exploitation-in-multinational-enterprises
Data Starved
& High Uncertainty
Data Rich
& Low
Uncertainty
30
App,Goal
Xnext
Probe Algorithm

Messaging
Platform
Bot’s NLP
Layer
Bot’s Backend
Layer
Chatbot architecture

• Many levels of correlated stack traces
• Identifying the root cause is hard and time consuming
32
Typical app failure in Spark

Let us see a better way

What is going on here?

Predictive
Model
Root cause
of the failure
App failure
App’s
Container
Logs
Error
Template
Extraction
Feature
vector

Predictive
Model
Root cause
of the failure
App failure
App’s
Container
Logs
Error
Template
Extraction
Error
Template
Extraction
Feature
vectors
Model
Learning
Container
Logs
Root cause labels
Logs from
millions of
app failures
Label
Generation
Feature
vector

Two ways to get root-cause labels
• Manual diagnosis by a domain expert
• Automatic injection of the root cause

Unravel’s large-scale lab framework for
automatic root cause analysis
Spark and multi-tenant Workloads:
- Variety of workloads: Batch, ML, SQL, Streaming, etc.
Failures:
- Large set of root causes learned from customers &
partners. Constantly updated
- Continuously inject these root causes to train & test
models for root-cause prediction
Environment:
- Lab created on demand on cloud or on-premises
- Workloads are run and failures are injected

Injecting “labeled” failures
Application
Execution
Application
Monitor
FAILED
Injected
Failure
Label
Labeled
Failures
• Invalid input
• Invalid memory configuration
• OOME: Java heap space
• OOME: GC overhead limit
• Container killed by YARN
• Runtime incompatibility
Injected failure examples:
• No space left on device
• Transformations inside other
transformations
• Runtime error
• Arithmetic error
• Invalid configuration settings
Input Feature
Extraction
39
Training
data

We created a Failure Taxonomy for Labels
Configuration
Errors
Data
Errors
Resource
Errors
Deployment
Errors
Root Node
Category of failure
Input Path
Not
Available
Number
Format
Exception
SparkSQL
JsonProcessing
Exception
…
Root cause labels

Extracting input features from logs
java.lang.OutOfMemoryError: Java heap space
at
scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:114)
at
scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:112)
at …
• Extracting stack traces and error messages
• Tokenize by class names and words
Tokens example:
java.lang.OutOfmemoryError Java heap space at
scala.reflect.ManifestFactory$$anon$9.newArray(Manife
st.scala:114)

Input feature extraction
• Bag of Words with TF-IDF
– Computes a vocabulary of words
– Uses TF-IDF to reflect importance of words in a document
• Doc2Vec
– Maps words, paragraphs, or documents to multi-dimensional vectors
– Evaluates the placement of words wrt neighboring words
– Uses a 3-layer neural network
42#Exp8SAIS

Predictive
Model
Root cause
of the failure
App failure
App’s
Container
Logs
Error
Template
Extraction
Error
Template
Extraction
Feature
vectors
Model
Learning
Container
Logs
Root cause labels
Logs from
millions of
app failures
Label
Generation
Feature
vector

Learning the predictive model
• Shallow Learning
– Logistic Regression
– Random forests
• Deep Learning
– Neural networks
44
• Training and testing with injected failures
• Test to train data set ratio 75% to 25%
• Models: logistic regression, random forests
80
85
90
95
100
TF-IDF Doc2Vec
AccuracyScore
[%]
Logistic Regression Random Forests

Messaging
Platform
Bot’s NLP
Layer
Bot’s Backend
Layer
The NLP element in the Chatbot
Algorithm
Compute
Storage

Extract
the intent
Intent =
AppAutoTune
Entities: {
AppName =
‘CEO report’,
TuningGoal =
Speedup }
Invoke app
autotuning
algorithm
How can I make
CEO report query
faster Tune an app
Fetch a metric
Generate a report
Set an alert
Diagnose a failure
…
Extract entities
for the intent
Take
action
The NLP element in the Chatbot

Many use cases can be addressed
• Who are the top resource-wasting users on the cluster?
• Which app is causing contention on the cluster?
• Why is my app stuck?
• Alert me if my query fails
• Which part of my query failed?
• Kill the sales report BI app if it uses more than $25
• And many more …

In summary
• AI-driven Spark Expert in a Bot!
– Makes you more productive
– Saves you time and money
Sign up for a free trial, we value your feedback!
http://unraveldata.com/free-trial
And yes, we are hiring @ Unravel
shivnath@unraveldata.com

DON’T FORGET TO RATE
AND REVIEW THE SESSIONS
SEARCH SPARK + AI SUMMIT

An AI-Powered Chatbot to Simplify Apache Spark Performance Management

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à An AI-Powered Chatbot to Simplify Apache Spark Performance Management

Similaire à An AI-Powered Chatbot to Simplify Apache Spark Performance Management (20)

Plus de Databricks

Plus de Databricks (20)

Dernier

Dernier (20)

An AI-Powered Chatbot to Simplify Apache Spark Performance Management