Artificial intelligence and machine learning models are growing increasingly available, but many models offer predictions that are difficult to understand, evaluate and ultimately act upon. We present how scalable interactive visualization can be used to amplify people’s ability to understand and interact with large-scale data and complex models. We sample from projects where interactive visualization has provided key leaps of insight, from increased model explorability with models trained on millions of instances (ActiVis deployed with Facebook), increased usability for non-experts about state-of-the-art AI (GAN Lab open-sourced with Google Brain; went viral!), and our latest work Summit, the first interactive system that scalably summarizes and visualizes what features a deep learning model has learned and how those features interact to make predictions. We conclude by highlighting the next visual analytics research frontiers in AI.
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribution
1. S C A L A B L E I N T E R A C T I V E T O O L S
I N T E R P R E T A T I O N & A T T R I B U T I O N
HUMAN-CENTERED AI
Polo Chau
Georgia Tech
Associate Professor, Computational Science & Engineering
ML Area Leader, College of Computing
Associate Director, MS Analytics
for
2. AI HI+
HUMAN
INTELLIGENCE
ARTIFICIAL
INTELLIGENCE
Scalable interactive tools to make sense of
complex large-scale datasets and models
Polo Club of Data Science
3. Human-Centered AI Cyber Security
Social Good & HealthLarge Graph Mining & Visualization
Polo Club of Data Science poloclub.github.io
Adversarial ML
4. Why should we make AI more
human-centered?
Accessible & Interpretable
for people who build and use machine learning
8. 8
Our ShapeShifter Attack: Stop Sign ! Person
Spotlighted in new DARPA GARD program [PKDD’18; with Intel]
Real Stop Sign
Printed Adversarial
Stop Sign
9. 8
Our ShapeShifter Attack: Stop Sign ! Person
Spotlighted in new DARPA GARD program [PKDD’18; with Intel]
10. SHIELD: Fast Practical Defense for
Deep Learning via JPEG Compression
[KDD’18 Audience Appreciation Award runner up; with intel]
9
11. We do NOT know
why AI attacks and defenses work
Actually nobody does
(e.g., which neurons are under attack?)
23. How to scale up to large datasets?
Challenge 2
?
24. How to make tools easy-to-use for
various users?
Challenge 3
EXPERTS NOVICES
[Hohman, Kahng, Pienta, Chau, TVCG, 2018]
PRACTITIONERS
25. Human-Centered AI by Visual Analytics
Our Research
GAN Lab
ActiVis
Visualization for Industry-Scale Models
Interactive Learning of Complex Models
ML Cube - Model comparison
- Activation analysis by subsets
- Experimentation with GANs
31. ActiVis Research Challenges
1. Many model parameters to visualize
2. Many data instances to analyze
3. Intensive computation for deployment
27
Visualization for industry-scale deep models
32. Location
How to visualize many model parameters?
28
Challenge #1
INPUT OUTPUTMODEL
Person
Location 81%
8%
Where is
Mercedes-Benz
Stadium
located?
Number 11%
many layers
33. Location
How to visualize many model parameters?
28
Challenge #1
INPUT OUTPUTMODEL
Person
Location 81%
8%
Where is
Mercedes-Benz
Stadium
located?
Number 11%
particularly
useful
many layers
Observation: No need to show everything
35. How to analyze many data instances?
30
Challenge #2
SUBSET-LEVELINSTANCE-LEVEL
Complementary
Useful for debugging Useful for large datasets
Observation: Two Analytics Patterns
How model behaves at
higher-level categorization
(e.g., by topic)?
How model responds to
individual instances?
39. Human-Centered AI by Visual Analytics
Our Research
GAN Lab
ActiVis
Visualization for Industry-Scale Models
Interactive Learning of Complex Models
ML Cube - Model comparison
- Activation analysis by subsets
- Experimentation with GANs
41. Challenge: Model Selection
Which model to use?
Baseline Model New Model
89.5% 90.1%overall accuracy
Age 20-39
Age 13-19 92.0% 97.0%
87.0% 69.0%
42. Comparison by Subsets with Data Cube
37
country
age
gender
age gender country
Model A
accuracy
Model B
accuracy
* * * 89.5% 90.1%
13-19 * * 92.0% 97.0%
20-39 * * 87.0% 69.0%
* F * 89.6% 89.9%
13-19 F * 91.0% 93.0%
13-19 F USA 91.1% 94.0%
* M USA 75.5% 74.0%
20-39 M Canada 87.2% 73.7%
How to scale to very large number of possible subsets?
46. Research Contributions from ActiVis & MLCube
How can we scale visualization to
industry-scale models and data?
1.Exploration from overview to details
2.Drilling down into specific parts of data
3.Combination of scalable & interactive methods
(Under review — discover interesting subsets automatically:
FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning)
47. Human-Centered AI by Visual Analytics
Our Research
GAN Lab
ActiVis
Visualization for Industry-Scale Models
Interactive Learning of Complex Models
ML Cube - Model comparison
- Activation analysis by subsets
- Experimentation with GANs
48. GAN Lab
Interactive understanding of
complex deep learning models
PAIR | People + AI Research Initiative
[Kahng, et al. IEEE VIS’18]
51. Generative Adversarial Networks (GANs)
46
“the most interesting idea in the last 10 years in ML”
- Yann LeCun
Face images generated by BEGAN [Berthelot et al., 2017]
53. Why GANs are hard?
A GAN uses two competing neural networks
48
Discriminator
spots fake
Police
spots fake bills
Generator
synthesizes outputs
Counterfeiter
makes fake bills
54. Why GANs are hard?
A GAN uses two competing neural networks
48
Discriminator
spots fake
Police
spots fake bills
Generator
synthesizes outputs
Counterfeiter
makes fake bills
55. Why GANs are hard?
A GAN uses two competing neural networks
48
Discriminator
spots fake
Police
spots fake bills
Generator
synthesizes outputs
Counterfeiter
makes fake bills
56. Why GANs are hard?
A GAN uses two competing neural networks
48
Discriminator
spots fake
Police
spots fake bills
Generator
synthesizes outputs
Counterfeiter
makes fake bills
57. Why GANs are hard?
A GAN uses two competing neural networks
48
Discriminator
spots fake
Police
spots fake bills
Generator
synthesizes outputs
Counterfeiter
makes fake bills
58. Why GANs are hard?
A GAN uses two competing neural networks
48
Discriminator
spots fake
Police
spots fake bills
Generator
synthesizes outputs
Counterfeiter
makes fake bills
59. GAN Lab Research Challenges
1. Conceptual understanding of GANs
2. Interactive model training
3. Easily accessible for students
Can we design an interactive tool for GANs?
61. What type of data to visualize?
50
Discriminator
(Police)
Generator
(Counterfeiter)
62. What type of data to visualize?
2D distribution, instead of high-dimensional images
50
Discriminator
(Police)
Generator
(Counterfeiter)
63. What type of data to visualize?
2D distribution, instead of high-dimensional images
51
Discriminator
(Police)
Generator
(Counterfeiter)
64. What type of data to visualize?
2D distribution, instead of high-dimensional images
51
Discriminator
(Police)
Generator
(Counterfeiter)
1. To focus on GAN’s main concepts
2. To easily visualize data distribution
Why 2D data points?
79. How to visualize the discriminator?
2D heatmap, to represent binary classification
56
Data points in this region
are likely real.
Data points are likely fake.
86. GAN Lab broadens education access
62
Conventional Deep Learning Visualization
in JavaScript
in Python with GPU
Model Training
Visualization
$$$
87. Everything done in browser, powered by TensorFlow.js
GAN Lab broadens education access
63
Accelerated by WebGL
in JavaScript
Visualization
also in JavaScript
Model Training
88. GAN Lab is Live!
64
30K visitors, 135 countries 1.9K Likes 800+ Retweets
Try at bit.ly/gan-lab
89. Research Contributions from GAN Lab
Can we design tools for non-experts to
understand complex deep models?
1.Visualization of overall structure & components
2.Interactive experimentation of training process
3.Accessible approach using browsers
90. Visual Analytics in Deep Learning:
An Interrogative Survey for the Next Frontiers
Fred Hohman, Minsuk Kahng, Robert Pienta, Polo Chau
TVCG 2018
!66
91. !67
Some Takeaways
1. Most tools aimed at expert users
2. Instance-based analysis
3. Inherently interdisciplinary
4. Lacks actionability
5. Evaluation is hard
6. State-of-the-art models not robust
bit.ly/va-dl-survey
92. How can we make AI more
human-centered?
Accessible & Interpretable
for people who build and use machine learning
93. How can we make AI more
human-centered?
Accessible & Interpretable
for people who build and use machine learning
Through the design of visualization tools
that are scalable, interactive & usable,
we can help users learn and interpret
large-scale complex ML systems.
94. Thanks!
Polo Chau Georgia Tech
HUMAN-CENTERED AI
S C A L A B L E I N T E R A C T I V E T O O L S
I N T E R P R E T A T I O N & A T T R I B U T I O N
for
