1. Deep Learning Lightning Talk 1
Deep Learning
State-of-the-art, powerful way
to do machine learning
Keynote Template

2. Deep Learning Lightning Talk 2
Our goal: Build awesome data products
User activity, business data,
images, text, audio…
Big Data technologies:
Hadoop, Spark, Hive, Pig, Storm, Impala…
Extract meaning from data and
incorporate it into product:
predicion, analytics, recommendations
!
Technology
!
Data
!
Machine Learning

3. Deep Learning Lightning Talk 3
!
Machine Learning
Machine Learning
There are tons of different machine learning algorithms
for different problems
Unsupervised methods: Customer segmentation
(clustering), dataset visualisation, dimensionality
reduction
⊞
Supervised methods: Predictions, classifications.
Example product: spam filtering
⊞
Recommendations, anomaly detection…
!
⊞

4. Deep Learning Lightning Talk 4
Difficult problems
Image
Recognition
"
Speech
Recognition
♫
Natural Language
Processing
$

5. Deep Learning Lightning Talk 5
Breakthrough: Deep Learning
And now it works ;)

6. Deep Learning Lightning Talk 6
Examples
! Skype Translator ! Google+ Photo Tagging
http://www.youtube.com/watch?v=eu9kMIeS0wQ
+ Voice recognition in Android 4.0+, Apple’s Siri, Baidu’s Image Search, and more…

7. Deep Learning Lightning Talk 7
A bit of theory
Deep Learning is a „bigger and badder” approach to neural networks, which are known since 80’
y= g(x ⊗ W)

8. Deep Learning Lightning Talk 7
A bit of theory
Deep Learning is a „bigger and badder” approach to neural networks, which are known since 80’
y= g(x ⊗ W)

9. Deep Learning Lightning Talk 7
A bit of theory
Deep Learning is a „bigger and badder” approach to neural networks, which are known since 80’
y= g(x ⊗ W)

10. Deep Learning Lightning Talk 7
A bit of theory
Deep Learning is a „bigger and badder” approach to neural networks, which are known since 80’
y= g(x ⊗ W)
Now we have much more computing
power to train large (and deep) networks
⊞
Now we know better regularization and
optimization methods
⊞
Now we have much more labeled data⊞
Now we can also train models with
unlabeled data
⊞

11. Deep Learning Lightning Talk 8
Why it works?
Let’s consider the problem of face recognition
That’s how we see it
0.2 0.0 0.1 1.0 1.0 0.1 0.4 0.8 1.0 ... 0.1 That’s how „machine” sees it

12. Deep Learning Lightning Talk 9
Why it works?
It’s much easier to infer that something is a face based on that it has two eyes and nose, than it has some
black pixels in lower left corner, and white area somewhere in the middle

13. Deep Learning Lightning Talk 10
A bit of practice
GPU Cluster

14. Deep Learning Lightning Talk 11
A bit of practice
GPU
Numerical operations are very efficient, up
to 100x faster than CPU
⊞
Single machine, no communication overhead⊞
Significant memory contraints, we can’t train
larger models
⊟

15. Deep Learning Lightning Talk 12
A bit of practice
TASK
one learning task, many workers
different parameters for each worker
PICK BEST MODEL Netflix style!
GPU
WORKER 1 WORKER 2 WORKER 3 WORKER 4

16. Deep Learning Lightning Talk 13
A bit of practice
Cluster

17. Deep Learning Lightning Talk 13
A bit of practice
Cluster
WORKER2
WORKER1
WORKER3
WORKER4
+ ASYNCHRONOUS PARAMETERS SERVER
Google style!

18. Deep Learning Lightning Talk 14
A bit of practice
Cluster
We can train much larger and more
powerful models
⊞
Scalable⊞
Poor resource utlization, even if we restrict
connectivity
⊟
Complicated⊟

19. Deep Learning Lightning Talk 15
Hype
NETFLIX MOVES INTO DEEP LEARNING
RESEARCH TO IMPROVE PERSONALIZATION
10 BREAKTHROUGH TECHNOLOGIES 2013
GIGAOM GUIDE TO DEEP LEARNING:
WHO’S DOING IT AND WHY IT MATTERS
NYU „DEEP LEARNING” PROFESSOR LECUN WILL HEAD
FACEBOOK’S NEW ARTIFICIAL INTELLIGENCE LAB
Geoffrey Hinton
Leading researcher in DL, his startup
was acquired by Google
Lookflow
Deep Learning image startup,
acquired by Yahoo
DeepMind
Deep Learning startup, acquired by
Google for 400 mln USD
Yan LeCun
Leading researcher in DL, hired by
Facebook to lead new AI lab.

20. Deep Learning Lightning Talk 16
Geoffrey Hinton
Leading researcher in DL, his startup
was acquired by Google
Lookflow
Deep Learning image startup,
acquired by Yahoo
DeepMind
Deep Learning startup, acquired by
Google for 400 mln USD
Yan LeCun
Leading researcher in DL, hired by
Facebook to lead new AI lab.
Hype

21. Deep Learning Lightning Talk 17
It’s not a silver bullet
It’s difficult.
Sometimes it’s better to use simpler method.
"
#
Nevertheless, it’s a very powerful technique, has attention of biggest IT
companies and brings us closer to real artificial intelligence
It requires substantial computing power and memory.
Sometimes it’s not feasible to use deep learning models, especially if we have to train them regularly
!
It’s kind of `black-box`
Sometimes we can’t draw conclusions from learned features

22. !
:)
THANKS
$ mateusz.buskiewicz@getbase.com
RESOURCES
MOOC: Neural Networks for Machine Learning
& https://www.coursera.org/course/neuralnets
DL Tutorials + sample code
& http://deeplearning.net/
Google+ Deep Learning Community
& https://plus.google.com/u/0/communities/112866381580457264725
Deep Learning Book by Yoshua Bengio (draft)
& http://www.iro.umontreal.ca/~bengioy/dlbook/
Deep Learning Libraries & Software
& http://deeplearning.net/software_links/