Introduction to LSTM, the deep learning algorithm behind Google Voice Transcriptions, explained without any mathematics equation. Mostly for non-technical audience without any data-science background.
9. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Expected
Learning by Backpropagation
Ground truth
10. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Prediction
Learning by Backpropagation
First iteration truth
11. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Prediction
Error
Learning by Backpropagation
12. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Prediction
Error
Update weights in every layer w/ an optimization algorithm
Learning by Backpropagation
13. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Prediction
Error
Update weights in every layer w/ an optimization algorithm
Retry prediction with updated weights
LeaLearning by Backpropagation
ning by Backpropagation
21. Human Talks Montpellier @lievAnastazia
Je vais passer les vacances en Occitanie.
Je veux bien apprendre à parler Occitan
22. Human Talks Montpellier @lievAnastazia
Use cases
- Time series
- Video processing
- Natural language processing
- Audio processing
- Sequence data in bio-informatics
- Automatic cars
- ...
23. Human Talks Montpellier @lievAnastazia
Variations of RNN & LSTM
- Bi-directional RNNs
- Hierarchical RNNS
- Depth Gated RNN
- GRU
- LSTM with peephole connections
- ...
24. Human Talks Montpellier @lievAnastazia
To go further ( or deeper :P )
- Andrej Karpathy blog
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
- Colah blog
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
- Books and courses on stanford.edu
- Articles of Jürgen Schmidhuber
- Recent Advances in Recurrent Neural Networks
https://arxiv.org/abs/1801.01078
25. Google voice transcriptions demystified:
Introduction to recurrent neural networks & LSTM
Lieva Anastasia @lievAnastazia
Human Talks Montpellier