Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Deep L...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Table ...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
exp(I(hk 
i0)) 
p(pk 
= 0|v) = 
1 
1 + 
P 
j02
exp(I(hk 
i0)) 
Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Conten...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Defini...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Featur...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Implem...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Implem...
Deep Learning 
Restricted Boltzmann Machine 
Deep Belief Network 
Convolutional RBM 
Convolutional DBN 
Conclusion 
Implem...
Prochain SlideShare
Chargement dans…5
×

sur

Deep learning presentation Slide 1 Deep learning presentation Slide 2 Deep learning presentation Slide 3 Deep learning presentation Slide 4 Deep learning presentation Slide 5 Deep learning presentation Slide 6 Deep learning presentation Slide 7 Deep learning presentation Slide 8 Deep learning presentation Slide 9 Deep learning presentation Slide 10 Deep learning presentation Slide 11 Deep learning presentation Slide 12 Deep learning presentation Slide 13 Deep learning presentation Slide 14 Deep learning presentation Slide 15 Deep learning presentation Slide 16 Deep learning presentation Slide 17 Deep learning presentation Slide 18 Deep learning presentation Slide 19 Deep learning presentation Slide 20 Deep learning presentation Slide 21 Deep learning presentation Slide 22 Deep learning presentation Slide 23 Deep learning presentation Slide 24 Deep learning presentation Slide 25 Deep learning presentation Slide 26 Deep learning presentation Slide 27 Deep learning presentation Slide 28 Deep learning presentation Slide 29 Deep learning presentation Slide 30 Deep learning presentation Slide 31 Deep learning presentation Slide 32 Deep learning presentation Slide 33 Deep learning presentation Slide 34 Deep learning presentation Slide 35 Deep learning presentation Slide 36 Deep learning presentation Slide 37 Deep learning presentation Slide 38 Deep learning presentation Slide 39 Deep learning presentation Slide 40 Deep learning presentation Slide 41 Deep learning presentation Slide 42 Deep learning presentation Slide 43
Prochain SlideShare
Deep Learning - The Past, Present and Future of Artificial Intelligence
Suivant
Télécharger pour lire hors ligne et voir en mode plein écran

11 j’aime

Partager

Télécharger pour lire hors ligne

Deep learning presentation

Télécharger pour lire hors ligne

Short introduction to deep learning and to the DLL Library (C++, https://github.com/wichtounet/dll). Nothing fancy.

Deep learning presentation

  1. 1. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Deep Learning Baptiste Wicht baptiste.wicht@gmail.com September 12, 2014 Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  2. 2. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Table of Contents 1 Deep Learning 2 Restricted Boltzmann Machine 3 Deep Belief Network 4 Convolutional RBM 5 Convolutional DBN 6 Conclusion Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  3. 3. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition History Usages Difficulties Contents 1 Deep Learning Definition History Usages Difficulties 2 Restricted Boltzmann Machine 3 Deep Belief Network 4 Convolutional RBM 5 Convolutional DBN 6 Conclusion Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  4. 4. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition History Usages Difficulties Definition Deep Learning (Wikipedia) Deep learning is a set of algorithms in machine learning that attempt to model high-level abstractions in data by using model architectures composed of multiple non-linear transformations Deep Learning (deeplearning.net) Deep Learning is a new area of Machine Learning research, which has been introduced with the objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence. Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  5. 5. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition History Usages Difficulties Definition (cont.d) Goal: Imitate the nature Set of algorithms Generally structures with multiple layers Often unsupervised feature learning Time-consuming training Sometimes large amount of data Generally complex data New name for an old thing hot topic Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  6. 6. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition History Usages Difficulties History 1960: Neural networks 1985: Multilayer Perceptrons 1986: Restricted Boltzmann Machine 1995: Support Vector Machine 2006: Hinton presents the Deep Belief Network (DBN) New interests in deep learning and RBM State of the art MNIST 2009: Deep Recurrent Neural Network 2010: Convolutional DBN 2011: Max-Pooling CDBN Many competitions won and state of the art results Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  7. 7. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition History Usages Difficulties Names Geoffrey Hinton Andrew Y. Ng Yoshua Bengio Honglak Lee Yann LeCun ... Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  8. 8. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition History Usages Difficulties Algorithms Deep Neural Networks Deep Belief Networks Convolutional Deep Belief Networks Deep SVM Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  9. 9. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition History Usages Difficulties Usages Text recognition Facial Expression Recognition Object Recognition Audio classification Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  10. 10. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition History Usages Difficulties Difficulties Large number of free variables Few insights on how to set them Complex to implement Large variations between papers Lot of refinements were proposed Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  11. 11. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Contents 1 Deep Learning 2 Restricted Boltzmann Machine Definition Training Units Variants 3 Deep Belief Network 4 Convolutional RBM 5 Convolutional DBN 6 Conclusion Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  12. 12. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Definition Restricted Boltzmann Machine Function: Learn a probability distribution over the input Generative stochastic neural network Visible and hidden neurons Neurons form a bipartite graph V visible units and visible biases H hidden units and hidden biases VxH weights Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  13. 13. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Definition (Cont.d) Binary units (Bernoulli RBM) p(hj = 1|v) = (cj + mX i viwi,j ) p(vi = 1|h) = (bi + nX j hjwi,j ) Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  14. 14. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Example Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  15. 15. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Example Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  16. 16. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Example Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  17. 17. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Example Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  18. 18. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Example Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  19. 19. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Usages Unsupervised feature learning Classification with other techniques (linear classifier, SVM, ...) Limited to one layer of abstraction Stacking for higher-level models and classification Deep Belief Network Deep Boltzmann Machines Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  20. 20. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Training Objective: Maximizing the log-likelihood Intractable Other methods have been developed: Markov Chain Monte Carlo (MCMC) (Too slow) Contrastive Divergence (CD) (Hinton) Persistent CD Mean-Field CD (mf-CD) Parallel Tempering Annealed Importance Sampling Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  21. 21. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Contrastive Divergence For each data point 1 Compute gradients g between t = k and t = k − 1 2 Add g to the weights and the biases Repeat for several epochs Experiments have shown that CD1 (k = 1) works well Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  22. 22. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Contrastive Divergence When to stop training ? 1 Proxies to log-likelihood: Reconstruction error Pseudo-likelihood (PCD) 2 Visual inspection of the filters Training is relatively fast Can be trained on GPU Hard to compare two RBMs Hard to test an implementation correctly Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  23. 23. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Contrastive Divergence Options Mini-batch training Momentum Weight decay Sparsity Target ... Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  24. 24. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Units RBM Was initially developed with binary units Different types of units can be used: Gaussian visible units for real-value inputs Softmax hidden unit for classification (last layer) Rectified Linear Unit (ReLU) units for hidden/visible Can be capped Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  25. 25. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Units Variants Variants Convolutional RBM (see later) mean-covariance RBM (mcRBM) Sparse RBM (SRBM) Third-Order RBM Spike And Slab RBM Nonnegative RBM ... Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  26. 26. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Contents 1 Deep Learning 2 Restricted Boltzmann Machine 3 Deep Belief Network Definition Training 4 Convolutional RBM 5 Convolutional DBN 6 Conclusion Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  27. 27. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Definition Deep Belief Network Generative graphical model Type of Deep Neural Network Multiple layer of hidden units Stack of RBMs Can be implemented with other autoencoders Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  28. 28. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Definition (Cont.d) Each RBM takes input from previous layer output Each layer forms a higher-level representation of the data Number of hidden units in each layer can be tuned Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  29. 29. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Training 1 Train each layer, from bottom to top, with Contrastive Divergence (Unsupervised) 2 Then treat the DBN as a MLP 3 If necessary, fine-tune the last layer for classification (Supervised) Back propagation nonlinear Conjugate Gradient method Limited Memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) Hessian-Free CG (Martens) Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  30. 30. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Probabilistic Max Pooling Contents 1 Deep Learning 2 Restricted Boltzmann Machine 3 Deep Belief Network 4 Convolutional RBM Definition Training Probabilistic Max Pooling 5 Convolutional DBN 6 Conclusion Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  31. 31. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Probabilistic Max Pooling Definition Convolutional RBM Motivation: Translation-invariance Scaling to full-size images Variant of RBM, concepts remain the same NV xNV binary visible units K groups of hidden units NKxNK binary hidden units per group Each group has a NWxNW filter (NW , NV − NH + 1) A bias bk for each hidden group A single bias c for all visible units Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  32. 32. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Probabilistic Max Pooling Definition (Cont.d) Binary units: p(hk j = 1|v) = (bk + (W~ k v v)j ) p(vi = 1|h) = (c + KX k (Wk f hk )i) Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  33. 33. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Probabilistic Max Pooling Training Contrastive Divergence Gradients computations are done with convolutions Same refinements can be used (weight decay, momentum, ...) CRBM is highly overcomplete Sparse learning is very important Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  34. 34. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Probabilistic Max Pooling Probabilistic Max Pooling Shrink the representation by a constant factor C Allows higher-level to be invariant to small translations Reduces computational effort Generative version of standard Max Pooling Pooling layer with K groups of pooling units Each group has NPxNP units NP , NH/C Each hidden block (CxC) is connected to exactly one pooling unit Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  35. 35. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Training Probabilistic Max Pooling Definition (Cont.d) Binary units: p(vi = 1|h) = (c + KX k (Wk f hk )i) I(hk j ) , bk + (W~ k v v)j p(hk j = 1|v) = exp(I(hk i )) 1 + P j02
  36. 36. exp(I(hk i0)) p(pk = 0|v) = 1 1 + P j02
  37. 37. exp(I(hk i0)) Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  38. 38. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Contents 1 Deep Learning 2 Restricted Boltzmann Machine 3 Deep Belief Network 4 Convolutional RBM 5 Convolutional DBN 6 Conclusion Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  39. 39. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Definition Stack of Convolutional RBM With or without Probabilistic Max Pooling Each RBM takes input from previous layer output Each layer forms a higher-level representation of the data Number of hidden units in each layer can be tuned Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  40. 40. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Feature Learning Source: Honglak Lee Each layer learns a different abstraction of features 1 Stroke 2 Parts of faces 3 Faces Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  41. 41. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Implementation Conclusion Contents 1 Deep Learning 2 Restricted Boltzmann Machine 3 Deep Belief Network 4 Convolutional RBM 5 Convolutional DBN 6 Conclusion Implementation Conclusion Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  42. 42. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Implementation Conclusion Implementation Deep Learning Library (DLL) https://github.com/wichtounet/dll RBM Binary, Gaussian, Softmax, ReLU units CD and PCD Momentum, Weight Decay, Sparsity Target Convolutional RBM Standard version Probabilistic Max Pooling Various units CD and PCD Momentum, Weight Decay, Sparsity Target Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  43. 43. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Implementation Conclusion Implementation DBN Pretraining with RBM Fine-tuning with Conjugate Gradient Fine-tuning with Stochastic Gradient Descent Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  44. 44. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Implementation Conclusion Future Work Use CDBN for text detection Convolutional DBN SVM classification layer for DBN Refinements New training methods for RBM/DBN Reduce compute time Maxout, Dropout Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  45. 45. Deep Learning Restricted Boltzmann Machine Deep Belief Network Convolutional RBM Convolutional DBN Conclusion Implementation Conclusion Conclusion Deep Learning solutions are very powerful State of the art in several problems , Still room for improvement , Still young solutions (hype) , However They are complex to implement / Free variables need to be configured with care / Results from paper are hard to reproduce / Heavy to train / Baptiste Wichtbaptiste.wicht@gmail.com Deep Learning
  • steven_liucx

    May. 10, 2017
  • JosephOcen

    Aug. 10, 2015
  • Michael_Alex

    Jun. 28, 2015
  • ssuser3f56de

    May. 2, 2015
  • readgyo

    Jan. 2, 2015
  • petroniocandido

    Dec. 19, 2014
  • paul_kyeong

    Oct. 18, 2014
  • pharrell

    Sep. 28, 2014
  • masatoshiyoshida

    Sep. 21, 2014
  • pmonster

    Sep. 17, 2014
  • nextschool

    Sep. 14, 2014

Short introduction to deep learning and to the DLL Library (C++, https://github.com/wichtounet/dll). Nothing fancy.

Vues

Nombre de vues

9 474

Sur Slideshare

0

À partir des intégrations

0

Nombre d'intégrations

5 511

Actions

Téléchargements

317

Partages

0

Commentaires

0

Mentions J'aime

11

×