Deep learning tutorial (i)
•Télécharger en tant que PPTX, PDF•
1 j'aime•320 vues
Deep learning progress, industry landscapes, and cutting-edge directions.
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (16)
Similaire à Deep learning tutorial (i)
Similaire à Deep learning tutorial (i) (20)
Dernier
Dernier (20)
Deep learning tutorial (i)
- 1. Deep Learning Tutorial (I) Introduction Guan Wang
- 3. Overview
- 4. Deep neural networks learn to do the following
- 5. Yearbook • prior 2012 • Machine learning was more about SVM, Graphical Models, non-parametric baysian, and simple hacks, e.g., decision tree, naïve bayesian, regressions, etc •2012~2013 • Deep Neural Network, big data, mature distributed computing architectures • Refreshing accuracy record in image recognition tasks • Feed forward Neural Networks, CNN, RBM •2014~2015 • Bridging CV, NLP and expanding to other domains • New architectures and new ML tasks • RNN, LSTM, RL •2016~future • Larger models on CV, NLP, keep expanding to other domains • Larger systems, larger models on CNN, LSTM, etc
- 7. Hardware Optimized for Neural Networks Google TPUNVIDIA
- 8. Deep Learning Service on cloud
- 9. Computer Vision (crowded market) Generic Algorithms & API providers •Special object recognition (face, etc) •General object recognition (image search, etc) •Moving object detection & recognition (pedestrian detection, etc) •Image understanding (visual QA, artify, etc) •Video understanding (video search, etc) Verticals •Satellite image analysis (understanding civil developments) •Home & office place security & surveillance •User interest analysis and high precision targeting (ads) •ADAS & Autonomous Driving •Robotics and drones •The list goes on and on
- 10. Natural Language Understanding (crowded market) Generic Algorithms & API providers •Personal assistance (x.ai, api.ai, etc) •Chat bots (facebook ecosystem, viv.ai, etc) •Knowledge understanding (IBM watson, etc) Verticals •Customer service •Travel management •Financial service •Smart homes •Connected cars •The list goes on and on
- 11. System Architecture Every good machine learning algorithm deserves its own system architecture. -- one of my mentors
- 12. Distributed Architectures Generic solvers •Stochastic Gradient Descent •Coordinate Descent •MCMC •ADMM •… Design Choices •sync or async •CPU or GPU cluster •Online training or offline training •... Examples •Parameter Server •DistBelief •Tensorflow Flexible solutions •CoreOS, etcd, Docker •Kubernetes •Pachyderm •Mesos •etc Big data ecosystem •Spark & tachyon •yarn •hdfs •etc Deep learning tools •Tensorflow •Torch-IPC •DistBelief •Petuum •GraphLab
- 13. Standalone Toolkits CPU with GPU speedup •Torch •Caffe •Theano •Tensorflow Good for research or small applications Embedded Support •Tiny-CNN •Tensorflow-embedded Good for phone apps, raspberry Pi, cars, drones, robotics
- 15. CNN CNN on Text Classification R-CNN, fast R-CNN Attention Model
- 16. LSTM Seq2seq model •Translation •Dialogue system •Time series analysis •Text classification
- 17. Memory Networks External memory on RNN •Translation •Dialogue system •QA
- 18. Reinforcement Learning Deep Q-Learning Function approximation on •Policy function •Value function
- 19. Deep Generative Model Adversarial networks
Notes de l'éditeur
- What deep learning can do Who are working on that. How are they working on that What are they working on
- Pre 2012: NN are powerful, but unstable. A great student that no one can teach effectively. 2012, when I started with deep learning. Unified ML model or at least part of the unified model Tools like caffe 2014 Merging effort from vision and language. 2016 Expanding to other domains, robotics, control, recommendation systems
- Since DL requires a lot of data & computation, the capability of computing is a competitive advantage