Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Deep Learning:

and Deep Data-Science
12	
  May	
  2015
@graphific
Roelof Pieters
www.csc.kth.se/~roelof/
roelof@kth.se Gra...
BUT FIRST…
CAT PERSON? DOG PERSON?
are you a…
in the next few minutes
DETECTOR
we ll be making a
main Libraries
•sckikit-learn (machine learning) 

http://scikit-learn.org
•caffe (deep learning) - for training deep neur...
Code is ahead, soon… 

I promise :)
“Data science is clearly a blend of the
hackers' art, statistics and machine
learning…”
—Hilary Mason & Chris Wiggins, 201...
(Drew Connoway 2010)
1 feature
> Features = Awesomeness
1 feature
2 features
> Features = Awesomeness
1 feature
2 features
too few features/dimensions = overfitting
1 feature
2 features
too few features/dimensions = overfitting
3 features
> Features = Awesomeness
More Features = Awesomeness!
1 feature
2 features 3 features
++ Data Needs also grow!
(picture by Dato)
(picture by Dato)
Deep Learning?
•A host of statistical machine learning
techniques
•Enables the automatic learning of feature
hierarchies
•...
(picture by Dato)
Deep Learning
Deep Learning?
•Manually designed features are often over-specified,
incomplete and take a long time to design and validate...
23
2006+ : The Deep Learning Conspirators
(chart by Clarifai)
Audio Recognition
Image Recognition
(chart by Clarifai)
Natural Langauge Processing
…
Natural Langauge Processing
… …
DL? How ?
almost at the code…
(picture by Dato)
Coding time!
our ingredients…
(picture by Dato)
Kaggle’s Cat vs Dog
dataset (25k dog/cat
pictures)
https://www.kaggle.com/c/dogs-vs-cats/data
Pretrained
Convolutional 

Neural Net (CNN)
(picture by Dato)
97% accuracy in < 1h
MultiLayer
Perceptron
Cooking Instructions1 Load Pretrained Net
features
image
2 Extract features for all training images
3. train MLP on those ...
Cooking Instructions1 Load Pretrained Net
https://github.com/BVLC/caffe/wiki/Model-Zoo
No Free Lunch… But Free Models!
# imports demo
# load pretrained deep neural net
(convnet from Krizhevsky et al.'s NIPS 2012 ImageNet classification paper)
demo
Cooking Instructions
features
image
2 Extract features for all training images
1 Load Pretrained Net
# feed image into the network and return internal feature
representation of layer fc6 demo
#extract features from images demo
(…)
#dump features as pickle file demo
Cooking Instructions1 Load Pretrained Net
features
image
2 Extract features for all training images
3. train MLP on those ...
Pylearn2:
Multilayer Perceptron (MLP) on top of
extracted features
#imports demo
#load earlier extracted features and labels
#convert to input that pylearn understands demo
# create

layers of

MLP
# with 

softmax

as final layer
# trainer
initialized

with SGD, 

momentum,

dropout
demo
# train/test splits demo
#start our MLP (pylearn experiment method)
(…)
demo
already after 5 min:
90%
1 hour
50%
accuracy
97%
1 hour
94%
start at iteration #2
accuracy
(…)
So are YOU more like a Dog or Cat?
DETECTOR
(I might put it up as a Flask site online, if people are
interested?)
What 

about

me?
EASY PIEZY…
THATS ALL!
as PhD candidate KTH/CSC:
“Always interested in discussing
Machine Learning, Deep
Architectures, Graphs, and
Language Tech...
Wanna Know More?
bit.ly/SthlmDL
• Theano - CPU/GPU symbolic expression compiler in
python (from LISA lab at University of Montreal).
http://deeplearning.n...
• RNNLM (Mikolov)

http://rnnlm.org
• NB-SVM

https://github.com/mesnilgr/nbsvm
• Word2Vec (skipgrams/cbow)

https://code....
• cuda-convnet2 (Alex Krizhevsky, Toronto) (c++/
CUDA, optimized for GTX 580) 

https://code.google.com/p/cuda-convnet2/
•...
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Prochain SlideShare
Chargement dans…5
×

Deep Learning as a Cat/Dog Detector

35 004 vues

Publié le

Talk given at PYCON Stockholm 2015
Intro to Deep Learning + taking pretrained imagenet network, extracting features, and RBM on top = 97 Accuracy after 1 hour (!) of training (in top 10% of kaggle cat vs dog competition)

  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy &amp; Proven Way to Build Good Habits &amp; Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • How I Cured My Uterine Fibroids? Reverse And Eliminate Uterine Fibroids, Safe &amp; Natural With Fast Results.. ▲▲▲ http://ishbv.com/fibroids7/pdf
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • I want my dog to listen and come every time I call! Dog Training Course Download▲▲▲ http://scamcb.com/brainydogs/pdf
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • video recording at https://www.youtube.com/watch?v=wBKfGaakFp8
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Deep Learning as a Cat/Dog Detector

  1. 1. Deep Learning:
 and Deep Data-Science 12  May  2015 @graphific Roelof Pieters www.csc.kth.se/~roelof/ roelof@kth.se Graph Technologies R&D roelof@graph-technologies.com slides online at:
 https://www.slideshare.net/roelofp/deep-learning-as-a-catdog-detector
  2. 2. BUT FIRST… CAT PERSON? DOG PERSON? are you a…
  3. 3. in the next few minutes DETECTOR we ll be making a
  4. 4. main Libraries •sckikit-learn (machine learning) 
 http://scikit-learn.org •caffe (deep learning) - for training deep neural nets (for today: loading a pre-trained one) 
 http://caffe.berkeleyvision.org •theano (efficient gpu-powered math)
 http://www.deeplearning.net/software/theano/ •ipython notebook
 http://ipython.org/notebook.html
  5. 5. Code is ahead, soon… 
 I promise :)
  6. 6. “Data science is clearly a blend of the hackers' art, statistics and machine learning…” —Hilary Mason & Chris Wiggins, 2010 Data Science ?
  7. 7. (Drew Connoway 2010)
  8. 8. 1 feature > Features = Awesomeness
  9. 9. 1 feature 2 features > Features = Awesomeness
  10. 10. 1 feature 2 features too few features/dimensions = overfitting
  11. 11. 1 feature 2 features too few features/dimensions = overfitting 3 features > Features = Awesomeness
  12. 12. More Features = Awesomeness! 1 feature 2 features 3 features
  13. 13. ++ Data Needs also grow!
  14. 14. (picture by Dato)
  15. 15. (picture by Dato)
  16. 16. Deep Learning? •A host of statistical machine learning techniques •Enables the automatic learning of feature hierarchies •Generally based on artificial neural networks
  17. 17. (picture by Dato) Deep Learning
  18. 18. Deep Learning? •Manually designed features are often over-specified, incomplete and take a long time to design and validate •Learned Features are easy to adapt, fast to learn
 •Deep learning provides a very flexible, (almost?) universal, learnable framework for representing world, visual and linguistic information. •Deep learning can learn unsupervised (from raw text/audio/images/whatever content) and supervised (with specific labels like positive/ negative) (as summarised by Richard Socher 2014)
  19. 19. 23 2006+ : The Deep Learning Conspirators
  20. 20. (chart by Clarifai) Audio Recognition
  21. 21. Image Recognition (chart by Clarifai)
  22. 22. Natural Langauge Processing …
  23. 23. Natural Langauge Processing … …
  24. 24. DL? How ? almost at the code…
  25. 25. (picture by Dato) Coding time! our ingredients…
  26. 26. (picture by Dato) Kaggle’s Cat vs Dog dataset (25k dog/cat pictures)
  27. 27. https://www.kaggle.com/c/dogs-vs-cats/data
  28. 28. Pretrained Convolutional 
 Neural Net (CNN)
  29. 29. (picture by Dato) 97% accuracy in < 1h MultiLayer Perceptron
  30. 30. Cooking Instructions1 Load Pretrained Net features image 2 Extract features for all training images 3. train MLP on those features
  31. 31. Cooking Instructions1 Load Pretrained Net
  32. 32. https://github.com/BVLC/caffe/wiki/Model-Zoo No Free Lunch… But Free Models!
  33. 33. # imports demo
  34. 34. # load pretrained deep neural net (convnet from Krizhevsky et al.'s NIPS 2012 ImageNet classification paper) demo
  35. 35. Cooking Instructions features image 2 Extract features for all training images 1 Load Pretrained Net
  36. 36. # feed image into the network and return internal feature representation of layer fc6 demo
  37. 37. #extract features from images demo
  38. 38. (…) #dump features as pickle file demo
  39. 39. Cooking Instructions1 Load Pretrained Net features image 2 Extract features for all training images 3. train MLP on those features
  40. 40. Pylearn2: Multilayer Perceptron (MLP) on top of extracted features #imports demo
  41. 41. #load earlier extracted features and labels #convert to input that pylearn understands demo
  42. 42. # create
 layers of
 MLP # with 
 softmax
 as final layer # trainer initialized
 with SGD, 
 momentum,
 dropout demo
  43. 43. # train/test splits demo
  44. 44. #start our MLP (pylearn experiment method) (…) demo already after 5 min:
  45. 45. 90% 1 hour 50% accuracy
  46. 46. 97% 1 hour 94% start at iteration #2 accuracy (…)
  47. 47. So are YOU more like a Dog or Cat? DETECTOR
  48. 48. (I might put it up as a Flask site online, if people are interested?) What 
 about
 me?
  49. 49. EASY PIEZY… THATS ALL!
  50. 50. as PhD candidate KTH/CSC: “Always interested in discussing Machine Learning, Deep Architectures, Graphs, and Language Technology” roelof@kth.se www.csc.kth.se/~roelof/ Data Science ConsultancyAcademic/Research roelof@graph-systems.com www.graph-technologies.com Gve Systems Graph Technologies 64 In Touch!
  51. 51. Wanna Know More? bit.ly/SthlmDL
  52. 52. • Theano - CPU/GPU symbolic expression compiler in python (from LISA lab at University of Montreal). http://deeplearning.net/software/theano/ • Pylearn2 - library designed to make machine learning research easy. http://deeplearning.net/software/ pylearn2/ • Torch - Matlab-like environment for state-of-the-art machine learning algorithms in lua (from Ronan Collobert, Clement Farabet and Koray Kavukcuoglu) http://torch.ch/ • more info: http://deeplearning.net/software links/ Wanna Play ? 66 Wanna Play ? General Deep Learning
  53. 53. • RNNLM (Mikolov)
 http://rnnlm.org • NB-SVM
 https://github.com/mesnilgr/nbsvm • Word2Vec (skipgrams/cbow)
 https://code.google.com/p/word2vec/ (original)
 http://radimrehurek.com/gensim/models/word2vec.html (python) • GloVe
 http://nlp.stanford.edu/projects/glove/ (original)
 https://github.com/maciejkula/glove-python (python) • Socher et al / Stanford RNN Sentiment code:
 http://nlp.stanford.edu/sentiment/code.html • Deep Learning without Magic Tutorial:
 http://nlp.stanford.edu/courses/NAACL2013/ 67 Wanna Play ? NLP
  54. 54. • cuda-convnet2 (Alex Krizhevsky, Toronto) (c++/ CUDA, optimized for GTX 580) 
 https://code.google.com/p/cuda-convnet2/ • Caffe (Berkeley) (Cuda/OpenCL, Theano, Python)
 http://caffe.berkeleyvision.org/ • OverFeat (NYU) 
 http://cilvr.nyu.edu/doku.php?id=code:start 68 Wanna Play ? Computer Vision

×