An Introduction to Deep Learning with Apache MXNet (November 2017)
1. Julien Simon, AI/ML Evangelist, EMEA
@julsimon
An introduction to Deep Learning
with Apache MXNet
2. Agenda
• The Advent of Deep Learning
• Deep Learning Applications
• Apache MXNet Demos (there will be code, ha!)
3. Artificial Intelligence: design software applications which
exhibit human-like behavior, e.g. speech, natural language
processing, reasoning or intuition
Machine Learning: teach machines to learn without being
explicitly programmed
Deep Learning: using neural networks, teach machines to
learn from complex data where features cannot be explicitly
expressed
9. Infrastructure CPU
Engines MXNet TensorFlow Caffe Theano Pytorch CNTK
Services
Amazon Polly
Chat
Platforms
IoT
Speech
Amazon Lex
Mobile
Amazon AI: Artificial Intelligence In The Hands Of Every Developer
Amazon
ML
Spark &
EMR
Kinesis Batch ECS
GPU
Amazon Rekognition
Vision
FPGA
10. Hardware innovation for Deep Learning
https://aws.amazon.com/blogs/aws/new-amazon-ec2-instances-with-up-to-8-nvidia-tesla-v100-gpus-p3/
https://devblogs.nvidia.com/parallelforall/inside-volta/
https://aws.amazon.com/fr/blogs/aws/now-available-compute-intensive-c5-instances-for-amazon-ec2/
Intel Skylake CPU
November 6th
Nvidia Volta GPU
October 25th
13. • 17,000 images from Instagram
• 7 brands
• Inception v3 model, pre-trained on ImageNet
• Fine-tuning with TensorFlow and EC2 GPU
instances
• Additional work on color extraction
https://technology.condenast.com/story/handbag-brand-and-color-detection
Image Classification
24. Apache MXNet
Programmable Portable High Performance
Near linear scaling
across hundreds of GPUs
Highly efficient
models for mobile
and IoT
Simple syntax,
multiple languages
Most Open Best On AWS
Optimized for
Deep Learning on
AWS
Accepted into the
Apache Incubator
25. 1. Image classification: using pre-trained models
Imagenet, multiple CNNs, MXNet
2. Image classification: fine-tuning a pre-trained model
CIFAR-10, ResNet-50, Keras + MXNet
3. Image classification: learning from scratch
MNIST, MLP & LeNet, MXNet
4. Machine Translation: translating German to English
News, LSTM, Sockeye + MXNet
32. Demo #4 – Machine Translation: German to English
• Sockeye
• 5.8M sentences (news headlines), 5 hours of training on 8 GPUs (p2)
./translate.sh "Chopin zählt zu den bedeutendsten Persönlichkeiten der
Musikgeschichte Polens .”
Chopin is one of the most important personalities of Poland’s history
./translate.sh "Hotelbetreiber müssen künftig nur den Rundfunkbeitrag
bezahlen, wenn ihre Zimmer auch eine Empfangsmöglichkeit bieten .”
in the future , hotel operators must pay only the broadcasting fee if their
rooms also offer a reception facility .
https://aws.amazon.com/blogs/ai/train-neural-machine-translation-models-with-sockeye/
33. Demo #5 – Image Generation with MNIST
https://medium.com/@julsimon/generative-adversarial-networks-on-apache-mxnet-part-1-b6d39e6b5df1
34. Anything you dream is fiction, and anything
you accomplish is science, the whole history
of mankind is nothing but science fiction.
Ray Bradbury