SlideShare a Scribd company logo

PyTorch Tutorial for NTU Machine Learing Course 2017

•Download as PPTX, PDF•
•
Yu-Hsun (lymanblue) Lin
Yu-Hsun (lymanblue) Lin

Introduction of PyTorch Explains PyTorch usages by a CNN example. Describes the PyTorch modules (torch, torch.nn, torch.optim, etc) and the usages of multi-GPU processing. Also gives examples for Recurrent Neural Network and Transfer Learning.

Data & Analytics
1 of 64
PyTorch Tutorial -NTU Machine Learning Course- Lyman Lin 林裕訓 Nov. 03, 2017 lymanblue[at]gmail.com
What is PyTorch? • Developed by Facebook – Python first – Dynamic Neural Network – This tutorial is for PyTorch 0.2.0 • Endorsed by Director of AI at Tesla
Installation • PyTorch Web: http://pytorch.org/
Packages of PyTorch Package Description torch a Tensor library like Numpy, with strong GPU support torch.autograd a tape based automatic differentiation library that supports all differentiable Tensor operations in torch torch.nn a neural networks library deeply integrated with autograd designed for maximum flexibility torch.optim an optimization package to be used with torch.nn with standard optimization methods such as SGD, RMSProp, LBFGS, Adam etc. torch.multiprocessing python multiprocessing, but with magical memory sharing of torch Tensors across processes. Useful for data loading and hogwild training. torch.utils DataLoader, Trainer and other utility functions for convenience torch.legacy(.nn/.optim) legacy code that has been ported over from torch for backward compatibility reasons This Tutorial
Outline • Neural Network in Brief • Concepts of PyTorch • Multi-GPU Processing • RNN • Transfer Learning • Comparison with TensorFlow
Neural Network in Brief • Supervised Learning – Learning a function f, that f(x)=y Data Label X1 Y1 X2 Y2 … … Trying to learn f(.), that f(x)=y
Neural Network in Brief WiData Neural Network Big Data Batch N Batch 1 Batch 2 Batch 3 … 1 Epoch N=Big Data/Batch Size
Neural Network in Brief WiData Label’ Neural Network Forward Big Data Batch N Batch 1 Batch 2 Batch 3 … 1 Epoch Forward Process: from data to label N=Big Data/Batch Size
Neural Network in Brief WiData Label’ Neural Network LabelLoss Forward Big Data Batch N Batch 1 Batch 2 Batch 3 … 1 Epoch Forward Process: from data to label N=Big Data/Batch Size
Neural Network in Brief Wi-> Wi+1Data Label’ Neural Network LabelLoss Optimizer Backward Forward Big Data Batch N Batch 1 Batch 2 Batch 3 … 1 Epoch Forward Process: from data to label Backward Process: update the parameters N=Big Data/Batch Size
Neural Network in Brief Wi-> Wi+1Data Label’ Neural Network LabelLoss Optimizer Backward Forward W Forward Backward Inside the Neural Network W W W W… Data Label’ Gradient
Neural Network in Brief Wi-> Wi+1Data Label’ Neural Network LabelLoss Optimizer Backward Forward W Forward Backward Inside the Neural Network W W W W… Data Gradient Data in the Neural Network - Tensor (n-dim array) - Gradient of Functions Label’
Concepts of PyTorch • Modules of PyTorch Wi-> Wi+1Data Label’ Neural Network LabelLoss Optimizer Backward Forward Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
Concepts of PyTorch • Modules of PyTorch • Similar to Numpy Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
Concepts of PyTorch • Modules of PyTorch • Operations – z=x+y – torch.add(x,y, out=z) – y.add_(x) # in-place Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
Concepts of PyTorch • Modules of PyTorch • Numpy Bridge • To Numpy – a = torch.ones(5) – b = a.numpy() • To Tensor – a = numpy.ones(5) – b = torch.from_numpy(a) Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
Concepts of PyTorch • Modules of PyTorch • CUDA Tensors • Move to GPU – x = x.cuda() – y = y.cuda() – x+y Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
Concepts of PyTorch • Modules of PyTorch Wi-> Wi+1Data Label’ Neural Network LabelLoss Optimizer Backward Forward Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
Neural Network in Brief Wi-> Wi+1Data Label’ Neural Network LabelLoss Optimizer Backward Forward W Forward Backward Inside the Neural Network W W W W… Data Gradient Data in the Neural Network - Tensor (n-dim array) - Gradient of Functions Label’
Concepts of PyTorch • Modules of PyTorch • Variable Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing Tensor data For Current Backward Process Handled by PyTorch Automatically
Concepts of PyTorch • Modules of PyTorch • Variable • x = Variable(torch.ones(2, 2), requires_grad=True) • print(x) • y = x + 2 • z = y * y * 3 • out = z.mean() • out.backward() • print(x.grad) Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network Define modules (must have) Build network (must have)
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have) [Channel, H, W]: 1x32x32->6x28x28
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have) [Channel, H, W]: 6x28x28
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have) [Channel, H, W]: 6x28x28 -> 6x14x14
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have) [Channel, H, W]: 6x14x14 -> 16x10x10
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have) [Channel, H, W]: 16x10x10
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have) [Channel, H, W]: 16x10x10 -> 16x5x5
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Flatten the Tensor Define modules (must have) Build network (must have) 16x5x5 Tensor: [Batch N, Channel, H, W]
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have)
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have)
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have)
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have)
• http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#define-the-network conv1 x relu pooling conv2 relu pooling fc1 relu fc2 relu fc3 Define modules (must have) Build network (must have)
Concepts of PyTorch • Modules of PyTorch • NN Modules (torch.nn) – Modules built on Variable – Gradient handled by PyTorch • Common Modules – Convolution layers – Linear layers – Pooling layers – Dropout layers – Etc… Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D – Example: – torch.nn.conv2d(in_channels=3, out_channels=16, kernel_size=3, padding=1)
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d Win Cin *: convolution
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin * 1st kernel *: convolution
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin Hout Wout 1 * 1st kernel = *: convolution
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin Hout Wout 1 * 1st kernel = *: convolution k=3d=1 s=1, moving step size p=1
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin Hout Wout 1 * 1st kernel = *: convolution k=3d=1 p=1 p=1 s=1, moving step size
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin Hout Wout 1 * 1st kernel = *: convolution k=3d=1 p=1 k=3 p=1 s=1, moving step size
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin Hout Wout 1 * 1st kernel = *: convolution k=3d=1 p=1 k=3 s=1 p=1 s=1, moving step size
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin Hout Wout 1 * 1st kernel Cout-th kernel k k Cin Hout Wout 1 * = = *: convolution … …
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin Hout Wout 1 * 1st kernel Cout-th kernel k k Cin Hout Wout 1 * = = Hout Wout Cout *: convolution … …
NN Modules • Convolution Layer – N-th Batch (N), Channel (C) – torch.nn.Conv1d: input [N, C, W] # moving kernel in 1D – torch.nn.Conv2d: input [N, C, H, W] # moving kernel in 2D – torch.nn.Conv3d: input [N, C, D, H, W] # moving kernel in 3D Hin Input for Conv2d k k Win Cin Cin * 1st kernel Cout-th kernel k k Cin * *: convolution … # of parameters
NN Modules • Linear Layer – torch.nn.Linear(in_features=3, out_features=5) – y=Ax+b
NN Modules • Dropout Layer – torch.nn.Dropout(p) – Random zeros the input with probability p – Output are scaled by 1/(1-p) If dropout here
NN Modules • Pooling Layer – torch.nn.AvgPool2d(kernel_size=2, stride=2, padding=0) – torch.nn.MaxPool2d(kernel_size=2, stride=2, padding=0) k=2d=1 k=2 p=0 s=2, moving step size s=2, moving step size
Concepts of PyTorch • Modules of PyTorch • NN Modules (torch.nn) – Modules built on Variable – Gradient handled by PyTorch • Common Modules – Convolution layers – Linear layers – Pooling layers – Dropout layers – Etc… Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
Concepts of PyTorch • Modules of PyTorch • Optimizer (torch.optim) – SGD – Adagrad – Adam – RMSprop – … – 9 Optimizers (PyTorch 0.2) • Loss (torch.nn) – L1Loss – MSELoss – CrossEntropy – … – 18 Loss Functions (PyTorch 0.2) Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-optim Define modules (must have) Build network (must have) What We Build?
http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-optim Define modules (must have) Build network (must have) What We Build? … … … D_in=1000 H=100 D_out=100 y_pred
http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-optim Define modules (must have) Build network (must have) What We Build? … … … D_in=1000 H=100 D_out=100 y_pred Optimizer and Loss Function Construct Our Model Don’t Update y (y are labels here)
http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-optim Define modules (must have) Build network (must have) … … … D_in=1000 H=100 D_out=100 y_pred Optimizer and Loss Function Reset Gradient Backward Update Step Construct Our Model What We Build? Don’t Update y (y are labels here)
Concepts of PyTorch • Modules of PyTorch • Basic Method – torch.nn.DataParallel – Recommend by PyTorch • Advanced Methods – torch.multiprocessing – Hogwild (async) Data: - Tensor - Variable (for Gradient) Function: - NN Modules - Optimizer - Loss Function - Multi-Processing
Multi-GPU Processing • torch.nn.DataParallel – gpu_id = '6,7‘ – os.environ['CUDA_VISIBLE_DEVICES'] = gpu_id – net = torch.nn.DataParallel(model, device_ids=[0, 1, 2]) – output = net(input_var) • Important Notes: – Device_ids must start from 0 – (batch_size/GPU_size) must be integer
Saving Models • First Approach (Recommend by PyTorch) • # save only the model parameters • torch.save(the_model.state_dict(), PATH) • # load only the model parameters • the_model = TheModelClass(*args, **kwargs) • the_model.load_state_dict(torch.load(PATH)) • Second Approach • torch.save(the_model, PATH) # save the entire model • the_model = torch.load(PATH) # load the entire model http://pytorch.org/docs/master/notes/serialization.html#recommended-approach-for-saving-a-model
Recurrent Neural Network (RNN) http://pytorch.org/tutorials/beginner/former_torchies/nn_tutorial.html#example-2-recurrent-net self.i2h input_size=50+20=70 input hidden output
Recurrent Neural Network (RNN) http://pytorch.org/tutorials/beginner/former_torchies/nn_tutorial.html#example-2-recurrent-net self.i2h input_size=50+20=70 input hidden output Same module (i.e. same parameters) among the time
Transfer Learning • Freeze the parameters of original model – requires_grad = False • Then add your own modules http://pytorch.org/docs/master/notes/autograd.html#excluding-subgraphs-from-backward
Comparison with TensorFlow Properties TensorFlow PyTorch Graph Static Dynamic (TensorFlow Fold) Dynamic Ramp-up Time - Win Graph Creation and Debugging - Win Feature Coverage Win Catch up quickly Documentation Tie Tie Serialization Win (support other lang.) - Deployment Win (Cloud & Mobile) - Data Loading - Win Device Management Win Need .cuda() Custom Extensions - Win Summarized from https://awni.github.io/pytorch-tensorflow/
Remind: Platform & Final Project Thank You~!

Recommended

Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance Segmentation
Dat Nguyen
 
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Simplilearn
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
Shuai Zhang
 
Generative Adversarial Networks (GAN)
Generative Adversarial Networks (GAN)Generative Adversarial Networks (GAN)
Generative Adversarial Networks (GAN)
Manohar Mukku
 
Multi Layer Perceptron & Back Propagation
Multi Layer Perceptron & Back PropagationMulti Layer Perceptron & Back Propagation
Multi Layer Perceptron & Back Propagation
Sung-ju Kim
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
佳蓉 倪
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
Fellowship at Vodafone FutureLab
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer Perceptron
Mostafa G. M. Mostafa
 

Recommended

Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance Segmentation
Dat Nguyen
 
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Simplilearn
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
Shuai Zhang
 
Generative Adversarial Networks (GAN)
Generative Adversarial Networks (GAN)Generative Adversarial Networks (GAN)
Generative Adversarial Networks (GAN)
Manohar Mukku
 
Multi Layer Perceptron & Back Propagation
Multi Layer Perceptron & Back PropagationMulti Layer Perceptron & Back Propagation
Multi Layer Perceptron & Back Propagation
Sung-ju Kim
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
佳蓉 倪
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
Fellowship at Vodafone FutureLab
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer Perceptron
Mostafa G. M. Mostafa
 
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Simplilearn
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Faster rcnn
Faster rcnnFaster rcnn
Faster rcnn
捷恩 蔡
 
Faster R-CNN - PR012
Faster R-CNN - PR012Faster R-CNN - PR012
Faster R-CNN - PR012
Jinwon Lee
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
남주 김
 
Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...
Universitat Politècnica de Catalunya
 
Flow based generative models
Flow based generative modelsFlow based generative models
Flow based generative models
수철 박
 
GoogLeNet Insights
GoogLeNet InsightsGoogLeNet Insights
GoogLeNet Insights
Auro Tripathy
 
artificial neural network
artificial neural networkartificial neural network
artificial neural network
Pallavi Yadav
 
backpropagation in neural networks
backpropagation in neural networksbackpropagation in neural networks
backpropagation in neural networks
Akash Goel
 
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial NetworksIntroduction to Generative Adversarial Networks
Introduction to Generative Adversarial Networks
BennoG1
 
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Universitat Politècnica de Catalunya
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial Networks
MLReview
 
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at GoogleDataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
Hakka Labs
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
NAVER Engineering
 
Object detection - RCNNs vs Retinanet
Object detection - RCNNs vs RetinanetObject detection - RCNNs vs Retinanet
Object detection - RCNNs vs Retinanet
Rishabh Indoria
 
Video Transformers.pptx
Video Transformers.pptxVideo Transformers.pptx
Video Transformers.pptx
Sangmin Woo
 
AlexNet
AlexNetAlexNet
AlexNet
Bertil Hatt
 
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement Learning
NAVER Engineering
 
Pytorch for tf_developers
Pytorch for tf_developersPytorch for tf_developers
Pytorch for tf_developers
Abdul Muneer
 
Deep Learning with PyTorch
Deep Learning with PyTorchDeep Learning with PyTorch
Deep Learning with PyTorch
Mayur Bhangale
 

More Related Content

What's hot

Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Simplilearn
 
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Universitat Politècnica de Catalunya
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial Networks
MLReview
 
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at GoogleDataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
Hakka Labs
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
NAVER Engineering
 

What's hot (20)

Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
 
Faster rcnn
Faster rcnnFaster rcnn
Faster rcnn
 
Faster R-CNN - PR012
Faster R-CNN - PR012Faster R-CNN - PR012
Faster R-CNN - PR012
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...
 
Flow based generative models
Flow based generative modelsFlow based generative models
Flow based generative models
 
GoogLeNet Insights
GoogLeNet InsightsGoogLeNet Insights
GoogLeNet Insights
 
artificial neural network
artificial neural networkartificial neural network
artificial neural network
 
backpropagation in neural networks
backpropagation in neural networksbackpropagation in neural networks
backpropagation in neural networks
 
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial NetworksIntroduction to Generative Adversarial Networks
Introduction to Generative Adversarial Networks
 
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial Networks
 
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at GoogleDataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
 
Object detection - RCNNs vs Retinanet
Object detection - RCNNs vs RetinanetObject detection - RCNNs vs Retinanet
Object detection - RCNNs vs Retinanet
 
Video Transformers.pptx
Video Transformers.pptxVideo Transformers.pptx
Video Transformers.pptx
 
AlexNet
AlexNetAlexNet
AlexNet
 
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement Learning
 

Viewers also liked

Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessment
Sayyid Ina
 
Environmental impact assessment m5
Environmental impact assessment m5Environmental impact assessment m5
Environmental impact assessment m5
Bibhabasu Mohanty
 
Environmental impact assessment concept
Environmental impact assessment conceptEnvironmental impact assessment concept
Environmental impact assessment concept
Intan Ayuna
 
Developing Guidelines for Public Participation on Environmental Impact Assess...
Developing Guidelines for Public Participation on Environmental Impact Assess...Developing Guidelines for Public Participation on Environmental Impact Assess...
Developing Guidelines for Public Participation on Environmental Impact Assess...
Ethical Sector
 

Viewers also liked (20)

Pytorch for tf_developers
Pytorch for tf_developersPytorch for tf_developers
Pytorch for tf_developers
 
Deep Learning with PyTorch
Deep Learning with PyTorchDeep Learning with PyTorch
Deep Learning with PyTorch
 
ENVIRONMENTAL IMPACT ASSESSMENT - EIA
ENVIRONMENTAL IMPACT ASSESSMENT - EIAENVIRONMENTAL IMPACT ASSESSMENT - EIA
ENVIRONMENTAL IMPACT ASSESSMENT - EIA
 
Environmental Impact Assessment - University of Winnipeg
Environmental Impact Assessment - University of WinnipegEnvironmental Impact Assessment - University of Winnipeg
Environmental Impact Assessment - University of Winnipeg
 
EIA thermal power plant
EIA thermal power plantEIA thermal power plant
EIA thermal power plant
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessment
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessment
 
13 environmental impact assessment
13 environmental impact assessment13 environmental impact assessment
13 environmental impact assessment
 
Environmental impact assessment m5
Environmental impact assessment m5Environmental impact assessment m5
Environmental impact assessment m5
 
Environmental impact assessment concept
Environmental impact assessment conceptEnvironmental impact assessment concept
Environmental impact assessment concept
 
Environmental impact assessment (eia) By Mr Allah Dad Khan Visiting Professor...
Environmental impact assessment (eia) By Mr Allah Dad Khan Visiting Professor...Environmental impact assessment (eia) By Mr Allah Dad Khan Visiting Professor...
Environmental impact assessment (eia) By Mr Allah Dad Khan Visiting Professor...
 
Environmental Audit and Environmental Impact Assessment
Environmental Audit and Environmental Impact AssessmentEnvironmental Audit and Environmental Impact Assessment
Environmental Audit and Environmental Impact Assessment
 
Seminar on Environmental Impact Assessment
Seminar on Environmental Impact AssessmentSeminar on Environmental Impact Assessment
Seminar on Environmental Impact Assessment
 
environmental impact assessment
environmental impact assessment environmental impact assessment
environmental impact assessment
 
Environmental Impact Assessment in Water Resources Projects
Environmental Impact Assessment in Water Resources ProjectsEnvironmental Impact Assessment in Water Resources Projects
Environmental Impact Assessment in Water Resources Projects
 
Environmental impact assessment methodology
Environmental impact assessment methodologyEnvironmental impact assessment methodology
Environmental impact assessment methodology
 
Developing Guidelines for Public Participation on Environmental Impact Assess...
Developing Guidelines for Public Participation on Environmental Impact Assess...Developing Guidelines for Public Participation on Environmental Impact Assess...
Developing Guidelines for Public Participation on Environmental Impact Assess...
 
Environmental impact assessment and life cycle assessment and their role in s...
Environmental impact assessment and life cycle assessment and their role in s...Environmental impact assessment and life cycle assessment and their role in s...
Environmental impact assessment and life cycle assessment and their role in s...
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessment
 
Planning and environmental impact assessment (EIA)
Planning and environmental impact assessment (EIA)Planning and environmental impact assessment (EIA)
Planning and environmental impact assessment (EIA)
 

Similar to PyTorch Tutorial for NTU Machine Learing Course 2017

Accelerating HPC Applications on NVIDIA GPUs with OpenACC
Accelerating HPC Applications on NVIDIA GPUs with OpenACCAccelerating HPC Applications on NVIDIA GPUs with OpenACC
Accelerating HPC Applications on NVIDIA GPUs with OpenACC
inside-BigData.com
 
Natural Language Processing with CNTK and Apache Spark with Ali Zaidi
Natural Language Processing with CNTK and Apache Spark with Ali ZaidiNatural Language Processing with CNTK and Apache Spark with Ali Zaidi
Natural Language Processing with CNTK and Apache Spark with Ali Zaidi
Databricks
 

Similar to PyTorch Tutorial for NTU Machine Learing Course 2017 (20)

[Update] PyTorch Tutorial for NTU Machine Learing Course 2017
[Update] PyTorch Tutorial for NTU Machine Learing Course 2017[Update] PyTorch Tutorial for NTU Machine Learing Course 2017
[Update] PyTorch Tutorial for NTU Machine Learing Course 2017
 
Numba: Array-oriented Python Compiler for NumPy
Numba: Array-oriented Python Compiler for NumPyNumba: Array-oriented Python Compiler for NumPy
Numba: Array-oriented Python Compiler for NumPy
 
Scaling Python to CPUs and GPUs
Scaling Python to CPUs and GPUsScaling Python to CPUs and GPUs
Scaling Python to CPUs and GPUs
 
Simple, fast, and scalable torch7 tutorial
Simple, fast, and scalable torch7 tutorialSimple, fast, and scalable torch7 tutorial
Simple, fast, and scalable torch7 tutorial
 
OpenPOWER Workshop in Silicon Valley
OpenPOWER Workshop in Silicon ValleyOpenPOWER Workshop in Silicon Valley
OpenPOWER Workshop in Silicon Valley
 
PyCon Estonia 2019
PyCon Estonia 2019PyCon Estonia 2019
PyCon Estonia 2019
 
Neural Network with Python & TensorFlow
Neural Network with Python & TensorFlowNeural Network with Python & TensorFlow
Neural Network with Python & TensorFlow
 
Accelerating HPC Applications on NVIDIA GPUs with OpenACC
Accelerating HPC Applications on NVIDIA GPUs with OpenACCAccelerating HPC Applications on NVIDIA GPUs with OpenACC
Accelerating HPC Applications on NVIDIA GPUs with OpenACC
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
Overview of Chainer and Its Features
Overview of Chainer and Its FeaturesOverview of Chainer and Its Features
Overview of Chainer and Its Features
 
Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications
 
Pytroch-basic.pptx
Pytroch-basic.pptxPytroch-basic.pptx
Pytroch-basic.pptx
 
Natural Language Processing with CNTK and Apache Spark with Ali Zaidi
Natural Language Processing with CNTK and Apache Spark with Ali ZaidiNatural Language Processing with CNTK and Apache Spark with Ali Zaidi
Natural Language Processing with CNTK and Apache Spark with Ali Zaidi
 
Tutorial: Image Generation and Image-to-Image Translation using GAN
Tutorial: Image Generation and Image-to-Image Translation using GANTutorial: Image Generation and Image-to-Image Translation using GAN
Tutorial: Image Generation and Image-to-Image Translation using GAN
 
Venkat ns2
Venkat ns2Venkat ns2
Venkat ns2
 
Machine learning with py torch
Machine learning with py torchMachine learning with py torch
Machine learning with py torch
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
Text classification in scikit-learn
Text classification in scikit-learnText classification in scikit-learn
Text classification in scikit-learn
 
The Effect of Hierarchical Memory on the Design of Parallel Algorithms and th...
The Effect of Hierarchical Memory on the Design of Parallel Algorithms and th...The Effect of Hierarchical Memory on the Design of Parallel Algorithms and th...
The Effect of Hierarchical Memory on the Design of Parallel Algorithms and th...
 

Recently uploaded

Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
AroojKhan71
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightCheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Delhi Call girls
 
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in Kishangarh
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in KishangarhDelhi 99530 vip 56974 Genuine Escort Service Call Girls in Kishangarh
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in Kishangarh
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Delhi Call girls
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
amitlee9823
 
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
shivangimorya083
 
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
Call Girls in Nagpur High Profile Call Girls
 

Recently uploaded (20)

Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
 
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightCheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
 
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in Kishangarh
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in KishangarhDelhi 99530 vip 56974 Genuine Escort Service Call Girls in Kishangarh
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in Kishangarh
 
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
 
Smarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptxSmarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptx
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
 
100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx
 
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
 
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdfMarket Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
 
Zuja dropshipping via API with DroFx.pptx
Zuja dropshipping via API with DroFx.pptxZuja dropshipping via API with DroFx.pptx
Zuja dropshipping via API with DroFx.pptx
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysis
 
Carero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptxCarero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptx
 
Halmar dropshipping via API with DroFx
Halmar dropshipping via API with DroFxHalmar dropshipping via API with DroFx
Halmar dropshipping via API with DroFx
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
 
Accredited-Transport-Cooperatives-Jan-2021-Web.pdf
Accredited-Transport-Cooperatives-Jan-2021-Web.pdfAccredited-Transport-Cooperatives-Jan-2021-Web.pdf
Accredited-Transport-Cooperatives-Jan-2021-Web.pdf
 
Ravak dropshipping via API with DroFx.pptx
Ravak dropshipping via API with DroFx.pptxRavak dropshipping via API with DroFx.pptx
Ravak dropshipping via API with DroFx.pptx
 
Invezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signalsInvezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signals
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptx
 

PyTorch Tutorial for NTU Machine Learing Course 2017

Editor's Notes

  1. http://pytorch.org/docs/master/notes/autograd.html