SlideShare une entreprise Scribd logo

capsule network

•
•
민
민기 정

CapsNet Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv, Dynamic Routing Between Capsules

Données & analyses
1  sur  31
Télécharger pour lire hors ligne
Dynamic Routing Between Capsules Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv LAB SEMINAR 1 2017.11.13 SNU DATAMINING CENTER MINKI CHUNG
TABLE OF CONTENTS ▸ Intuition ▸ Problems of ConvNet ▸ How brain works, Inverse graphics ▸ Capsule Theory ▸ CapsNet ▸ Capsule ▸ CapsNet architecture ▸ Experiment ▸ Classification on MNIST ▸ Reconstruction on MNIST ▸ Dimension perturbation on MNIST ▸ Discussion 2
INTUITION ▸ Problems of ConvNet ▸ How brain works, Inverse graphics ▸ Capsule Theory 3
PROBLEMS OF CONVNET 4 ▸ ConvNet Architecture PROBLEMS IS ‘POOLING’ https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc Obtain translational, rotational invariance
PROBLEMS OF CONVNET 5 ▸ @REDDIT, MACHINE LEARNING https://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/clyj4jv/
PROBLEMS OF CONVNET 6 ▸ WHAT IS THIS PICTURE? https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952
PROBLEMS OF CONVNET 7 ▸ HOW ABOUT THIS? https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952
PROBLEMS OF CONVNET 8 ▸ NEED EQUIVARIANCE, NOT INVARIANCE https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952
HOW BRAIN WORKS, INVERSE GRAPHICS 9 ▸ Constructing a visual image from some internal hierarchical representation of geometric data ▸ Internal representation is stored in computer’s memory as arrays of geometrical objects and matrices that represent relative positions and orientation of these objects ▸ Special software takes that representation and converts it into an image on the screen. This is called rendering ▸ Brains, in fact, do the opposite of rendering. Hinton calls it inverse graphics: Visual information received by eyes, they deconstruct a hierarchical representation of the world around us and try to match it with already learned patterns and relationships stored in the brain ▸ Key idea is that representation of objects in the brain does not depend on view angle COMPUTER GRAPHICS https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b
CAPSULE THEORY 10 ▸ In 3D graphics, relationships between 3D objects can be represented by a so- called pose, which is in essence translation plus rotation ▸ Capsule approach: It incorporates relative relationships between objects (Internal representation) and it is represented numerically as a 4D pose matrix ▸ by ‘Dynamic Routing’ (more details later) ▸ allows capsules to communicate with each other and create representations similar to scene graphs in computer graphics https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b YOU CAN EASILY RECOGNIZE THAT THIS IS THE STATUE OF LIBERTY, EVEN THOUGH ALL THE IMAGES SHOW IT FROM DIFFERENT ANGLES
CAPSULE THEORY 11 ▸ Benifits: ▸ Better understanding 3D Space ▸ Achieve state-of-the art performance by only using a fraction of the data that a CNN would use ▸ In order to learn to tell digits apart, the human brain needs only a couple of dozens of examples, hundreds at most, while CNN need tens of thousands of examples https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b
CAPSNET ▸ Capsule ▸ CapsNet architecture ▸ Experiment 12
CAPSULE 13 ▸ Comparison with traditional neuron https://www.zhihu.com/question/67287444/answer/251460831 V VEC LENGTH WORKS LIKE PROBABILITY ACTIVATION OF NEXT CAPSULE DYNAMIC ROUTING
CAPSNET ARCHITECTURE 14 ARCHITECTURE Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules CONV CAPS.CONV CAPS.FC DYNAMIC ROUTING 8X 32 X MNIST LOCAL FEATURE DETECTION 6*6*32=1152 CAPSULES, EACH HAS 8 PROPERTIES 10 CAPSULES (CLASS), EACH HAS 16 PROPERTIES DEEPER MEANS MORE COMPLEX, DIMENSION SHOULD INCREASE
CAPSNET ARCHITECTURE 15 ▸ naturomics github CAPSNET-TENSORFLOW CAPS.CONVCONV CONV X 32 MNIST X 8 https://github.com/naturomics/CapsNet-Tensorflow X 32 X 8 CAPS.FC CAPS.CONV CAPS.FC DYNAMIC ROUTING
CAPSNET ARCHITECTURE 16 ▸ Place-coded Capsule ▸ Concatenate (=8 different regular conv layers) ▸ Consider each feature map as capsule (6*6*32=1152 capsules with 8 properties) CAPS.CONV, PRIMARYCAPS CAPS.CONV X 32 MNIST X 8 https://github.com/naturomics/CapsNet-Tensorflow DIRECTION
CAPSNET ARCHITECTURE 17 ▸ Place-coded Capsule ▸ Concatenate (=8 different regular conv layers) ▸ Consider each feature map as capsule (6*6*32=1152 capsules with 8 properties) ▸ Use squashing function in the end CAPS.CONV, PRIMARYCAPS CAPS.CONV X 32 MNIST X 8 https://github.com/naturomics/CapsNet-Tensorflow
CAPSNET ARCHITECTURE 18 ▸ Rate-coded capsules ▸ caps: 1152 → 10 ▸ vec-len: 8 → 16 ▸ Dynamic Routing CAPS.FC, DIGITCAPS https://github.com/naturomics/CapsNet-Tensorflow X 32 MNIST X 8 CAPS.FC DYNAMIC ROUTING DYNAMIC ROUTING
CAPSNET ARCHITECTURE 19 ▸ Dynamic Routing ▸ Top-down feedback ▸ Routing by agreement ▸ Works like attention CAPS.FC, DIGITCAPS https://github.com/naturomics/CapsNet-Tensorflow IF MULTIPLE PREDICTIONS AGREE, HIGHER LEVEL CAPSULE BECOMES ACTIVE VEC LENGTH WORKS LIKE PROBABILITY ACTIVATION OF NEXT CAPSULE COUPLING COEFFICIENTS TOPDOWN FEEDBACK: IF RELATION EXISTS COUPLING COEFFICIENTS INCREASE AGREEMENT
CAPSNET ARCHITECTURE 20 ▸ Dynamic Routing CAPS.FC, DIGITCAPS https://github.com/naturomics/CapsNet-Tensorflow X 32 MNIST X 8 CAPS.FC DYNAMIC ROUTING 3 ITERATIONS WILL DO
EXPERIMENT ▸ Classification on MNIST ▸ Reconstruction on MNIST ▸ Dimension perturbation on MNIST 21
EXPERIMENT 22 ▸ Introduce first three ▸ Classification on MNIST (99.75%, conv 99.61%) ▸ Reconstruction on MNIST ▸ Dimension Perturbation on MNIST ▸ Robustness to Affine Transformation on MNIST (79%, conv 66%) ▸ Classification on MultiMNIST (5% error) ▸ Classification on CIFAR 10 (10.6% error - ZFNet) ▸ Classification on SVHN (4.3% error) Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules
EXPERIMENT 23 ▸ 99.75% (baseline 99.61%) 1. CLASSIFICATION ON MNIST Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules
EXPERIMENT 24 ▸ 2. RECONSTRUCTION ON MNIST Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules
EXPERIMENT 25 ▸ 3. DIMENSION PERTURBATION ON MNIST Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules
DISCUSSION 26
_ 27 ▸ Capsule(Vector), ▸ Not conventional neuron(Scalar) NOVELTY
_ 28 ▸ Still use regular conv layer at first for local feature extraction ▸ Capsule cannot extract local feature? STILL USE CONV LAYER HOW TO RESTRICT TO GET CERTAIN FEATURE? ▸ Disentangling features ▸ How to obtain ‘certain features’?
ANY Q? 29
REFERENCE ▸ Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules (https:// arxiv.org/abs/1710.09829) ▸ Geoffrey Hinton et al., Matrix Capsules With EM Routing, Under review as a conference paper at ICLR 2018 (https:// openreview.net/pdf?id=HJWLfGWRb) ▸ https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b ▸ https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc ▸ https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952 ▸ https://github.com/naturomics/CapsNet-Tensorflow ▸ https://www.zhihu.com/question/67287444/answer/251460831 ▸ https://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/clyj4jv/ ▸ Geoffrey Hinton: "Does the Brain do Inverse Graphics?” (https://www.youtube.com/watch? v=TFIMqt0yT2I&feature=youtu.be) ▸ Geoffrey Hinton talk "What is wrong with convolutional neural nets ?” (https://www.youtube.com/watch? v=rTawFwUvnLE&t=1214s) ▸ https://www.youtube.com/watch?v=u50nqWMQe1k 30
END OF DOCUMENT 31

Recommandé

Capsule Networks
Capsule NetworksCapsule Networks
Capsule Networks
Charles Martin
 
Introduction to Capsule Networks (CapsNets)
Introduction to Capsule Networks (CapsNets)Introduction to Capsule Networks (CapsNets)
Introduction to Capsule Networks (CapsNets)
Aurélien Géron
 
Dynamic Routing Between Capsules
Dynamic Routing Between CapsulesDynamic Routing Between Capsules
Dynamic Routing Between Capsules
Kyuhwan Jung
 
KNN Algorithm - How KNN Algorithm Works With Example | Data Science For Begin...
KNN Algorithm - How KNN Algorithm Works With Example | Data Science For Begin...KNN Algorithm - How KNN Algorithm Works With Example | Data Science For Begin...
KNN Algorithm - How KNN Algorithm Works With Example | Data Science For Begin...
Simplilearn
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Convolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsConvolutional Neural Network and Its Applications
Convolutional Neural Network and Its Applications
Kasun Chinthaka Piyarathna
 
Batch normalization presentation
Batch normalization presentationBatch normalization presentation
Batch normalization presentation
Owin Will
 

Recommandé

Capsule Networks
Capsule NetworksCapsule Networks
Capsule Networks
Charles Martin
 
Introduction to Capsule Networks (CapsNets)
Introduction to Capsule Networks (CapsNets)Introduction to Capsule Networks (CapsNets)
Introduction to Capsule Networks (CapsNets)
Aurélien Géron
 
Dynamic Routing Between Capsules
Dynamic Routing Between CapsulesDynamic Routing Between Capsules
Dynamic Routing Between Capsules
Kyuhwan Jung
 
KNN Algorithm - How KNN Algorithm Works With Example | Data Science For Begin...
KNN Algorithm - How KNN Algorithm Works With Example | Data Science For Begin...KNN Algorithm - How KNN Algorithm Works With Example | Data Science For Begin...
KNN Algorithm - How KNN Algorithm Works With Example | Data Science For Begin...
Simplilearn
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Convolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsConvolutional Neural Network and Its Applications
Convolutional Neural Network and Its Applications
Kasun Chinthaka Piyarathna
 
Batch normalization presentation
Batch normalization presentationBatch normalization presentation
Batch normalization presentation
Owin Will
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep Learning
Mohamed Loey
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
佳蓉 倪
 
1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...
1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...
1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...
PyData
 
ViT (Vision Transformer) Review [CDM]
ViT (Vision Transformer) Review [CDM]ViT (Vision Transformer) Review [CDM]
ViT (Vision Transformer) Review [CDM]
Dongmin Choi
 
Deep Generative Models
Deep Generative ModelsDeep Generative Models
Deep Generative Models
Mijung Kim
 
Geometric Deep Learning
Geometric Deep Learning Geometric Deep Learning
Geometric Deep Learning
PetteriTeikariPhD
 
CNN Machine learning DeepLearning
CNN Machine learning DeepLearningCNN Machine learning DeepLearning
CNN Machine learning DeepLearning
Abhishek Sharma
 
Lecture11 - neural networks
Lecture11 - neural networksLecture11 - neural networks
Lecture11 - neural networks
Albert Orriols-Puig
 
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Preferred Networks
 
Variational Autoencoder
Variational AutoencoderVariational Autoencoder
Variational Autoencoder
Mark Chang
 
Introduction to capsule networks
Introduction to capsule networksIntroduction to capsule networks
Introduction to capsule networks
Andrii Babii
 
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Transfer learning-presentation
Transfer learning-presentationTransfer learning-presentation
Transfer learning-presentation
Bushra Jbawi
 
Support Vector Machines for Classification
Support Vector Machines for ClassificationSupport Vector Machines for Classification
Support Vector Machines for Classification
Prakash Pimpale
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketaki
Ketaki Patwari
 
Deep Learning - Overview of my work II
Deep Learning - Overview of my work IIDeep Learning - Overview of my work II
Deep Learning - Overview of my work II
Mohamed Loey
 
Graph neural networks overview
Graph neural networks overviewGraph neural networks overview
Graph neural networks overview
Rodion Kiryukhin
 
Pr045 deep lab_semantic_segmentation
Pr045 deep lab_semantic_segmentationPr045 deep lab_semantic_segmentation
Pr045 deep lab_semantic_segmentation
Taeoh Kim
 
Graph Neural Network in practice
Graph Neural Network in practiceGraph Neural Network in practice
Graph Neural Network in practice
tuxette
 
Transfer Learning
Transfer LearningTransfer Learning
Transfer Learning
Hichem Felouat
 
[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo
JaeJun Yoo
 
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
NAVER Engineering
 

Contenu connexe

Tendances

Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 

Tendances (20)

Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep Learning
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
 
1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...
1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...
1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...
 
ViT (Vision Transformer) Review [CDM]
ViT (Vision Transformer) Review [CDM]ViT (Vision Transformer) Review [CDM]
ViT (Vision Transformer) Review [CDM]
 
Deep Generative Models
Deep Generative ModelsDeep Generative Models
Deep Generative Models
 
Geometric Deep Learning
Geometric Deep Learning Geometric Deep Learning
Geometric Deep Learning
 
CNN Machine learning DeepLearning
CNN Machine learning DeepLearningCNN Machine learning DeepLearning
CNN Machine learning DeepLearning
 
Lecture11 - neural networks
Lecture11 - neural networksLecture11 - neural networks
Lecture11 - neural networks
 
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
 
Variational Autoencoder
Variational AutoencoderVariational Autoencoder
Variational Autoencoder
 
Introduction to capsule networks
Introduction to capsule networksIntroduction to capsule networks
Introduction to capsule networks
 
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
 
Transfer learning-presentation
Transfer learning-presentationTransfer learning-presentation
Transfer learning-presentation
 
Support Vector Machines for Classification
Support Vector Machines for ClassificationSupport Vector Machines for Classification
Support Vector Machines for Classification
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketaki
 
Deep Learning - Overview of my work II
Deep Learning - Overview of my work IIDeep Learning - Overview of my work II
Deep Learning - Overview of my work II
 
Graph neural networks overview
Graph neural networks overviewGraph neural networks overview
Graph neural networks overview
 
Pr045 deep lab_semantic_segmentation
Pr045 deep lab_semantic_segmentationPr045 deep lab_semantic_segmentation
Pr045 deep lab_semantic_segmentation
 
Graph Neural Network in practice
Graph Neural Network in practiceGraph Neural Network in practice
Graph Neural Network in practice
 
Transfer Learning
Transfer LearningTransfer Learning
Transfer Learning
 

Similaire à capsule network

캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
NAVER Engineering
 
(Research Note) Delving deeper into convolutional neural networks for camera ...
(Research Note) Delving deeper into convolutional neural networks for camera ...(Research Note) Delving deeper into convolutional neural networks for camera ...
(Research Note) Delving deeper into convolutional neural networks for camera ...
Jacky Liu
 
Distributed Systems Theory for Mere Mortals - Software Craftsmanship Turkey
Distributed Systems Theory for Mere Mortals - Software Craftsmanship TurkeyDistributed Systems Theory for Mere Mortals - Software Craftsmanship Turkey
Distributed Systems Theory for Mere Mortals - Software Craftsmanship Turkey
Ensar Basri Kahveci
 
"Designing CNN Algorithms for Real-time Applications," a Presentation from Al...
"Designing CNN Algorithms for Real-time Applications," a Presentation from Al..."Designing CNN Algorithms for Real-time Applications," a Presentation from Al...
"Designing CNN Algorithms for Real-time Applications," a Presentation from Al...
Edge AI and Vision Alliance
 
Distance-based bias in model-directed optimization of additively decomposable...
Distance-based bias in model-directed optimization of additively decomposable...Distance-based bias in model-directed optimization of additively decomposable...
Distance-based bias in model-directed optimization of additively decomposable...
Martin Pelikan
 

Similaire à capsule network (20)

[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo
 
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
 
Illustrative Introductory CNN
Illustrative Introductory CNNIllustrative Introductory CNN
Illustrative Introductory CNN
 
Distributed deep learning
Distributed deep learningDistributed deep learning
Distributed deep learning
 
Capsule Networks
Capsule NetworksCapsule Networks
Capsule Networks
 
Distributed Systems Theory for Mere Mortals - Topconf Dusseldorf October 2017
Distributed Systems Theory for Mere Mortals - Topconf Dusseldorf October 2017Distributed Systems Theory for Mere Mortals - Topconf Dusseldorf October 2017
Distributed Systems Theory for Mere Mortals - Topconf Dusseldorf October 2017
 
Capsules Network Overview
Capsules Network OverviewCapsules Network Overview
Capsules Network Overview
 
Deep Neural Networks 
that talk (Back)… with style
Deep Neural Networks 
that talk (Back)… with styleDeep Neural Networks 
that talk (Back)… with style
Deep Neural Networks 
that talk (Back)… with style
 
Paper Reviews on Visual Attention
Paper Reviews on Visual AttentionPaper Reviews on Visual Attention
Paper Reviews on Visual Attention
 
02.cnn - CNN 파헤치기 3탄
02.cnn - CNN 파헤치기 3탄02.cnn - CNN 파헤치기 3탄
02.cnn - CNN 파헤치기 3탄
 
(Research Note) Delving deeper into convolutional neural networks for camera ...
(Research Note) Delving deeper into convolutional neural networks for camera ...(Research Note) Delving deeper into convolutional neural networks for camera ...
(Research Note) Delving deeper into convolutional neural networks for camera ...
 
Distributed Systems Theory for Mere Mortals - Software Craftsmanship Turkey
Distributed Systems Theory for Mere Mortals - Software Craftsmanship TurkeyDistributed Systems Theory for Mere Mortals - Software Craftsmanship Turkey
Distributed Systems Theory for Mere Mortals - Software Craftsmanship Turkey
 
Lausanne 2019 #2
Lausanne 2019 #2Lausanne 2019 #2
Lausanne 2019 #2
 
IRJET-Breast Cancer Detection using Convolution Neural Network
IRJET-Breast Cancer Detection using Convolution Neural NetworkIRJET-Breast Cancer Detection using Convolution Neural Network
IRJET-Breast Cancer Detection using Convolution Neural Network
 
Intro. Capsule network
Intro. Capsule networkIntro. Capsule network
Intro. Capsule network
 
"Designing CNN Algorithms for Real-time Applications," a Presentation from Al...
"Designing CNN Algorithms for Real-time Applications," a Presentation from Al..."Designing CNN Algorithms for Real-time Applications," a Presentation from Al...
"Designing CNN Algorithms for Real-time Applications," a Presentation from Al...
 
Distance-based bias in model-directed optimization of additively decomposable...
Distance-based bias in model-directed optimization of additively decomposable...Distance-based bias in model-directed optimization of additively decomposable...
Distance-based bias in model-directed optimization of additively decomposable...
 
Accelerating Science with Generative Adversarial Networks
Accelerating Science with Generative Adversarial NetworksAccelerating Science with Generative Adversarial Networks
Accelerating Science with Generative Adversarial Networks
 
Artificial Intelligence Applications in Petroleum Engineering - Part I
Artificial Intelligence Applications in Petroleum Engineering - Part IArtificial Intelligence Applications in Petroleum Engineering - Part I
Artificial Intelligence Applications in Petroleum Engineering - Part I
 
Neural Networks and Deep Learning: An Intro
Neural Networks and Deep Learning: An IntroNeural Networks and Deep Learning: An Intro
Neural Networks and Deep Learning: An Intro
 

Dernier

Delhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
shivangimorya083
 
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
shivangimorya083
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
amitlee9823
 
Generative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and MilvusGenerative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and Milvus
Timothy Spann
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceBDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
Delhi Call girls
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
amitlee9823
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Delhi Call girls
 
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
Call Girls in Nagpur High Profile Call Girls
 
FESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfFESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdf
MarinCaroMartnezBerg
 

Dernier (20)

Introduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptxIntroduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptx
 
Midocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFxMidocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFx
 
Delhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
 
100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx
 
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls CP 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
 
CebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxCebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptx
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
Generative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and MilvusGenerative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and Milvus
 
Invezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signalsInvezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signals
 
Data-Analysis for Chicago Crime Data 2023
Data-Analysis for Chicago Crime Data 2023Data-Analysis for Chicago Crime Data 2023
Data-Analysis for Chicago Crime Data 2023
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysis
 
Smarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptxSmarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptx
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Accredited-Transport-Cooperatives-Jan-2021-Web.pdf
Accredited-Transport-Cooperatives-Jan-2021-Web.pdfAccredited-Transport-Cooperatives-Jan-2021-Web.pdf
Accredited-Transport-Cooperatives-Jan-2021-Web.pdf
 
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceBDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
 
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
 
FESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfFESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdf
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptx
 

capsule network

  • 1. Dynamic Routing Between Capsules Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv LAB SEMINAR 1 2017.11.13 SNU DATAMINING CENTER MINKI CHUNG
  • 2. TABLE OF CONTENTS ▸ Intuition ▸ Problems of ConvNet ▸ How brain works, Inverse graphics ▸ Capsule Theory ▸ CapsNet ▸ Capsule ▸ CapsNet architecture ▸ Experiment ▸ Classification on MNIST ▸ Reconstruction on MNIST ▸ Dimension perturbation on MNIST ▸ Discussion 2
  • 3. INTUITION ▸ Problems of ConvNet ▸ How brain works, Inverse graphics ▸ Capsule Theory 3
  • 4. PROBLEMS OF CONVNET 4 ▸ ConvNet Architecture PROBLEMS IS ‘POOLING’ https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc Obtain translational, rotational invariance
  • 5. PROBLEMS OF CONVNET 5 ▸ @REDDIT, MACHINE LEARNING https://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/clyj4jv/
  • 6. PROBLEMS OF CONVNET 6 ▸ WHAT IS THIS PICTURE? https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952
  • 7. PROBLEMS OF CONVNET 7 ▸ HOW ABOUT THIS? https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952
  • 8. PROBLEMS OF CONVNET 8 ▸ NEED EQUIVARIANCE, NOT INVARIANCE https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952
  • 9. HOW BRAIN WORKS, INVERSE GRAPHICS 9 ▸ Constructing a visual image from some internal hierarchical representation of geometric data ▸ Internal representation is stored in computer’s memory as arrays of geometrical objects and matrices that represent relative positions and orientation of these objects ▸ Special software takes that representation and converts it into an image on the screen. This is called rendering ▸ Brains, in fact, do the opposite of rendering. Hinton calls it inverse graphics: Visual information received by eyes, they deconstruct a hierarchical representation of the world around us and try to match it with already learned patterns and relationships stored in the brain ▸ Key idea is that representation of objects in the brain does not depend on view angle COMPUTER GRAPHICS https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b
  • 10. CAPSULE THEORY 10 ▸ In 3D graphics, relationships between 3D objects can be represented by a so- called pose, which is in essence translation plus rotation ▸ Capsule approach: It incorporates relative relationships between objects (Internal representation) and it is represented numerically as a 4D pose matrix ▸ by ‘Dynamic Routing’ (more details later) ▸ allows capsules to communicate with each other and create representations similar to scene graphs in computer graphics https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b YOU CAN EASILY RECOGNIZE THAT THIS IS THE STATUE OF LIBERTY, EVEN THOUGH ALL THE IMAGES SHOW IT FROM DIFFERENT ANGLES
  • 11. CAPSULE THEORY 11 ▸ Benifits: ▸ Better understanding 3D Space ▸ Achieve state-of-the art performance by only using a fraction of the data that a CNN would use ▸ In order to learn to tell digits apart, the human brain needs only a couple of dozens of examples, hundreds at most, while CNN need tens of thousands of examples https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b
  • 12. CAPSNET ▸ Capsule ▸ CapsNet architecture ▸ Experiment 12
  • 13. CAPSULE 13 ▸ Comparison with traditional neuron https://www.zhihu.com/question/67287444/answer/251460831 V VEC LENGTH WORKS LIKE PROBABILITY ACTIVATION OF NEXT CAPSULE DYNAMIC ROUTING
  • 14. CAPSNET ARCHITECTURE 14 ARCHITECTURE Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules CONV CAPS.CONV CAPS.FC DYNAMIC ROUTING 8X 32 X MNIST LOCAL FEATURE DETECTION 6*6*32=1152 CAPSULES, EACH HAS 8 PROPERTIES 10 CAPSULES (CLASS), EACH HAS 16 PROPERTIES DEEPER MEANS MORE COMPLEX, DIMENSION SHOULD INCREASE
  • 15. CAPSNET ARCHITECTURE 15 ▸ naturomics github CAPSNET-TENSORFLOW CAPS.CONVCONV CONV X 32 MNIST X 8 https://github.com/naturomics/CapsNet-Tensorflow X 32 X 8 CAPS.FC CAPS.CONV CAPS.FC DYNAMIC ROUTING
  • 16. CAPSNET ARCHITECTURE 16 ▸ Place-coded Capsule ▸ Concatenate (=8 different regular conv layers) ▸ Consider each feature map as capsule (6*6*32=1152 capsules with 8 properties) CAPS.CONV, PRIMARYCAPS CAPS.CONV X 32 MNIST X 8 https://github.com/naturomics/CapsNet-Tensorflow DIRECTION
  • 17. CAPSNET ARCHITECTURE 17 ▸ Place-coded Capsule ▸ Concatenate (=8 different regular conv layers) ▸ Consider each feature map as capsule (6*6*32=1152 capsules with 8 properties) ▸ Use squashing function in the end CAPS.CONV, PRIMARYCAPS CAPS.CONV X 32 MNIST X 8 https://github.com/naturomics/CapsNet-Tensorflow
  • 18. CAPSNET ARCHITECTURE 18 ▸ Rate-coded capsules ▸ caps: 1152 → 10 ▸ vec-len: 8 → 16 ▸ Dynamic Routing CAPS.FC, DIGITCAPS https://github.com/naturomics/CapsNet-Tensorflow X 32 MNIST X 8 CAPS.FC DYNAMIC ROUTING DYNAMIC ROUTING
  • 19. CAPSNET ARCHITECTURE 19 ▸ Dynamic Routing ▸ Top-down feedback ▸ Routing by agreement ▸ Works like attention CAPS.FC, DIGITCAPS https://github.com/naturomics/CapsNet-Tensorflow IF MULTIPLE PREDICTIONS AGREE, HIGHER LEVEL CAPSULE BECOMES ACTIVE VEC LENGTH WORKS LIKE PROBABILITY ACTIVATION OF NEXT CAPSULE COUPLING COEFFICIENTS TOPDOWN FEEDBACK: IF RELATION EXISTS COUPLING COEFFICIENTS INCREASE AGREEMENT
  • 20. CAPSNET ARCHITECTURE 20 ▸ Dynamic Routing CAPS.FC, DIGITCAPS https://github.com/naturomics/CapsNet-Tensorflow X 32 MNIST X 8 CAPS.FC DYNAMIC ROUTING 3 ITERATIONS WILL DO
  • 21. EXPERIMENT ▸ Classification on MNIST ▸ Reconstruction on MNIST ▸ Dimension perturbation on MNIST 21
  • 22. EXPERIMENT 22 ▸ Introduce first three ▸ Classification on MNIST (99.75%, conv 99.61%) ▸ Reconstruction on MNIST ▸ Dimension Perturbation on MNIST ▸ Robustness to Affine Transformation on MNIST (79%, conv 66%) ▸ Classification on MultiMNIST (5% error) ▸ Classification on CIFAR 10 (10.6% error - ZFNet) ▸ Classification on SVHN (4.3% error) Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules
  • 23. EXPERIMENT 23 ▸ 99.75% (baseline 99.61%) 1. CLASSIFICATION ON MNIST Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules
  • 24. EXPERIMENT 24 ▸ 2. RECONSTRUCTION ON MNIST Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules
  • 25. EXPERIMENT 25 ▸ 3. DIMENSION PERTURBATION ON MNIST Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules
  • 27. _ 27 ▸ Capsule(Vector), ▸ Not conventional neuron(Scalar) NOVELTY
  • 28. _ 28 ▸ Still use regular conv layer at first for local feature extraction ▸ Capsule cannot extract local feature? STILL USE CONV LAYER HOW TO RESTRICT TO GET CERTAIN FEATURE? ▸ Disentangling features ▸ How to obtain ‘certain features’?
  • 30. REFERENCE ▸ Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, 10, 2017, Arxiv. Dynamic Routing Between Capsules (https:// arxiv.org/abs/1710.09829) ▸ Geoffrey Hinton et al., Matrix Capsules With EM Routing, Under review as a conference paper at ICLR 2018 (https:// openreview.net/pdf?id=HJWLfGWRb) ▸ https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b ▸ https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc ▸ https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952 ▸ https://github.com/naturomics/CapsNet-Tensorflow ▸ https://www.zhihu.com/question/67287444/answer/251460831 ▸ https://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/clyj4jv/ ▸ Geoffrey Hinton: "Does the Brain do Inverse Graphics?” (https://www.youtube.com/watch? v=TFIMqt0yT2I&feature=youtu.be) ▸ Geoffrey Hinton talk "What is wrong with convolutional neural nets ?” (https://www.youtube.com/watch? v=rTawFwUvnLE&t=1214s) ▸ https://www.youtube.com/watch?v=u50nqWMQe1k 30