Layers in Deep Learning&Caffe layers (model architecture )
1. Layers in Deep Learning
&
Caffe layers
(model architecture )
Farshid PirahanSiah
Dec 2016
2. Layers in Deep Learning
• Convolution
• 2D
• Activation
• Sigmoid
• Tanh
• ReLU
• Pooling
• Max
• Average
• elementwise
• Sum
• Product
• Max
• Two tensors
• LRN
• Cross-channel
• Fully-connected
• With bias
• Without bias
• softMax
• Cross-channel
• Deconvolution
• Output
• Prob
• Softmax
3. Layers in Caffe
• Vision Layers
• particular operation to some region of the input to produce a corresponding region of
the output.
• other layers (with few exceptions) ignore the spatial structure of the input
• Loss Layers
• comparing an output to a target and assigning cost to minimize.
• Activation / Neuron Layers
• element-wise operators, taking one bottom blob and producing one top blob of the
same size
• Data Layers
• (LevelDB or LMDB), directly from memory, or, when efficiency is not critical, from files
on disk in HDF5 or common image formats
• Common Layers
4. Layers in Caffe
• Vision Layers
• Convolution
• Pooling
• Local Response Normalization (LRN)
• Im2col
• Loss Layers
• Softmax
• Sum-of-Squares / Euclidean
• Hinge / Margin
• Sigmoid Cross-Entropy
• Infogain
• Accuracy and Top-k
• Activation / Neuron Layers
• ReLU / Rectified-Linear and Leaky-ReLU
• Sigmoid
• TanH / Hyperbolic Tangent
• Absolute Value
• Power
• BNLL
• Data Layers
• Database
• In-Memory
• HDF5 Input
• HDF5 Output
• Images
• Windows
• Dummy
• Common Layers
• Inner Product
• Splitting
• Flattening
• Reshape
• Concatenation
• Slicing
• Elementwise Operations
• Argmax
• Softmax
• Mean-Variance Normalization
5. Layer Activations
• “Activation functions transform the weighted sum of inputs that goes into
the artificial neurons. “
• “These functions should be non-linear to encode complex patterns of the
data. “