12. Environment Setting
We bought …
GTX 980(4GB)
Power(600W)
740 thousand ₩
We bought …
Motherboard, CPU,
SSD, Case
690 thousand ₩
Our Motherboard was
Micro-ATX, So …
13. Environment Setting
We bought …
GTX 980(4GB)
Power(600W)
740 thousand ₩
We bought …
Motherboard, CPU,
SSD, Case
690 thousand ₩
Our Motherboard was
Micro-ATX, So …
Memory was only 4GB
So…
14. Environment Setting
We bought …
GTX 980(4GB)
Power(600W)
740 thousand ₩
Our Motherboard was
Micro-ATX, So …
We bought …
Motherboard, CPU,
SSD, Case
690 thousand ₩
Memory was only 4GB
So…
We bought …
Memory 16GB
128 thousand ₩
15. Environment Setting
We bought …
GTX 980(4GB)
Power(600W)
740 thousand ₩
We bought …
Motherboard, CPU,
SSD, Case
690 thousand ₩
We bought …
Memory 16GB
128 thousand ₩
Our Motherboard was
Micro-ATX, So …
Memory was only 4GB
So…
16. Setting Deep Learning Toolbox
Keras(ver0.1): Theano-based library Caffe: fastest, nice framework
Python, No Configuration file(!) C++, protobuf
19. Data source
All labeled 20 styles
60% training, 20% validation, 20% test
Positive example : clean
Negative example : not clean
80,000 images
Curated by Flickr Group[Kerayev. et.al]
22. Convolutional Neural Network
We made theano-based CNN model and tested it
too slow, need more data
128 batch, SGD optimize
INPUT(256x256)
CONV(32,3x3)
CONV(32,3x3)
MAXPOOLING(4x4)
CONV(32,3x3)
CONV(32,3x3)
MAXPOOLING(4x4)
FC(8192-2048)
FC(2048-20)
SOFTMAX
Our CNN Model (keras)
23. Convolutional Neural Network
CaffeNet without fine-tuning
too slow, need more data (!)
128 batch, SGD optimize
INPUT(256x256)
CONV(48,2x2)
MAXPOOLING(2x2)
CONV(128,3,3)
MAXPOOLING(2x2)
CONV(192,3x3)
CONV(192,3x3)
CONV(128,3x3)
MAXPOOLING(2x2)
FC(10,816-4,096)
FC(4,096-4,096)
FC(4,096-2000)
SOFTMAX
CaffeNet (Modified AlexNet)
24. Convolutional Neural Network
No Fine-tuning, No Future
Let’s not contribute to global warming
128 batch, SGD optimize
INPUT(256x256)
CONV(48,2x2)
MAXPOOLING(2x2)
CONV(128,3,3)
MAXPOOLING(2x2)
CONV(192,3x3)
CONV(192,3x3)
CONV(128,3x3)
MAXPOOLING(2x2)
FC(10,816-4,096)
FC(4,096-4,096)
FC(4,096-2000)
SOFTMAX
CaffeNet (Modified AlexNet)
25. Convolutional Neural Network
Fine-tuning CNN (1)
Dataset MIT Places Database (2.5 million images, 205 category)
Model Deeply Supervised Net (DSN)
Deeply-Supervised Nets [Chen-Yu Lee, Saining Xie]
26. Convolutional Neural Network
Fine-tuning CNN (2)
Dataset ImageNet (1.2 million images into 1000 category)
Model CaffeNet (replication of AlexNet)
Caffe: Convolutional Architecture for Fast Feature Embedding [Yangqing Jia, Evan Shelhamer]
27. Convolutional Neural Network
Fine-tuning CNN (3)
Dataset ImageNet (1.2 million images into 1000 category)
Model VGG 16-layer Net
VERY DEEP CONVOLUTIONAL NETWORKS FOR LARGE-SCALE IMAGE RECOGNITION [Karen Simonyan, Andrew Zisserman]
30. Hand-crafted Features
GIST Color Histogram
3 scale, {8, 8, 4} orientation
4 x 4 grid
960 dimension
8 bins, 1 patch for Channel
48 dimension
Color Variance
4 x 4 grid for each channel
48 dimension
40. Final Touch
Now we have every building blocks
Any remaining variation?
Classifier
pre-training
Top5 Score
Handcraft Features
ConvNet Features
Bright
Romantic
Serene
Sunny
Macro
Color Histogram
GIST Descriptor
Color Variance
Datasets
41. Final Touch
Let’s compare combination of features
Classifier
pre-training
Top5 Score
Handcraft Features
ConvNet Features
Bright
Romantic
Serene
Sunny
Macro
Color Histogram
Color Variance
Datasets
GIST Descriptor
42. Final Touch
Let’s compare combination of features
CNN (4,096) GIST Color Histogram Color Variance Acc
COMB 1 O O O O 0.3945
COMB 2 O O O 0.3941
COMB 3 O O 0.3949
COMB 4 O 0.3942
Accuracy for various feature combinations
CNN GIST
4,096 960
1 by 5,056 Feature Vector
Final feature combination
Top1 Accuracy. 0.395 > 0.368 (benchmark)