Laplacian Pyramid of Generative Adversarial Networks (LAPGAN) - NIPS2015読み会 #nips_yomi
1. 1
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
NIPS2015 読み会
Deep Generative Image Models using a
Laplacian Pyramid of Adversarial Networks
濱田晃一
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
@hamadakoichi
論文紹介
2016/1/20
2. 2
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
4. 4
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
http://papers.nips.cc/paper/5773-deep-generative-image-models-using-a-laplacian-pyramid-of-adversarial-networks.pdf
5. 5
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
各周波数帯ごとの Generative Adversarial Network を作り
解像度を上げていく画像生成を実現する
低解像度
画像
高解像度
画像
元画像 生成画像サンプル
Laplacian Pyramid of Generative Adversarial Networks(LAPGAN)
6. 6
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of Generative Adversarial Networks(LAPGAN)
各周波数帯ごとの Generative Adversarial Network を作り
解像度を上げていく画像生成を実現する
学習
生成
7. 7
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of Generative Adversarial Networks(LAPGAN)
Visual Turing Test : 人が画像を見て、本物か生成画像か評価
40%の生成画像に関して、本物画像と見間違う
Visual Turing Test (CIFAR10)
40%
本物画像と判定された割合
8. 8
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
http://soumith.ch/eyescream/
Project Page
Laplacian Pyramid of Generative Adversarial Networks(LAPGAN)
9. 9
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Poster発表 & Deep Learning Symposiumでの Oral発表
Posterでも19時から24時までずっと人だかりができ続けていた
Laplacian Pyramid of Generative Adversarial Networks(LAPGAN)
10. 10
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
11. 11
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
12. 12
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Generative Adversarial Network (GAN)
Generative Adversarial Nets(GAN)
Goodfellow+, NIPS2014
13. 13
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Generative Adversarial Network (GAN)
Generator(生成器)と Discriminator(識別器)を戦わせ
生成精度を向上させる
識別器: “本物画像”と “生成器が作った偽画像”を識別する
生成器: 生成画像を識別器に“本物画像”と誤識別させようとする
(Goodfellow+, NIPS2014, Deep Learning Workshop, Presentation)
14. 14
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Generative Adversarial Network (GAN)
Minimax Objective function
Discriminator が
「本物画像」を「本物」と識別
(Goodfellow+, NIPS2014, Deep Learning Workshop, Presentation)
Discriminator が
「生成画像」を「偽物」と識別する
Discriminatorは
正しく識別しようとする
(最大化)
Generatorは Discriminator に誤識別させようとする(最小化)
Generator(生成器)と Discriminator(識別器)を戦わせ
生成精度を向上させる
15. 15
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Generative Adversarial Network (GAN)
SGDで Discriminator k回、Generator 1回と交互に学習
MCMC等のSampling必要なく、Back Propagation で解ける
Discriminatorでの
最大化(k 回)
Generator での
最小化(1回)
16. 16
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
17. 17
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Conditional Generative Adversarial Network (GAN)
Minimax Objective function
Conditional Generative Adversarial Nets(CGAN)
Mirza+, 2014, Gauthier, 2014
18. 18
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
19. 19
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of GAN(LAPGAN)
各周波数帯の画像特徴ごとにGANを学習
周波数帯ごとの画像特徴を生成・合成し、解像度を上げていく
学習
生成
20. 20
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of GAN(LAPGAN)
各周波数帯ごとのGANを生成し
解像度を上げた画像生成をしていく
低解像度
画像
高解像度
画像
元画像 生成画像サンプル
21. 21
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid (Burt & Adelson, 1983)
各周波数帯の画像特徴
downsampling
upsampling
Low pass filtered images
Band-pass filtered images
http://eric-yuan.me/image-pyramids/
22. 22
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of GAN(LAPGAN)
Laplacian Pyramid で各周波数帯ごとに
低周波数画像が与えられたときの Conditional GANを学習
学習 ・Generator と Discriminator は ConvNet
・Generator は低周波数画像から高周波数画像の生成を学習する
Minimax Objective function
23. 23
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of GAN(LAPGAN)
周波数帯ごとの画像特徴を生成・合成し
高解像度の画像を生成していく
生成
24. 24
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of GAN(LAPGAN)
Class Conditional LAPGAN
http://soumith.ch/eyescream/
1 hot coding
of class label
25. 25
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of GAN(LAPGAN)
Model Architecture
10万画像
・CIFAR10: 32x32 pixel, 10万画像, 10 Class
・STL10 : 96x96 pixel, 10万画像, 10 Class
26. 26
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Laplacian Pyramid of GAN(LAPGAN)
Model Architecture
1000万画像
・LSUN: 64x64pixelにDownsample, 1000万画像, 10 Class
27. 27
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Experiments
画像生成
低解像度
画像
高解像度
画像
元画像 生成画像サンプル
28. 28
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Experiments
Visual Turing Test : 人に画像を見せ本物か生成画像か評価
40%の生成画像に関して、本物画像と見間違う
Log Likelihood
Visual Turing Test (CIFAR10)
40%
本物画像と判定された割合
29. 29
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Experiments
生成画像のModel比較
CIFAR10
最近傍の
実画像
生成画像
CC-LAPGAN: Class Conditional LAPGAN
30. 30
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Experiments
Class Conditional LAPGAN での各Classの画像生成
31. 31
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Experiments
256x256 pixel 画像を 再帰的に生成
http://soumith.ch/eyescream/
32. 32
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
33. 33
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Deep Convolutional GAN (DCGAN)
Unsupervised Representation Learning with Deep Convolutional
Generative Adversarial Networks(Radford+, arXiv:1511.06434)
Convolution/DecomvolutionのGAN
34. 34
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Deep Convolutional GAN (DCGAN)
クリアな画像生成
Unsupervised Representation Learning with Deep Convolutional
Generative Adversarial Networks(Radford+, arXiv:1511.06434)
画像演算
クリアな画像生成・潜在空間での画像演算
35. 35
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Deep Convolutional GAN (DCGAN)
・顔イラスト生成:https://github.com/mattya/chainer-DCGAN
・漢字生成:http://genekogan.com/works/a-book-from-the-sky.htmlse characters
・花生成:https://twitter.com/vintermann/status/675599478494208000
DCGAN応用・適用例
顔イラスト 花生成
36. 36
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Variational AutoEncoder GAN (VAEGAN)
Autoencoding beyond pixels using a learned similarity
metric(Larsen+, arXiv:1512.09300)
VAE + GAN
37. 37
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Variational AutoEncoder GAN (VAEGAN)
Autoencoding beyond pixels using a learned similarity
metric(Larsen+, arXiv:1512.09300)
VAEGAN with visual attribute vectors
38. 38
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
39. 39
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Deep Visual Analogy Making
Deep Visual Analogy Making (Reed+)
変換則の学習
40. 40
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Deep Convolutional Inverse Graphics Network
Deep Convolutional Inverse Graphics Network(Kulcarni+)
VAEのConvolution拡張。3Dモデルの連続変換を学習
41. 41
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
42. 42
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Summary: LAPGAN
各周波数帯ごとの Generative Adversarial Network を作り
解像度を上げていく画像生成を実現する
学習
生成
43. 43
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
各周波数帯ごとの Generative Adversarial Network を作り
解像度を上げていく画像生成を実現する
低解像度
画像
高解像度
画像
元画像 生成画像サンプル
Summary: LAPGAN
44. 44
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Summary: LAPGAN
Visual Turing Test : 人が画像を見て、本物か生成画像か評価
40%の生成画像に関して、本物画像と見間違う
Visual Turing Test (CIFAR10)
40%
本物画像と判定された割合
45. 45
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References
46. 46
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Reference
-Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y.
(2014). Generative adversarial nets. In NIPS 2014.
-Gauthier, J. (2014). Conditional generative adversarial nets for convolutional face generation. Class
Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter semester 2014.
-Mirza, M. and Osindero, S.(2014). Conditional generative adversarial nets. arXiv preprint, arXiv:1411.1784.
-Denton, E., Chintala, S., Szlam, A., and Fergus, R.(2015) Deep Generative Image Models using a Laplacian
Pyramid of Adversarial Networks. In NIPS 2015.
Conditional GAN
GAN
LAPGAN
Recent GANs
-Larsen, A. B. L., Sønderby, S. K., Winther, O.(2015). Autoencoding beyond pixels using a learned
similarity metric. arXiv preprint, arXiv:1512.09300.
-Radford, A., Metz, L., and Chintala, S.(2015)Unsupervised Representation Learning with Deep Convolutional
Generative Adversarial Networks. arXiv preprint, arXiv:1511.0643.
-Reed, S., Zhang, Y., Zhang, Y., and Lee, H.(2015) Deep Visual Analogy-Making. In NIPS 2015.
-Kulkarni, T. D., Whitney, W., Kohli, P., Tenenbaum, J. B.(2015). Deep Convolutional Inverse Graphics
Network. In NIPS 2015.
Other Generative Image Models in NIPS 2015
47. 47
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
AGENDA
◆Laplacian Pyramid of GAN(LAPGAN)
◆Recent extended GANs
◆Other Deep Generative Models in NIPS2015
◆Generative Adversarial Network(GAN)
◆Conditional GAN(CGAN)
◆Summary
◆References