26. 教師あり学習(supervised learning)
入力データ x と教師信号 t のペアが学習対象
出力 y が t と一致するように {w,θ} を調整
y
x1
w2
w1
t
x2
x2
x1(0,0) (1,0)
(0,1)
(1,1)
AND 素子
1 →
1 → 青だと思います
赤です
34. Perceptronの学習(1): 誤り訂正学習
Hebb 学習則
教師 t と出力 y の関係により w を修正
目標: {φn, tn} が与えられたとき y(φn) = tn としたい
正解 tn と答え yn が不一致のときのみパラメータを修正
解が存在する場合,正しい解に収束する
yφ
w
t
46. 多層ネットワークの学習の難しさ
誤り訂正教師信号 t は1階層であれば対応可能
中間層 z1, z2 に対する教師信号はどう生成する?
u
yx1 y
x2
Σ
u
θ
w1
w2
u
yx1 y
x2
Σ
u
θ
w1
w2
u
yx1 y
x2
Σ
u
θ
w1
w2
x1
x2
z1
z2
y
tz1
θ
z2
θ
u
u
w1
(2)
w2
(2)
OK!
w11
(1)
w22
(1)
w12
(1)
w21
(1)
NG!
47. Error Back-propagation
単純 Perceptron の線形分離問題
→ 階層性による打破
多層Perceptron (MLP)の学習則
基本アイディアは勾配法
微分の連鎖則を活用
x0
x1
xD
z0
z1
zM
y1
yK
w
(1)
MD
w
(2)
KM
w
(2)
10
hidden units
inputs outputs
51. MLP の勾配学習
MLP の勾配学習に線形性ではなく微分可能性では?
微分の連鎖則(chain-rule) を適用
多層に意味を持たせるためには
微分可能な非線形活性化関数であれば良い
u
yx1 y
x2
Σ
u
θ
w1
w2
u
yx1 y
x2
Σ
u
θ
w1
w2
u
yx1 y
x2
Σ
u
θ
w1
w2
x1
x2
z1
z2
y
tw11
(1)
w22
(1)
w12
(1)
w21
(1)
w1
(2)
w2
(2)
z1
u
z2
u
y
u
@E(w)
@w(1)
22
@E(w)
@w(2)
2
z1
u
z2
u
y
u
62. 視覚野(Ventral pathway)の性質
視覚野: 階層構造を持ち,階層ごとに異なる視覚課題の解決
初期視覚野: 狭い受容野,単純な特徴抽出
Simple Cell,Complex Cellの存在
高次視覚野: 広い受容野,中程度に複雑な特徴に選択的
V1
V2
V4
PITCIT
Ventral Pathway
AIT
TEO
TE
V1
V2
V3 VP
V4 MT VA/V4
PIT
AIT/CIT 8 TF
LIP MST DPL VIP
7a
V3A
V1
V4
V2
IT
Small receptive field
Edge, Line segment
detector
Large receptive field
Face, Complex feature
detector
?
?
[Felleman+91, DiCarlo+12, Kruger+13]
uition of basic (mostly biological) terms used
ng sections. Most data we present in the
obtained from macaque monkeys because
siological knowledge stems from investiga-
primate brain consists of approximately
eas, the human brain probably contains as
reas.3
There is a general consensus that the
ry and motor areas in the monkey are
the corresponding areas in the human brain.
everal other cortical areas in the monkey have
FOR COMPUTER VISION? 1849
ations (summarized from [44]). Box and font sizes are
In summary, in this paper we want to argue that deep
hierarchies are an appropriate concept to achieve a general,
robust, and versatile computer vision system. Even more
importantly, we want to present relevant insights about the
hierarchical organization of the primate visual system for
computer vision scientists in an accessible way. We are
aware that some of our abstractions are rather crude from
the neurophysiological point of view and that we have left
out important details of the processes occurring at the
different levels,2
but we hope that such abstractions and the
holistic picture given in this paper will help to foster
productive exchange between the two fields.
The paper is organized as follows: In Section 2, we will
touch upon the aspects of the primate visual system that are
relevant to understand and model the processing hierarchy.
also give an intuition of basic (mostly biological) terms used
in the following sections. Most data we present in the
following were obtained from macaque monkeys because
most neurophysiological knowledge stems from investiga-
tions on these.
While the primate brain consists of approximately
100 cortical areas, the human brain probably contains as
many as 150 areas.3
There is a general consensus that the
primary sensory and motor areas in the monkey are
homologous to the corresponding areas in the human brain.
Furthermore, several other cortical areas in the monkey have
an identified homologue in the human (e.g., MT/MST,
Anterior Intraparietal Area (AIP)). These areas can be viewed
as landmarks that can be used to relate other cortical areas in
KRU¨ GER ET AL.: DEEP HIERARCHIES IN THE PRIMATE VISUAL CORTEX: WHAT CAN WE LEARN FOR COMPUTER VISION? 1849
Fig. 2. Simplified hierarchical structure of the primate’s visual cortex and approximate area locations (summarized from [44]). Box and font sizes are
relative to the area size.
64. 初期視覚野の性質
線分やエッジなどの成分に反応
Simple cell: 方位,位相に敏感
Complex cell: 位相には許容的
Simple Cell
Phase Sensitive
Orientation Selective
Receptive Field
Input Stimulus
Fire Not FireNot Fire
Phase InsensitiveComplex Cell
Receptive Field
Input Stimulus
Fire Not FireFire
V1
V2
V4
PITCIT
Ventral Pathway
AIT
TEO
TE
V1
V4
V2
IT
Small receptive field
Edge, Line segment
detector
Large receptive field
Face, Complex feature
detector
?
?
Simple Cell
Phase Sensitive
Orientation Selective
Receptive Field
Input Stimulus
Fire Not FireNot Fire
Phase InsensitiveComplex Cell
Receptive Field
Input Stimulus
Fire Not FireFire
65. Hubel-Wiesel 階層仮説
Simple Cell の出力合成で,
Complex cell は説明可能
(Hubel & Wiesel 59)
Simple Cell
Phase Sensitive
Orientation Selective
Receptive Field
Input Stimulus
Fire Not FireNot Fire
Phase InsensitiveComplex Cell
Receptive Field
Input Stimulus
Fire Not FireFire