6. u(l+1)
= W(l+1)
z(l)
+ b(l+1)
z(l+1)
= f(u(l+1)
)
z(1)
= x
x2
x783
x784
x1
x0
z10u10
z1u1
z2u1
z9u9
z100
0
u1000
z999u999
u1 z1
u2 z2
z0
z100
0
u1000
z999u999
u1 z1
u2 z2
z0
Neural Network
(1)
= x
z(l+1)
= f(u(l+1)
)
bW
M
1
N
1
N
M
1
N
W, b
y = softmax(z(L)
)
(l = 1, 2, · · · , L 1, L)
softmaxk(z(L)
) =
exp(u
(L)
k )
PK
j=1 exp(u
(L)
j )
L(w) =
NY
n=1
KY
k=1
(yk(x, w))
7.
8. x =
✓
x1
x2
◆
W(1)
=
w
(1)
11 w
(1)
12
w
(1)
21 w
(1)
22
!
W(2)
=
w
(2)
11 w
(2)
12
w
(2)
21 w
(2)
22
!
b(1)
=
b
(1)
1
b
(1)
2
!
b(2)
=
b
(2)
1
b
(2)
2
!
https://github.com/matsuken92/Qiita_Contents/blob/master/General/mathcafe_20161212.ipynb
9. x =
✓
x1
x2
◆
W(1)
=
w
(1)
11 w
(1)
12
w
(1)
21 w
(1)
22
!
W(2)
=
w
(2)
11 w
(2)
12
w
(2)
21 w
(2)
22
!
b(1)
=
b
(1)
1
b
(1)
2
!
b(2)
=
b
(2)
1
b
(2)
2
!
✔
https://github.com/matsuken92/Qiita_Contents/blob/master/General/mathcafe_20161212.ipynb