19. Double backprop
x F z
y
・・・ L
class F(FunctionNode):
def forward(self, x, y):
return x * x + y
def backward(self, x, y, gz):
return 2 * gz * x, gz
NumPy, CuPy
Note: The interface is simplified from actual implementation.
chainer.Variable
-> Creates CG
19
20. Double backprop
x F z
y
gx Grad F gz
gy
・・・ L
Backprop!
=∂L/∂z=∂L/∂x
=∂L/∂y
1.0
=∂L/∂L
Mul
x
gz
y
gx
gy
*2
20
25. Double backprop
x f z
gxGrad f
Computes the differentiation of L = G(f(x), ∇f(x)) with respect to x
L = G(f(x), ∇f(x))
25
26. Double backprop
x f z
gxGrad f
・・・ L
Computes the differentiation of L = G(f(x), ∇f(x)) with respect to x
L = G(f(x), ∇f(x))
26
27. Double backprop
x f z
gxGrad f
・・・ L
Backprop!
ggx
Double
Grad f
∂L/∂x
1.0gzGrad f
Computes the differentiation of L = G(f(x), ∇f(x)) with respect to x
L = G(f(x), ∇f(x))
27