Author: Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, Cheng-Lin Liu Origin: https://arxiv.org/abs/1708.05552 Related: https://github.com/number9473/nn-algorithm/issues/262 > BlockQNN

1. Practical Block-wise Neural
Network Architecture
Generation
Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, Cheng-Lin Liu. CVPR 2018.
Yu Kai Huang

2. Outline
● Neural Network Architecture Generation
● BlockQNN: Automated Network Design Framework
○ Q-Learning
○ Reward Shaping
○ Early Stop Strategy
○ Epsilon-greedy Strategy, Experience Replay
● Experiments
○ Comparison with
■ hand-crafted networks
■ auto-generated networks
○ Generalizability
■ CIFAR-10, CIFAR-100
■ ImageNet

3. Neural Network Architecture Generation

4. “... a high-performance network architecture generally
possesses a tremendous number of possible configurations
about the number of layers, hyperparameters in each layer
and type of each layer.”

5. “It is hence infeasible for manually exhaustive search, and the
design of successful hand-crafted networks heavily rely on
expert knowledge and experience.”

6. “constructing the network in a smart and automatic manner
remains an open problem.”

7. “... some recent works have attempted computer-aided or
automated network design [2, 37], there are several
challenges still unsolved”
● huge search space and heavy computational costs
● hard to transfer to other tasks or generalize to another
dataset with different input data sizes.

8. BlockQNN: Automated Network Design
Framework

9. “... we introduce the block-wise network generation, i.e., we
construct the network architecture as a flexible stack of
personalized blocks rather tedious per-layer network piling.”
● the generated network owns a powerful generalization to
other task domains or different datasets.

11. Designing Network Blocks With Q-Learning
action
state
reward

12. Network Structure Code (NSC)
● 5-D NSC vector:
○ (layer index, operation type, kernel size, Pred1, Pred2)

13. Q-learning process

14. Q-learning process
“We model the layer selection process as a Markov Decision Process ... To find
the optimal architecture, we ask our agent to maximize its expected reward over
all possible trajectories ...”

15. Q-learning process
“For this maximization problem, it is usually to use recursive Bellman Equation to
optimality.”
“... we define the maximum total expected reward to be Q∗(st, a) ...”
α: learning rate
γ: discount factor
rt: intermediate reward
st: current state
sT: final state

16. Shaped Intermediate Reward
“Since the reward rt cannot be explicitly measured in our task, we use reward
shaping [22] to speed up training.”

17. Early Stop Strategy

18. Early Stop Strategy
“... which means that some good blocks unfortunately perform worse than bad
blocks when stop training early.”

19. Early Stop Strategy
“we notice that the FLOPs and density of the corresponding blocks have a
negative correlation with the final accuracy. Thus, we redefine the reward function
as:”
FLOPs: computational complexity of the
block.
Density: the edge number divided by the
dot number in DAG of the block.
µ and ρ: balance the weights of FLOPs and
Density.

20. Training Details
● Epsilon-greedy Strategy
○ “With epsilon-greedy strategy, the random action is taken with probability ϵ and the greedy
action is chosen with probability 1-ϵ.”
● Experience Replay
○ “Within a given interval, i.e. each training iteration, the agent samples 64 blocks with their
corresponding validation accuracies from the memory and updates Q-value 64 times.”
● BlockQNN Configuration
○ learning rate α, discount factor γ, hyperparameters µ and ρ, maximum layer index, ...

21. Experiments

22. Q-learning performance
“The accuracy goes up with the epsilon decrease and the top models are all found
in the final stage, show that our agent can learn to generate better block structures
instead of random searching.”

23. Q-learning result with different NSC

24. Block-QNN’s
results
compare with
state-of-
the-art

25. The required computing resource and time

26. Comparisons on ImageNet-1K Dataset.