Group normalization

1. Group Normalization
Fengda Zhu

2. Overview
1. BN has memory constraints
2. feature channels have correlatives

3. Group Normalization

4. Formulation

5. Implementation

6. Experiments

7. Experiments

8. Experiments

9. Experiments

10. Experiments

11. Experiments

12. Experiments

13. Experiments

14. Discussion
1. BN has been so influential that many state-of- the-art systems and their hyper-
parameters have been de- signed for it, which may not be optimal for GN-
based models.
2. Sense GN is related to LN and IN, two normalization methods that are
particularly successful in training recurrent (RNN/LSTM) or generative (GAN)
models. This suggests us to study GN in those areas in the future.