This document discusses optimizations for deep neural network (DNN) hardware architectures. It describes common DNN layers like convolutional and recurrent layers. It evaluates DNN architectures based on metrics like accuracy, throughput, latency, energy efficiency and hardware cost. It introduces the roofline model for analyzing computational intensity and performance bottlenecks. Different techniques are proposed to optimize the multiply-accumulate (MAC) operations at the core of DNNs, like leveraging parallel processing elements, quantization, pruning and rearranging computations. The MIT Eyeriss architecture is highlighted as an example of an optimized hardware design that maps DNN computations efficiently onto a configurable array of processing elements.

1.
DNN

3. DNN MAC

4. • Fully Connected Layer
• ,
• Convolutional Layer
• , ,
• , Convolutional Neural Network (CNN)
•
• Recurrent Layer
•
• Recurrent Neural Network (RNN)
• ,
• Attention Layer/Mechanism
• Attention + ,
• Transformer [Vaswani, NeurIPS 2017]

5. :

10. DNN

11. : OPS/W
• (Accuracy
• (Throughput)
• (Latency)
• (Energy and Power)
• (Hardware Cost)
• (Flexibility)
• (Scalability)

12. Roofline
• , DNN
•
•

13. DNN
(Operational Intensity)
• VGG16 15000/(527+58) = 25 FLOP/Byte
• MobileNet v1 : 542/(16+58) = 7 FLOP/Byte

14. • Nvidia GTX 1080Ti 11.3 Tops, 484 GB/s
• 11300/484 = 23.3 Flop/Byte
https://zhuanlan.zhihu.com/p/34204282

15. DNN
•
• MAC
• ,
•
• MAC ( )
• (Processing Elements; PE)
• MAC -> PE ->
• PE
• PE ( )
• PE
• Batch Size

16. PE
https://arxiv.org/abs/1807.07928

17. DLA
• (Accuracy
• ,
• , , DNN
• (Throughput)
• DNN , / , PE
• (Latency)
•
• (Energy and Power)
• DNN , ,
• (Hardware Cost)
• On-chip , PE ,
• (Flexibility)
• DNN

18. https://arxiv.org/abs/1807.07928

19. DNN

21. Gauss

24. • MACs
• -> MACs
•
•
• MACs ,
• , Run length coding
• CPU/GPU , ,
• DNNs ->

26. /

30. MAC

31. CPU GPU
• :
• MACs:
• PE : , ,
• :
• MAC
• PEs
• large aggregate instructions (SIMD/SIMT), MACs
• MACs
• PEs >
• PEs > float 32 to float 16 / int 8
• PE
• PEs ->
• ->

32. DNN

33. DNN

36. •
https://people.csail.mit.edu/emer/papers/2016.06.isca.eyeriss_architecture.pdf

37. Weight

38. Output

39. Input

40. Row

41. Row

42. DNN
• MIT 's Eyeriss

43. • + ,
•
• /
• /
•
• ,
•

44. • DNN , .
• DNN ,
, .
• DNN , :
, , , , ,