This document discusses using neural networks for image compression. It describes traditional image compression techniques and architectures, as well as different types of neural networks like convolutional neural networks (CNNs) and recurrent neural networks (RNNs). CNNs have been used in autoencoders to obtain a latent space representation of an image for compression. RNNs have also been proposed for neural network image coding frameworks to leverage their memory capabilities. Examples show neural networks can achieve comparable or better compression than traditional techniques like JPEG at lower bitrates.

1. Reti neurali e compressione immagine
---
Neural Network Image Coding
Cristian Perra
Dipartimento di Ingegneria Elettrica ed Elettronica, UdR CNIT
Università di Cagliari
cristian.perra@unica.it

2. Overview
Image compression
Neural network
Neural network image coding
Conclusions

3. Image compression
Systems and applications
Acquisition, processing, rendering of high quality images
Constraints
Limited capabilities of transmission networks
Limited processing power
Limited storage capacities
Solution
Exploiting data redundancy: Spatial (Prediction), Visual
(quantization), Statistical (transform coding, entropy
coding)
Lossless/Lossy coding architecture

4. Image compression
tools
Color space transform / chrominance subsampling
Block transform (e.g. DCT in JPEG, POT-PCT in JPEG-XR)
DPCM (e.g. DC prediction in JPEG)
Image transform (e.g. wavelet in JPEG2000)
Quantization (e.g. quantization matrix in JPEG, quantization function in
JPEG2000)
Run-length coding (e.g. JPEG)
Variable Length Coding (e.g. Huffman-like tables in JPEG), Arithmetic Coding
Embedded Block Coding with Optimized Truncation (e..g EBCOT in JPEG2000)
…

5. Image Compression Architectures
Transform
/ Prediction
Quantization Entropy Coding
Entropy
Decoding
Inverse
Quantization
Inverse
T/P
...001010101101010101010...
Coding performance improvement within the traditional coding architectures are very challenging

6. Neural Network
Neural networks are in the recent years becoming
the main tool in artificial intelligent
Neural networks are proposed as additional tool for
image coding architectures
Neural networks are proposed for a paradigm shift in
image coding where the full “encoding-decoding”
architecture is modelled as a neural network

7. Neural network
Multiple layers of processing
units (neuron / perceptron)
Weighted connections from
previously activated neurons
activate the current neuron
Activation function for
intermediate layers for
achieving non-linearity

8. Neural network
training
Training problem: learning the model
parameters (weights)
Backpropagation: procedure to solve the
training problem
Training fitting:
Underfitting ➔ High training errors
Overfitting ➔ High test errors

9. Neural Network Techniques
Multilayer Perceptrons (MLPs)
Random Neural Networks
…
Convolutional Neural Networks (CNNs)
Recurrent Neural Networks (RNNs, LSTMs, GRUs)
Generative Adversarial networks (GANs, cGANs)

10. Convolutional Neural Network
• Outperform the traditional algorithms by a
huge margin in high-level computer vision tasks such as
• image classification
• object detection
• super-resolution
• compression artefact reduction
• CNN adopts the convolution operation to characterize the correlation
between neighbouring pixels, and the cascaded convolution operations
well conform the hierarchical statistical properties of natural images

11. Autoencoder
Autoencoders (AE) are a family of NN
for which the input is the same as the
output. They work by compressing the
input into a latent-space
representation, and then reconstructing
the output from this representation.

12. Convolutional Autoencoder (CAE)
Autoencoder applied to images:
replacing fully connected layers by convolutional layers and pooling layers
(input from “wide and thin” to “narrow and thick”)
the network extract visual features from the images, and therefore obtains a
latent space representation
reconstruction process uses upsampling and convolutions
Convolutional Autoencoder (CAE)

13. Convolutional Autoencoder
CNN
analysis
CNN
synthesis
Latent Space

14. Convolutional Autoencoder Training
Input Training CAE Output

15. Convolutional Autoencoder Reconstruction
Reconstruction: lower quality (blurry, …)
CNN
analysis
CNN
synthesis
Latent Space

16. Neural Network Coding Framework (1/2)
𝑥 ∈ ℝ 𝑛
ො𝑥 = 𝑔𝑠(ො𝑦, 𝜽) ො𝑦 = 𝑞
𝑦 = 𝑔 𝑎(𝑥, 𝝓)
𝑞 ∈ ℤ 𝑛
𝐻 𝑃𝑞 < 𝑅
Ƹ𝑧 = 𝑔 𝑝(ො𝑥)
𝑔𝑠, 𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑟𝑖𝑐 𝑠𝑦𝑛𝑡ℎ𝑒𝑠𝑖𝑠
𝑡𝑟𝑎𝑛𝑠𝑓𝑜𝑟𝑚
𝑔 𝑎, 𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑟𝑖𝑐 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠
𝑡𝑟𝑎𝑛𝑠𝑓𝑜𝑟𝑚
z = 𝑔 𝑝(𝑥)
𝐷 = 𝑑(𝑧, Ƹ𝑧)
𝐿 𝜃, 𝜙, 𝜆 = 𝑅 𝜃, 𝜙 + 𝜆𝐷(𝜃, 𝜙) (from) Ballé, Laparra, e Simoncelli, «End-to-End
Optimized Image Compression».

17. CNN Coding framework (2/2)
𝑦 = 𝑔 𝑎(𝑥, 𝝓) ො𝑥 = 𝑔𝑠(ො𝑦, 𝜽)
1,709,313
parameters
(from) Ballé, Laparra, e Simoncelli, «End-to-End
Optimized Image Compression».

18. Recurrent Neural Network Based Coding
NN with memory to store the recent behaviours
memory units in RNN have the connections to
themselves, which transmit transformed
information from the execution in the past
changes the behaviour of the current forward
process to adapt to the context of current input

19. Recurrent Neural Network Coding Framework
(from) Toderici et al., «Full Resolution Image Compression with Recurrent Neural Networks».

20. Generative Adversarial Networks
NN Encoder NN Generator
NN Generator
NN
Discriminator
fake
real
Add
Noise

21. NN Compression Examples
• CNN
• GAN

22. CNN
0.35
bpp
JPEG
0.35
bpp

23. GAN
0.0651
bpp
SOURCE

24. GAN (0.072 bpp)

25. Conclusions
Pros
Content adaptivity
Larger receptive field
Texture and feature representation
Cons
Fixed compression rate based on the size of a bottleneck layer
Computational complexity
Memory consumption

26. Contact
Cristian PERRA
Department of Electrical and Electronica Engineering
CNIT Lab Cagliari
University of Cagliari, Italy
cperra@ieee.org, cristian.perra@unica.it
This work is licensed under the Creative Commons Attribution – NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/