3. Fundamentals of images
An image consists of pixels (picture elements)
Each pixel represents luminance (and colour)
Typically, 8-bits per pixel
3
5. Fundamentals of images
A TV frame is about 640x480 pixels
If each pixels is represented by 8-bits for each
colour, then the total image size is
640×480*3=921,600 bytes or ≈7.4Mbits
At 30 frames per second, this would be
≈ 220Mbits/second
5
8. Fundamentals of images
Do we need all these bits?
No!
The previous example illustrated the eye’s sensitivity
to luminance
We can build a perceptual model
Only code what is important to the human visual
system (HVS)
Usually a function of spatial frequency
8
9. Fundamentals of images
Discrete cosine transform
Coefficients are approximately uncorrelated
Except DC term
C.f. original 8×8 pixel block
Concentrates more power in the low frequency
coefficients
Computationally efficient
Block-based DCT
Compute DCT on 8×8 blocks of pixels
9
12. JPEG
JPEG (Joint Photographic Experts Group) is an ISO
/IEC group of experts that develops and maintains
standards for a suite of compression algorithms for
computer image files.
Together with the Graphic Interchange Format (GIF)
and Portable Network Graphics (PNG) file formats,
the JPEG is one of the image file formats supported
on the World Wide Web, usually with the file suffix of
".jpg". You can create a progressive JPEG that is
similar to an interlaced GIF.
12
13. Fundamentals of JPEG
13
DCT Quantizer Entropy
coder
IDCT Dequantizer Entropy
decoder
Compressed
image data
Encoder
Decoder
14. Fundamentals of JPEG
JPEG works on 8×8 blocks
Extract 8×8 block of pixels
Convert to DCT domain
Quantize each coefficient
Different stepsize for each coefficient
Based on sensitivity of human visual system
Order coefficients in zig-zag order
Entropy code the quantized values
14
17. Fundamentals of JPEG
Entropy coding
Run length encoding followed by
Huffman
Arithmetic
DC term treated separately
Differential Pulse Code Modulation (DPCM)
2-step process
1. Convert zig-zag sequence to a symbol sequence
2. Convert symbols to a data stream
17
18. Fundamentals of JPEG
Modes
Sequential
Progressive
Spectral selection
Send lower frequency coefficients first
Successive approximation
Send lower precision first, and subsequently refine
Lossless
Hierarchical
Send low resolution image first
18
24. MPEG
Consider a sequence of n frames of video.
It consists of:
I-frames
P-frames
B-frames
A sequence of one I-frame followed by P- and B-
frames is known as a GOP
Group of Pictures
E.g. IBBPBBPBBPBBP
24
25. MPEG
I-frames
Intraframe coded
No motion compensation
P-frames
Interframe coded
Motion compensation
Based on past frames only
B-frames
Interframe coded
Motion compensation
Based on past and future frames
25
26. MPEG
Like JPEG, the DC term is treated separately
DPCM
B-frame compression high
Need buffer and delay
26
27. MPEG-1
MPEG-1 was designed for coding progressive video
at a transmission rate of about 1.5 million bits per
second. It was designed specifically for Video-CD
and CD-i media.
MPEG-1 audio layer-3 MP3 has also evolved from
early MPEG work.
27
28. MPEG-2
MPEG-2 was designed for coding interlaced images
at transmission rates above 4 million bits per second.
MPEG-2 is used for digital TV broadcast and DVD.
An MPEG-2 player can handle MPEG-1 data as well.
28
29. MPEG-3
A proposed MPEG-3 standard, intended for High
Definition TV (HDTV), was merged with the MPEG-2
standard when it became apparent that the MPEG-
2 standard met the HDTV requirements.
29
30. MPEG-4
MPEG-4 is a much more ambitious standard and
addresses speech and video
synthesis, fractal geometry, computer visualization,
and an artificial intelligence AI approach to
reconstructing images.
MPEG-4 addresses a standard way for authors to
create and define the media objects in a
multimedia presentation, how these can be
synchronized and related to each other in
transmission, and how users are to be able to
interact with the media objects.
30
31. MPEG-21
MPEG-21 provides a larger, architectural framework for
the creation and delivery of multimedia. It defines
seven key elements:
Digital item declaration
Digital item identification and declaration
Content handling and usage
Intellectual property management and protection
Terminals and networks
Content representation
Event reporting
31