Introductory lecture for Optimisation and Compression

1. Battle of the Codecs
In Digital Film and TV
James Uren
Independent Training Consultant,
Senior Lecturer Broadcasting,
Creative Director 3D Stories
Optimisation and Compression
Ravensbourne College of Design & Communication
24 November 2011

3. Demistifying Codecs
 The technologies of digital film
 So many codecs...
 The technology behind compression
 The future...
2

4. Digital Images Defined in Pixels
 Standard-Definition (SDTV)
 Europe: 720 x 576 pixels/frame. US/Japan: 720 x 480 pixels/frame
 High-Definition (HDTV)
 1920 x 1080 pixels/frame (Also called CIF (Common Image Format)). 1280 x 720 pixels/frame
 Digital Film and D-cinema
 Full-aperture 2K is 2048 x 1536 (gives a 1.33:1 (4:3) image)
 Academy 2K is 1828 x 1080 (For a 1.85:1 image)
 Full-aperture 4K is 4096 x 3072 pixels/frame
4

5. Digital Image Pixel Resolutions
720 x 576
1280 x 720
1920 x 1080
2048 x 1536
4096 x 3072
5

6. Film?
• What resolution is film? Grain...

11. Film?
• So resolution unknown - is 4K enough?
• So in film we have:
• Digital Intermediate (DI) for post-production
• Digital cinema projection
• Digital Capture - the last piece of the puzzle.

12. HD Capture
 CCD (Charge Coupled Device) or CMOS (Complementary
Metal-Oxide Silicon) sensor options:
 CMOS now lower noise than CCDs
 CMOS allows A/D & processing on-chip
 CMOS can have >100 million transistors...
ARRIFLEX D21
 Examples:
The ARRIFLEX D21 uses a 35mm 6 megapixel (2.9K) CMOS
imager
The SI 2K has a 2/3” 2.5 megapixel (2K) CMOS imager
SI-2K
10

13. HD, 2K 4K + Imaging
CMOS Active Pixel Sensor
Photodiode
12

14. HD, 2K 4K + Imaging
Single-chip colour sensor
Typically use a Bayer filter layout
developed at Kodak.
13

15. HD, 2K 4K + Imaging
CMOS imager integrated
circuit architecture
http://micro.magnet.fsu.edu/primer/digitalimaging/cmosimagesensors.html
14

16. Compression
• Why?
• Bandwidth,
• Storage
• Workflow
• Lossless vs. lossy
• Proxies

17. Inter-Frame Compression
• MPEG is an inter-frame
codec and forms the
backbone of many codecs
(e.g. H.264)
• Exploits commonality
between frames
• Sends one full ‘I Frame’
then predicts through the
Group of Pictures (GOP)

18. Intra-Frame Compression
• Compressing each frame independently
• Also known as ‘I Frame only’
• Examples include ProRes, DNxHD
• Best for offline editing as no GOP reconstruction required

19. So many codecs...

20. Terminology
• Resolution (e.g. 4K)
• Anamorphic pixels
• Raw data (e.g. RAW / R3D)
• Bit depth (e.g. 12-bit)
• Chroma sampling (e.g. 4:4:4)
• Bandwidth / data rate / bit rate (e.g. 100Mbit/s)
• Frame rate / scan type (e.g. 50i / 25PsF*)
• Audio sampling frequency (e.g. 48kHz)
• Wrapper

21. Example Codec
• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz

22. Example
• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz
Codec

23. Example
• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz
Colour bandwidth ratios

24. Example
• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz
High Quality
- the bit rate is higher - 220 Mbit/s

25. Example
• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz
Pixel resolution
(HD Anamorphic)

26. Example
• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz
Frame rate and
scan type

27. Example
• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz
Audio sampling
frequency

28. Interlaced or Progressive?
 Interlace
 Each image (frame) comprises two interlaced fields
 Any movement between the fields causes artefacts
 Introduced for TV because it halves transmission bandwidth
 Used in 50i (Europe), 59.94i (USA/Japan) SDTV/HDTV
 Progressive
 Each image is complete & contains all the pixels
 Can produce judder motion artefact at 24p or 25p
 50p gives best motion portrayal
 Higher frame rates would be better still (See BBC research into 150 fps)
6

29. Interlaced or Progressive?
 Not straightforward: Imagers do not scan the image, neither do flat panel displays
 Interlace introduces artefacts and reduces vertical resolution
 What the experts say:
 “All data reduction systems (such as MPEG) give better results at lower bitrates with progressive sources”
 “Progressive scanning gives improved motion portrayal”
 “1080i offers better static resolution, 720p better motion portrayal”
 (EBU The Review No.301)
7

30. Picture Impairments
Too much compression can introduce impairments:
Blocking
Mosquito Noise / Wings
Contouring
Smearing
Freezes
Breathing
42

31. .bmp
964k
43

32. .jpg
28k
44

33. But...
• ...this is only if you compress to much.
• ProRes 4444 at 250Mbit/s is indistinguishable from
uncompressed...
• ...and much less data

34. D-Cinema
The DCI (Digital Cinema Initiatives) was set up in 2002
Aim was to set standards for a viable D-Cinema system
Members include Disney, Fox, MGM, Paramount, Sony Pictures Entertainment
(Columbia), Universal & Warner Bros.
Adopted JPEG2000 compression
36

35. D-Cinema
JPEG2000 uses wavelet compression
JPEG2000 does not divide the picture into discrete blocks
D-cinema uses a mild level of compression to avoid artefacts (~250Mbit/s)
37

36. JPEG2000 Examples
39

37. The Future
• Requirements on compression reduces with Moore’s Law
• But this is countered by increasing resolutions and frame
rates
• Compression is here to stay, but offline and online continue
to blur

38. Round-Up
• A complete digital pipeline from capture to display
• Don’t fear compression
• Great pictures mean a high bitrate end-to-end, but too high and
your pipeline starts to get expensive... But not for long.

39. Battle of the Codecs
In Digital Film and TV
James Uren
Independent Training Consultant,
Senior Lecturer Broadcasting,
Creative Director 3D Stories
Ravensbourne College of Design & Communication