1. Influence of Chromatic
Information on
Quality of Experience
Presented by
António M. G. Pinheiro
In collaboration with:
Marco Bernardo, Manuela Pereira, Paulo Fiadeiro
U.B.I.
Covilhã, Portugal
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria

2. Influence of Chromatic Information on QoE
• What is the most natural image?
• One of them would look like the natural image
(if this projector did not change the true colors).
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

3. OUTLINE
• Introduction
• Analysis of the Influence of Chromatic Distortion
• Objective Metrics applied to Chromatic Distortion
• Modeling QoE for Chromatic Distortions
• Content Influence
• Conclusions
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

4. Introduction
• Human sensibility to chromatic information:
• Humans reveal very low sensibility!
• Moreover, humans also see colors as they expect
them to be.
• The same object is perceived in the early morning, at noon
and at sunset, as having the same colors.
• However, that is not the case for a common cam!!!
If illumination changes, color components also change.
• This phenomenon is called Color Constancy.
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

5. Introduction
• The best example of low chromatic sensibility:
Analog Color TV
• Color YUV system is used
• Most of the bandwidth is given to Y (Gray-Level)
• U and V, the chromatic components, just get a small
portion of the bandwidth
Image, and
Y,U,V
components
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

6. Introduction
• Color TV: Chromatic components sub-sampling
• NTSC (USA) sub-sampling is 4:1:1
• PAL sub-sampling is 4:2:2
(4 columns for Y and 2 for the 2 Chromatic components)
• Digital TV also uses 4:2:0 (2x2 of Y and to 1 for the 2 Chroma.)
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

7. Analysis of the Influence of Chromatic Distortion
• Hyperspectral Images Databases
• True color knowledge (33 wavelength)
• CIE 1976 (L*a*b) Color space
• a* and b* are the chromatic components; L* - Luminosity
• Device Independent color space.
• Perceptually uniform - a change of the same amount in a color value should produce a
change of about the same visual importance.
Hence, an induced error is proportional to the human color perception
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

8. Analysis of the Influence of Chromatic Distortion
• Study
• To avoid artifacts, colors have been clustered by similarity
(the Kmeans algorithm was used – K=4)
• Chromatic Errors applied to each cluster (only in the components a*,b*)
• Random error directions in (a*,b*) with magnitudes of 3, 6, 9, 12, 15.
• 5 different examples for each of the 5 different images
(result in 125 images plus the initial ones).
• Component L* (Luminosity) kept constant.
∆
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

9. Analysis of the Influence of Chromatic Distortion
• Methodology for the Subjective Assessment
• SSCQE – Single Stimulus Continuous Quality Evaluation
• Individuals asked to rate images based on their naturalness
• Medium Opinion Score
• 102 individuals (54 females)
• Normal color vision assessed
before the subjective testing.
Remarks:
• Requires a careful planning
• Long time requirements
• True color display
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

10. Analysis of the Influence of Chromatic Distortion
• How to do subjective tests?
• ITU standards
• ITU-T Recommendation BT 500-12: Methodology for the subjective
assessment of the quality of television pictures. Technical report, ITU
Telecom. Standard. Sector, 2009.
• Some Consensus:
• Scale should give a sensation of
a continuous scale.
• Results discretized in a set of levels.
(studies say 10 levels is a good choice,
but a trade-off between the number
of tests and individuals is required).
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

11. Influence of Chromatic Distortion on QoE
∆=6
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

12. Influence of Chromatic Distortion on QoE
∆=9
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

13. Influence of Chromatic Distortion on QoE
∆ = 15
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

14. Influence of Chromatic Distortion on QoE
∆=9
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

15. Influence of Chromatic Distortion on QoE
∆=9
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

16. Influence of Chromatic Distortion on QoE
∆ = 15
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

17. Analysis of the Influence of Chromatic Distortion
• Example 1 10
8
6
MOS
4
2
0
∆=0 ∆=3 ∆=6 ∆=9 ∆=12 ∆=15
Image 3
1
29 30
1 2 3
28 0.8 4
27 5
26 0.6 6
25 0.4 7
24 0.2 8
0
23 9
22 10
21 11
20 12
19 13
18 17 15 14
16
+STD MOS -STD
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

18. Analysis of the Influence of Chromatic Distortion
• Example 2 10
8
6
MOS
4
2
0
∆=0 ∆=3 ∆=6 ∆=9 ∆=12 ∆=15
Image 2
1
29 30
1 2 3
28 0.8 4
27 5
26 0.6 6
25 0.4 7
24 0.2 8
0
23 9
22 10
21 11
20 12
19 13
18 17 15 14
16
+STD MOS -STD
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

19. Objective Metrics applied to Chromatic Distortion
• Some typical full reference objective quality measures
were analysed and compared with the MOS:
• PSNR (Peak Signal-to-Noise Ratio)
• MSSIM (Mean Structural SIMilarity index)
• VIFP (Visual Information Fidelity measure in Pixel domain)
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
PSNR MSSIM VIFP MOS PSNR MSSIM VIFP MOS
Urban image Country image
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

20. Modeling QoE for Chromatic Distortions
• QoE Influence Factor: Any characteristic of a user,
system, service, application, or context whose actual
state or setting may have influence on the Quality
of Experience for the user.
• Human: humans perceive colors in a different way;
• System: usually represent colors differently
(easy to observe in a displayed picture after printing).
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

21. Modeling QoE for Chromatic Distortions
• QoE Influence Feature: A perceivable, recognized and
namable characteristic of the individual’s experience of
a service which contributes to its quality.
• Direct Perception Level
• Service Level
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

22. Content Influence
• Considering the initial example:
• The last picture is the initial image
• However, the middle picture has more saturated colors,
creating a sensation of Spring/Summer time.
• What about the first one? Early morning?!!!
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

23. Content Influence
∆=9
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

24. Content Influence
∆=9
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

25. Content Influence
∆=9
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

26. Content Influence
∆=9
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

27. Content Influence
∆=9
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

28. Content Influence
∆ = 12
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

29. Content Influence
∆ = 12
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

30. Content Influence
∆ = 12
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

31. Content Influence
∆ = 12
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

32. Content Influence
∆ = 12
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

33. Content Influence
∆ = 15
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

34. Content Influence
∆ = 15
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

35. Content Influence
∆ = 15
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

36. Content Influence
∆ = 15
M M
O O
S S
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.

37. Conclusions
• Human have low sensibility to chromatic distortion:
• It is well known that humans can identify errors larger than 2.2
in the CIE1976 (L*a*b) Color Space.
• In our tests, color errors less than 6 are very well tolerated.
• The MOS values can be simulated by objective full reference
metrics like MSSIM and also VIFP.
• The chromatic errors produce different error perception,
depending of the color transformation.
• Usually more saturated colors are very tolerated.
Colloquium on Quality of Experience in Multimedia Systems and Services
November 23, 2012, Klangenfurt, Austria
U.B.I.