Presentation MM-4099, Adapting game content to the viewing environment, by Noman Hashim at the AMD Developer Summit (APU13) November 11-13, 2013.

1. ADAPTING
GAME
CONTENT
TO
THE
VIEWING
ENVIRONMENT
WWW.APICAL.CO.UK
1

2. YOU
CAN'T
WATCH
A
MOVIE
BY
THE
POOL…CAN
YOU?
Amazon Kindle “Poolside” commercial Fall 2010
2
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
2

3. THE
CHALLENGE
OF
THE
MOBILE
VIEWING
ENVIRONMENT
Difficult
for
emissive
display
to
compete
with
sun’s
brightness!
O[en
need
to
reduce
display
brightness
to
conserve
power
3
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
3

4. PROBLEM
STATEMENT
Display
contrast
ra]o
is
affected
independently
from
the
bright
and
dark
ends
of
the
range
Display
contrast
ra]o
must
be
adapted
independently
Full
brightness
Dark
room
4
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
Reduced
brightness
Dark
room
Reduced
brightness
Bright
room
4

5. CONTRAST
RATIO
CHANGES
FROM
1000:1
TO
50:1
WITH
AMBIENT
LIGHT
Source:
hap://www.displaymate.com/Tablet_ShootOut_3.htm
5
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
5

6. SOME
CONTENT
IS
HDR
WHILE
DISPLAY
IS
VERY
LDR
6
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
6

7. THIS
ALSO
MEANS
THAT
DISPLAYS
NEED
TO
BE
BRIGHT
AND
BURN
POWER
Windows
8
recommended
Excessive
brightness:
wastes
power
Display
brightness
(nits)
160
140
120
100
Not
bright
enough
to
see
images/
movies/
games
80
60
40
20
0
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Ambient
light
(lux)
From
embedded
Ambient
Light
sensor
(ALS)
7
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
7

8. HOW
ABOUT
ADJUSTING
GAMMA?
8
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
8

9. YOU
CAN’T
CHANGE
GAMMA
WITHOUT
CHANGING
COLORS
Increasing
gamma
makes
image
less
colorful
and
reduces
contrast
–”wash
out”
9
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
9

10. ASSERTIVE
DISPLAY
Asser]ve
Display
Core
ALS
AD_CALC
VIDEO
PIPELINE
RGB
10
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
iridix_display
gamma
LUT
perceptual
dither
RGB
10

11. ASSERTIVE
DISPLAY
Content
adapted
to
display
contrast
ra]o
Same
power
consump]on
Display
at
max
brightness:
contrast
ra]o
too
low
for
mul]media
Sample
condi]ons:
Ambient
light
1400
lux
–indoors
in
daylight
11
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
11

12. PLAY
AN
IMMERSIVE
GAME
IN
ANY
CONDITIONS
Asser]ve
Display:
Content
adapted
to
display
contrast
ra]o
Same
power
consump]on
Display
at
max
brightness:
contrast
ra]o
too
low
for
mul]media
Sample
condi]ons:
Ambient
light
1400
lux
–indoors
in
daylight
12
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
12

13. THE
4
SCREENS
Asser]ve
Display
ON
(No
specific
calibra]on)
Asser]ve
Display
OFF
Ambient
light
200
lux.
Smartphone:
full
brightness.
TV:
Standard
Mode
13
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
13

14. WHY
IS
ASSERTIVE
DISPLAY
EFFECTIVE?
The
key
to
Asser]ve
Display
is:
Brightness
can
be
set
according
to
preference
and
power
saving
Contrast
raDo
is
always
correct
for
op]mal
mul]media
viewability
It
enables
independent
adjustment
of
bright
and
dark
regions
of
each
frame
So
it
decouples
brightness
from
contrast
raDo
14
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL

15. ASSERTIVE
DISPLAY:
XBOX
CALL
OF
DUTY
15
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
15

16. ASSERTIVE
DISPLAY:
NOKIA
LUMIA
1520
16
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
16

17. IMPLEMENTATION
OF
ASSERTIVE
DISPLAY
ON
A
HETEROGENOUS
PROCESSOR
Headline
results
on
Exynos
5410:
Frame
size
Maximum
frame
rate
(single
core)
CPU
(Neon)
only
1080P
(2MP)
17
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
OpenGL
ES2.0
OpenCL
4fps
28fps
45fps
17

18. VIDEO
PROCESSING
ENGINE:
PIXEL-­‐BY-­‐PIXEL
TONE
MAPPING
18
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
18

19. VIDEO
PROCESSING
ENGINE:
PIXEL-­‐BY-­‐PIXEL
TONE
MAPPING
Contrast
in
highlights
enhanced
Contrast
in
midtones
preserved
Contrast
in
shadows
enhanced
iridix
calculates
a
unique
tone
curve
for
each
pixel
of
each
frame
19
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
19

20. ASSERTIVE
DISPLAY
INVOLVES
A
GPU-­‐FRIENDLY
IMAGE
TRANSFORM
y Use
of
GPU
effec]ve
since
many
pixel
transforma]ons
can
be
done
in
parallel
y However,
transform
is
calculated
per-­‐pixel
adap]vely
on-­‐the-­‐fly,
so
beyond
simple
texture
mapping
⇒ Requires
mul]ple
CPU
and
GPU
passes
with
intermediate
data
store
per
frame
⇒ Main
reason
for
going
from
GL
to
CL
20
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
20

21. LIMITATIONS
OF
OPENGL:
INTERMEDIATE
MEMORY
ACCESSES
GPU
memory
CPU
memory
Image
frame
Resize
Resized
frame
Resized
frame
Transforma]on
LUT
Transforma]on
LUT
Post-­‐processed
transforma]on
LUT
Upscaled
Transforma]on
LUT
Transformed
frame
DISPLAY
21
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
21

22. BENEFITS
OF
OPENCL
1. Intermediate
memory
accesses
With
Open-­‐CL
compliant
hardware,
CPU
and
GPU
memory
can
be
shared,
removing
mul]ple
memory
transfers
required
by
the
algorithms
2. Intermediate
data
precision
Even
if
input
data
is
8-­‐bit,
intermediate
calcula]ons
require
at
least
12
bit
precision.
OpenGL
is
inefficient
for
such
opera]ons
when
mul]ple
processing
stages
are
needed
3. Transforma]on
of
different
color
planes
We
apply
a
different
transform
to
Y,U
and
V
planes.
In
OpenCL
these
can
be
done
by
a
single
kernel,
whereas
in
OpenGL
separate
kernels
are
required
22
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
22

23. PERFORMANCE
COMPARISONS
AD
v3
ImplementaDon
Maximum
pixel
rate
Power
consumpDon
at
1080P30
Full
HW
IP
core
(28nm
TSMC
LP)
4K2K
at
60fps
3mW
CPU
only
(Exynos
5410)
1080P
at
4fps
N/A
OpenGL
ES2.0
(Exynos
5410)
1080P
at
28fps
112mW
OpenCL
(Exynos
5410)
1080P
at
45fps
78mW
y GPU-­‐based
processing
is
much
beaer
than
CPU,
but
dedicated
hardware
is
s]ll
best
‒ Partly
because
HW
core
does
not
require
system
memory
access
23
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
23

24. CONCLUSIONS
y An
immersive
experience
can
be
achieved
on
normal
displays
in
all
condi]ons
by
means
of
advanced
image
processing
y Latest
processors
with
CPU/GPUs,
including
those
in
PS4
and
Xbox
One,
can
run
these
kind
of
algorithms
in
real-­‐]me
y Extensions
of
this
approach
to
deliver
cinema-­‐like
experiences
for
both
video
and
gaming
are
in
development
24
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL

25. DISCLAIMER
&
ATTRIBUTION
The
informa]on
presented
in
this
document
is
for
informa]onal
purposes
only
and
may
contain
technical
inaccuracies,
omissions
and
typographical
errors.
The
informa]on
contained
herein
is
subject
to
change
and
may
be
rendered
inaccurate
for
many
reasons,
including
but
not
limited
to
product
and
roadmap
changes,
component
and
motherboard
version
changes,
new
model
and/or
product
releases,
product
differences
between
differing
manufacturers,
so[ware
changes,
BIOS
flashes,
firmware
upgrades,
or
the
like.
AMD
assumes
no
obliga]on
to
update
or
otherwise
correct
or
revise
this
informa]on.
However,
AMD
reserves
the
right
to
revise
this
informa]on
and
to
make
changes
from
]me
to
]me
to
the
content
hereof
without
obliga]on
of
AMD
to
no]fy
any
person
of
such
revisions
or
changes.
AMD
MAKES
NO
REPRESENTATIONS
OR
WARRANTIES
WITH
RESPECT
TO
THE
CONTENTS
HEREOF
AND
ASSUMES
NO
RESPONSIBILITY
FOR
ANY
INACCURACIES,
ERRORS
OR
OMISSIONS
THAT
MAY
APPEAR
IN
THIS
INFORMATION.
AMD
SPECIFICALLY
DISCLAIMS
ANY
IMPLIED
WARRANTIES
OF
MERCHANTABILITY
OR
FITNESS
FOR
ANY
PARTICULAR
PURPOSE.
IN
NO
EVENT
WILL
AMD
BE
LIABLE
TO
ANY
PERSON
FOR
ANY
DIRECT,
INDIRECT,
SPECIAL
OR
OTHER
CONSEQUENTIAL
DAMAGES
ARISING
FROM
THE
USE
OF
ANY
INFORMATION
CONTAINED
HEREIN,
EVEN
IF
AMD
IS
EXPRESSLY
ADVISED
OF
THE
POSSIBILITY
OF
SUCH
DAMAGES.
ATTRIBUTION
©
2013
Advanced
Micro
Devices,
Inc.
All
rights
reserved.
AMD,
the
AMD
Arrow
logo
and
combina]ons
thereof
are
trademarks
of
Advanced
Micro
Devices,
Inc.
in
the
United
States
and/or
other
jurisdic]ons.
SPEC
is
a
registered
trademark
of
the
Standard
Performance
Evalua]on
Corpora]on
(SPEC).
Other
names
are
for
informa]onal
purposes
only
and
may
be
trademarks
of
their
respec]ve
owners.
25
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL