Virtual Reality (VR360): Production Framework for the Combination of High Dynamic Range Imaging (HDRI) and Spherical Panorama towards Lean Approach

Researcher: Zi Siang See
Production Framework for the Combination of HDRI
and Spherical Panorama towards Lean Approach
Email (University Tunku Abdul Rahman) seezs@utar.edu.my
Email (Industrial collaboration): zisiang@reina.com.my
University Tunku Abdul Rahman
Faculty of Creative Industries
Centre for Immersive Technology and Creativity
Department of Multimedia Design and Animation
Centre of Immersive Technology and Creativity
Department of Multimedia Design and Animation
Research Talk: Zi Siang See
Virtual Reality 360 (VR360)
collaborators from

Research Presentation
1) Introduction (VR360)
a. Aim, Objectives, Research Problem
b. Literature Review and Related Work
2) Research Design and Method
3) Data Collection and Analysis
4) Discussion and Conclusion
collaborators from

Professional Background:
Industrial involvements since 2003/04
Industrial Specialization: Creative and Advertising Imaging
Research Area: High Dynamic Range Imaging (HDRI)
Spherical Panorama
Augmented Reality (AR)
Creative and Advertising Processes

Work: McDonald’s Malaysia
Creative: NagaDDB
Operation: Quality Ingredient Story
Creative Imaging: Reina Imaging

Researcher: Zi Siang SeeDigital Imaging (DI) image manipulation for advertising usage, high technical finishing | Presenter: Zi Siang See

Researcher: Zi Siang SeeDigital Imaging (D() for Advertising Visuals| Presenter: Zi Siang See

Researcher: Zi Siang SeeAdvertising visual | Presenter: Zi Siang See

Researcher: Zi Siang SeeObject Virtual Reality (Object VR) | Presenter: Zi Siang See

Researcher: Zi Siang SeeVirtual Reality 360 (VR360) | Presenter: Zi Siang See
Virtual Reality 360 (VR360)

Introduction
Multimedia (Combination of Elements)
• Text
• Image
• Sound
• Video
• Animation Industries
• Creative and advertising
• Corporate communications visuals
• Architectural imaging
• Cultural heritage preservation
• Scientific achieve
Research - Virtual Reality 360 (VR360)
1) High Dynamic Range Imaging (HDRI)
2) Spherical Panorama

Introduction: HDRI & Spherical Panorama
Obstacles and
Issues
• High Dynamic Range Imaging (HDRI)
• Spherical Panorama
Towards Lean
Approach
• Inaccurate reproduction of location-based subjects in image
• Architectural context as case study subject of location-based
content for data collection and analysis
Proposed
Production
Framework
• To have real-world likelihood photographic outcome
reproduced by HDRI and spherical panorama with reduced
imaging variables caused by various obstacles and issues.
production processes

Figure: combined luminance (EV) range being covered by HDRI outcome.
Darker images
(luminance) Brighter images
(luminance)
Introduction: HDRI
• High Dynamic Range Imaging (HDRI) in photography technique has
a goal to increase the dynamic range resulting an improved
luminance reproduced in shadow and highlight areas (Reinhard et
al, 2010; Debevec and Malik, 1997; Fairchild, 2007), usually by
combining multiple exposures consist of darker and brighter images.

• Possible major obstacles and issues of HDRI that involves multiple
exposures can include ghosting effect and misalignment issue due
to inconsistent acquisitions (Reinhard et al, 2010)
Figure: HDRI with visual abnormalities (by Reina).
Introduction: HDRI
Multiple exposures for HDRISingle exposure (LDR)
Visual abnormality

Introduction: HDRI
• ‘The HDR Photographic Survey’ by Fairchild (2007) has
demonstrated HDRI reproduction may require massive amount of
multiple exposures, however may speculate to have production
wastage or technical redundancy.
(1) (2) (3)
(4) (5) (6)
(7) (8) (9)
(a) (b)
Massive amount of multiple
exposures, may have
wastage or redundancy.
For example, some image
sequences have exceed the
need of constructing HDRI.
Figure: example taken from ‘The HDR Photographic Survey’ by Fairchild (2007)

Figure: Multirow production process to photographically create a spherical imagery
Processing (Stitching)
Multirow setup, to have
multiple photographs
covering 360 degrees.
Spherical
panorama
rendering
Introduction: Spherical Panorama
• Spherical panorama photography imaging technique aims to acquire
wider panoramic visibility in image usually unable to be acquired
from the conventional single angle photograph image (Benosman
and Kang, 2001; Brown and Lowe, 2006; Felinto et al, 2012), this
can be arranged by combining multiple angle images.

• Microsoft launched a feature called the ‘Street Side’ in Bing Map on
November 2010 in competition to Google Map with ‘Street View’
featuring spherical panorama interactivity, however numerous
content published have shown high amount of visible imaging
errors and insufficient of dynamic range.
Figure: (a) “Street View” feature in Google Map. (b) “Street Side” feature in Microsoft Bing Map.
(a) (b) Visual abnormalityVisual abnormality

• Major difficulty for spherical panorama reproduction may occur, this
can include limited dynamic range reproduced and parallax error
(Brown and Lowe, 2006; Felinto et al, 2012)
Figure: panoramic imagery with visual abnormality (by Reina).
Visual abnormality
Limited dynamic range
Parallax error

Research Presentation
1) Introduction (VR360)
a. Research Aim, Objectives
b. Literature Review and Related Work
2) Research Design and Method
3) Data Collection and Analysis
4) Discussion and Conclusion

Research Design and Method
Technical VariablesUnknown Variables
Review of Literature and Related Work
Data collection and analysis with experiment samplings
HDRI Spherical Panorama
Study: Reduction of Imaging
Variables (Obstacles, Issues)
Optimization towards Lean Approach
HDRI and Spherical Panorama
Propose Production Framework:
Combination HDRI and Spherical Panorama
Discussion and Conclusion
1 2
3

Research Design and Method:
Lean Thinking:
What and how is the production framework of lean approach in the
context for the combination of HDRI and spherical panorama?
• Value: Value of an outcome (or a product) specified by target user
• Value Stream: Value-creating activities to be remained and waste to be reduced
• Flow: Flow of production processes for possible rearrangement
• Pull: Any new value to be refined according to new requirements (continuously)
• Perfection: Effort of continuous improvement and study
According to Lean Thinking (Womack and Jones, 1996)

High Dynamic Range Imaging (HDRI)
HDRI Technical
Configurations• Observe and study the obstacles
and issues.
• Study: The process of multiple
exposures for HDRI (Debevec and
Malik, 1997; Reinhard et al, 2010;
Fairchild, 2007; Larson, 1997).
• Quantify the dynamic range for
observation (Reinhard et al, 2010;
Imatest, 2012; Gardner, 2012).
• Exploration attempts to verify the
potential dynamic range can be
extended by digital negative – RAW
(Hill, 2010; Gakken Co, 2008).
• Produce samplings of HDRI during
the data collection and analysis
process for location-based images
using architectural context as case
study subject.
• Industrial convention DSLR system
(D3x)
• Quantify and verify the latitude of
the projected dynamic range covered
by the RAW image file of D3X at
ISO100
• Observe usable dynamic range
observed (pixel value reading)
-2EV (expanded) 0EV (neutral) +2EV (expanded)

Spherical Panorama
Spherical
Panorama
Technical
Configurations• Observe and study the obstacles
and issues.
• Considerations about the spherical
panorama photographic production
process (Felinto et al, 2012; Guan et
al, 2009; Hoshi and Kato, 2006;
Andrews, 2003).
• Study: The multirow configuration
for spherical panorama production
processes that is to acquire and
combine multiple angle images
(Brown and Lowe, 2006; Chen, 1995;
Gawthrop, 2007).
• Produce samplings of spherical
panorama during the data collection
and analysis process for location-
based images using architectural
context as case study subject.
• Industrial convention DSLR system
(D3x)
• 16mm Fish-eye, Manfrotto 303SPH
for multirow setup.
Multirow setup, to
have multiple angle
image sequence
covering 360 degrees.
1 x Zenith Angle
6 x Horizontal
Angle(s)
1 x Nadir Angle

• Related studies (Felinto et al 2012, Gawthrop 2007, Brown and
Lowe 2006; Schulz, 2012) have shown that photographic
combination for HDRI and spherical panorama can be possible,
however conditionally tolerating with various obstacles and issues
as reported in previous literature and work.
Spherical Panorama Multirow HDRI requires multiple exposures
-4, -3, -2, -1, 0, +1, +2, +3, +4 (EV)
VR360: HDRI & Spherical Panorama
(Existing Production Processes, according related studies)

Data Collection and Analysis Progress
1

Data Collection & Analysis:
Measuring Dynamic Range of Instrument Used
• Quantify and verify the latitude of the projected dynamic range
covered by the digital negative of instrument used (D3X at ISO100).
• Different RAW processors tested (native, extended dynamic range).
pixelvalue
Sequence of exposure(s)
RAW Processor: C1Pro RAW Processor: ACR RAW Processor: ViewNX

Digital
Negative
(RAW)
Observations • The study implies that extending
high dynamic range is possible with
digital negative image by using
single acquisition as shown in the
initial result of measuring dynamic
range with instruments used.
• Similar to the idea brought up by
Debevec and Malik (1997) about
extending dynamic range can be
possible from conventional single
film processing - multiple scanning.
• The finding in the study about
native EV of dynamic range
reproduction was observed to have
similar approximated usable
exposure values with the RAW
produced by ISO100 according to the
test by Rehm (2009) with the same
camera equipment tested using
calibrated Stouffer Step Wedge.
• Extending Dynamic Range from
Digital Negative (RAW), observation.
• (-2 EV and +2 EV, resulting an
increment of 8.5EV to 12.5EV)
Extended Dynamic Range
Data taken from RAW Processor: C1Pro
• 8.5EV, contrast ratio 362:1
• 12.5EV, contrast ratio 5793:1 2f-stopContrast ratio =

High Dynamic Range Imaging (HDRI) with single acquisition
Figure: HDRI by retrieving extended high dynamic range from digital negative image
Blended result,
HDRI expanded from single
source of RAW.
Proposed Method and Apparatus:
monotonic mask
from RGB
channel
monotonic mask
from RGB
channel
(inverted)
masking on the
extended (-2EV) layer
on top of the neutral
(0EV) layer
masking on the
extended (+2EV) layer
on top of the neutral
(0EV) layer
RAW processing, EV parameter specified -2EV (expanded) 0EV (neutral) +2EV (expanded)
Apply Image: 50% opacity
on top of each layer.
(can be any sequence)
Example of single exposure required for this process
Global or local HDRI
reproduction. For
local HDRI, blurred
mask can be applied
(optional)
Global or local HDRI
reproduction. For
local HDRI, blurred
mask can be applied
(optional)

High Dynamic Range Imaging (HDRI) with single acquisition
Blended result,
HDRI expanded from single
source of RAW.
Proposed Method and Apparatus:
Single
Acquisition
(RAW)
Process: Extending
Dynamic Range
Extended
+2EV
Original
0EV
Extended
-2EV
HDRI from
single source
(simplified production process illustration)

(a) Conventional LDR Photo (b) Proposed HDRI from single acquisition (c) HDRI from Multiple Exposures
-2EV +2EV0EV
Proposed HDRI
using single acquisition
(HDRI extended from RAW)
HDRI (Multiple exposures)
Sampling(s)

(a) Conventional LDR Photo
(b) Proposed HDRI produced
from single acquisition
Can have moderate extended
dynamic range reproduced, least
production processes and free
from imaging errors (zero visual
abnormality).
(c) HDRI reproduced from
multiple exposures technique
Observations: HDRI
Visual abnormalities

-2EV +2EV0EV
(a) LDR (b) HDRI, single
acquisition
(c) HDRI, multiple
exposures
-2EV +2EV0EV
Proposed HDRI produced
Proposed HDRI produced
(a) LDR (b) HDRI, single
acquisition
(c) HDRI, multiple
exposures
Sampling(s) Sampling(s)

Ghosting effect /issue of HDRI
reproduction from source of
multiple exposures
HDRI reproduction from source of single
acquisition where dynamic range can be
extended from Digital Negative (RAW)
(achieve result of reduced imaging errors
while needing least production processes
and time)
Moving objects, plants, weather
Observation: HDRI
(Obstacles and Issues)

Outcome: HDRI with single acquisition using digital negative
• Reduced production processes and imaging variables.
• Avoided visual abnormalities (issues from multiple exposures).
• Imagery features can be preserved moderately in high contrast scene.
Discussion: HDRI
Single
Acquisition
(RAW)
Process: Extending
Dynamic Range
Extended
+2EV
Original
0EV
Extended
-2EV
HDRI from
single source
HDRI that requires single acquisition
Proposed Production Process DesignExisting Process
HDRI that requires multiple exposures
-4, -3, -2, -1, 0, +1, +2, +3, +4 (EV)

2

multiple angles of multirow configuration,
covering horizontal, zenith and nadir angles.
(However having nadir angle difficulty)
Spherical Panorama Sampling(s)

Parallax error
Nadir angle difficulty
(unwanted imagery content )
Observations: Spherical Panorama (cubic projection)
Insufficient dynamic range
Sampling(s)

(a) Spherical Panorama
(nadir angle difficulty resulting
visual abnormality is a mistake
that requires rectification)
(b) Spherical Panorama
(post-processing correction)
Observations: Spherical Panorama (rectification required)
Visual abnormalities after post processing correction (manual image manipulation)
Sampling(s)

• Need to have higher luminance (dynamic range) in multiple angles.
• Suggested to avoid parallax error & ghosting effect (acquisition)
Critical consideration
• Nadir angle difficulty
(if rectification required?)
• Consistency of white balance and exposures in multiple angles.
(a) (b)
Post
Processing
Correction
(Manual
Correction
Required)
Consistency white balance
Is critical for the process
Observation: Spherical Panorama
(Obstacles and Issues)

Outcome: Spherical Panorama Multirow Configuration Redesign
• Redesign that aims to reduce mistake that requires rectification
• Redesign that can reproduce the nadir (bottom) angle adequately
from the original source of the location-based subject in image.
Discussion: Spherical Panorama
Multirow with reduced obstacles and issues
Proposed Production Process DesignExisting Process
Multirow with various obstacles and issues

3

Total images required = 72 images (simulated scenario)
• 8 angles required (image sequence)
• 9 exposures required for each angle
Spherical Panorama configuration covering the
immersive 360 degrees.
HDRI that requires multiple exposures
• 9 exposures required for each angle
-4, -3, -2, -1, 0, +1, +2, +3, +4 (EV)
Combination of HDRI and Spherical Panorama
(Existing Production Process, according related studies)
Figure: combination of HDRI and spherical panorama, existing production processes.

Total images required = 11 images (sampling scenario)
• 1 exposure (RAW) required for each angle
Figure: Proposed production framework towards lean approach for the combination of HDRI and spherical panorama according to
previous studies and related works.
HDRI that requires single source acquisition
• 1 physical exposure required for each angle
(Extended dynamic range -2EV and +2EV can be
reproduced from RAW processing)
Spherical panorama configuration covering
the immersive 360 degrees.
Single
Acquisition
(RAW)
Process: Extending
Dynamic Range
Extended
+2EV
Original
0EV
Extended
-2EV
HDRI from
single source
Combination of HDRI and Spherical Panorama
(Proposed Production Framework towards Lean Approach)

Production Framework: HDRI and Spherical Panorama
multiple angles facilitated by HDRI required
for producing the spherical panorama
(zero visual abnormality)
Successful sampling(s)

Observations: HDRI and Spherical Panorama
(a) LDR photo image (b) HDRI from single source acquisition (c) HDRI from multiple exposures
Moderate extended high dynamic range,
reduced overall production processes.

HDRI ghosting effect
using multiple
exposures method.
HDRI reproduction
from single acquisition,
digital negative (RAW)
visual abnormality is
eliminated or avoided
(for multiple angles
required for spherical
panorama)

Multiple angles of multirow configuration
(from the proposed production framework)
with reduced error or mistakes that requires
rectifications, including nadir angle reproduction.
Nadir Angle:
Reproduction from the original
location-based scene, minimal
or no post-production
correction required
Sampling(s)

Inconsistent HDRI local adaptation for different angles, for spherical panorama
(multiple exposures: 9 image sequences for each angle)
Can produce consistent multiple angles of HDRI, for spherical panorama
(single acquisition: extended high dynamic range from RAW)
Sampling(s)
Sampling(s)

Elements from Lean Approach HDRI Spherical Panorama
Waste: mistakes which requires
rectification
Obstacle avoided
(Ghosting, Misalignment)
Obstacle minimized,
minimal correction
Waste: excessive productions objects or
volume that remaindered as goods pile up
Single acquisition,
reduced storage and time
Irrelevant
Waste: processing steps which are actually
not needed
Single acquisition
(instead of multiple)
New multi angles
configuration
Waste: redundant human movements of
production processes or transportation
Less activity
Faster production
process
Waste: waiting time for an processed
materials to be used in the next process
which has not been delivered in time
Reduced to single
acquisition (instead of
multiple exposures)
Single acquisition only
from HDRI for each
angle
Waste: Production outcome which do not
meet the needs according to the value
defined by the end user or customer
Moderate extended
dynamic range limited to
capability of DNG/RAW
Avoided nadir error,
geometrically
accurate/reproduced
Discussion: HDRI and Spherical Panorama
Improvements made towards Lean Approach
Table: Improvements with lean approach for the combination of HDRI and spherical panorama
Updated reference: Lean Lexicon (The Lean Enterprise Institution, 2008) and Lean Thinking (Womack and Jones, 1996)

Field Implementation: Corporate communications image for advertising visual (SP Setia, 2013)
Field Implementation

Discover MINI
Enter at your own risk of losing control

Conclusion
Proposed
Production
Framework
Before
optimization
The proposed production framework towards lean approach has
shown that the outcome of spherical panorama facilitated with
HDRI can be reproduced towards improved photographic
reproduction with least visual abnormalities. This is possible with
reduced production processes, wastes, obstacles and issues.

• Zi Siang See, Adrian David Cheok. (2014) Virtual Reality 360 Interactive Panorama Reproduction Obstacles and
Issues. Virtual Reality, Springer [indexed by SCOPUS]
• Zi Siang See, Khairul Hazrin Hashim, Kamarulzaman Ab Aziz, Adrian David Cheok (work-in-progress). Spherical
Panorama and High Dynamic Range Imaging: Lean Reproduction Approach. [Journal submission /review in
progress]
• Zi Siang See, Khairul Hazrin Hashim, Harold Thwaites, Lee Xia Sheng, Ooi Wooi Har. (2012). Retrieving
Extended High Dynamic Range From Digital Negative Image - An Experiment On Architectural Photo Imaging.
World Academy of Science, Engineering and Technology, Issue 68 August 2012, pp. 2158-2166. [indexed by
EBSCO]
• Zi Siang See, Lee Xia Sheng. (2011). Non-bias Architectural Image Archive using High Dynamic Range
Approach. 2nd International Conference on Research and Innovation in Information Systems – 2011 (ICRIIS’11)
[CD Proceedings], Kuala Lumpur, Malaysia, 23-24 November. [indexed by SCOPUS, IEEE Xplorer]
• Zi Siang See. (2011). Non-bias Scientific Imaging Archive for Urban Morphology Research. Exchanging Views on
How to Sustain Urban and Rural Environments in Aspects of Technology, Social and Ecology in Green Innovation
Research Group (GIRG) [CD-ROM], Universiti Technologi Malaysia (UTM), Malaysia.
Publications & Collaborations

Recommendations and Suggestions
for Future Research
• Explore production framework adaptation with new technologies available.
• Collaboration with academic and field researchers.
Figure: Mobile device (Android) featuring spherical panorama functionality. Technical areas of HDRI and spherical panorama are
currently the industrial interests for digital equipment developers and multimedia content producers.

Prototype (program name): See Through 360
• Prototype configuration design: See Through 360
• Capability to acquire completeness of multiple angle images including
horizontal, zenith and nadir.
Figure: “See Through 360” prototype configuration customization, program funded by Reina Imaging (2014).
(a) (b) (c)

Prototype (program name): See Through 360
• Prototype configuration design: See Through 360
• Capability to acquire completeness of multiple angle images including
horizontal, zenith and nadir.
Figure: “See Through 360” prototype configuration customization, program funded by Reina Imaging (2014).
(a) (b) (c)
Leveling
rotation nadir

Thank you
Research Discussion
Research Track
www.zisiangsee.com/research

Recommendations and Suggestions
for Future Research
• Augmented Reality Process using Using Spherical Panorama
facilitated with High Dynamic Range Imaging (HDRI)
• Cultural Heritage Reproduction Using Spherical Panorama
facilitated with High Dynamic Range Imaging (HDRI)
• Macro Virtual Reality 360 Using Spherical Panorama facilitated with
• Interactive Advertising Visual Using Spherical Panorama facilitated
with High Dynamic Range Imaging (HDRI)

Virtual Reality (VR360): Production Framework for the Combination of High Dynamic Range Imaging (HDRI) and Spherical Panorama towards Lean Approach

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (8)

Similaire à Virtual Reality (VR360): Production Framework for the Combination of High Dynamic Range Imaging (HDRI) and Spherical Panorama towards Lean Approach

Similaire à Virtual Reality (VR360): Production Framework for the Combination of High Dynamic Range Imaging (HDRI) and Spherical Panorama towards Lean Approach (20)

Dernier

Dernier (20)

Virtual Reality (VR360): Production Framework for the Combination of High Dynamic Range Imaging (HDRI) and Spherical Panorama towards Lean Approach

Notes de l'éditeur