SlideShare une entreprise Scribd logo
1  sur  22
On Constructing z-Dimensional
DIBR-Synthesized Images
What is DIBR?
 DIBR stands for Depth Image Based Rendering
 Image-Based Rendering (IBR) is an emerging technology which enables the
synthesis of novel realistic images of a scene from virtual viewpoints, using a
collection of available images. The applications of IBR can be found in various
situations such as virtual reality, telepresence, thanks to the complexity and
performance advantage over model-based techniques, which bases on complex 3-
D geometric models, material properties and lightening conditions of the scene
 DIBR is IBR technique which maps each color pixel in a reference view to a 2D grid
location in the virtual view, using disparity information provided by the
corresponding depth pixel.
What is Rendering and Z Dimension?
 Rendering is the process of generating an image from a 2D or 3D model (or
models in what collectively could be called a scene file) by means of computer
programs.
 Three-dimensional space (also: 3-space or, rarely, tri-dimensional space) is a
geometric setting in which three values (called parameters) are required to
determine the position of an element (i.e., point). This is the informal meaning of
the term dimension.
CONSTRUCTION OF Z DIMENSION
 To construct this new image type, we first perform a new DIBR pixel-mapping for z-dimensional
camera movement.
 We then identify expansion holes—a new kind of missing pixels unique in z-dimensional DIBR-
mapped images—using a depth layering procedure.
 To fill expansion holes we formulate a patch-based maximum a posteriori problem, where the
patches are appropriately spaced using diamond tiling.
 Leveraging on recent advances in graph signal processing, we define a graph-signal smoothness
prior to regularize the inverse problem.
 Finally, we design a fast iterative reweighted least square algorithm to solve the posed problem
efficiently. Experimental results show that our z-dimensional synthesized images outperform
images rendered by a native modification
Virtual Camera
 a virtual camera system aims at controlling a camera or a set of cameras to display
a view of a 3D virtual world. Camera systems where their purpose is to show the
action at the best possible angle; more generally, they are used in 3D virtual worlds
when a third person view is required.
Constructing z-Dimensional DIBR-Synthesized
Images
Divided into three Sections
1.DIBR (Depth Image Based Rendering)
2.Image Super Resolution
3.Graph Based Image Processing
Depth Image Based Rendering
 Color-plus-depth format , consisting of one or more color and depth image pairs
from different viewpoints, is a widely used 3D scene representation. Using this
format, low-complexity DIBR view synthesis procedure such as 3D warping [ can be
used to create credible virtual view images, with the aid of in painting algorithms to
complete disocclusion holes
 In this work, we assume that enough pixels from one or more reference view(s)
have been transmitted to the decoder for virtual view synthesis, and we focus only
on the construction of z-dimensional DIBR-synthesized images given received
reference view pixels.
Image Super Resolution
 Increase in object size due to large z-dimensional virtual camera movement is
analogous to increasing the resolution (super-resolution (SR)) of the whole image.
However, during z-dimensional camera motion an object closer to the camera
increases in size faster than objects farther away, while in SR, resolution is increased
uniformly for all spatial regions in the image.
 For the above reason, we cannot directly apply conventional image SR techniques
[30] in rectangular pixel grid to interpolate the synthesized view. Further, recent
non-local SR techniques such as leveraging on self-similarity of natural images that
require an exhaustive search of similar patches throughout an image tend to be
computationally expensive. In contrast, our interpolation scheme performs only
iterative local filtering, and thus is significantly more computation-efficient.
Graph Based Image Processing
 GSP is the study of signals that live on structured data kernels described by graphs
, leveraging on spectral graph theory for frequency analysis of graph-signals.
 Graph-signal priors have been derived for inverse problems such as denoising ,
interpolation , bit-depth enhancement and de-quantization.
 In this work, we assume the latter case and construct a suitable graph G from
available DIBR-synthesized pixels for joint denoising/interpolation of pixels in a
target patch.
SYSTEM OVERVIEW
 Interactive Free Viewpoint Streaming System
 DIBR
 Rounding Noise in mapped pixels
 Identification of expansion holes
Interactive Free Viewpoint Streaming System
Depth-Image-Based Rendering
Rounding Noise in DIBR-Mapped Pixels
Identification of Expansion Holes
CONCLUSION
 Unlike typical free viewpoint system that considers only synthesis of virtual views
shifted horizontally along the x dimension via DIBR, in this paper we consider in
addition construction of z-dimensional DIBR-synthesized images. In such far-to-
near viewpoint synthesis, there exists a new type of missing pixels called expansion
holes – where objects close to the camera will increase in size and simple pixel-to-
pixel mapping in DIBR from reference to virtual view will result in missing pixel
areas – that demand a new interpolation scheme.
THANKS
 P. Merkle, A. Smolic, K. Mueller, and T. Wiegand, “Multi-view video plus depth
representation and coding,
 A. Chuchvara, M. Georgiev, and A. Gotchev, “CPU-efficient free view synthesis based
on depth layering,” in Proc. 3DTV-Conf: True Vis. - Capture, Transmiss. Display 3D
Video, Jul. 2014,
 M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal
Process. Mag., vol. 28, no. 1, pp. 67–76, Jan. 2011.
REFERENCES

Contenu connexe

Tendances

COMPLETE END-TO-END LOW COST SOLUTION TO A 3D SCANNING SYSTEM WITH INTEGRATED...
COMPLETE END-TO-END LOW COST SOLUTION TO A 3D SCANNING SYSTEM WITH INTEGRATED...COMPLETE END-TO-END LOW COST SOLUTION TO A 3D SCANNING SYSTEM WITH INTEGRATED...
COMPLETE END-TO-END LOW COST SOLUTION TO A 3D SCANNING SYSTEM WITH INTEGRATED...
ijcsit
 
study Diffusion Curves: A Vector Representation for Smooth-Shaded Images
study Diffusion Curves: A Vector Representation for Smooth-Shaded Imagesstudy Diffusion Curves: A Vector Representation for Smooth-Shaded Images
study Diffusion Curves: A Vector Representation for Smooth-Shaded Images
Chiamin Hsu
 
mvitelli_ee367_final_report
mvitelli_ee367_final_reportmvitelli_ee367_final_report
mvitelli_ee367_final_report
Matt Vitelli
 

Tendances (18)

3-d interpretation from single 2-d image for autonomous driving
3-d interpretation from single 2-d image for autonomous driving3-d interpretation from single 2-d image for autonomous driving
3-d interpretation from single 2-d image for autonomous driving
 
3 d display-methods-in-computer-graphics(For DIU)
3 d display-methods-in-computer-graphics(For DIU)3 d display-methods-in-computer-graphics(For DIU)
3 d display-methods-in-computer-graphics(For DIU)
 
Deep learning for image video processing
Deep learning for image video processingDeep learning for image video processing
Deep learning for image video processing
 
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...
[3D勉強会@関東] Deep Reinforcement Learning of Volume-guided Progressive View Inpa...
 
Object Detection for Service Robot Using Range and Color Features of an Image
Object Detection for Service Robot Using Range and Color Features of an ImageObject Detection for Service Robot Using Range and Color Features of an Image
Object Detection for Service Robot Using Range and Color Features of an Image
 
Object detection for service robot using range and color features of an image
Object detection for service robot using range and color features of an imageObject detection for service robot using range and color features of an image
Object detection for service robot using range and color features of an image
 
GRPHICS02 - Creating 3D Graphics
GRPHICS02 - Creating 3D GraphicsGRPHICS02 - Creating 3D Graphics
GRPHICS02 - Creating 3D Graphics
 
GRPHICS05 - Rendering (2)
GRPHICS05 - Rendering (2)GRPHICS05 - Rendering (2)
GRPHICS05 - Rendering (2)
 
Optic flow estimation with deep learning
Optic flow estimation with deep learningOptic flow estimation with deep learning
Optic flow estimation with deep learning
 
6 texture mapping computer graphics
6 texture mapping computer graphics6 texture mapping computer graphics
6 texture mapping computer graphics
 
Normal Mapping / Computer Graphics - IK
Normal Mapping / Computer Graphics - IKNormal Mapping / Computer Graphics - IK
Normal Mapping / Computer Graphics - IK
 
Multimedia content based retrieval in digital libraries
Multimedia content based retrieval in digital librariesMultimedia content based retrieval in digital libraries
Multimedia content based retrieval in digital libraries
 
COMPLETE END-TO-END LOW COST SOLUTION TO A 3D SCANNING SYSTEM WITH INTEGRATED...
COMPLETE END-TO-END LOW COST SOLUTION TO A 3D SCANNING SYSTEM WITH INTEGRATED...COMPLETE END-TO-END LOW COST SOLUTION TO A 3D SCANNING SYSTEM WITH INTEGRATED...
COMPLETE END-TO-END LOW COST SOLUTION TO A 3D SCANNING SYSTEM WITH INTEGRATED...
 
study Diffusion Curves: A Vector Representation for Smooth-Shaded Images
study Diffusion Curves: A Vector Representation for Smooth-Shaded Imagesstudy Diffusion Curves: A Vector Representation for Smooth-Shaded Images
study Diffusion Curves: A Vector Representation for Smooth-Shaded Images
 
Deep learning for 3-D Scene Reconstruction and Modeling
Deep learning for 3-D Scene Reconstruction and Modeling Deep learning for 3-D Scene Reconstruction and Modeling
Deep learning for 3-D Scene Reconstruction and Modeling
 
mvitelli_ee367_final_report
mvitelli_ee367_final_reportmvitelli_ee367_final_report
mvitelli_ee367_final_report
 
Deep vo and slam ii
Deep vo and slam iiDeep vo and slam ii
Deep vo and slam ii
 
ei2106-submit-opt-415
ei2106-submit-opt-415ei2106-submit-opt-415
ei2106-submit-opt-415
 

En vedette (7)

Mahendran Chap 06 Output
Mahendran Chap 06 OutputMahendran Chap 06 Output
Mahendran Chap 06 Output
 
Sergey A. Sukhanov, "3D content production"
Sergey A. Sukhanov, "3D content production"Sergey A. Sukhanov, "3D content production"
Sergey A. Sukhanov, "3D content production"
 
1240934 634561225344818750
1240934 6345612253448187501240934 634561225344818750
1240934 634561225344818750
 
Computer graphics1
Computer graphics1Computer graphics1
Computer graphics1
 
Graphics Output Hardware Devices
Graphics Output Hardware DevicesGraphics Output Hardware Devices
Graphics Output Hardware Devices
 
Graphics input and output devices
Graphics input and output devicesGraphics input and output devices
Graphics input and output devices
 
I/O devices - Computer graphics
I/O devices -  Computer graphicsI/O devices -  Computer graphics
I/O devices - Computer graphics
 

Similaire à On constructing z dimensional Image By DIBR Synthesized Images

Stereo Correspondence Algorithms for Robotic Applications Under Ideal And Non...
Stereo Correspondence Algorithms for Robotic Applications Under Ideal And Non...Stereo Correspondence Algorithms for Robotic Applications Under Ideal And Non...
Stereo Correspondence Algorithms for Robotic Applications Under Ideal And Non...
CSCJournals
 
10.1109@ICCMC48092.2020.ICCMC-000167.pdf
10.1109@ICCMC48092.2020.ICCMC-000167.pdf10.1109@ICCMC48092.2020.ICCMC-000167.pdf
10.1109@ICCMC48092.2020.ICCMC-000167.pdf
mokamojah
 
Focused Image Creation Algorithms for digital holography
Focused Image Creation Algorithms for digital holographyFocused Image Creation Algorithms for digital holography
Focused Image Creation Algorithms for digital holography
Conor Mc Elhinney
 
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrate...
 Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrate... Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrate...
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrate...
AIRCC Publishing Corporation
 
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrated...
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrated...Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrated...
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrated...
AIRCC Publishing Corporation
 

Similaire à On constructing z dimensional Image By DIBR Synthesized Images (20)

3-d interpretation from single 2-d image III
3-d interpretation from single 2-d image III3-d interpretation from single 2-d image III
3-d interpretation from single 2-d image III
 
Stereo Correspondence Algorithms for Robotic Applications Under Ideal And Non...
Stereo Correspondence Algorithms for Robotic Applications Under Ideal And Non...Stereo Correspondence Algorithms for Robotic Applications Under Ideal And Non...
Stereo Correspondence Algorithms for Robotic Applications Under Ideal And Non...
 
10.1109@ICCMC48092.2020.ICCMC-000167.pdf
10.1109@ICCMC48092.2020.ICCMC-000167.pdf10.1109@ICCMC48092.2020.ICCMC-000167.pdf
10.1109@ICCMC48092.2020.ICCMC-000167.pdf
 
3-d interpretation from single 2-d image IV
3-d interpretation from single 2-d image IV3-d interpretation from single 2-d image IV
3-d interpretation from single 2-d image IV
 
[DL輪読会]ClearGrasp
[DL輪読会]ClearGrasp[DL輪読会]ClearGrasp
[DL輪読会]ClearGrasp
 
Cj36511514
Cj36511514Cj36511514
Cj36511514
 
3-d interpretation from single 2-d image for autonomous driving II
3-d interpretation from single 2-d image for autonomous driving II3-d interpretation from single 2-d image for autonomous driving II
3-d interpretation from single 2-d image for autonomous driving II
 
Dense Visual Odometry Using Genetic Algorithm
Dense Visual Odometry Using Genetic AlgorithmDense Visual Odometry Using Genetic Algorithm
Dense Visual Odometry Using Genetic Algorithm
 
Image Interpolation Techniques with Optical and Digital Zoom Concepts -semina...
Image Interpolation Techniques with Optical and Digital Zoom Concepts -semina...Image Interpolation Techniques with Optical and Digital Zoom Concepts -semina...
Image Interpolation Techniques with Optical and Digital Zoom Concepts -semina...
 
Large scale 3 d point cloud compression using adaptive radial distance predic...
Large scale 3 d point cloud compression using adaptive radial distance predic...Large scale 3 d point cloud compression using adaptive radial distance predic...
Large scale 3 d point cloud compression using adaptive radial distance predic...
 
Gesture Recognition Based Mouse Events
Gesture Recognition Based Mouse EventsGesture Recognition Based Mouse Events
Gesture Recognition Based Mouse Events
 
OBJECT DETECTION FOR SERVICE ROBOT USING RANGE AND COLOR FEATURES OF AN IMAGE
OBJECT DETECTION FOR SERVICE ROBOT USING RANGE AND COLOR FEATURES OF AN IMAGEOBJECT DETECTION FOR SERVICE ROBOT USING RANGE AND COLOR FEATURES OF AN IMAGE
OBJECT DETECTION FOR SERVICE ROBOT USING RANGE AND COLOR FEATURES OF AN IMAGE
 
Focused Image Creation Algorithms for digital holography
Focused Image Creation Algorithms for digital holographyFocused Image Creation Algorithms for digital holography
Focused Image Creation Algorithms for digital holography
 
Web Image Retrieval Using Visual Dictionary
Web Image Retrieval Using Visual DictionaryWeb Image Retrieval Using Visual Dictionary
Web Image Retrieval Using Visual Dictionary
 
Web Image Retrieval Using Visual Dictionary
Web Image Retrieval Using Visual DictionaryWeb Image Retrieval Using Visual Dictionary
Web Image Retrieval Using Visual Dictionary
 
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...
 
A Wireless Network Infrastructure Architecture for Rural Communities
A Wireless Network Infrastructure Architecture for Rural CommunitiesA Wireless Network Infrastructure Architecture for Rural Communities
A Wireless Network Infrastructure Architecture for Rural Communities
 
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrate...
 Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrate... Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrate...
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrate...
 
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrated...
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrated...Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrated...
Complete End-to-End Low Cost Solution to a 3D Scanning System with Integrated...
 
Simulation of collision avoidance by navigation
Simulation of collision avoidance by navigationSimulation of collision avoidance by navigation
Simulation of collision avoidance by navigation
 

Dernier

Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Dernier (20)

The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 

On constructing z dimensional Image By DIBR Synthesized Images

  • 2. What is DIBR?  DIBR stands for Depth Image Based Rendering  Image-Based Rendering (IBR) is an emerging technology which enables the synthesis of novel realistic images of a scene from virtual viewpoints, using a collection of available images. The applications of IBR can be found in various situations such as virtual reality, telepresence, thanks to the complexity and performance advantage over model-based techniques, which bases on complex 3- D geometric models, material properties and lightening conditions of the scene  DIBR is IBR technique which maps each color pixel in a reference view to a 2D grid location in the virtual view, using disparity information provided by the corresponding depth pixel.
  • 3. What is Rendering and Z Dimension?  Rendering is the process of generating an image from a 2D or 3D model (or models in what collectively could be called a scene file) by means of computer programs.  Three-dimensional space (also: 3-space or, rarely, tri-dimensional space) is a geometric setting in which three values (called parameters) are required to determine the position of an element (i.e., point). This is the informal meaning of the term dimension.
  • 4. CONSTRUCTION OF Z DIMENSION  To construct this new image type, we first perform a new DIBR pixel-mapping for z-dimensional camera movement.  We then identify expansion holes—a new kind of missing pixels unique in z-dimensional DIBR- mapped images—using a depth layering procedure.  To fill expansion holes we formulate a patch-based maximum a posteriori problem, where the patches are appropriately spaced using diamond tiling.  Leveraging on recent advances in graph signal processing, we define a graph-signal smoothness prior to regularize the inverse problem.  Finally, we design a fast iterative reweighted least square algorithm to solve the posed problem efficiently. Experimental results show that our z-dimensional synthesized images outperform images rendered by a native modification
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10. Virtual Camera  a virtual camera system aims at controlling a camera or a set of cameras to display a view of a 3D virtual world. Camera systems where their purpose is to show the action at the best possible angle; more generally, they are used in 3D virtual worlds when a third person view is required.
  • 11.
  • 12. Constructing z-Dimensional DIBR-Synthesized Images Divided into three Sections 1.DIBR (Depth Image Based Rendering) 2.Image Super Resolution 3.Graph Based Image Processing
  • 13. Depth Image Based Rendering  Color-plus-depth format , consisting of one or more color and depth image pairs from different viewpoints, is a widely used 3D scene representation. Using this format, low-complexity DIBR view synthesis procedure such as 3D warping [ can be used to create credible virtual view images, with the aid of in painting algorithms to complete disocclusion holes  In this work, we assume that enough pixels from one or more reference view(s) have been transmitted to the decoder for virtual view synthesis, and we focus only on the construction of z-dimensional DIBR-synthesized images given received reference view pixels.
  • 14. Image Super Resolution  Increase in object size due to large z-dimensional virtual camera movement is analogous to increasing the resolution (super-resolution (SR)) of the whole image. However, during z-dimensional camera motion an object closer to the camera increases in size faster than objects farther away, while in SR, resolution is increased uniformly for all spatial regions in the image.  For the above reason, we cannot directly apply conventional image SR techniques [30] in rectangular pixel grid to interpolate the synthesized view. Further, recent non-local SR techniques such as leveraging on self-similarity of natural images that require an exhaustive search of similar patches throughout an image tend to be computationally expensive. In contrast, our interpolation scheme performs only iterative local filtering, and thus is significantly more computation-efficient.
  • 15. Graph Based Image Processing  GSP is the study of signals that live on structured data kernels described by graphs , leveraging on spectral graph theory for frequency analysis of graph-signals.  Graph-signal priors have been derived for inverse problems such as denoising , interpolation , bit-depth enhancement and de-quantization.  In this work, we assume the latter case and construct a suitable graph G from available DIBR-synthesized pixels for joint denoising/interpolation of pixels in a target patch.
  • 16. SYSTEM OVERVIEW  Interactive Free Viewpoint Streaming System  DIBR  Rounding Noise in mapped pixels  Identification of expansion holes
  • 17. Interactive Free Viewpoint Streaming System
  • 19. Rounding Noise in DIBR-Mapped Pixels
  • 21. CONCLUSION  Unlike typical free viewpoint system that considers only synthesis of virtual views shifted horizontally along the x dimension via DIBR, in this paper we consider in addition construction of z-dimensional DIBR-synthesized images. In such far-to- near viewpoint synthesis, there exists a new type of missing pixels called expansion holes – where objects close to the camera will increase in size and simple pixel-to- pixel mapping in DIBR from reference to virtual view will result in missing pixel areas – that demand a new interpolation scheme.
  • 22. THANKS  P. Merkle, A. Smolic, K. Mueller, and T. Wiegand, “Multi-view video plus depth representation and coding,  A. Chuchvara, M. Georgiev, and A. Gotchev, “CPU-efficient free view synthesis based on depth layering,” in Proc. 3DTV-Conf: True Vis. - Capture, Transmiss. Display 3D Video, Jul. 2014,  M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal Process. Mag., vol. 28, no. 1, pp. 67–76, Jan. 2011. REFERENCES