The European project CR-PLAY involve severals academic and industrial partner to increase the research and the integration of the Image Based Rendering and the Video Based Rendering (also the delighting/relighting). Here is a resume of my engineer work in that context. From the first IBR C++ linux prototype (spixelWarp), we developed a common IBR framework to ease the creation of novel IBR/VBR algorithms on more platforms (Windows, Mac, mobile but also in virtual reality center like a CAVE). When an algorithm is mature and stable, there is a transfer to Unity 4/5 to ease the way to create video games.

1. An innovative mixed pipeline for
videogames development
Sophia Antipolis Mai 2014
Technical contributions
George Drettakis – INRIA – scientific project manager
Jerome Esnault – INRIA – R&D engineer

2. WP1
Capture of real
world data
WP2
Image Based
rendering (IBR)
WP3
Video Based
rendering (VBR)
WP4
Design and Development
(Requirements, Functional Specifications, Prototypes)
WP5
Formative and Summative Evaluation
WP6
Disseminatio
n activities
and
Exploitation
planning
WP7
Management and
coordination
VBR Tech.Req.IBR Tech.Req.Req.
Tools Feedback
Results
Results
Feedback
Results
Feedback
Feedback
MVE Tech

3. Partners Involved
Link IBR to MVE project
Research collaborations
Link IBR pipeline to
Unity3D Engine
Research
collaborations

4. WP2: Image Based Rendering framework : IBR_COMMON

5. [1] Gaurav Chaurasia, Sylvain Duchêne, Olga Sorkine-Hornung, George Drettakis,
Depth Synthesis and Local Warps for Plausible Image-based Navigation,
ACM Transactions on Graphics, Volume 32, 2013
Details here: http://www-sop.inria.fr/reves/Basilic/2013/CDSD13/
WP2: Image Based Rendering framework : IBR_COMMON
https://youtu.be/YbQDC589eb0

6. WP2: Image Based Rendering framework : IBR_COMMON

7. WP2: Image Based Rendering framework : IBR_COMMON

8. WP2: Image Based Rendering framework : IBR_COMMON

9. WP2: Image Based Rendering framework : IBR_COMMON

10. PMVS
superpixel
Depth synthesis
Depth
MVE
Multi-View
Algorithms
Choose :
Cameras parameters
Undistorted images
3D point cloud
Mesh
Depth maps images
Clipping planes
Over segmented images
Neighbors graph
Improved depth maps
Bundler
VisualSFM
Structure from
motion Algorithms
Choose :
CLOUD (gforge) :
- Update needed?
- auto download
Bundler package
VisualSFM package
PMVS package
MVE package
IBR package
Pictures
STEPS
IBR dataset
IBR dataset generation
process
(reconstruction tool)
WP4: Integration and technical validation : IBR PRE-PROCESS

11. Load bundler
Load images
Load mesh
Solver Rendering loop
ShaderLab
IBR Unity scripts
components
Depth data
Superpixel data
Solver structure
IBR Unity
binary plugin Unity game
developper
GUI editor
IBR dataset
WP4: Integration and technical validation : IBR RUNTIME

12. Final prototypes (IBR, VBR and a game which combine both). Realized with technical partners
https://youtu.be/s90PaaOMW74