SlideShare une entreprise Scribd logo
1  sur  97
Télécharger pour lire hors ligne
www.crs4.it/vic/




 Tecnologie di Visual Computing
       per i Beni Culturali


R. Pintus
CRS4 Visual Computing
R. Pintus – CRS4/ViC, October 2012




Tecnologie per i beni culturali
• Focus: digitalizzazione accurata (forma e
  colore) di siti e manufatti + …
  – Partire dai dati: Acquisizione -> Trattamento !
  – Modelli misurabili
• Molti usi oltre la visualizzazione
  – Riproduzione materica
  – Studio di opere d’arte
  – Documentazione in-situ di scavi archeologici
  – Supporto al restauro e sua documentazione
  – Valorizzazione
R. Pintus – CRS4/ViC, October 2012




Tecnologie per i beni culturali
• Le quantità di dati prodotte dai moderni sensori
  sono però difficili da trattare, archiviare,
  distribuire, visualizzare
  – Scalabilità!
• Tecniche attuali sub-ottimali
  – Costi, tempi, qualità
• Bisogno di ricerca in tecnologie abilitanti
  scalabili
  –   Acquisizione
  –   Processamento geometrico
  –   Visualizzazione
  –   …
R. Pintus – CRS4/ViC, October 2012




Tecnologie per i beni culturali




•   Come acquisire e processare efficacemente forma e colore di
    siti e manufatti?
    – Tecniche di fusione multi-sensore, stream-processing, multiresolution, external
      memory algorithms, parallel programming, GPGPUs
•   Come archiviare e distribuire efficacemente i modelli?
    – Multiresolution, adaptive streaming, compression
•   Come visualizzarli efficacemente?
    – Multiresolution, adaptive rendering, out-of-core methods, GPU programming,
      parallelization, rasterization, ray-casting
•   Come esplorarli?
    – Novel 3D displays, specific interaction techniques
    – Portable devices
R. Pintus – CRS4/ViC, October 2012




Alcuni esempi
•   Allineamento geometria/colore
•   Colorazione di modelli 3D
•   Fusione di dati e ricostruzione geometrica
•   Visualizzazione scalabile ed interattiva
•   Distribuzione di dati in rete
•   Esplorazione su display innovativi

(… e molto altro)
R. Pintus – CRS4/ViC, October 2012




Our Goal




                                            6
R. Pintus – CRS4/ViC, October 2012




Modelling vs Acquisition
                            Modelling
                            Subjective Reality




        Acquisition
        Objective reality
                                                                       7
R. Pintus – CRS4/ViC, October 2012




3D Reconstruction
• Acquire geometry and color
• A lot of techniques
  – Structured light, laser scanning (triangulation or
    time-of-flight), photometric stereo, shape-from-X,
    …
• Which technique?
  –   Object type (big/small, material….)
  –   Cost
  –   Accuracy/Resolution
  –   Time
  –   Complexity



                                                                        8
R. Pintus – CRS4/ViC, October 2012




Outline
• 3D Reconstruction Techniques

• 3D Reconstruction Pipeline
  – Photo mapping/blending
  – Printing


• Case study




                                                                  9
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring machines
         (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X              • Transmissive
           –   Stereo                     – Computed Tomography (CT)
           –   Multiview                  – Transmissive Ultrasound
           –   Silhouettes         • Reflective
           –   Focus/Defocus              – Non-Optical Methods
                                                » reflective ultrasound, radar, sonar, MRI
                                          – Time-of-Flight
                                          – Triangulation
                                                » laser striping
                                                » structured lighting
                                          – Photometric Stereo


                                                                                             10
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring
         machines (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X              • Transmissive
           –   Stereo                     – Computed Tomography (CT)
           –   Multiview                  – Transmissive Ultrasound
           –   Silhouettes         • Reflective
           –   Focus/Defocus              – Non-Optical Methods
                                                » reflective ultrasound, radar, sonar, MRI
                                          – Time-of-Flight
                                          – Triangulation
                                                » laser striping
                                                » structured lighting
                                          – Photometric Stereo


                                                                                             11
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
      • rulers, calipers, pantographs, coordinate measuring machines
        (CMM), AFM
 • Non-Contact
   – Passive                   – Active
      • Shape-from-X               • Transmissive
          –    Stereo                     – Computed Tomography (CT)
          –    Multiview                  – Transmissive Ultrasound
          –    Silhouettes         • Reflective
          –    Focus/Defocus              – Non-Optical Methods
                                                » reflective ultrasound, radar, sonar, MRI
                                          – Time-of-Flight
                                          – Triangulation
                                                » laser striping
                                                » structured lighting
                                          – Photometric Stereo


                                                                                             12
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Stereo




                                                    13
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Stereo




                                                    14
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
      • rulers, calipers, pantographs, coordinate measuring machines
        (CMM), AFM
 • Non-Contact
   – Passive                   – Active
      • Shape-from-X               • Transmissive
          –    Stereo                     – Computed Tomography (CT)
          –    Multiview                  – Transmissive Ultrasound
          –    Silhouettes         • Reflective
          –    Focus/Defocus              – Non-Optical Methods
                                                » reflective ultrasound, radar, sonar, MRI
                                          – Time-of-Flight
                                          – Triangulation
                                                » laser striping
                                                » structured lighting
                                          – Photometric Stereo


                                                                                             15
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Multiview




                                                       16
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Multiview




                                                       17
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
      • rulers, calipers, pantographs, coordinate measuring machines
        (CMM), AFM
 • Non-Contact
   – Passive                   – Active
      • Shape-from-X               • Transmissive
          –    Stereo                     – Computed Tomography (CT)
          –    Multiview                  – Transmissive Ultrasound
          –    Silhouettes         • Reflective
          –    Focus/Defocus              – Non-Optical Methods
                                                » reflective ultrasound, radar, sonar, MRI
                                          – Time-of-Flight
                                          – Triangulation
                                                » laser striping
                                                » structured lighting
                                          – Photometric Stereo


                                                                                             18
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Silhouettes




                                                         19
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
      • rulers, calipers, pantographs, coordinate measuring machines
        (CMM), AFM
 • Non-Contact
   – Passive                   – Active
      • Shape-from-X               • Transmissive
          –    Stereo                     – Computed Tomography (CT)
          –    Multiview                  – Transmissive Ultrasound
          –    Silhouettes         • Reflective
          –    Focus/Defocus              – Non-Optical Methods
                                                » reflective ultrasound, radar, sonar, MRI
                                          – Time-of-Flight
                                          – Triangulation
                                                » laser striping
                                                » structured lighting
                                          – Photometric Stereo


                                                                                             20
R. Pintus – CRS4/ViC, October 2012



Taxonomy – Depth from
focus/defocus




                                                        21
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring machines
         (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X             • Transmissive
           –   Stereo                  – Computed Tomography (CT)
           –   Multiview               – Transmissive Ultrasound
           –   Silhouettes        • Reflective
           –   Focus/Defocus           – Non-Optical Methods
                                             » reflective ultrasound, radar, sonar, MRI
                                       – Time-of-Flight
                                       – Triangulation
                                             » laser striping
                                             » structured lighting
                                       – Photometric Stereo


                                                                                          22
R. Pintus – CRS4/ViC, October 2012




 Taxonomy – Transmissive
Computed Tomography




                      Density Function



                            Trasmissive Ultrasound


                                                                         23
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring machines
         (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X             • Transmissive
           –   Stereo                 – Computed Tomography (CT)
           –   Multiview              – Transmissive Ultrasound
           –   Silhouettes        • Reflective
           –   Focus/Defocus          – Non-Optical Methods
                                            » reflective ultrasound, radar, sonar, MRI
                                      – Time-of-Flight
                                      – Triangulation
                                            » laser striping
                                            » structured lighting
                                      – Photometric Stereo


                                                                                         24
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Non-Optical
           Non-




 Ultrasound    Radar                 MRI


                                                         25
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring machines
         (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X             • Transmissive
           –   Stereo                 – Computed Tomography (CT)
           –   Multiview              – Transmissive Ultrasound
           –   Silhouettes        • Reflective
           –   Focus/Defocus          – Non-Optical Methods
                                            » reflective ultrasound, radar, sonar, MRI
                                      – Time-of-Flight
                                      – Triangulation
                                            » laser striping
                                            » structured lighting
                                      – Photometric Stereo


                                                                                         26
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Time-of-Flight
           Time-of-




     2d    5.0m
t=      =           ≈ 17ns
      c 3 × 108 m s




                                                             27
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring machines
         (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X             • Transmissive
           –   Stereo                 – Computed Tomography (CT)
           –   Multiview              – Transmissive Ultrasound
           –   Silhouettes        • Reflective
           –   Focus/Defocus          – Non-Optical Methods
                                           » reflective ultrasound, radar, sonar, MRI
                                      – Time-of-Flight
                                      – Triangulation
                                           » laser striping
                                           » structured lighting
                                      – Photometric Stereo


                                                                                        28
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Laser Striping




                                                        29
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring machines
         (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X             • Transmissive
           –   Stereo                 – Computed Tomography (CT)
           –   Multiview              – Transmissive Ultrasound
           –   Silhouettes        • Reflective
           –   Focus/Defocus          – Non-Optical Methods
                                           » reflective ultrasound, radar, sonar, MRI
                                      – Time-of-Flight
                                      – Triangulation
                                           » laser striping
                                           » structured lighting
                                      – Photometric Stereo


                                                                                        30
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Structured Lighting




                                                        31
R. Pintus – CRS4/ViC, October 2012




Taxonomy (non-destructive)
         (non-

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring machines
         (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X             • Transmissive
           –   Stereo                 – Computed Tomography (CT)
           –   Multiview              – Transmissive Ultrasound
           –   Silhouettes        • Reflective
           –   Focus/Defocus          – Non-Optical Methods
                                            » reflective ultrasound, radar, sonar, MRI
                                      – Time-of-Flight
                                      – Triangulation
                                            » laser striping
                                            » structured lighting
                                      – Photometric Stereo


                                                                                         32
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Photometric Stereo




                                                       33
R. Pintus – CRS4/ViC, October 2012




Photometric Stereo – SEM




                                                           34
R. Pintus – CRS4/ViC, October 2012




Taxonomy – Photometric Stereo




                                                       35
R. Pintus – CRS4/ViC, October 2012




Taxonomy SEM

 • Contact
   – Direct Measurements
       • rulers, calipers, pantographs, coordinate measuring machines
         (CMM), AFM
 • Non-Contact
   – Passive                   – Active
       • Shape-from-X              • Transmissive
           –   Stereo                     – Computed Tomography (CT)
           –   Multiview                  – Transmissive Ultrasound
           –   Silhouettes         • Reflective
           –   Focus/Defocus              – Non-Optical Methods
                                                » reflective ultrasound, radar, sonar, MRI
                                          – Time-of-Flight
                                          – Triangulation
                                                » laser striping
                                                » structured lighting
                                          – Photometric Stereo


                                                                                             36
R. Pintus – CRS4/ViC, October 2012




Cultural Heritage
• Techniques
  –   Triangulation (laser scanner)
  –   Time of Flight
  –   Texture Mapping
  –   Multi-view reconstruction
  –   Photometric Stereo
• Deal with multiple acquisitions
• Manage a huge amount of data for visualization
  purposes




                                                                      37
R. Pintus – CRS4/ViC, October 2012




3D Reconstruction Pipeline
Real Object          Acquisition Devices
                                               Photos




3D Digital Model                                Geometry
                   === Processing ===
                         -Cleaning
                         - Merging
                     - Photo Alignment
                     - Color Projection
                            -…
                                                                          38
R. Pintus – CRS4/ViC, October 2012




3D Reconstruction Pipeline
•   Real Model Inspection (onsite)

•   Scans design (offsite/onsite)

•   Acquisition (onsite)

•   Alignment (offsite)

•   Editing (offsite)

•   Merge (offsite)

•   Texture (offsite)

•   Final Model (offsite)

•   3D Printing (offsite)




                                                                     39
R. Pintus – CRS4/ViC, October 2012




3D Reconstruction Pipeline
•   Real Model Inspection (onsite)

•   Scans design (offsite/onsite)

•   Acquisition (onsite)

•   Alignment (offsite)

•   Editing (offsite)

•   Merge (offsite)

•   Texture (offsite)

•   Final Model (offsite)

•   3D Printing (offsite)




                                                                     40
R. Pintus – CRS4/ViC, October 2012




Goal
• Fast and low-cost technique for creating
  accurate colored models
• Acquisition
  – 3D – laser scanners
  – Color – digital cameras
• Mapping photo-to-geometry
  – Fast and Robust Semi-Automatic Registration of Photographs
    to 3D Geometry
• Photo blending
  – A Streaming Framework for Seamless Detailed Photo
    Blending on Massive Point Clouds
www.crs4.it/vic/




                          Photo Mapping



Ruggero Pintus, Enrico Gobbetti, and Roberto Combet. “Fast and Robust Semi-Automatic
Registration of Photographs to 3D Geometry”. In The 12th International Symposium on Virtual
Reality, Archaeology and Cultural Heritage, October 2011.
R. Pintus – CRS4/ViC, October 2012



              Problem Statement

3D Geometry                              Unordered Set
                                        Of n Uncalibrated
                                             Photos




              n Camera Poses
              (2D/3D Registration)
R. Pintus – CRS4/ViC, October 2012




Related work
• Manual selection of 2D-3D matches
  – Massive user intervention – Tiring and time-consuming
• Automatic feature matching
  – Not robust enough for a generic dataset
• Semi-automatic statistical correlation
  – Point cloud attributes not always provided
• Geometric multi-view reconstruction
  – 2D-3D problem    3D-3D registration task
  – dense and ordered frame sequence
• Our contribution
  – Minimize user intervention / Large datasets / Semi-
    automatic / Multi-view based approach / No Attributes
R. Pintus – CRS4/ViC, October 2012



                   Input Data

User Dense 3D n Photos   • Dense Geometry
                           – Point cloud, triangle
                             mesh, etc.
  SfM Reconstruction
                           – No attributes
                           – No particular features
  Coarse Registration    • n photos
                           – Naïve constraints:
     Refinement               • Blur, Noise, Under- or
                                over-exposured
                           – Sufficient overlap
     Output Data
R. Pintus – CRS4/ViC, October 2012



                   Multi-
                   Multi-view

User Dense 3D n Photos   • Bundler [Snavely et al.
                           2006]
                           – SfM system for unordered
  SfM Reconstruction         image collections
                           – http://phototour.cs.washingto
                             n.edu/bundler/
  Coarse Registration    • Output
                           – A sparse point cloud
                           – n camera poses
     Refinement            – SIFT keypoints (projections of
                             sparse 3D points)

     Output Data
R. Pintus – CRS4/ViC, October 2012



                   Coarse registration

User Dense 3D n Photos   • Register two point clouds
                           with different:
                            – scales
                            – reference frames
  SfM Reconstruction        – resolutions
                         • Automatic methods are not
                           robust and efficient enough
  Coarse Registration    • User aligns few images (one
                           or more) to the dense
                           geometry
     Refinement          • Affine transformation is
                           applied to all cameras and
                           sparse points
     Output Data
R. Pintus – CRS4/ViC, October 2012




Refinement
User Dense 3D n Photos                                                              Pj
                              C1
                         Q (C2 , p j )                   s1, j
                                                                             pj
  SfM Reconstruction
                                    s2 , j                                  NN F ( p j )


  Coarse Registration

                         Q (C2 , NN F ( p j ))
     Refinement                             C2



                         E (C , P ) = ∑∑ vij Q (Ci , NN F ( p j )) − si , j
                                             N P NC
                                                                                           2
     Output Data                             j =1 i =1
R. Pintus – CRS4/ViC, October 2012



                   Refinement

User Dense 3D n Photos   • Sparse Bundle
                           Adjustment (SBA)
                           – Constants – SIFT keypoints,
  SfM Reconstruction         dense 3D points
                           – Variables – Camera poses,
                             sparse 3D points
  Coarse Registration      – SBA
                              • A Generic SBA C/C++ Package
                                Based on the Levenberg-
                                Marquardt Algorithm
     Refinement               • http://www.ics.forth.gr/~loura
                                kis/sba/

     Output Data
R. Pintus – CRS4/ViC, October 2012



                   Output data

User Dense 3D n Photos   • n camera poses

  SfM Reconstruction     • Input of photo
                           blending
                           – n photos
  Coarse Registration
                           – n camera poses
                           – Dense 3D geometry
     Refinement


     Output Data
R. Pintus – CRS4/ViC, October 2012




Results – Photo mapping
www.crs4.it/vic/




                          Photo Blending



Ruggero Pintus, Enrico Gobbetti, and Marco Callieri. A Streaming Framework for Seamless Detailed
Photo Blending on Massive Point Clouds. In Proc. Eurographics Area Papers. Pages 25- 32, 2011.
R. Pintus – CRS4/ViC, October 2012



              Problem Statement

Point Cloud      Calibrated
                  Photos
R. Pintus – CRS4/ViC, October 2012



              Problem Statement

Point Cloud      Calibrated
                  Photos
   P
R. Pintus – CRS4/ViC, October 2012



              Problem Statement

                 Calibrated      Colored
Point Cloud
                  Photos        Point Cloud
   P
R. Pintus – CRS4/ViC, October 2012



                       Problem Statement

                              Calibrated               Colored
    Point Cloud
                               Photos                 Point Cloud
       P




•    Problem    Unlimited size of 3D model (Gpoints) and unlimited
     number of images
R. Pintus – CRS4/ViC, October 2012




Related work
• State-of-the-art techniques
   – Image quality estimation
   – Stitching or blending
• Data representation
   – Triangle meshes – exploit connectivity
   – Meshless approaches
      • Both triangle meshes and point clouds
• Memory settings
   – All in-core – no massive geometry/images
   – 3D in-core and images out-of-core – no massive geometry
   – All out-of-core – Low performances
• Our contribution
   – Blending function / Streaming framework / Massive point cloud /
     Adaptive geometry refinement
R. Pintus – CRS4/ViC, October 2012




Pipeline




                                          Masked
                     Per-pixel
 Photo     Stencil                        Per-pixel
                      Weight
                                           Weight
R. Pintus – CRS4/ViC, October 2012




Simple blending
R. Pintus – CRS4/ViC, October 2012



Edge extraction and Distance
Transform
R. Pintus – CRS4/ViC, October 2012




Smooth weight
R. Pintus – CRS4/ViC, October 2012




Smooth weight
R. Pintus – CRS4/ViC, October 2012




Single band blending
R. Pintus – CRS4/ViC, October 2012




Multi band blending
R. Pintus – CRS4/ViC, October 2012




Adaptive point refinement
R. Pintus – CRS4/ViC, October 2012




Adaptive point refinement
R. Pintus – CRS4/ViC, October 2012




Adaptive point refinement
R. Pintus – CRS4/ViC, October 2012




Adaptive point refinement
R. Pintus – CRS4/ViC, October 2012




Results


            David      •   Callieri et. al 2008 – David
                           28M
          470Mpoints         –   Disk space occupancy –
                                 6.2GB
                             –   Computation time – 15.5
                                 hours
R. Pintus – CRS4/ViC, October 2012




Results – Church’s Apse




  14 Mpoint Geometry      40 photos
R. Pintus – CRS4/ViC, October 2012




Results – Church’s Apse
R. Pintus – CRS4/ViC, October 2012




Results – Grave
                            21 photos
8 Mpoint Geometry
R. Pintus – CRS4/ViC, October 2012




Results – Grave
R. Pintus – CRS4/ViC, October 2012




Results
R. Pintus – CRS4/ViC, October 2012




Results
R. Pintus – CRS4/ViC, October 2012




Results




          David
       470Mpoints
Image size – 19456x53248
         1Gpixel
R. Pintus – CRS4/ViC, October 2012




Results
R. Pintus – CRS4/ViC, October 2012




Conclusion
•   Image-to-geometry registration approach
•   Minimum user intervention
•   No constraints on geometry, attributes and features
•   Specific robust cost function and SBA

•   Out-of-core photo blending approach (Point clouds of unlimited size)
•   Incremental color accumulation (Unlimited number of images)
•   Smooth weight function (Seamless color blending)
•   Streaming framework (Performance improvement)
•   Adaptive point refinement

•   Future work
    – Automatic sparse-to-dense geometry registration
    – Interactive blending - adding and removing images in an interactive tool
    – Fast visual check of previous alignment step
R. Pintus – CRS4/ViC, October 2012




Conclusion
•   Low cost
    – Personal computer
    – Digital camera
    – Decreased manual intervention
•   Open Source / Free Software
    – Bundler – SfM reconstruction –
      http://phototour.cs.washington.edu/bundler/
    – Sparse Bundle Adjustment – SBA – Minimization –
      http://www.ics.forth.gr/~lourakis/sba/
    – Opengl / GLSL shaders – Rendering – http://www.opengl.org/
    – Qt – Interface – http://qt.nokia.com/
    – Opencv – Manual registration – http://opencv.willowgarage.com/wiki/
    – Spaceland Library – Geometric computation –
      http://spacelib.sourceforge.net/
    – IIPImage – Web-based Viewer – http://iipimage.sourceforge.net/
R. Pintus – CRS4/ViC, October 2012




3D Printing




                                             80
R. Pintus – CRS4/ViC, October 2012




Printing Process

 • Original model

 • Slice
   representation

 • Layer by layer
   deposition

 • Cleaning

 • Printed model
                                                   81
R. Pintus – CRS4/ViC, October 2012




Printing Process

 • Original model

 • Slice
   representation

 • Layer by layer
   deposition

 • Cleaning

 • Printed model
                                                   82
R. Pintus – CRS4/ViC, October 2012




Printing Process

 • Original model

 • Slice
   representation

 • Layer by layer
   deposition

 • Cleaning

 • Printed model
                                                   83
R. Pintus – CRS4/ViC, October 2012




Printing Process

 • Original model

 • Slice
   representation

 • Layer by layer
   deposition

 • Cleaning

 • Printed model
                                                   84
R. Pintus – CRS4/ViC, October 2012




Printing Process

 • Original model

 • Slice
   representation

 • Layer by layer
   deposition

 • Cleaning

 • Printed model
                                                   85
R. Pintus – CRS4/ViC, October 2012




Geometry processing




                                                     86
R. Pintus – CRS4/ViC, October 2012




Geometry processing




                                                     87
R. Pintus – CRS4/ViC, October 2012




Geometry processing




                                                     88
R. Pintus – CRS4/ViC, October 2012




Sub-
Sub-surface scattering




                                                        89
R. Pintus – CRS4/ViC, October 2012




Color enhancement




                                                   90
R. Pintus – CRS4/ViC, October 2012




Color enhancement




                                                   91
R. Pintus – CRS4/ViC, October 2012




Color enhancement




                                                   92
R. Pintus – CRS4/ViC, October 2012




Conclusioni
• Lavorare su dati misurati è un pre-
  requisito di molti lavori (tutti?) nel
  contesto dei beni culturali
  – Applicazioni specialistiche o per grande pubblico
• Le moderne tecnologie di acquisizione
  consentono di acquisire una grande
  quantità di informazioni (forma e colore)
  – Laser scanning, camere digitali, ecc.
• Uso potenziale vasto!
  – Valorizzazione, restauro, studio, ecc.
R. Pintus – CRS4/ViC, October 2012




Conclusioni
• Queste quantità di dati sono però difficili
  da trattare, archiviare, distribuire,
  visualizzare
  – Scalabilità!
• Tecniche attuali sub-ottimali
  – Costi, tempi, qualità
R. Pintus – CRS4/ViC, October 2012




Conclusioni
• Il CRS4 è impegnato in attività di ricerca
  per migliorare le tecnologie…
  – Stato dell’arte internazionale
  – Collaborazioni e ricadute locali
     • PMI, Contro Restauro SS, Soprintendenze, CNR, UniCA

• … e per applicarle a casi concreti
  – Collaborazioni multidisciplinari!
R. Pintus – CRS4/ViC, October 2012




Conclusioni




                                              96
R. Pintus – CRS4/ViC, October 2012




Questions & Contacts

                   • CRS4 – VIC
                     www.crs4.it/vic/

                   • Ruggero Pintus
                     ruggero@crs4.it

Contenu connexe

Plus de CRS4 Research Center in Sardinia

Plus de CRS4 Research Center in Sardinia (20)

The future is close
The future is closeThe future is close
The future is close
 
The future is close
The future is closeThe future is close
The future is close
 
Presentazione Linea B2 progetto Tutti a Iscol@ 2017
Presentazione Linea B2 progetto Tutti a Iscol@ 2017Presentazione Linea B2 progetto Tutti a Iscol@ 2017
Presentazione Linea B2 progetto Tutti a Iscol@ 2017
 
Iscola linea B 2016
Iscola linea B 2016Iscola linea B 2016
Iscola linea B 2016
 
Sequenziamento Esomico. Maria Valentini (CRS4), Cagliari, 18 Novembre 2015
Sequenziamento Esomico. Maria Valentini (CRS4), Cagliari, 18 Novembre 2015Sequenziamento Esomico. Maria Valentini (CRS4), Cagliari, 18 Novembre 2015
Sequenziamento Esomico. Maria Valentini (CRS4), Cagliari, 18 Novembre 2015
 
Near Surface Geoscience Conference 2015, Turin - A Spatial Velocity Analysis ...
Near Surface Geoscience Conference 2015, Turin - A Spatial Velocity Analysis ...Near Surface Geoscience Conference 2015, Turin - A Spatial Velocity Analysis ...
Near Surface Geoscience Conference 2015, Turin - A Spatial Velocity Analysis ...
 
GIS partecipativo. Laura Muscas e Valentina Spanu (CRS4), Cagliari, 21 Ottobr...
GIS partecipativo. Laura Muscas e Valentina Spanu (CRS4), Cagliari, 21 Ottobr...GIS partecipativo. Laura Muscas e Valentina Spanu (CRS4), Cagliari, 21 Ottobr...
GIS partecipativo. Laura Muscas e Valentina Spanu (CRS4), Cagliari, 21 Ottobr...
 
Alfonso Damiano (Università di Cagliari) ICT per Smart Grid
Alfonso Damiano (Università di Cagliari) ICT per Smart Grid Alfonso Damiano (Università di Cagliari) ICT per Smart Grid
Alfonso Damiano (Università di Cagliari) ICT per Smart Grid
 
Big Data Infrastructures - Hadoop ecosystem, M. E. Piras
Big Data Infrastructures - Hadoop ecosystem, M. E. PirasBig Data Infrastructures - Hadoop ecosystem, M. E. Piras
Big Data Infrastructures - Hadoop ecosystem, M. E. Piras
 
Big Data Analytics, Giovanni Delussu e Marco Enrico Piras
 Big Data Analytics, Giovanni Delussu e Marco Enrico Piras  Big Data Analytics, Giovanni Delussu e Marco Enrico Piras
Big Data Analytics, Giovanni Delussu e Marco Enrico Piras
 
Dinamica Molecolare e Modellistica dell'interazione di lipidi col recettore P...
Dinamica Molecolare e Modellistica dell'interazione di lipidi col recettore P...Dinamica Molecolare e Modellistica dell'interazione di lipidi col recettore P...
Dinamica Molecolare e Modellistica dell'interazione di lipidi col recettore P...
 
Innovazione e infrastrutture cloud per lo sviluppo di applicativi web e mobil...
Innovazione e infrastrutture cloud per lo sviluppo di applicativi web e mobil...Innovazione e infrastrutture cloud per lo sviluppo di applicativi web e mobil...
Innovazione e infrastrutture cloud per lo sviluppo di applicativi web e mobil...
 
ORDBMS e NoSQL nel trattamento dei dati geografici parte seconda. 30 Sett. 2015
ORDBMS e NoSQL nel trattamento dei dati geografici parte seconda. 30 Sett. 2015ORDBMS e NoSQL nel trattamento dei dati geografici parte seconda. 30 Sett. 2015
ORDBMS e NoSQL nel trattamento dei dati geografici parte seconda. 30 Sett. 2015
 
Sistemi No-Sql e Object-Relational nella gestione dei dati geografici 30 Sett...
Sistemi No-Sql e Object-Relational nella gestione dei dati geografici 30 Sett...Sistemi No-Sql e Object-Relational nella gestione dei dati geografici 30 Sett...
Sistemi No-Sql e Object-Relational nella gestione dei dati geografici 30 Sett...
 
Elementi di sismica a riflessione e Georadar (Gian Piero Deidda, UNICA)
Elementi di sismica a riflessione e Georadar (Gian Piero Deidda, UNICA)Elementi di sismica a riflessione e Georadar (Gian Piero Deidda, UNICA)
Elementi di sismica a riflessione e Georadar (Gian Piero Deidda, UNICA)
 
Near Surface Geoscience Conference 2014, Athens - Real-­time or full­‐precisi...
Near Surface Geoscience Conference 2014, Athens - Real-­time or full­‐precisi...Near Surface Geoscience Conference 2014, Athens - Real-­time or full­‐precisi...
Near Surface Geoscience Conference 2014, Athens - Real-­time or full­‐precisi...
 
SmartGeo/Eiagrid portal (Guido Satta, CRS4)
SmartGeo/Eiagrid portal (Guido Satta, CRS4)SmartGeo/Eiagrid portal (Guido Satta, CRS4)
SmartGeo/Eiagrid portal (Guido Satta, CRS4)
 
Luigi Atzori Metabolomica: Introduzione e review di alcune applicazioni in am...
Luigi Atzori Metabolomica: Introduzione e review di alcune applicazioni in am...Luigi Atzori Metabolomica: Introduzione e review di alcune applicazioni in am...
Luigi Atzori Metabolomica: Introduzione e review di alcune applicazioni in am...
 
Mobile Graphics (part2)
Mobile Graphics (part2)Mobile Graphics (part2)
Mobile Graphics (part2)
 
Mobile Graphics (part1)
Mobile Graphics (part1)Mobile Graphics (part1)
Mobile Graphics (part1)
 

Dernier

Dernier (20)

Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source Milvus
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 

Seminario Ruggero Pintus, 4-10-2012

  • 1. www.crs4.it/vic/ Tecnologie di Visual Computing per i Beni Culturali R. Pintus CRS4 Visual Computing
  • 2. R. Pintus – CRS4/ViC, October 2012 Tecnologie per i beni culturali • Focus: digitalizzazione accurata (forma e colore) di siti e manufatti + … – Partire dai dati: Acquisizione -> Trattamento ! – Modelli misurabili • Molti usi oltre la visualizzazione – Riproduzione materica – Studio di opere d’arte – Documentazione in-situ di scavi archeologici – Supporto al restauro e sua documentazione – Valorizzazione
  • 3. R. Pintus – CRS4/ViC, October 2012 Tecnologie per i beni culturali • Le quantità di dati prodotte dai moderni sensori sono però difficili da trattare, archiviare, distribuire, visualizzare – Scalabilità! • Tecniche attuali sub-ottimali – Costi, tempi, qualità • Bisogno di ricerca in tecnologie abilitanti scalabili – Acquisizione – Processamento geometrico – Visualizzazione – …
  • 4. R. Pintus – CRS4/ViC, October 2012 Tecnologie per i beni culturali • Come acquisire e processare efficacemente forma e colore di siti e manufatti? – Tecniche di fusione multi-sensore, stream-processing, multiresolution, external memory algorithms, parallel programming, GPGPUs • Come archiviare e distribuire efficacemente i modelli? – Multiresolution, adaptive streaming, compression • Come visualizzarli efficacemente? – Multiresolution, adaptive rendering, out-of-core methods, GPU programming, parallelization, rasterization, ray-casting • Come esplorarli? – Novel 3D displays, specific interaction techniques – Portable devices
  • 5. R. Pintus – CRS4/ViC, October 2012 Alcuni esempi • Allineamento geometria/colore • Colorazione di modelli 3D • Fusione di dati e ricostruzione geometrica • Visualizzazione scalabile ed interattiva • Distribuzione di dati in rete • Esplorazione su display innovativi (… e molto altro)
  • 6. R. Pintus – CRS4/ViC, October 2012 Our Goal 6
  • 7. R. Pintus – CRS4/ViC, October 2012 Modelling vs Acquisition Modelling Subjective Reality Acquisition Objective reality 7
  • 8. R. Pintus – CRS4/ViC, October 2012 3D Reconstruction • Acquire geometry and color • A lot of techniques – Structured light, laser scanning (triangulation or time-of-flight), photometric stereo, shape-from-X, … • Which technique? – Object type (big/small, material….) – Cost – Accuracy/Resolution – Time – Complexity 8
  • 9. R. Pintus – CRS4/ViC, October 2012 Outline • 3D Reconstruction Techniques • 3D Reconstruction Pipeline – Photo mapping/blending – Printing • Case study 9
  • 10. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 10
  • 11. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 11
  • 12. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 12
  • 13. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Stereo 13
  • 14. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Stereo 14
  • 15. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 15
  • 16. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Multiview 16
  • 17. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Multiview 17
  • 18. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 18
  • 19. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Silhouettes 19
  • 20. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 20
  • 21. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Depth from focus/defocus 21
  • 22. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 22
  • 23. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Transmissive Computed Tomography Density Function Trasmissive Ultrasound 23
  • 24. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 24
  • 25. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Non-Optical Non- Ultrasound Radar MRI 25
  • 26. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 26
  • 27. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Time-of-Flight Time-of- 2d 5.0m t= = ≈ 17ns c 3 × 108 m s 27
  • 28. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 28
  • 29. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Laser Striping 29
  • 30. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 30
  • 31. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Structured Lighting 31
  • 32. R. Pintus – CRS4/ViC, October 2012 Taxonomy (non-destructive) (non- • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 32
  • 33. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Photometric Stereo 33
  • 34. R. Pintus – CRS4/ViC, October 2012 Photometric Stereo – SEM 34
  • 35. R. Pintus – CRS4/ViC, October 2012 Taxonomy – Photometric Stereo 35
  • 36. R. Pintus – CRS4/ViC, October 2012 Taxonomy SEM • Contact – Direct Measurements • rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM • Non-Contact – Passive – Active • Shape-from-X • Transmissive – Stereo – Computed Tomography (CT) – Multiview – Transmissive Ultrasound – Silhouettes • Reflective – Focus/Defocus – Non-Optical Methods » reflective ultrasound, radar, sonar, MRI – Time-of-Flight – Triangulation » laser striping » structured lighting – Photometric Stereo 36
  • 37. R. Pintus – CRS4/ViC, October 2012 Cultural Heritage • Techniques – Triangulation (laser scanner) – Time of Flight – Texture Mapping – Multi-view reconstruction – Photometric Stereo • Deal with multiple acquisitions • Manage a huge amount of data for visualization purposes 37
  • 38. R. Pintus – CRS4/ViC, October 2012 3D Reconstruction Pipeline Real Object Acquisition Devices Photos 3D Digital Model Geometry === Processing === -Cleaning - Merging - Photo Alignment - Color Projection -… 38
  • 39. R. Pintus – CRS4/ViC, October 2012 3D Reconstruction Pipeline • Real Model Inspection (onsite) • Scans design (offsite/onsite) • Acquisition (onsite) • Alignment (offsite) • Editing (offsite) • Merge (offsite) • Texture (offsite) • Final Model (offsite) • 3D Printing (offsite) 39
  • 40. R. Pintus – CRS4/ViC, October 2012 3D Reconstruction Pipeline • Real Model Inspection (onsite) • Scans design (offsite/onsite) • Acquisition (onsite) • Alignment (offsite) • Editing (offsite) • Merge (offsite) • Texture (offsite) • Final Model (offsite) • 3D Printing (offsite) 40
  • 41. R. Pintus – CRS4/ViC, October 2012 Goal • Fast and low-cost technique for creating accurate colored models • Acquisition – 3D – laser scanners – Color – digital cameras • Mapping photo-to-geometry – Fast and Robust Semi-Automatic Registration of Photographs to 3D Geometry • Photo blending – A Streaming Framework for Seamless Detailed Photo Blending on Massive Point Clouds
  • 42. www.crs4.it/vic/ Photo Mapping Ruggero Pintus, Enrico Gobbetti, and Roberto Combet. “Fast and Robust Semi-Automatic Registration of Photographs to 3D Geometry”. In The 12th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, October 2011.
  • 43. R. Pintus – CRS4/ViC, October 2012 Problem Statement 3D Geometry Unordered Set Of n Uncalibrated Photos n Camera Poses (2D/3D Registration)
  • 44. R. Pintus – CRS4/ViC, October 2012 Related work • Manual selection of 2D-3D matches – Massive user intervention – Tiring and time-consuming • Automatic feature matching – Not robust enough for a generic dataset • Semi-automatic statistical correlation – Point cloud attributes not always provided • Geometric multi-view reconstruction – 2D-3D problem 3D-3D registration task – dense and ordered frame sequence • Our contribution – Minimize user intervention / Large datasets / Semi- automatic / Multi-view based approach / No Attributes
  • 45. R. Pintus – CRS4/ViC, October 2012 Input Data User Dense 3D n Photos • Dense Geometry – Point cloud, triangle mesh, etc. SfM Reconstruction – No attributes – No particular features Coarse Registration • n photos – Naïve constraints: Refinement • Blur, Noise, Under- or over-exposured – Sufficient overlap Output Data
  • 46. R. Pintus – CRS4/ViC, October 2012 Multi- Multi-view User Dense 3D n Photos • Bundler [Snavely et al. 2006] – SfM system for unordered SfM Reconstruction image collections – http://phototour.cs.washingto n.edu/bundler/ Coarse Registration • Output – A sparse point cloud – n camera poses Refinement – SIFT keypoints (projections of sparse 3D points) Output Data
  • 47. R. Pintus – CRS4/ViC, October 2012 Coarse registration User Dense 3D n Photos • Register two point clouds with different: – scales – reference frames SfM Reconstruction – resolutions • Automatic methods are not robust and efficient enough Coarse Registration • User aligns few images (one or more) to the dense geometry Refinement • Affine transformation is applied to all cameras and sparse points Output Data
  • 48. R. Pintus – CRS4/ViC, October 2012 Refinement User Dense 3D n Photos Pj C1 Q (C2 , p j ) s1, j pj SfM Reconstruction s2 , j NN F ( p j ) Coarse Registration Q (C2 , NN F ( p j )) Refinement C2 E (C , P ) = ∑∑ vij Q (Ci , NN F ( p j )) − si , j N P NC 2 Output Data j =1 i =1
  • 49. R. Pintus – CRS4/ViC, October 2012 Refinement User Dense 3D n Photos • Sparse Bundle Adjustment (SBA) – Constants – SIFT keypoints, SfM Reconstruction dense 3D points – Variables – Camera poses, sparse 3D points Coarse Registration – SBA • A Generic SBA C/C++ Package Based on the Levenberg- Marquardt Algorithm Refinement • http://www.ics.forth.gr/~loura kis/sba/ Output Data
  • 50. R. Pintus – CRS4/ViC, October 2012 Output data User Dense 3D n Photos • n camera poses SfM Reconstruction • Input of photo blending – n photos Coarse Registration – n camera poses – Dense 3D geometry Refinement Output Data
  • 51. R. Pintus – CRS4/ViC, October 2012 Results – Photo mapping
  • 52. www.crs4.it/vic/ Photo Blending Ruggero Pintus, Enrico Gobbetti, and Marco Callieri. A Streaming Framework for Seamless Detailed Photo Blending on Massive Point Clouds. In Proc. Eurographics Area Papers. Pages 25- 32, 2011.
  • 53. R. Pintus – CRS4/ViC, October 2012 Problem Statement Point Cloud Calibrated Photos
  • 54. R. Pintus – CRS4/ViC, October 2012 Problem Statement Point Cloud Calibrated Photos P
  • 55. R. Pintus – CRS4/ViC, October 2012 Problem Statement Calibrated Colored Point Cloud Photos Point Cloud P
  • 56. R. Pintus – CRS4/ViC, October 2012 Problem Statement Calibrated Colored Point Cloud Photos Point Cloud P • Problem Unlimited size of 3D model (Gpoints) and unlimited number of images
  • 57. R. Pintus – CRS4/ViC, October 2012 Related work • State-of-the-art techniques – Image quality estimation – Stitching or blending • Data representation – Triangle meshes – exploit connectivity – Meshless approaches • Both triangle meshes and point clouds • Memory settings – All in-core – no massive geometry/images – 3D in-core and images out-of-core – no massive geometry – All out-of-core – Low performances • Our contribution – Blending function / Streaming framework / Massive point cloud / Adaptive geometry refinement
  • 58. R. Pintus – CRS4/ViC, October 2012 Pipeline Masked Per-pixel Photo Stencil Per-pixel Weight Weight
  • 59. R. Pintus – CRS4/ViC, October 2012 Simple blending
  • 60. R. Pintus – CRS4/ViC, October 2012 Edge extraction and Distance Transform
  • 61. R. Pintus – CRS4/ViC, October 2012 Smooth weight
  • 62. R. Pintus – CRS4/ViC, October 2012 Smooth weight
  • 63. R. Pintus – CRS4/ViC, October 2012 Single band blending
  • 64. R. Pintus – CRS4/ViC, October 2012 Multi band blending
  • 65. R. Pintus – CRS4/ViC, October 2012 Adaptive point refinement
  • 66. R. Pintus – CRS4/ViC, October 2012 Adaptive point refinement
  • 67. R. Pintus – CRS4/ViC, October 2012 Adaptive point refinement
  • 68. R. Pintus – CRS4/ViC, October 2012 Adaptive point refinement
  • 69. R. Pintus – CRS4/ViC, October 2012 Results David • Callieri et. al 2008 – David 28M 470Mpoints – Disk space occupancy – 6.2GB – Computation time – 15.5 hours
  • 70. R. Pintus – CRS4/ViC, October 2012 Results – Church’s Apse 14 Mpoint Geometry 40 photos
  • 71. R. Pintus – CRS4/ViC, October 2012 Results – Church’s Apse
  • 72. R. Pintus – CRS4/ViC, October 2012 Results – Grave 21 photos 8 Mpoint Geometry
  • 73. R. Pintus – CRS4/ViC, October 2012 Results – Grave
  • 74. R. Pintus – CRS4/ViC, October 2012 Results
  • 75. R. Pintus – CRS4/ViC, October 2012 Results
  • 76. R. Pintus – CRS4/ViC, October 2012 Results David 470Mpoints Image size – 19456x53248 1Gpixel
  • 77. R. Pintus – CRS4/ViC, October 2012 Results
  • 78. R. Pintus – CRS4/ViC, October 2012 Conclusion • Image-to-geometry registration approach • Minimum user intervention • No constraints on geometry, attributes and features • Specific robust cost function and SBA • Out-of-core photo blending approach (Point clouds of unlimited size) • Incremental color accumulation (Unlimited number of images) • Smooth weight function (Seamless color blending) • Streaming framework (Performance improvement) • Adaptive point refinement • Future work – Automatic sparse-to-dense geometry registration – Interactive blending - adding and removing images in an interactive tool – Fast visual check of previous alignment step
  • 79. R. Pintus – CRS4/ViC, October 2012 Conclusion • Low cost – Personal computer – Digital camera – Decreased manual intervention • Open Source / Free Software – Bundler – SfM reconstruction – http://phototour.cs.washington.edu/bundler/ – Sparse Bundle Adjustment – SBA – Minimization – http://www.ics.forth.gr/~lourakis/sba/ – Opengl / GLSL shaders – Rendering – http://www.opengl.org/ – Qt – Interface – http://qt.nokia.com/ – Opencv – Manual registration – http://opencv.willowgarage.com/wiki/ – Spaceland Library – Geometric computation – http://spacelib.sourceforge.net/ – IIPImage – Web-based Viewer – http://iipimage.sourceforge.net/
  • 80. R. Pintus – CRS4/ViC, October 2012 3D Printing 80
  • 81. R. Pintus – CRS4/ViC, October 2012 Printing Process • Original model • Slice representation • Layer by layer deposition • Cleaning • Printed model 81
  • 82. R. Pintus – CRS4/ViC, October 2012 Printing Process • Original model • Slice representation • Layer by layer deposition • Cleaning • Printed model 82
  • 83. R. Pintus – CRS4/ViC, October 2012 Printing Process • Original model • Slice representation • Layer by layer deposition • Cleaning • Printed model 83
  • 84. R. Pintus – CRS4/ViC, October 2012 Printing Process • Original model • Slice representation • Layer by layer deposition • Cleaning • Printed model 84
  • 85. R. Pintus – CRS4/ViC, October 2012 Printing Process • Original model • Slice representation • Layer by layer deposition • Cleaning • Printed model 85
  • 86. R. Pintus – CRS4/ViC, October 2012 Geometry processing 86
  • 87. R. Pintus – CRS4/ViC, October 2012 Geometry processing 87
  • 88. R. Pintus – CRS4/ViC, October 2012 Geometry processing 88
  • 89. R. Pintus – CRS4/ViC, October 2012 Sub- Sub-surface scattering 89
  • 90. R. Pintus – CRS4/ViC, October 2012 Color enhancement 90
  • 91. R. Pintus – CRS4/ViC, October 2012 Color enhancement 91
  • 92. R. Pintus – CRS4/ViC, October 2012 Color enhancement 92
  • 93. R. Pintus – CRS4/ViC, October 2012 Conclusioni • Lavorare su dati misurati è un pre- requisito di molti lavori (tutti?) nel contesto dei beni culturali – Applicazioni specialistiche o per grande pubblico • Le moderne tecnologie di acquisizione consentono di acquisire una grande quantità di informazioni (forma e colore) – Laser scanning, camere digitali, ecc. • Uso potenziale vasto! – Valorizzazione, restauro, studio, ecc.
  • 94. R. Pintus – CRS4/ViC, October 2012 Conclusioni • Queste quantità di dati sono però difficili da trattare, archiviare, distribuire, visualizzare – Scalabilità! • Tecniche attuali sub-ottimali – Costi, tempi, qualità
  • 95. R. Pintus – CRS4/ViC, October 2012 Conclusioni • Il CRS4 è impegnato in attività di ricerca per migliorare le tecnologie… – Stato dell’arte internazionale – Collaborazioni e ricadute locali • PMI, Contro Restauro SS, Soprintendenze, CNR, UniCA • … e per applicarle a casi concreti – Collaborazioni multidisciplinari!
  • 96. R. Pintus – CRS4/ViC, October 2012 Conclusioni 96
  • 97. R. Pintus – CRS4/ViC, October 2012 Questions & Contacts • CRS4 – VIC www.crs4.it/vic/ • Ruggero Pintus ruggero@crs4.it