1.
Enriching 3D point cloud data with
Artificial Intelligence
@mrcabellom
mrcabellom@gmail.com
Research Engineer @ Plain Concepts
Rodrigo Cabello

2.
The procedures to convert point clouds into application-specific deliverables
are very costly in time/manual intervention. It is necessary to develop
processes that extract the essential information automatically to create
valuable data for decision-making.

3.
3D representation
• Better understanding of our environment. 3D
data can provide more dimensional
information.
• 3D models can:
• Represent the features of virtually any
object.
• Represent complex objects with a finite
number of elements. (Point Cloud)
• Improve the decision-making process.
• Design error reduction:
• industry and building sector.

4.
Data representation
Point Cloud
Voxels
(Volumetric Pixels)
Meshes
Pixels
2D image 3D Model

5.
RGB-D provides a 2,5D information
3D data representation

6.
3D data representation - NERF

7.
Point cloud

8.
• A point cloud is a set of data points in space.
• Points may represent a 3D shape or object.
• Each point position has its set of Cartesian
coordinates (X, Y, Z).
• Points can include attributes like RGB, intensity,
and classification.
• Point clouds are generally produced by 3D
scanners (LIDAR) or by photogrammetry
software.
An example of a 1.2 billion data point cloud render of Beit Ghazaleh,
What is a point cloud?

9.
Artificial
intelligence in
point cloud

10.
Millions of points Unstructured data
Multiple file formats Noisy Sensors
Challenges

11.
Point cloud + AI
• Automated insights extraction in large point clouds.
• ML-assisted capacity can help reduce human errors by automatically pre-labeling.
Classification, custom
object detection, and
segmentation.
Object detection in the 2D
image and 3D projection.
Synthetic
Data
Generation
Deep Learning

12.
Object detection – Cube RCNN

13.
Engineer
Data Ingestion
Data Cleansing/
Transformation
Model Training/
Building
Model Testing Model Deployment
Test Data
1 2 3
4
4
4 5
Model feedback Loop
Train Test Loop
AI Workflow
Version 1
Azure Machine Learning
Model registry

14.
• Point cloud
• Data transformation to 3D voxel grid
projections.
• Data normalization (scale)
• Translation, rotation, and permutation
invariance.
• Sort input into canonical order.
• Large point clouds. Memory consumption.
• Slices, segmentation
• Downsampling
Data preprocesing - Deep Learning

15.
Data cleaning
• Pass-through filter.
• Statistical outlier removal.
• Radius outlier.

16.
Downsampling
• Voxel Grid Filter. Centroid
• Farthest point sample

17.
Semantic
segmentation
Object detection and
localization
Segmentation
Classification
Computer vision tasks
PointNet++
PointCNN
SE-PseudoGrid
PointPillars
PointRCNN
FCAF3D
RANSAC
Correspondence
Grouping
K-Means, DBSCAN
PAConv
PointNetTransformers

18.
Industrial applications

19.
01 Create the digital version of the environment.
Automatic digital twin

20.
01 Create the digital version of the environment.
Automatic digital twin
02 Extract main environment insights.

21.
01 Create the digital version of the environment.
Automatic digital twin
02 Extract main environment insights.
Detect and recognize specific objects.
03

22.
01 Create the digital version of the environment.
Transform environment and insights
into 3D objects.
Automatic digital twin
02 Extract main environment insights.
Detect and recognize specific objects.
03
04

23.
01 Create the digital version of the environment.
Transform environment and insights
into 3D objects.
Automatic digital twin
02 Extract main environment insights.
Detect and recognize specific objects.
03
04
IoT devices simulator
05

24.
Questions & Answers