Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Sensor fusion of LiDAR and Camera for real time object detection - talk version

Autonomous vehicles primary need is to have a robust and precise perception of the surrounding environment. To achieve accurate results in autonomous driving a combination of several different sensors is used. This technique is commonly known as sensor fusion.
However, a critical aspect of autonomous driving perception is also the reactivity: we do not only need to perform sensor fusion properly, but we also have to respect real time requirements. The latter are fundamental to guarantee predictability of the system, avoid anomalous situations and prevent hazards.
Performing fusion online is not trivial because sensors returns a lot of data and process them may be time consuming. The solution we present is an alignment of the sensors, which combines information from LiDAR and cameras, that is performed once in preprocessing and allow us to exploit the precomputed matching in real time.

  • Identifiez-vous pour voir les commentaires

  • Soyez le premier à aimer ceci

Sensor fusion of LiDAR and Camera for real time object detection - talk version

  1. 1. Sensor fusion for autonomous driving perception Bagni Fabio 228594@studenti.unimore .it San Francisco - 15 May 2019 1
  2. 2. Fabio Bagni San Francisco - 15 May 2019 Autonomous driving perception Precision and reactivity Risks of accidents, traffic laws respect. 2
  3. 3. Fabio Bagni San Francisco - 15 May 2019 Embedded platforms 3
  4. 4. Fabio Bagni San Francisco - 15 May 2019 Goals Real time 3D detection ● High performances ○ required 10 Hz frequency ● High precision ● Real time on embedded platforms 4
  5. 5. Fabio Bagni San Francisco - 15 May 2019 UNIMORE HiPeRT prototype 5
  6. 6. Fabio Bagni San Francisco - 15 May 2019 Sensors ● LiDAR ○ 3D point cloud ● Cameras ○ images Heterogeneous outputs 6
  7. 7. Fabio Bagni San Francisco - 15 May 2019 Sensor fusion What is sensor fusion: ● Matching of different sensors views Why sensor fusion: ● Heterogeneous information of environment 7
  8. 8. Fabio Bagni San Francisco - 15 May 2019 Fields of view LiDAR: 360 degrees around the vehicle Camera: wide angle, disposed do cover 360 degrees. 8 Cameras LiDAR
  9. 9. Fabio Bagni San Francisco - 15 May 2019 Deep learning approach Camera: ● Bird’s Eye View estimation ● Object detection LiDAR ● Bird’s Eye View calculation ● Object detection 9 Image Point cloud 3D detection 3D detection Fusion
  10. 10. Fabio Bagni San Francisco - 15 May 2019 Deep learning approach time Image stitching for multiple camera fusion 10 ➔ Under10 Hz Notenoughresponsive 1. Ming Liang, Bin Yang, Shenlong Wang and Raquel Urtasun: Deep Continuous Fusion for Multi-Sensor3D ObjectDetection.In: ECCV (2018).
  11. 11. Fabio Bagni San Francisco - 15 May 2019 Static alignment sensor fusion Alignment of LiDAR with each camera individually. 11 Calibration Preprocessing
  12. 12. Fabio Bagni San Francisco - 15 May 2019 Features extraction Matching is made by a calibration with a perforated panel. Holes are detected by all sensors. Holes’ centers are used as features to be matched. 12
  13. 13. Fabio Bagni San Francisco - 15 May 2019 Cylindrical projection Fields of view matching is made projecting sensors’ outputs on the surface of a cylinder. 13
  14. 14. Fabio Bagni San Francisco - 15 May 2019 Projections complexity LiDAR cylindrical projection: Depthmap O(n) : one operation per point 14 Camera cylindrical projection O(n) : one operation per pixel PARALLELIZABLE
  15. 15. Fabio Bagni San Francisco - 15 May 2019 Calibration results 15
  16. 16. Fabio Bagni San Francisco - 15 May 2019 Colored points Alignment of each point with the corresponding pixel on the image. 16
  17. 17. Fabio Bagni San Francisco - 15 May 2019 Object detection Neural network for object detection on camera frame Clustering of LiDAR points Bounding boxes matching 17
  18. 18. Fabio Bagni San Francisco - 15 May 2019 Object detection results 18
  19. 19. Fabio Bagni San Francisco - 15 May 2019 Object detection average time 19 LiDAR output : 10 Hz Time limit : 100 ms
  20. 20. Thanks for your attention Fabio Bagni - 228594@studenti.unimore.it 20 Fabio Bagni San Francisco - 15 May 2019
  21. 21. 21
  22. 22. Fabio Bagni San Francisco - 15 May 2019 Homography Homography allow to translate camera projection plane on the LiDAR projection plane. This method make cylindrical surfaces match. 22

×