For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit-magney For more information about embedded vision, please visit: http://www.embedded-vision.com Phil Magney, founder of VSI Labs, presents the "Improving the Safety and Performance of Automated Vehicles Through Precision Localization" tutorial at the May 2019 Embedded Vision Summit. How does a self-driving car know where it is? Magney explains how autonomous vehicles localize themselves against their surroundings through the use of a variety of sensors along with precision maps. He explores the pros and cons of various localization methods and the trade-offs associated with each of them. He also examines the challenges of keeping mapping assets fresh and up-to-date through crowdsourcing.

1. © 2019 VSI Labs
Improving the Safety & Performance of
Automated Vehicles Through Precision
Localization
Phil Magney
VSI Labs
May 2019

2. © 2019 VSI Labs
VSI’s Range of Research
2
Industry Research
& Advisory
Testing &
Demonstration
Engineering
Services
• Applied research on active safety and
automated driving since 2014
• Supporting R&D & planning for automotive,
suppliers and technology industry
• VSI Offers various research portals

3. © 2019 VSI Labs
Automated Vehicle Trajectories
Series Production
Low Speed Shuttles
Robo-Taxis
3

4. © 2019 VSI Labs
Automated Driving Today – Series
Production Cars
Automated driving is born
from active safety systems
(active lane keeping,
adaptive cruise control, etc.)
Most L2 vehicles use
camera and radar
Lane centering, no organic
path planning
Localization – most L2
vehicles rely on SLAM
methods
4

5. © 2019 VSI Labs
Automated Driving
Today -- Shuttles
Driverless Shuttles – low
speed vehicles being
developed as fully automated
people movers
Shuttles operate on a “virtual
rail”
Shuttles will also use various
other sensors for collision
avoidance
Localization -- Shuttles use
correction services to
localization against a pre-
mapped course
5

6. © 2019 VSI Labs
Automated Driving
Today -- Robo Taxis
Many companies are
developing L4 “robo-taxis”
for deployment in metro
areas under a ride sharing
model
Robo-taxis use many
sensors including redundant
camera, Lidar, Radar, HD
maps, etc.
Localization -- Precision
localization against HD
maps, mainly using lidar
6

7. © 2019 VSI Labs
The Tech Behind Automated
Vehicle Systems
Sensors
Processors
Software
7

8. © 2019 VSI Labs
Anatomy of an Automated Vehicle
8

9. © 2019 VSI Labs
The Automation Pipeline
9

10. © 2019 VSI Labs
Sensor
Configurations
L1
Active Safety Features
L2
Lane Level
Automation
L2+
Multi Lane Automation
L3
Traffic Jam Pilot and
Hwy Pilot
L4/L5
Robo-taxi / MaaS
Camera Front facing
Rear facing (passive)
Front facing
Rear facing (passive)
Front facing x 2
Side facing x 2
Rear facing
Driver facing
Front facing x 2
Side facing x 2
Rear facing
Driver facing
Front facing x 3
Side facing x 4
Rear facing x 2
Thermal x 2
Driver facing
Radar Med-long range front
Short range side x 2
Med-long range front
Short range side x 2
Med-long range front
Short range side x 2
Short range front
Med-long range
Short range side x 2
Short range front
Med-long range
Short range side x 2
Imaging radar x 2
Ultrasonic 360 degree 360 degree 360 degree 360 degree 360 degree
LiDAR Solid state Solid state x 2 Solid state x 2
360 degree x 2
HD Maps ADAS model ADAS model
Lane model
Localization model
ADAS model
Lane model
Localization model
ADAS model
Lane model
Localization model
Inertial IMU IMU IMU IMU IMU
GNSS + Correction RTK RTK RTK
Total Sensors 7 8 16 18 28
Notes: This are proxy estimates based on OEM and fleets planned announcements. Sensor distribution may vary greatly among
OEMs. Maps are counted as a sensor as well as IMU, GNSS devices. Surround view applications are not included in this chart. Ultra
sonic is counted as one rather than discrete units. L3 – L5 are largely development sensor configurations, final production may
differ.
Automotive Sensor Proxy
10

11. © 2019 VSI Labs
Improving Safety Through
Precision Maps
11

12. © 2019 VSI Labs
Limitations of Vision-based Automation (basic L2)
12

13. © 2019 VSI Labs
Limitations of AVs in Poor Weather?
13
Sensors Fail
• Camera & Lidar fail quickly
• Radar still robust
Infrastructure Fails
• Lanes lines covered
• Localization assets hard to recognize
if they are covered in snow
What Does it Take?
• Precision localization – through
correction services
• V2I – through embedded cellular,
DSRC
• Virtual Infrastructure – Precision
Maps (lanes models, intersections,
and other metadata)
• Low Grip Algorithm

14. © 2019 VSI Labs
Absolute Localization (Ground Truth)
14
• GPS based accuracy is not enough and suffers from
outages
• Heighten the fixes with correction (RTK services) to w/I a
few centimeters
• Once base accuracy is enough, then you need to
localized against it between fixes
• Wheel odometry uses a starting position and wheel
speed to estimate a position from a fixed point
• Inertial measurement unit (IMU) senses the movement of
the vehicle in time via yaw, pitch, roll axis and magnetic
• Visual odometry estimates vehicle motion from a
sequence of camera frames

15. © 2019 VSI Labs
Relative Localization With Landmarks
• Landmark localization -
matches landmarks and
objects (signs, poles, and
other permanent objects
• In landmark-based
approaches the objects are
classified
• The challenge is the creation
and maintenance of these
maps, requires human
annotation
15
Source: Towards Data Science

16. © 2019 VSI Labs
Relative Localization With Lane Models
• Lane models provide virtual lane
and centerline trajectories from
which to localize against
• This is particularly important when
lanes lines are obstructed (i.e.
covered in snow)
• Lane models is vital to multilane
highways and intersection
traversals
• The challenge is the creation and
maintenance of these maps
16

17. © 2019 VSI Labs
What are Precision Maps & Who Supplies Them
• Road model (ADAS Map)
• Topology
• Routing
• Speed attributes, etc.)
• Lane Models
• Lane geometry
• Polylines
• Trajectories
• Localization Layer
• Landmarks
• Signs, barriers, poles etc.
• Edges and boundaries
• Voxels
• Confidence Index
17

18. © 2019 VSI Labs
How to Keep the Data Fresh
• Mapping data are always
changing
• Temporary lane closures or
detours
• Changes to the infrastructure
(bridges, landmarks, poles,
lights)
• How to detect changes, collect,
then validate those changes
• Requires fleets to crowd
source
• Requires connected vehicles
• Requires standardization of
message sets
18

19. © 2019 VSI Labs
Elements of Mapping Assets
19
Trajectories
• Geocoded Paths
Lane Models
• Lane Geometry
ADAS Map
• Speeds/Slopes/Curves
Localization Layers
Raw Images
Raw Point Clouds
Vixels (Voxels + RGB data)
Voxels (clusters of points)
Compressed Point Cloud
Object List (landmarks)
VolumeofData
Sense and React Sense and Align Sense and Position
HERE HD Live Maps

20. © 2019 VSI Labs
Closing Remarks
• Automation in series production is an extension of
ADAS.
• Couple camera and radar and you have L2
capabilities.
• Adding precision maps substantially improves the
performance and safety of automated vehicles:
• Mapping assets to improve localization
• Lane models tell you where you can and cannot
operate!
• ADAS attributes add speeds, curve, slope and
other important metadata
• Shuttles are a pragmatic approach to automation
• No path planning as you are using a recorded
path to create a virtual rail
• Robo-taxis will apply everything we have talked
about times three
• They will be highly constrained (largely because of
SOTIF – Safety Of the Intended Function)
20

21. © 2019 VSI Labs
Contact
21
Resources
www.VSL-Labs.com
Sign Up for Free White Papers
Request Services Agreement
Client Portal Registration Page
Contact Phil Magney
+1-952-215-1797