Talk made during Transportation Research Arena 2014 (Paris). This talk present the lessons learned by the French team during the first Large Scale Field Operational Test in Europe (EuroFOT).

1. Impact evaluation of speed
regulation systems using naturalistic
driving data: The EuroFOT example.
Saint Pierre, Guillaume, IFSTTAR, France
Tattegrain, Hélène, IFSTTAR, France
Val, Clément, CEESAR, France
* guillaume.saintpierre@ifsttar.fr
STS N°48 TRA2014 Paris 14-17 avril 2014

2. Saint Pierre G., Tattegrain H., Val C.
Introduction
 Increasing penetration of driving assistance systems
 Needs to measure theirs impacts during real uses
 Several projects launched recently (FP7 funded)
Field operational tests (FOT)
FESTA methodology
Naturalistic driving data
 Many challenges were adressed, and many lessons
learned
 Let’s come back to the french EuroFOT experience
The first large scale FOT, ended in 2012
STS N°48 TRA2014 Paris 14-17 avril 2014 2

3. Saint Pierre G., Tattegrain H., Val C.

4. Saint Pierre G., Tattegrain H., Val C.
EuroFOT in France
 VMC handled by
CEESAR
• 35 drivers using their
own car in the west of
Paris, during 6 months
• Light instrumentation
 5 identical cars
replaced the subject
ones three times
 Full instrumentation
(incl. Video)
 545 000 km of data
analysed
STS N°48 TRA2014 Paris 14-17 avril 2014 4
Low Level High Level
Vehicles used 35 drivers’
owned
vehicles.
5 vehicles owned by
CEESAR and loaned to
participants
CTAG datalogger 2
Max 4 CAN Channels
GPS
GPRS data transfer
●
2 channels
used
●
●
●
4 channels used
●
(not used : manual
transfer)
TRW AC20 radar
(not part of standard vehicle
equipment)
● ●
VideoLogger
(custom made for CEESAR,
H.264)
●
Cameras
(B&W, SuperHAD Exview)
4
Mobileye AWS
(added, with special firmware)
●
Smarteye Eyetracker ●

5. Saint Pierre G., Tattegrain H., Val C.
Experimental design
 Dream
 Reality
STS N°48 TRA2014 Paris 14-17 avril 2014 5
 Questionnaires were
administred 4 times
 Rotation of fully
equiped vehicules
among participants
Month
1
Month
2
Month
3
Month
4
Month
5
Month
6
Month
7
Month
8
Month
9
Month
10
Month
11
Month
12
Baseline Treatment Treatment
Screening,
Time 1
Time 2 Time 3 (a) Time 3 (b)
Time 4,
Debriefing

6. Saint Pierre G., Tattegrain H., Val C.
Lessons learned (1)
 Recruitment needs car owners database
acces to be efficient
 GPRS data transfert problematic, consider
UMTS instead,
 Simplify experimental plan
NDS style
« instrument and forget »
STS N°48 TRA2014 Paris 14-17 avril 2014 6

7. Saint Pierre G., Tattegrain H., Val C.
Some issues for ND data
 Data reduction
• Reduce or aggregate continuous data to a significant level
 Data modelling
• Avoid comparisons between heterogeneous datasets
• Control for exposure
• Take into account the intrinsic correlation present in the
data (repeated measures framework)
 Deal with rare events
• Post processing detection of “safety related events”
 Results extrapolation
• Transform events based analyses into casualties reduction
STS N°48 TRA2014 Paris 14-17 avril 2014 7

8. Saint Pierre G., Tattegrain H., Val C.
EuroFOT « solutions »
 Data reduction
 Identify homogeneous sections of data
 Split sections in identical time epochs (10-30 sec.)
 Data modelling
 Use suitable statistical models (GEE, GLMM, instead of ANOVA)
 Produce Odds ratios results
 Deal with rare events
 Automatically detect candidates events (triggers, system use
etc...)
 Confirm identification by video + Annotation
 Extract corresponding baseline and do some stats...
 Results extrapolation
 Speed & Accidents relationships
STS N°48 TRA2014 Paris 14-17 avril 2014 8

9. Saint Pierre G., Tattegrain H., Val C.
Key results
Behavior, acceptance, usage
 CC usage does not vary significantly over time.
 SL usage does not vary significantly over time.
 Drivers tend to use more one of the two systems.
 CC usage favorable driving conditions
 SL usage adverse conditions (ex. Night)
STS N° TRA2014 Paris 14-17 avril 2014 9

10. Saint Pierre G., Tattegrain H., Val C.
Safety: Events based
analysis
STS N° TRA2014 Paris 14-17 avril 2014 10
 Safety events are rare: Odds ratios can be interpreted as
relative risk
 SL associated with less frequent safety related events (SRE)
 CC associated with less SRE, except over-speeding

11. Saint Pierre G., Tattegrain H., Val C.
Lessons learned (2)
 Baseline selection/definition for each RQ
hypothesis is crucial
Needs to control for external factors (traffic, visibility)
 A data aggregation method is needed
It has an impact on the analysis
 Scaling up proove to be very difficult
Various methods tried during FOTs
None is perfect
 Events based analysis (EBA)
applicable to any system which impact is related to
the occurrence of this event
STS N°48 TRA2014 Paris 14-17 avril 2014 11

12. Saint Pierre G., Tattegrain H., Val C.
Conclusion & recommendations
for future FOT
 Why using NDS ?
 To get precise estimates of safety related events frequency
(with/without system)
 Identify systems usage context
 Identify systems misuses and potential countermeasures
 Limitations
 Difficult to get a representative panel
 Very hard to extrapolate to casualties reductions
 Further works
 Identify important measures for road safety
 Increase panel size and representativity
 Define/quantify safety critical events
STS N°48 TRA2014 Paris 14-17 avril 2014 12

13. Saint Pierre G., Tattegrain H., Val C.STS N°48 TRA2014 Paris 14-17 avril 2014 13
Thank you for your attention
Guillaume SAINT PIERRE
Guillaume.saintpierre@ifsttar.fr
COSYS/LIVIC
Components & systems department
Interaction vehicles/drivers/infrastructure research unit