Semantic Technologies for Vehicle Data - Defense

SEMANTIC TECHNOLOGIES FOR VEHICLE DATA
HOW CAN AUTOMOTIVE SERVICES BENEFIT FROM SEMANTICS IN THEIR DATA MODEL?
Benjamin Klotz
klotz@eurecom.fr
Academic supervisors: Christian Bonnet, Raphael Troncy
Industry supervisors: Daniel Wilms, Martin Arend, Michael Würtenberger

SEMANTIC TECHNOLOGIES FOR VEHICLE DATA
Semantic Technologies for Vehicle Data | Benjamin Klotz | 24/09/2019 3
Freedom? Driving experience?
Risk prediction in cross-
domain environments
Trajectory data mining
DATAStandard
Access
Understanding
Interoperability

PLAN
01 02
03
04
Motivation
Complex vehicle modeling
Automatic contextualization
Interactions in the Web of Things
05
Conclusion and future perspectives

MOTIVATION

DATA AROUND THE AUTOMOTIVE DOMAIN
Maps
POIs: “Home”, “Work”
Personal information
Contacts information
Weather dataTraffic data
Accident reports
Navigation

SENSOR DATA IN THE AUTOMOTIVE DOMAIN
Front camera
Radar
Tire pressure sensor
Park assistant
Steering angle sensor
Wheel speed sensor
Blind spot detection
Adaptive cruise control
Temperature sensor
Oil temperature sensor
Vehicle height sensor
{"name":"accelerator_pedal_position","value":0,"timestamp":1361454211.483000}
{"name":"fuel_level","value":23.478279,"timestamp":1361454211.485000}
{"name":"torque_at_transmission","value":1,"timestamp":1361454211.488000}
{"acceleratorPedal":{"position":"4095","ecoPosition":"3"},"brakeContact":"16","sp
eedActual":"0“}, "timeStamp":"2018-01-10T17:01:27.297Z",}
Signal name?
Units?
Datetime?

INTEROPERABILITY IN A FRAGMENTED IOT ECOSYSTEM
Auto
WG
Technology providers

SCOPE OF THIS WORK
Semantic Technologies
- Data access
- Data integration
- Data modeling (Knowledge engineering)
- Data processing for inferences
- Interoperability
- Standardization
- Security
- Privacy
- …
“OEM wantto interactwith cars,understand theircontext, and have carsinteracting with otherthings.”
Main topics
Out of scope in this thesis
Manifold Air
Pressure
map

Interactions
Prediction Trajectory
Data model Knowledge generation/extraction
Access Data model
OVERVIEW
VSS: Vehicle Signal Specification (GENIVI)
VISS: Vehicle Information Server Spec (W3C)
VIAS: Vehicle Information API Spec (W3C)
Dataaccess
API,carserver
Enrichment,
annotation
Applications and analytics
Motivating examples
Data
abstraction
Expert-
fixed rules
RQ1: How can we provide an interoperable access and description of vehicle data ?
RQ2: How can we learn human-understandable concepts from sensor data ?
RQ3: How can we enable the integration of vehicle-generated data with external dataset
using ontologies and schemata ?
VISS, VIAS VSS ontology Signal quality
Driving context
ontology
Machine
Learning models
WoT concepts
& interactions
Context-based
recommendation
Aggressive
driving
Maneuvers1a 1b 1c
2a
2b 2c
3

02
COMPLEX VEHICLE MODELING

Access Data model
OVERVIEW
Dataaccess
API,carserver
Data
abstraction
RQ1: How can we provide an interoperable access and description of vehicle data ?
1a 1b 1c

REQUIREMENTS: COMPETENCY QUESTIONS
Get information about attributes and signals on connected vehicles
Complete list: https://github.com/klotzbenjamin/vss-ontology
Telematics
Garage/diagnosis Seamless experience
32 competency questions…
Attributes
What type of fuel does this car need?
What is the model of this car?
How old is this car?
What type of transmission does this car have?
Signals and sensors
Is there a signal measuring the steering wheel angle?
How many different speedometers does this car contain?
Dynamic signals
What is the current gear?
What is the local temperature on the driver side?
E-commerce
… generated from domain needs
on vehicle signals and attributes

1A) CURRENT SOLUTIONS FOR CAR DATA ACCESS PROVIDED BY OEM
BMW Labs
[1] https://labs.bmw.com/
[2] https://developer.mercedes-benz.com/,
[3] http://www.porsche-next-oi-competition.com/
[4] https://developer.psa-peugeot-citroen.com/inc/
[5] https://www.w3.org/auto/wg/
Web services [1] APIs in development [2-4]
Standards in development
ISO 20078:
neutral server
ISO 22901: data
model for
diagnosis
VISS, VIAS [5]
“Gen 2”
In the current charter
V2X communication
Competing standards:
- 5G (cellular network)
- ITS G5 (peer-to-peer)
Interconnected fleets will rely on them

1B) STATE OF THE ART: VEHICLE SIGNAL SPECIFICATION (VSS)
Figure:
−451 branches
−1103 leaves:
− 43 attributes
− 1060 signals: including
− (700 seat-related),
− 268 with unitSignal/Attribute
Body
Weight
Raindetection Intensity
Type: UInt8
Unit: percent
Description: “…”
Value: restriction
or free
ADAS
Cabin
Chassis
Drivetrain
OBD
Vehicle
Attribute
Signal
Signal entries
Examples:
Gearbox-sensed speed: .Drivetrain.Transmission.Speed
Engine speed: .Drivetrain.Engine.Speed
GPS-sensed speed: .Cabin.Infotainment.Speed
Left door lock: .Body.Row1.Door.Left.IsLocked
Right mirror tilt: .Cabin.Mirror.Right.Tilt
https://github.com/GENIVI/vehicle_signal_specification

1B) STATE OF THE ART: SEMANTIC TECHNOLOGIES FOR CONNECTED THINGS
Connected things
- Have sensors/actuators
- Produce/consume signals
- Have communication means
Early attempts :
SensorML1, XML description of sensors and measurement processes, 2007-2014
Semantic Sensor Network2 (SSN), expressive representation of sensors and
observations, 2004
Limitations : SSN was never really accepted in the industry (big, dependent on DUL…), complex
standardization…
1: http://www.opengeospatial.org/standards/sensorml
2: http://purl.oclc.org/NET/ssnx/ssn
The Sensor, Observation, Sample, and Actuator (SOSA) ontology was released in 2017, it
is:
- Lightweight
- Self-contained
- Domain-independent
- Simple central pattern for SSN
https://www.w3.org/TR/vocab-ssn/

1B) HYPOTHESIS: SOSA PATTERN FOR VEHICLE SIGNALS
sosa:ObservableProperty
sosa:Sensor
sosa:Observationsosa:isObservedBy
rdf:type
sosa:Result
22.2 km/h^^cdt:speed
22.2^^xsd:double
qudt:KilometerPerHour
qudt:numericValuequdt:unit
sosa:phenomenonTime
geo:lat
:Speed
- Definition of a signal
- Definition of a sensor
- Formally-defined units
- Geolocation
No formal definition of:
- “speed” or other observable properties
- “speedometer” or other car sensors/actuators
- “Vehicle” or vehicle parts
BUT
"2018-04-18T13:36:12Z"^^xsd:dateTime
geo:lon
sosa:hasSimpleResult
43.614386
7.071125

1B) VSSO DEVELOPMENT
VSS
Add sensors and
actuators
Reuse design patterns
- SSN/SOSA
- QUDT (unit)
Fixing
problems
Generate definition of
VSS concepts
Manually validate and
clean the generated
ontology
VSSontology (VSSo)
Fixing problems
1. VSS concepts have unique names
2. All signals are either observable, actuatable or both
3. Different signals can yield the same phenomenon (e.g. speed)
4. All branches are part of the top “vsso:Vehicle” branch
5. All position-dependent branches have a property “position”
Benjamin Klotz, Raphael Troncy, Daniel Wilms, and Christian Bonnet. VSSo: A Vehicle Signal and Attribute
Ontology. 9th International Semantic Sensor Networks Workshop (SSN), Monterey, California, October 2018.

1B) VSSO EXAMPLE
sosa:Observable
Property
sosa:Sensor
sosa:Observation
sosa:isObservedBy
:Speed
rdf:type
sosa:Result
22.2 km/h^^cdt:speed
22.2^^xsd:double
qudt:Unit
qudt:unit
sosa:hasSimple
Result
vsso:Speedometer
vsso:Speed
vsso:ObservableSignal
rdfs:subClassOf
rdfs:subClassOf
rdfs:subClassOf
vsso:Transmission
vsso:Drivetrain
vsso:Internal
Combustion
vss:partOf vss:partOf
vss:hasSignal
vsso:Branch
rdfs:subClassOf
4 Nm^^cdt:torque
vss:maxTorque
rdf:type
https://ci.mines-stetienne.fr/lindt/
Maxime Lefrançois, Antoine Zimmermann Supporting Arbitrary Custom Datatypes in RDF and SPARQL, In
Proc. Extended Semantic Web Conference, ESWC, Heraklion, Greece, 2016

1B) VSSO CONTRIBUTION AND EVALUATION
VSSo: a Vehicle Signal and Attribute ontology (http://automotive.eurecom.fr/vsso)
−OWL ontology of DL expressivity: ALUHOI+
−483 classes (~300 signals); 63 properties (~50 attributes)
−Reuse SSN/SOSA modeling patterns
Evaluation:
Hypothesis: VSSo data enables SPARQL queries answering the set of competency questions
Dataset: simulated (random) values for 19 signals and 23 fixed attributes on a sliding window of 3
seconds
Experiment: set 2 SPARQL endpoints with VSSo data (with 1 vehicle, with a fleet of 3 vehicles)
http://automotive.eurecom.fr/simulator/query
http://automotive.eurecom.fr/simulator/fleetquery

1B) EVALUATION
VSSo expressivity: most requirements can be translated into SPARQL queries
What are the dimension of this car?
SELECT ?length ?width ?height
WHERE { ?branch vsso:length ?length;
vsso:width ?width;
vsso:height ?height.}
SELECT DISTINCT ?localTemperature ?value ?position ?time
WHERE { ?wheel a vsso:SteeringWheel;
vsso:steeringWheelSide ?steeringWheelSide.
?branch a vsso:LocalHVAC;
vsso:position ?position;
vsso:hasSignal ?localTemperature.
?localTemperature a vsso:LocalTemperature.
FILTER regex(str(?steeringWheelSide),str(?position))
?obs a sosa:Observation;
sosa:observedProperty ?localTemperature;
sosa:hasSimpleResult ?value;
sosa:phenomenonTime ?time.
}
ORDER BY DESC(?time)
LIMIT 1
What is the current temperature on the
driver side?
90% of competency
questions can be answered
http://automotive.eurecom.fr/simulator/query
http://automotive.eurecom.fr/simulator/fleetquery

1B) VSS2
Attribute
Branch
Vehicle
Branch
VehicleIdentification
Branch
VIN
Attribute
Body
Branch
Drivetrain
Branch
Signal
Branch
Body
Branch
Drivetrain
Branch
Vehicle
Branch
AverageSpeed
Signal
Private
Branch
BMW
Branch
PrivateSignal
??
HMI
Branch
Vehicle
Body ADAS Cabin Chassis Drivetrain OBD
Private branches and leaves should:
- Overwrite pre-existing concepts
- Extend the VSS tree
VSS needs consistent position
patterns
VSS1 | VSS2

Vehicle
sensor
data
Sampling
Contract
TransmissionMarketplace
Privacy
1C) REMAINING COMPONENTS
https://www.w3.org/community/autowebplatform/wiki/Data_tf
https://at.projects.genivi.org/wiki/pages/viewpage.action?pageId=34963516
Degradation
Retention
Compression
Quality
Business
considerations
Consent
Policies
Sensitivity
Access
control
Signal quality: many
components
Work items at the
W3C Automotive
Data Task Force

03
AUTOMATIC CONTEXTUALIZATION OF DRIVING DATA

Access Data model
OVERVIEW
Dataaccess
API,carserver
Enrichment,
annotation
Data
abstraction
Driving context
ontology
1a 1b 1c
2a

REQUIREMENTS FOR A DRIVING CONTEXT MODEL
- Sub-domains clustered based on their common models and output of classifiers.
- Centered around drivers, passengers, vehicles and roads.
28 competency questions
Domain states
What is the weather at this location/time?
What is the mental state of this passenger?
What is the level of fluidity on this road?
Domain events
What is this hazard?
Who is involved in it?
Cross-domain states
Am I currently overspeeding?
What is the most common mental state of the driver on the current trajectory?
Cross-domain events
What the cause of congestion?
How many tti:TurnLeft maneuvers have the driver done the last hour?
Driving context domains
Complete list: https://github.com/klotzbenjamin/vdc-ontology

2A) CONTRIBUTION: DRIVING CONTEXT MODELING PATTERN
Hazard
Ontologies [3]
Mental state
Ontologies [2]
Traffic
Ontologies [7]
Weather
Ontologies [6]
Places
Ontologies [5]
Road
Ontologies [4]
Driving context
Modeling pattern
Spatiotemporal
Events
and States
[1] http://automotive.eurecom.fr/vsso
[2] https://bioportal.bioontology.org/ontologies/MFOEM
[3] https://transportdisruption.github.io/
[4] http://ci.emse.fr/opensensingcity/ns/sca/vocabulary_81/
[5] http://mapserv.kt.agh.edu.pl/ontologies/osm.owl
[6] https://ci.mines-stetienne.fr/seas/WeatherOntology
[7] http://vocab.datex.org/terms/
VSSo
(=SSN/SOSA+VSS)
~ 300 signals
~ 50 attributes
~ 80 branches
Car signal
Ontologies [1]

2A) MODELING CHOICES: STATES AND EVENTS
Alignment with domain ontologies/vocabularies
owl:equivalentClass
On states of features of interest
States Events
Benjamin Klotz, Raphael Troncy, Daniel Wilms, and Christian Bonnet. VSSo – A vehicle signal
and attribute ontology for the Web of Things. Semantic Web journal, 2019.

2A) DRIVING CONTEXT CONTRIBUTION AND EVALUATION
VDC: a Vehicle Driving Context ontology (http://automotive.eurecom.fr/vdc)
−OWL ontology of DL expressivity: ALC
−22 classes and 3 object properties
−Re-use SSN/SOSA modeling patterns and the event ontology
Evaluation:
Hypothesis: VDC data enables SPARQL queries answering the set of competency questions
Datasets converted from:
− Collected from a BMW research car: 3 minutes, 3k observations on 10 signals, 103 maneuvers instances
− Public dataset of stress, mental load linked to car signals, maneuvers and road types: 45 minutes, 13k
observations on 6 states, 31 maneuvers
Experiment: set up a SPARQL endpoint with VDC data
Benjamin Klotz, Raphaël Troncy, Daniel Wilms, and Christian Bonnet. A driving context ontology for
making sense of cross-domain driving data. In BMW Summer school, Raitenhaslach, Germany, 2018
Schneegass, S., Pfleging, B., Broy, N., Heinrich, F., & Schmidt, A. (2013, October). A data set of real world driving to assess driver workload. In
Proceedings of the 5th international conference on automotive user interfaces and interactive vehicular applications (pp. 150-157). ACM.

2A) EVALUATION
VDC expressivity: most requirements can be translated into SPARQL queries
Am I overspeeding?
86% of competency
questions can be answered
SELECT ?overspeeding
WHERE {
?segment a datacron:TrajectoryPart.
FILTER NOT EXISTS {
?segment datacron:directlyPrecedes ?nextSegment
}
signal a tti:SpeedLimit.
?observation a sosa:Observation;
sosa:hasObservedProperty ?signal;
sosa:hasSimpleResult ?speedLimit;
geo:location ?segment.
SELECT ?speed WHERE { ?signal a vsso:VehicleSpeed.
sosa:hasSimpleResult ?speed;
sosa:hasPhenomenonTime ?time.
}
ORDER BY DESC (?time)
LIMIT 1
BIND( IF(?speed>?speedLimit,"yes","no") as ?overspeeding )
}
SELECT ?flow
WHERE {
?signal a datex:TrafficFlow.
sosa:hasSimpleResult ?flow;
sosa:hasPhenomenonTime ?time.
}ORDER BY DESC (?time)
LIMIT 1
What is the local traffic flow?

Access Data model
OVERVIEW
Dataaccess
API,carserver
Enrichment,
annotation
Motivating examples
Data
abstraction
Expert-
fixed rules
Driving context
ontology
Machine
Learning models
Context-based
recommendation
Aggressive
driving
Maneuvers1a 1b 1c
2a
2b 2c

STATE OF THE ART: PREDICTION FROM VEHICLE DATA
- Aggressiveness
- Drowsiness
- Driving style
- Diagnosis
- Topology
- Marks
- Potholes
- Obstacles
- Weather
- Emotions
- Stress
- Mental load
- Frustration
- Distraction
- Maneuvers
- Intents
Vehicle machine learning
In-car learning
Behavior Mental State Environment
Trajectory
patterns
Fleet
learning
Data sources:
- Car sensors
- Smartphones/cameras
- Physiological sensors

2B) AGGRESSIVE MANEUVER CLASSIFIER
Protocol:
- A driver told to drive safely on roads of to do dangerous maneuvers on a track
- A copilot is using a car application to annotate the recorded data
- Sequence: 10 measurements
- Split 70/30
AUCRF=0.99
AUCRNN=0.998
Evaluation
Dataset
- ~13 hours of recorded data
- ~3,5 hours of aggressive driving
- 183 maneuvers
- Sampling period of 0,5s
Daniel Alvarez Coello, Benjamin Klotz, Daniel Wilms, Jorge Marx Gómez, and Raphaël Troncy. Modeling
dangerous driving events based on in-vehicle data using Random Forest and Recurrent Neural Network. In 1st
International Workshop on Data Driven Intelligent Vehicle Applications (DDIVA), Paris, France, 2019

2B) RECONSTRUCTION OF DANGEROUS SITUATIONS
Important difference between safe and aggressive Track shape and speed far from public roads
2 biases
Daniel Alvarez Coello, Benjamin Klotz, Daniel Wilms, Jorge Marx Gómez, and Raphaël Troncy. Modeling
dangerous driving events based on in-vehicle data using Random Forest and Recurrent Neural Network. In 1st
International Workshop on Data Driven Intelligent Vehicle Applications (DDIVA), Paris, France, 2019

2C) STATE OF THE ART: MOBILITY DATA MINING
Spatiotemporal trajectories
Semantic trajectories
Main patterns Places/Regions
Connectivity
Mining
Movement
prediction
CharacterizationOutlier detection
Place
recommendation
Jean Damascene Mazimpaka and Sabine Timpf. Trajectory data mining-a review
of methods and applications. Journal of Spatial Information Science, 2016.

2C) CONTRIBUTION SEMANTIC TRAJECTORIES GENERATION
http://automotive.eurecom.fr/trajectory
Benjamin Klotz, Raphael Troncy, Daniel Wilms, and Christian Bonnet. Generating semantic trajectories using
a car signal ontology. In The Web Conference (WebConf'18), Demo Track, April 23-27, 2018, Lyon, France.
Rule example
acceleration/deceleration/
constant speed with a +/-
5km/h between observations
Comparative table of
ontology combination
from the state of the art
Demonstration: threshold-based Horn rules
Hypothesis: we can enable semantic trajectory
enrichments with signal values for generic applications

2B-2C) COMBINING DATA INTEGRATION AND INFERENCES
Semantic Technologies for Vehicle Data | Benjamin Klotz | 24/09/2019 Page 37
p1
p2
p3 p4 p5
p6
p7
p8
p9
p10
p11
p12
Aggressive behavior
Neutral
Safe behavior
Traffic
jam
Close
event
Closed road
Generalization of semantic trajectories with a driving context
Statistical learning from vehicle data
Data integration from external sources
Rule-based inferences
Spatiotemporal traces Contextually-enriched trajectories
Example: mental load & maneuver
events

2B-2C)DATA INTEGRATION AND ROUTE SELECTION: POI RECOMMENDATIONS
Recommendation of places to stop and rest on
long trips (http://drivescover.eurecom.fr)
According to driving features:
- Traffic
- Weather
- (opening) time
- Ratings
POI score:
score = [WeightClass *Class]*hoursOpen
*(Weather + Rating + Traffic)
Evaluation: contextual driving features enable
driver recommendations
- 18 journeys (35 stops) by giving on a poll by 18
users: rank the top between 10-20 POIs
- Average P@4=31,5%, R@4=14,1%
Klotz, B., Lisena, P., Troncy, R., Wilms, D., & Bonnet, C. (2017, October). DriveSCOVER: A Tourism
Recommender System Based on External Driving Factors. In International Semantic Web

2B-2C) CONTEXT-BASED ROUTE RECOMMENDER: DEMONSTRATION
Scope: from scattered data to risk-centric semantic
trajectories
− Vehicle data and its inferences (location, destination,
aggressiveness) combined with contextual data (routing,
traffic, weather)
− Semantic intended trajectories, include a quantified risk
and query it to find the “best route” (safe and short)
Assumption: quantifying risky driving features
− “Expert views of police officers, lay views of the driving
public, and official road accident records” to quantify the
contribution of contextual features to driving danger
(percentage of those features reported in accidents)
Variables: age, gender, experience, aggressiveness, fatigue,
time, locations, road information
Current position
Destination
Jonathan J. Rolison, Shirley Regev, Salissou Moutari, Aidan Feeney, What are the factors that
contribute to road accidents? An assessment of law enforcement views, ordinary drivers’
opinions, and road accident records, Accident Analysis & Prevention. Volume 115. 2018
Vehicle data +
demographics
Best N possible
routes
External context:
weather, traffic
Driving danger
features
Best route +
explanation
origin

The fastest route is safe A safer alternative is better
2B-2C) CONTEXT-BASED ROUTE RECOMMENDER: IMPLEMENTATION
Patterns
- Semantic Trajectories: DatAcron
- Driving Context: VDC,
- Risk: Hazard pattern (VoCamp)
Inferences
- Triple matching and
generation
- SPARQL Construct
Best route=trade-off between
- Routing time T
- Normalized risk factor R
Minimize f(T,R)
Experiment: show usability and explainability of an ontology-based route recommender
2 possible outputs
Provide a list of
differentiating
risk factors
Evaluation:
- Test different functions f to have about as many of both outputs
- F(T,R)=TR² is a working heuristic
- Test different time and driver profiles with fixed origin and destination
- Check output information validity in the semantic trajectory
Heuristics/parameters
Experience threshold: 10 years (half the average license age)
Snow/rain threshold: 4cm (rule of thumb on wheel sinkage)
Visibility maximum: 10km (METAR scale upper limit)
Night/day factor: 0.8 (arbitrary heuristics)
Road type risks: 0.1 to 1 (arbitrary heuristics) from average
stress impact [1]
[1] Schneegass, S., Pfleging, B., Broy, N., Heinrich, F., & Schmidt, A. (2013, October). A data set of
real world driving to assess driver workload. In Proceedings of the 5th international conference on
automotive user interfaces and interactive vehicular applications (pp. 150-157). ACM.

04
INTERACTIONS IN THE WEB OF THINGS

Interactions
Access Data model
OVERVIEW
Dataaccess
API,carserver
Enrichment,
annotation
Motivating examples
Data
abstraction
Expert-
fixed rules
RQ3: How can we enable the integration of vehicle-generated data with external dataset
using ontologies and schemata ?
Driving context
ontology
Machine
Learning models
WoT concepts
& interactions
Context-based
recommendation
Aggressive
driving
Maneuvers1a 1b 1c
2a
2b 2c
3

Semantic Technologies for Vehicle Data | Benjamin Klotz | 24/09/2019
W3C WoT servient
43
STATE OF THE ART: SEMANTIC TECHNOLOGIES FOR DATA INTEGRATION
Limited vehicle sensor/actuator access
SSN/SOSA
Domain
ontology
AutoWG
Web
Devices
IoT platforms

STATE OF THE ART: WEB OF THINGS
ContextDevice
Features of
Interest
(domain-
specific
metadata)
Thing
(Car)
Capabilities
(HVAC)
Interactions
(Action heater)
Data
(data schema)
Amelie Gyrard. Designing cross-domain semantic Web of things
applications. PhD thesis, EURECOM, 2015

STATE OF THE ART: WOT ONTOLOGY
−Define a wot:Thing
−Centered on wot:interactionPattern
− Properties
− Actions
− Events
−Use dataSchema
− Literal value
− wot:DataType
− om:Unit_of_measure
http://iot.linkeddata.es/def/wot/index-en.html

AUTOMOTIVE WEB THINGS: CHALLENGES
Domain vs Nature of
−Things
−Interactions
Complexity of vehicles
−Different access control and
security
−Different expertise
"@id": "property/acceleration“,
"@type": ["Property","vsso:LongitudinalAcceleration","iot:Property"],
Data access
−External hardware
−Legacy solutions

3) CONTRIBUTION: COMPLEX THINGS IN THE WOT
Differing Safety, Security and Privacy ?
TD: full vehicle
@context
securityDefinitions
Interactions:
1. property/media-
volume
2. action/stop-HVAC
3. event/sow-engine-oil
…
2548. action/write-message
TD: main
TD: safety-critical
TD: HVAC TD: infotainment
@context
securityDefinitions
Interactions:
1. property/media-
volume
…
Interactions:
1. action/stop-HVAC
…
TD: Engine
@type
@type
Interactions:
1. event/sow-engine-
oil
…
securityDefinitions
TD: privacy-critical
securityDefinitions
Requirements included in the latest charter update (June 2019)

3) CONTRIBUTION: ALIGNMENT WOT – SOSA FOR THE AUTOMOTIVE DOMAIN
Modeling pattern:
i. Vehicles are Things
ii. Signals are properties
Read-write depending on the signal type
iii. Actuatable signals are actions
iv. DataSchema use the domain
Units
Benjamin Klotz, Soumya Kanti Datta, Daniel Wilms, Raphael Troncy, and Christian Bonnet. A car as a semantic web
thing: Motivation and demonstration. In 2nd Global Internet of Things Summit (GIoTS'18), Bilbao, Spain, June 2018.

3) CONTRIBUTION: ALIGNMENT FOR EVENTS
Event modeling pattern:
i. Mapped to the Event Ontology
ii. Wot:Event≡event:Event
iii. The FOI (Vehicle) is an Agent
Evaluation:
- Hypothesis: a combination of WoT and VDC/VSSo enables cross-
domain interactions with vehicle attributes, signals and events
- Comparative table with state of the art survey
Benjamin Klotz, Raphael Troncy, Daniel Wilms, and Christian Bonnet. VSSo – A vehicle signal
and attribute ontology for the Web of Things. Semantic Web journal, 2019.

3) DEMONSTRATIONS (1): REAL CAR, DIAGNOSIS SIGNALS (2017)
Usage:
Behind a web browser, we can control
the windows, doors and honk
Benjamin Klotz. Binding the web of things with LwM2M for a vehicular use
case. OpenDay of the W3C Web of Things Working Group, June 2017

3) DEMONSTRATIONS (2): TRACES AND APPLICATIONS
Automatic applications of rules and interactions of web things:
- If a door is open while speed>0 then trigger a warning
- If longitudinal and lateral acceleration are high (dangerous driving), then turn on a red LED
- If the coordinates of the car are close to a fixed destination, control a garage door an light
Application servient
Car servient
(mocked up)
Check
rules Other servient
Read properties
Write properties
https://www.youtube.com/watch?v=pjgTLPlAsKQ
https://www.youtube.com/watch?v=zkL8Cdgy8PE

Legacy
3) DEMONSTRATION (3): INTUITIVE INTERFACE ON LEGACY VEHICLES
Device servient
Vehicle Web API
TD
Application servient
HTTPS
Flow
Vehicle
Node
HTTPS
Benjamin Klotz, Daniel Alvarez Coello, Daniel Wilms, and Raphaël Troncy. Abstracting and Interacting with Vehicles in
the Web of Things. 2nd W3C Workshop on the Web of Things The Open Web to Challenge IoT Fragmentation, 2019

05
CONCLUSION AND FUTURE PERSPECTIVES

CONCLUSION
RQ1: How can we provide an
interoperable access and description of
vehicle data ?
RQ2: How can we learn human-
understandable concepts from sensor
data ?
RQ3: How can we enable the
integration of vehicle-generated data
with external dataset using ontologies
and schemata ?
Data models
- VSSo (http://automotive.eurecom.fr/vsso)
- VDC (http://automotive.eurecom.fr/vdc)
- Alignment VSSo/VDC/WoT (axioms to publish)
Datasets: semantic trajectories
- VSSo data: 5 trajectories, 8 signals, 16k observations
(http://automotive.eurecom.fr/trajectory)
- VDC data: 2 trajectories, 16 states, 16k observations,
130 events
Classifiers (Internal BMW)
- Aggressive driving prediction
- Maneuvers prediction
Demonstrations
- DrIveSCOVER
- Semantic trajectory annotation
- WoT vehicles

REFLECTION
Focus: usage and benefits of vehicle data with semantic technologies
− This research is a step in making connected vehicles more interoperable and understandable for
application development
Need for open standards
− not necessarily for a full solution for connected vehicles
Need to find the right level of abstraction from data itself
−Limited computing resource embedded in vehicles, and the technical requirements to describe vehicles for
the industry

REFLECTION
Semantic Technologies for Vehicle Data | Benjamin Klotz | 24/09/2019 56https://www.w3.org/auto/

FUTURE WORK: SHORT TERM
Ongoing research optimizing the usage of VSSo/VDC
− Connected Thesis on “Scalable reasoning on data of heterogenous sources and environments in an
automotive context” started in 2018 at BMW by Daniel Alvarez.
Experiment with development departments of OEMs the usage of VSSo and VDC ontology
Possible work items benefiting from standardization:
− W3C
− Web of Things: complex things requirements description and best practices
− Automotive Data task force: data contract, sampling, quality, marketplaces…
− VSS2: specification to implement and new vehicle server to propose
− GENIVI
− Connected services community to join standardization efforts
− OGC
− Invitation to an OGC meeting to join efforts on Mobile Location Service (MLS) Domain Working Group
(DWG)

LONG TERM PERSPECTIVES
Research goals
−Explore pattern alignments of best practices of signal metadata (marketplace, sampling, policies…) with
VSSo
−Assess the best practices for VDC (granularity, domain expert rules) and formulate recommendations for
future VDC versions
−Develop testbeds for semantic trajectories and optimize recommender systems
Push the current work further to product development and global open standards.
− Involve more parties in the standardization of semantic technologies for connected vehicles, including
suppliers and data consumers, and provide the missing standard for formal vehicle signals and attributes
based on VSSo
− Push developer communities and OEMs to apply those standards in developed products at a larger
scales

PUBLICATIONS AND CONTRIBUTIONS TO STANDARDS (1)
Journals and book chapters (2)
− Benjamin Klotz, Raphael Troncy, Daniel Wilms, and Christian Bonnet. VSSo – A vehicle signal and attribute ontology for the Web of
Things. Semantic Web journal, 2019.
− Mahda Noura, Amélie Gyrard, Benjamin Klotz, Raphaël Troncy and Soumya Kanti Datta. How to understand better "smart vehicle"?
Knowledge Extraction for the Automotive Sector Using Web of Things, chapter SIoT 2020: Semantic IOT: Theory and Applications -
Interoperability, Provenance and Beyond. Studies in Computational Intelligence. Springer, 2020. Submitted
International Conferences and workshops (6)
− Daniel Wilms, Benjamin Klotz and Adnan Bekan. Enabling innovation through open standards in the transportation domain. W3C
Workshop on Transportation Data, 2019
− Daniel Alvarez Coello, Benjamin Klotz, Daniel Wilms, Jorge Marx Gómez, and Raphaël Troncy. Modeling dangerous driving events based
on in-vehicle data using Random Forest and Recurrent Neural Network. In 1st International Workshop on Data Driven Intelligent Vehicle
Applications (DDIVA), Paris, France, 2019
− Benjamin Klotz, Daniel Alvarez Coello, Daniel Wilms, and Raphaël Troncy. Abstracting and Interacting with Vehicles in the Web of Things.
2nd W3C Workshop on the Web of Things The Open Web to Challenge IoT Fragmentation, 2019
− Benjamin Klotz. Cross-domain interactions for connected cars, 2018. Keynote at the 8th International Conference of Consumer
Electronics
− Benjamin Klotz, Soumya Kanti Datta, Daniel Wilms, Raphael Troncy, and Christian Bonnet. A Car as a Semantic Web Thing: Motivation and
Demonstration. In 2nd Global Internet of Things Summit (GIoTS), Bilbao, Spain, 2018
− Benjamin Klotz, Raphael Troncy, Daniel Wilms, and Christian Bonnet. VSSo – A vehicle signal and attribute ontology. In 9th International
Semantic Sensor Networks Workshop (SSN), Monterey, USA, 2018

PUBLICATIONS AND CONTRIBUTIONS TO STANDARDS (2)
Demonstrations and Posters at international conferences (3)
− Benjamin Klotz, Raphaël Troncy, Daniel Wilms, and Christian Bonnet. A driving context ontology for making sense of cross-domain
driving data. In BMW Summer school, Raitenhaslach, Germany, 2018
− Benjamin Klotz, Raphaël Troncy, Daniel Wilms, and Christian Bonnet. Generating Semantic Trajectories Using a Car Signal Ontology. In
The Web Conference (WWW), Demo Track, pages 135138, Lyon, France, 2018
− Benjamin Klotz, Pasquale Lisena, Raphaël Troncy, Daniel Wilms, and Christian Bonnet. DrIveSCOVER: A tourism recommender system
based on external driving factors. In International Semantic Web Conference (Posters, Demos & Industry Tracks), Vienna, Austria, 2017
Contributions to Standards (7)
− Benjamin Klotz. W3C Automotive WG Activities - Next generation specification and VSS. GENIVI All Member Meeting & W3C
Automotive Working Group F2F Meeting, May 2019. Munich, Germany
− Benjamin Klotz. Ontology alignment for automotive Web Things. Iotschema Community, November 2018. Online Call
− Benjamin Klotz, Daniel Wilms, and Ulf Björkengren. VSS2.0 proposal. W3C Automotive Working Group at the TPAC, October 2018.
Lyon, France
− Benjamin Klotz. WoT and JSON-LD for Vehicle Data. W3C Automotive Working Group, October 2018. Online Call
− Benjamin Klotz. VSSo: a car signal ontology - Extension of the Vehicle Signal Specification. GENIVI All Member Meeting & W3C
Automotive Working Group F2F Meeting, April 2018. Munich, Germany
− Benjamin Klotz. Semantic Technologies for Vehicle Data - VSS proposal. GENIVI Networking Expert Group, March 2018. Online call
− Benjamin Klotz. Binding the web of things with LwM2M for a vehicular use case. OpenDay of the W3C Web of Things Working Group,
June 2017

QUESTIONS ?
THANK YOU FOR YOUR ATTENTION
61
Benjamin Klotz
 klotz@eurecom.fr
Academic supervisors: Christian Bonnet, Raphaël Troncy
Industry supervisors: Daniel Wilms, Martin Arend, Michael Würtenberger

Semantic Technologies for Vehicle Data - Defense

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Semantic Technologies for Vehicle Data - Defense

Similaire à Semantic Technologies for Vehicle Data - Defense (20)

Dernier

Dernier (20)

Semantic Technologies for Vehicle Data - Defense