Invited Talk, International Semantic Intelligence Conference (ISIC 2021) New Delhi, India - February 25, 2021
The objective of this talk is, from a pragmatic point of view, to show examples of applications using semantic technologies in whose development the presenter has been involved. The talk will start showing a toy application, used in the classroom for teaching purposes but that illustrates all aspects of a typical development benefiting from semantic technologies. From integrating different data sources to creating an end-user interface. Then, the talk moves to bigger, real-world projects ranging from the application of semantic technologies for copyright management to media monitoring for plant health threats.
1. Invited Talk:
A pragmatic view on Semantic Technologies
Roberto García, Universitat de Lleida, Spain
International Semantic Intelligence Conference (ISIC 2021)
New Delhi, India - February 25, 2021
3. Motivation & Outline
• Illustrate that even
“a little semantics goes a long way”
James Hendler, circa 1997
https://www.cs.rpi.edu/~hendler/LittleSemanticsWeb.html
• Do so through applications using semantic technologies
I have participated in:
• Game of Thrones
• Example project I use in the classroom
• MedISys Plant Health Threats
• Media Monitoring project for the European Food Safety Authority
• InVID Social Media Verification
• European research project about media verification and reuse for
journalistic purposes
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 3
4. Example 1: Classroom Project
• Example project to show what is expected from
the project they should deliver at the end
• Motivation:
application that supports readers of Game of
Thrones books (especially those that have seen
the TV series)
• Characters, houses they are loyal too, books they appear
in, picture showing series actor playing the character,...
• Added value by using semantic technologies:
• Reduced cost by integrating multiple existing data sources
• CSV, SPARQL, Web pages,…
• Facilitate the development of apps that allow exploring the
data
• Ease conceptualisation and maintenance by reusing
existing vocabularies (ontologies)
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 4
5. Reuse Existing Data
• Kaggle dataset (https://www.kaggle.com/mylesoneill/game-of-thrones)
• character-deaths.csv
• Structure: name, allegiance, death year,...
nobility (1: true | 0: false), appear in book (1: true | 0: false)
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 5
13. Tabular to Semantic Data
• Generate unique identifiers as URIs
• Independent from data source
• Table to RDF triples (subject –predicate object/literal)
• Rows
correspond to the same subject identified by URI
• Columns
correspond to subject properties
• Cells
correspond to objects (relationships) or literals (attributes)
• objets, sujects, and properties replace tabular value with URI
• Jon Snow http://mydomain.org/persons/Jon_Snow
• literales, text and make data type explicit if available
• 299 "299"^^<http://www.w3.org/2001/XMLSchema#gYear>
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 13
14. Tabular to Semantic Data
• Example for one row:
@PREFIX : <http://mydomain.org/persons/> .
@PREFIX families: <http://mydomain.org/families/> .
@PREFIX got: <http://mydomain.org/got-ontology/> .
@PREFIX foaf: <http://xmlns.com/foaf/0.1/> .
@PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@PREFIX dbo: <http://dbpedia.org/ontology/> .
@PREFIX dbp: <http://dbpedia.org/property/> .
@PREFIX dbr: <http://dbpedia.org/resource/> .
...
:Robert_Baratheon rdf:type dbo:Nobel, dbo:FictionalCharacter ;
rdfs:label "Robert Baratheon"@en ;
foaf:name "Robert Baratheon"@en ;
dbo:allegiance families:House_Baratheon ;
dbp:genre "Male" ;
got:deathChapter "47"^^<http://www.w3.org/2001/XMLSchema#integer> ;
owl:sameAs dbr:Robert_Baratheon ;
...
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 14
15. Automate Transformation
• Many tools Tabular or Relational DB to RDF
• For tabular: OpenRefine
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 15
16. Additional Sources
• Character’s pictures https://www.hbo.com/game-of-thrones/cast-and-crew
• Get picture, plus characters’ names to reconcile with
previous RDF data using OpenRefine
• Automatic integration using same URI for characters
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 16
17. Final Result
• Automatic User Interface
to explore the data
• Generation driven by
data structure
https://rhizomer.rhizomik.net/datasets/got
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 17
18. Example 2: EFSA Project
• Multilingual Ontology for Plant Health Threats Media Monitoring
• Development and testing of the media monitoring tool MedISys for
the early identification and reporting of existing and emerging
plant health threats
• Timing (duration): January 2014 – June 2016 (2.5 years)
• Funding: EFSA
• Objectives:
• Collate new and appropriate media information sources
• Multilingual ontology for the global identification of emerging new
plant health threats to be appended to MedISys
• English, Spanish, Italian, French, Dutch, German, Portuguese, Russian, Chinese and Arabic
• Develop and test strategies to monitor re-emerging plant health
threats on global and regional scale
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 18
19. Proposed Approach
• Approach based on the use of semantics and ontologies
• Ontology: key component of the developed system that structures and
provides knowledge about plant health threats
• Knowledge captured from existing sources and experts
• Guides applications for
• Knowledge capture
• Indirect sources search
• Terms translation
• Media monitoring
categories generation
An ontology is a formal, explicit specification of a shared conceptualisation.
is
means
implies expressed in
terms of
Abstract model of
portion of world
Machine-readable
and understandable
Based on a
consensus
Concepts,
properties,...
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 19
20. Ontology Generation
• Ontology Skeleton
• Collected 140 pests/diseases from EPPO Alerts, 2000/29-1-
A-1 and EU Emergency Control Measures
• 117 linked to UniProt Taxonomy:
• Taxonomical information, scientific/common/other names,…
• 47 linked also to Wikipedia
• Common names in
multiple languages
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 20
21. Ontology Generation
• Plant Health Threats Ontology
• Enrich ontology with affected crops, hosts, vectors, symptoms
expressions…
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 21
22. Ontology Enrichment
• Plant Health Threats Ontology
• All concepts linked to labels in different languages
• Extract as keywords for MedISys or Web search filters,…
• Example: “Maladie de Pierce” OR ( “grapevine” AND
“sharpshooter” )
Xylella fastidiosa
Gammaproteobacteria
Nerium oleander,
Prunus salicina, Medicago
sp., Sorghum halepense,…
Homalodisca coagulata,
Graphocephala sp.,
Oncometopia sp.,
Draeculacephala sp.,…
Grapevine, Citrus, Olive,
Almond, Peach, Coffee,…
subClassOf
vector
host
crop
“Pierce's disease”, “Citrus
variegated chlorosis” en
“Maladie de Pierce” fr
“葉緣焦枯病菌” zn
“Glassy-winged sharpshooter”,
“Spittlebugs”, “Froghoppers”,
“Planthoppers”,… en
“vite” it,… …
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 22
23. Ontology Enrichment
• Ontology Editor
• Assist experts during the knowledge capture process
23
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021
24. Assisted Knowledge Capture
• Ontology Editor – forms with assistance
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 24
25. Assisted Knowledge Capture
• Ontology Editor - autocomplete
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 25
26. Example 3: InVID Project
• H2020 project InVID, In Video Veritas
• Verification of Social Media Video Content for the News
Industry
• https://www.invid-project.eu
• Reuse of User Generated Video from Social Media for
journalistic purposes
• Discovering social media about current events
• Video verification to avoid fake news
• Request reuse, check licensing, negotiate terms, sign
agreements,… even economic compensation
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 26
27. Objectives
• Sophisticated models for copyright information:
• Rights status
• Reuse terms
• Negotiation
• Copyright agreements
• Trust and confidence on rights statements
• Potentially legally binding
(time stamp, signatures, tamper proof,…)
• Proposed approach:
• Semantic Web: rich information modelling and reasoning
• Blockchain: immutable and accountable information storage
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 27
28. Copyright Ontology
• Copyright knowledge representation
• Copyright Ontology based on the
fundamental ontological distinctions:
• Abstract: intangible
• Process: happens,
temporal stages
(action, event,…)
• Object: can be defined
independent of time
(includes digital objects)
Victor Hugo’s
Les Misérables
Abstract
Objects
Processes
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 28
29. Copyright Ontology
• Also capture the dynamic parts of
the copyright value chain
• Actions performed
by value chain participants
• Plus consumer actions:
• Buy, Attend, Access,
Play, Tune,…
• Plus licensing actions:
• Agree/Disagree
• Transfer, Attribute,…
Victor Hugo’s
Les Misérables
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021
Creator Actor Producer Broadcaster User
Motion Picture
Script
Adaptation Performance
manifest perform record
Communication
broadcast
adapt
Literary Work
tune
29
30. Copyright Ontology
• Model full details of an action,
its dimensions like as a verb in
a sentence (roles):
• who performs it,
what is manipulated,
when, where…
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021
Role Kind Main Role Description
who schema:agent
The direct performer or driver of the
action (animate or inanimate)
schema:participant
Other co-agents that participated in the
action indirectly, for instance a recipient
what schema:object
The object upon which the action is
carried out
schema:result The result produced in the action
where schema:location Where an action takes place
schema:fromLocation
The original location of the object or the
agent before the action
schema:toLocation
The final location of the object or the
agent after the action
when schema:startTime
When the action started or the time it is
expected to start
schema:endTime
When the action finished or the time it is
expected to end
pointInTime
The point in time when the action
happens
duration
The amount of time the action requires to
complete
with schema:instrument
The object that helps the agent perform
the action
why aim The reason or objective of the action
how manner The way the action is carried out
if condition
Something that must hold or happen
before the action starts
then consequence
Something that must hold or happen after
the action is completed
Who?
What?
When? Where?
30
31. Check UGV Rights Status
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021
https://rights.invid.udl.cat
31
32. Request Reuse
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021
Current video, plus
all future YouTube videos by content owner or
in any social network linked to InVID profile
32
33. License Reasoning
• Streamline licensing
• License to organisation or everyone
• License future videos
• Semantic representation
of agreements
• Semantic queries to check
previous agreements
• Including territories,
timeframes
or revocations
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021
InVID Rights
Management
Rights
Database
JSON-LD
Semantic
Repository
Semantic
Copyright
Management
33
34. Store Agreement
• JSON-LD serialisation of a
Reuse Agreement:
• Grants any member of the Daily Planet
permission to republish a YouTube
video whose owner is the Google user
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021
{
"@context": {
"@vocab": "http://invid.udl.cat/ontology/",
"cro": "http://rhizomik.net/ontologies/copyrightonto.owl#",
"schema": "http://schema.org/"
},
"@id": "…/reuseAgreements/1", "@type": "cro:Agree",
"cro:when": "2019-02-16T15:15:00Z",
"cro:who": [
{
"@id": ”…/inVIDUsers/1", "@type": "schema:Person",
"schema:name": "Clark Kent",
"schema:email": "journalist@invid-project.eu",
"schema:memberOf": {
"@id": "…/organizations/1",
"@type": "schema:Organization",
"schema:name": "Daily Planet"
}
},
{
"@id": "…/contentOwners/1", "@type": "schema:Person",
"username": "user”, "schema:email": "user@gmail.com",
"schema:name": "Google User"
} ],
"cro:what": {
"@id": "…/reuseTerms/1", "@type": "cro:MakeAvailable",
"schema:startTime": "2019-03-01T10:44:00Z",
"schema:endTime": "2019-05-01T10:44:00Z",
"cro:who": { "@id": "…/organizations/1" },
"cro:what": {
"@id": "…/youTubeVideos/_5l7vn1QdKM", "@type": "YouTubeVideo",
"user": {
"@id": "…/youTubeChannels/MyChannel", "@type": "YouTubeChannel",
"contactURL": "http://www.youtube.com/channel/MyChannel/about",
"contentOwner": { "@id": "…/contentOwners/1" }
}
}
}
}
34
36. SPARQL for License Reasoning
• SPARQL standard for semantic
queries
• Check intended reuse against
existing agreements
• Encapsulate complexities minimising
implementation cost
• Flexibility and scalability
• Example:
• Active agreements, not disagreed,
with Make Available term
• what: restricted to the YouTube
video _5l7vn1QdKM
• startTime: 2019-11-15
• who: is InVIDUser 2, Organization 1
or any organization InVIDUser 2 is a
member of
• where: Spain or a region Spain is
contained in
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021
PREFIX …
SELECT DISTINCT ?isAuthorized ?why
WHERE {
?agree rdf:type cro:Agree ;
cro:what ?term ; cro:when ?agreeDate
FILTER ( xsd:dateTime( ?agreeDate) <= now() )
OPTIONAL {
?disagree rdf:type cro:Disagree ;
cro:what ?term ; cro:when ?disagreeDate
FILTER ( xsd:dateTime( ?disagreeDate) <= now() )
}
BIND((bound(?agree) && (!bound(?disagree))) AS ?isAuthorized)
BIND(if(bound( ?disagree), ?disagree, ?agree) AS ?why)
?term rdf:type cro:MakeAvailable .
?term cro:what <…/youTubeVideos/_5l7vn1QdKM> .
?term schema:startTime ?start FILTER ("2019-11-15" >= ?start)
…
{
{ ?term cro:who <…/inVIDUsers/2>}
UNION
{ ?term cro:who <…/organizations/1>}
UNION
{ ?term cro:who ?organization .
<…/inVIDUsers/2> schema:memberOf ?organization }
}
{
{ ?term cro:where "Spain"}
UNION
{ ?term cro:where ?regionName .
?region rdfs:label ?regionName .
?country rdfs:label "Spain" .
?country (schema:containedInPlace)+ ?region
}
}
}
36
37. Trustful Agreements
• Use Ethereum Smart Contracts
• Blockchain as a global shared computer
• Immutable transactions (executed in all nodes)
• Encode rules guaranteed to execute
• Smart contract keeps track of semantic agreements
• Participants digitally sign negotiation steps, last by both (agreement)
• Identity management using uPort mobile app
• Self-Sovereign Identities (e.g. email attestations)
• Transaction signing: scan QR code
• Optional: remuneration using cryptocurrency wallet
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 37
https://www.uport.me
Rights
Database
InVID Rights
Management
Distributed
Ledger
Agreement
Time Stamping
Accountability
Auditability
Tamper Proof
Identify
Attestations
38. Thank you for your attention
Questions?
More details:
http://rhizomik.net/~roberto
?
Notes de l'éditeur
(“Objectives“ refers to the whole duration of the project; “focus of year 1“ narrows it down to the first year. Be consistent with the DoA.)