Deciding which RDF vocabulary terms to use when modeling data as Linked Open Data (LOD) is far from trivial. We propose "TermPicker" as a novel approach enabling vocabulary reuse by recommending vocabulary terms based on various features of a term. These features include the term’s popularity, whether it is from an already used vocabulary, and the so-called schema-level pattern (SLP) feature that exploits which terms other data providers on the LOD cloud use to describe their data. We apply Learning To Rank to establish a ranking model for vocabulary terms based on the utilized features. The results show that using the SLP-feature improves the recommendation quality by 29% to 36% considering the Mean Average Precision and the Mean Reciprocal Rank at the first five positions compared to recommendations based on solely the term’s popularity and whether it is from an already used vocabulary.

1. TermPicker: Enabling the Reuse of
Vocabulary Terms by Exploi:ng Data
from the Linked Open Data Cloud
Johann Schaible, Thomas Go2ron, and Ansgar Scherp.
at ESWC 2016

2. Problem statement
2
§ When modeling LOD, it is accustomed to reuse
vocabulary terms (à classes and proper;es)
§ However, it is a challenging task
swrc:Publication
http://ex.com/001
http://ex.com/002
http://ex.com/003
http://ex.com/p/001
http://ex.com/p/002
http://ex.com/p/003
swrc:Person
?
8
>><
>>:
dc:creator
foaf:maker
dcterms:creator
swrc:author
Need for Vocabulary Term
Recommenda;ons

3. Term recommenda:ons
based on…?
3
§ rdfs:domain, rdfs:range, and other informa;on
encoded in vocabularies?
§ Popularity of a vocabulary term?
§ Classes and proper;es from domain specific
vocabularies?
§ etc.
Which vocabulary terms did other data providers
on the LOD cloud use in a similar scenario?

4. How to capture a Scenario?
4
§ A scenario is defined by vocabulary terms used
for a part of a model à paKerns on schema level
Example:
slp = ({swrc:Publication}, {dc:creator}, {foaf:Person})
Resources of type swrc:PublicaFon are connected to
resources of type foaf:Person via the property dc:creator
slp = (sts, ps, ots)
Schema-Level PaIerns (SLPs)
A tuple describing the connec;on between two
sets of classes via a set of proper;es
In General:

5. Vocabulary Term
Recommenda:ons Based on LOD
5
Recommender of vocabulary terms:{x1, ..., xn}
query input I
query-SLP: slpq = ({mo:SoloMusicArtist}, ?, ?)
Feature Computation
{F(slpq, x1), ..., F(slpq, xn)}{F(slpq, x1), ..., F(slpq, xn)}
II
Ranking Model
III
%({F(slpq, x1), ..., F(slpq, xn)})
query output IV
Classes for subject:
properties:
Clases for object:<..., mo:Record, mo:MusicGroup,...>
<..., mo:MusicArtist, foaf:Person,...>
<..., foaf:made,..., mo:member of,...>

6. Overview
6
Feature Computation
{F(slpq, x1), ..., F(slpq, xn)}{F(slpq, x1), ..., F(slpq, xn)}
Recommender of vocabulary terms:
Ranking Model
query input
query output IV
I
II
III
Classes for subject:
properties:
Clases for object:
query-SLP:
{x1, ..., xn}
%({F(slpq, x1), ..., F(slpq, xn)})
<..., mo:Record, mo:MusicGroup,...>
slpq = ({mo:SoloMusicArtist}, ?, ?)
<..., mo:MusicArtist, foaf:Person,...>
<..., foaf:made,..., mo:member of,...>

7. Feature Computa:on:
The SLP-Feature
7
slpq = ({mo:SoloMusicArtist}, {}, {})
If slpq ✓ slpi (slpi 2 SLPLOD)
Then Sets of recommendations: slpi slpq
Collabora;ve
Filtering
Classes for subject: < mo:MusicArtist, dbo:Actor >
Properties: < mo:member of, foaf:made, mo:recorded >
Classes for object: < mo:MusicBand, mo:Record >
SLPLOD = {({mo:SoloMusicArtist, mo:MusicArtist}, {mo:member of}, {mo:MusicBand})
({mo:SoloMusicArtist, dbo:Actor}, {foaf:made, mo:recorded}, {mo:Record})
({foaf:Person}, {foaf:knows}, {foaf:Person})
}
SLPLOD = SPLs computed from datasets on the LOD cloud

8. Feature Computa:on:
State of the Art Features1
8
Feature Definition of the Feature
f1 Number of datasets on the LOD cloud using the recommendation
candidate x
f2 Number of datasets on the LOD cloud using the vocabulary Vx of
recommendation candidate x
f3 Total number of occurrences of recommendation candidate x on the
LOD cloud
f4 Whether the recommendation candidate x is from a vocabulary that
is already used in query-SLP slpq
f1 f3: Reusing popular vocabularies/vocabulary terms
f4: Reusing vocabulary terms from the same vocabulary
1) Schaible, GoKron, and Scherp: Survey on Common Strategies of Vocabulary Reuse in Linked Open Data Modeling
(ESWC 2104)

9. Overview
9
Feature Computation
{F(slpq, x1), ..., F(slpq, xn)}{F(slpq, x1), ..., F(slpq, xn)}
Recommender of vocabulary terms:
Ranking Model
query input
query output IV
I
II
III
Classes for subject:
properties:
Clases for object:
query-SLP:
{x1, ..., xn}
%({F(slpq, x1), ..., F(slpq, xn)})
<..., mo:Record, mo:MusicGroup,...>
slpq = ({mo:SoloMusicArtist}, ?, ?)
<..., mo:MusicArtist, foaf:Person,...>
<..., foaf:made,..., mo:member of,...>

10. Calcula:ng a Ranking Model
10
How to weight
the feature
values?
§ Learning to Rank (L2R):
} Family of supervised machine learning algorithms
based on data with relevance annota;ons
} state of the art in IR to compute a generalized ranking
model over a given set of features
} Ranking model is derived by observing correla;ons
between feature values and candidate relevance
F f1 f2 f3 f4 SLP-feature
(slpq, x1) 7 9 20 1 4
(slpq, x2) 3 3 5 0 6
(slpq, x3) 10 20 80 0 2
(slpq, x4) 4 20 29 1 4

11. Overview
11
Feature Computation
{F(slpq, x1), ..., F(slpq, xn)}{F(slpq, x1), ..., F(slpq, xn)}
Recommender of vocabulary terms:
Ranking Model
query input
query output IV
I
II
III
Classes for subject:
properties:
Clases for object:
query-SLP:
{x1, ..., xn}
%({F(slpq, x1), ..., F(slpq, xn)})
<..., mo:Record, mo:MusicGroup,...>
slpq = ({mo:SoloMusicArtist}, ?, ?)
<..., mo:MusicArtist, foaf:Person,...>
<..., foaf:made,..., mo:member of,...>

12. 12
Evalua:on
§ Baseline POP: Reuse popular vocabulary terms
} Based on features:
§ Baseline SAME: Reuse terms from same vocabulary
} Based on features:
§ SLP-feature-based: U;lizing the SLP-feature
} Based on features:
What is the benefit of the SLP-feature?
f1 f3
f1 f4
f1 f4 (+ SLP-feature )

13. 13
Evalua:on Procedure
§ Offline evalua;on with hidden informa;on
§ Measuring quality of recommenda;ons
} Mean Average Precision (MAP )
} Mean Reciprocal Rank at the first 5 posi;on (MRR@5)
§ Use of the RankLib2 library
slpq = ({mo:SoloMusicArtist}, {foaf:made}, {mo:Record})
Example:
Randomly hidden term: “foaf:made”
< foaf:name, mo:remixed, foaf:made, ... >Result list:
2) hKps://sourceforge.net/p/lemur/wiki/RankLib/

14. 14
Evalua:on Data for
Recommenda:ons
§ Two evalua;ons based on BTC 20143 and DyLDO4
BTC 2014 DyLDO
# of triples first 34 mio. (reduce overhead) 10.8 mio
# of PLDs 3, 500 382
# of distinct terms 5.5 mio. 2.3 mio.
# of vocabularies 1, 500 600
# of computed SLPs 227, 000 118, 000
§ 10-fold leave-one-out valida;on based on PLDs
3) hKp://km.aid.kit.edu/projects/btc-2014/
4) hKp://swse.deri.org/dyldo/

15. 15
Results – Box Plots MAP
L2R Algorithm
Features used

16. 16
Discussion
§ Using “from same vocabulary”-feature not
significant
} à only in few cases terms from same vocabulary are
used
§ Using SLPs significant improvement (ca. 35% in
MAP)
} à already now: looking at how others model their data
§ BeKer performance on BTC 2014
} More data in BTC 2014 to train the ranking model
} 37% more relevant candidates

17. 17
Conclusion
§ Using SLPs, relevant recommenda;ons are ranked
significantly higher in the result list
} Can aid the engineer even more in modeling data in a
way how other data providers do
§ Using L2R, the more relevant candidates correlate
with a feature, the beKer the results
However,
Offline evalua;ons do not observe actual
user behavior à online evalua;on needed5
5) Performed in Schaible, Szekely, Scherp: “Comparing Vocabulary Term Recommenda;ons using Associa;on Rules
and Learning To Rank: A User Study” at ESWC 2016

18. Thank You!
18
Tool URL: hKp://termpicker.lodrec.org
Evalua;on data and raw results: hKps://github.com/WanjaSchaible/l2r_eval_material