In other words, Managing Completeness over Web Data.

1. Enabling Fine-grained
RDF Data Completeness Assessment
Fariz Darari, Simon Razniewski, Radityo E. Prasojo, Werner Nutt
KRDB, Free University of Bozen-Bolzano, Italy
ICWE 2016
Lugano, Switzerland
June 8, 2016
Supported by the project MAGIC, funded by the province of Bolzano
Managing Completeness over Web Data June 8, 2016 1 / 31

2. Quality of Web Data: Completeness
How complete are Web data sources?
Managing Completeness over Web Data June 8, 2016 2 / 31

3. How complete is Wikidata for Apollo 11’s crew?
Managing Completeness over Web Data June 8, 2016 3 / 31

4. NASA says . . .
Managing Completeness over Web Data June 8, 2016 4 / 31

5. Wikidata is complete for Apollo 11’s crew!
Managing Completeness over Web Data June 8, 2016 5 / 31

6. Wikidata supports a special form of
completeness statement
Managing Completeness over Web Data June 8, 2016 6 / 31

7. Completeness Statements
Syntax:
Compl(s, p, ?o)
Managing Completeness over Web Data June 8, 2016 7 / 31

8. Completeness Statements
Syntax:
Compl(s, p, ?o)
Semantics:
Graph G has Compl(s, p, ?o)
Managing Completeness over Web Data June 8, 2016 7 / 31

9. Completeness Statements
Syntax:
Compl(s, p, ?o)
Semantics:
Graph G has Compl(s, p, ?o)
↓
G is complete for all p-values of s that exist in reality
Managing Completeness over Web Data June 8, 2016 7 / 31

10. Usages of Completeness Statements
Tracking data completion progress of KB contributors
Managing Completeness over Web Data June 8, 2016 8 / 31

11. Usages of Completeness Statements
Tracking data completion progress of KB contributors
Providing statistics about completeness of KBs
Example: For 25% of Swiss cantons, Wikidata is complete
for their official languages.
Managing Completeness over Web Data June 8, 2016 8 / 31

12. Usages of Completeness Statements
Tracking data completion progress of KB contributors
Providing statistics about completeness of KBs
Example: For 25% of Swiss cantons, Wikidata is complete
for their official languages.
Checking query completeness
Managing Completeness over Web Data June 8, 2016 8 / 31

13. Checking Query Completeness
GA99: graph about the space mission A99
Managing Completeness over Web Data June 8, 2016 9 / 31

14. Checking Query Completeness
GA99: graph about the space mission A99
P1: query for schools of the children of A99’s crew
{ (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
Managing Completeness over Web Data June 8, 2016 9 / 31

15. Checking Query Completeness
GA99: graph about the space mission A99
P1: query for schools of the children of A99’s crew
{ (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
Evaluating P1 over GA99 gives one answer mapping:
{?cr → Chan, ?ch → Dani, ?sc → USI}
Managing Completeness over Web Data June 8, 2016 9 / 31

16. Checking Query Completeness
GA99: graph about the space mission A99
P1: query for schools of the children of A99’s crew
{ (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
Evaluating P1 over GA99 gives one answer mapping:
{?cr → Chan, ?ch → Dani, ?sc → USI}
Is P1 complete over GA99?
Managing Completeness over Web Data June 8, 2016 9 / 31

17. Checking Query Completeness
GA99: graph about the space mission A99
P1: query for schools of the children of A99’s crew
{ (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
Evaluating P1 over GA99 gives one answer mapping:
{?cr → Chan, ?ch → Dani, ?sc → USI}
Is P1 complete over GA99? We don’t know!
Managing Completeness over Web Data June 8, 2016 9 / 31

18. Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness statements consisting of
C1 = Compl(A99, crew, ?o)
Managing Completeness over Web Data June 8, 2016 10 / 31

19. Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness statements consisting of
C1 = Compl(A99, crew, ?o)
C2 = Compl(Bob, child, ?o)
Managing Completeness over Web Data June 8, 2016 11 / 31

20. Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness statements consisting of
C1 = Compl(A99, crew, ?o)
C2 = Compl(Bob, child, ?o)
C3 = Compl(Chan, child, ?o)
Managing Completeness over Web Data June 8, 2016 12 / 31

21. Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness statements consisting of
C1 = Compl(A99, crew, ?o)
C2 = Compl(Bob, child, ?o)
C3 = Compl(Chan, child, ?o)
C4 = Compl(Dani, school, ?o)
Managing Completeness over Web Data June 8, 2016 13 / 31

22. Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness statements consisting of
C1 = Compl(A99, crew, ?o)
C2 = Compl(Bob, child, ?o)
C3 = Compl(Chan, child, ?o)
C4 = Compl(Dani, school, ?o)
Is P1 complete over GA99 wrt. CA99?
Managing Completeness over Web Data June 8, 2016 14 / 31

23. Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
C1 matches the first triple of P1

24. Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
C1 matches the first triple of P1 → Complete for Pc
1 = (A99, crew, ?cr)

25. Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
C1 matches the first triple of P1 → Complete for Pc
1 = (A99, crew, ?cr)
Instantiating the rest of P1 with the answers of Pc
1 gives:
P2 = { (Bob, child, ?ch), (?ch, school, ?sc) }
P3 = { (Chan, child, ?ch), (?ch, school, ?sc) }
Managing Completeness over Web Data June 8, 2016 15 / 31

26. Checking Query Completeness
P2 = { (Bob, child, ?ch), (?ch, school, ?sc) }
C2 matches the first triple of P2

27. Checking Query Completeness
P2 = { (Bob, child, ?ch), (?ch, school, ?sc) }
C2 matches the first triple of P2 → Complete for Pc
2 = (Bob, child, ?ch)

28. Checking Query Completeness
P2 = { (Bob, child, ?ch), (?ch, school, ?sc) }
C2 matches the first triple of P2 → Complete for Pc
2 = (Bob, child, ?ch)
Instantiating the rest of P2 with the answers of Pc
2 gives: nothing
Complete for P2
Managing Completeness over Web Data June 8, 2016 16 / 31

29. Checking Query Completeness
P3 = { (Chan, child, ?ch), (?ch, school, ?sc) }
C3 matches the first triple of P3

30. Checking Query Completeness
P3 = { (Chan, child, ?ch), (?ch, school, ?sc) }
C3 matches the first triple of P3 → Complete forPc
3 = (Chan, child, ?ch)

31. Checking Query Completeness
P3 = { (Chan, child, ?ch), (?ch, school, ?sc) }
C3 matches the first triple of P3 → Complete forPc
3 = (Chan, child, ?ch)
Instantiating the rest of P3 with the answers of Pc
3 gives:
P4 = { (Dani, school, ?sc) }
Managing Completeness over Web Data June 8, 2016 17 / 31

32. Checking Query Completeness
P4 = { (Dani, school, ?sc) }
C4 matches the only triple of P4

33. Checking Query Completeness
P4 = { (Dani, school, ?sc) }
C4 matches the only triple of P4 → Complete for the whole P4
Managing Completeness over Web Data June 8, 2016 18 / 31

34. Checking Query Completeness
P4 = { (Dani, school, ?sc) }
C4 matches the only triple of P4 → Complete for the whole P4
Conclusion: We found complete matches
for all query instantiations from P1
Managing Completeness over Web Data June 8, 2016 18 / 31

35. Checking Query Completeness
P4 = { (Dani, school, ?sc) }
C4 matches the only triple of P4 → Complete for the whole P4
Conclusion: We found complete matches
for all query instantiations from P1
→ P1 is complete over GA99 wrt. CA99
Managing Completeness over Web Data June 8, 2016 18 / 31

36. Algorithm for Checking Query Completeness
Input: P query, G graph, C set of completeness statements
Output: true iff P is complete wrt. G and C
P ← {P}
while P = ∅ do
choose and remove P0 ∈ P
Pc
0 ← FindMatch(P0, C)
if Pc
0 = ∅
return false
else
Prest
0 ← P0 Pc
0
P ← P ∪ {µPrest
0 | µ ∈ Pc
0 G}
return true
Managing Completeness over Web Data June 8, 2016 19 / 31

37. Experimental Questions
What is the relationship between the number of query answers
and completeness checking time?
How do query evaluation time and completeness checking
time compare?
Is there a difference between completeness checking time
for complete and incomplete cases?
Managing Completeness over Web Data June 8, 2016 20 / 31

38. Experimental Setup
Graph: Wikidata
Managing Completeness over Web Data June 8, 2016 21 / 31

39. Experimental Setup
Graph: Wikidata
Queries: Three sets of path queries with an increasing number of
query results (3 sets x 40 queries)
Pmot = { ($c$, mother, ?w), (?w, mother, ?x), (?x, mother, ?y) }
Pcre = { ($c$, crew, ?w), (?w, mission, ?x), (?x, operator, ?y) }
Pdiv = { ($c$, division, ?w), (?w, division, ?x), (?x, area, ?y) }
Managing Completeness over Web Data June 8, 2016 21 / 31

40. Experimental Setup
Graph: Wikidata
Queries: Three sets of path queries with an increasing number of
query results (3 sets x 40 queries)
Pmot = { ($c$, mother, ?w), (?w, mother, ?x), (?x, mother, ?y) }
Pcre = { ($c$, crew, ?w), (?w, mission, ?x), (?x, operator, ?y) }
Pdiv = { ($c$, division, ?w), (?w, division, ?x), (?x, area, ?y) }
Completeness statements:
Complete case: generated by traversing the query structure
(1.7 mio statements)
Incomplete case: drop randomly 20% of the statements
in the complete case
Managing Completeness over Web Data June 8, 2016 21 / 31

41. Experimental Setup
Implementation: Java with the Apache Jena library
Completeness statement matching = standard Java HashMap
Triple store = Jena-TDB
Machine: 2.4 GHz laptop with 8 GB memory
Managing Completeness over Web Data June 8, 2016 22 / 31

42. Experimental Results
The more the query results, the longer the completeness checks
Managing Completeness over Web Data June 8, 2016 23 / 31

43. Experimental Results
The more the query results, the longer the completeness checks
Though slower than query evaluation, in an absolute scale
completeness checking performs reasonably well (at most 35 ms)
Managing Completeness over Web Data June 8, 2016 23 / 31

44. Experimental Results
The more the query results, the longer the completeness checks
Though slower than query evaluation, in an absolute scale
completeness checking performs reasonably well (at most 35 ms)
Complete cases are slower than incomplete cases
Managing Completeness over Web Data June 8, 2016 23 / 31

45. Practical Applications of Completeness Statements
How complete are Web data sources?
To answer the question, we need to provide
A way to annotate complete parts of a data source using
completeness statements
Ways to utilize the completeness statements to give insights
on how complete the data source is
Managing Completeness over Web Data June 8, 2016 24 / 31

46. COOL-WD: COmpleteness toOL for WikiData
We have developed
a demo of completeness management tool for Wikidata
COOL-WD provides ways to
annotate complete parts of Wikidata
utilize completeness statements to do completeness
aggregation and query completeness assessment
Managing Completeness over Web Data June 8, 2016 25 / 31

47. COOL-WD: Detailed Features
Management of completeness statements
Adding or removing completeness statements of any property of a
Wikidata entity
Viewing an entity page with its completeness annotations
Aggregation of completeness statements
Assessment of query completeness
Managing Completeness over Web Data June 8, 2016 26 / 31

48. COOL-WD: Architecture
SPARQL Endpoint MediaWiki API
COOL-WD
Engine
COOL-WD
User Interface
HTTP RequestsData Access Web Browsing
SPARQL Queries API Calls
Completeness DB
Managing Completeness over Web Data June 8, 2016 27 / 31

49. COOL-WD: Demo
http://cool-wd.inf.unibz.it/
Managing Completeness over Web Data June 8, 2016 28 / 31

50. Conclusions
We developed a sound and complete algorithm
for query completeness checking wrt. an RDF graph and
completeness statements
Managing Completeness over Web Data June 8, 2016 29 / 31

51. Conclusions
We developed a sound and complete algorithm
for query completeness checking wrt. an RDF graph and
completeness statements
The algorithm can be generalized to consider a more general form
of completeness statements: Compl(P) where P is a basic graph
pattern (BGP)
Managing Completeness over Web Data June 8, 2016 29 / 31

52. Conclusions
We developed a sound and complete algorithm
for query completeness checking wrt. an RDF graph and
completeness statements
The algorithm can be generalized to consider a more general form
of completeness statements: Compl(P) where P is a basic graph
pattern (BGP)
We evaluated completeness checking performance
Managing Completeness over Web Data June 8, 2016 29 / 31

53. Conclusions
We developed a sound and complete algorithm
for query completeness checking wrt. an RDF graph and
completeness statements
The algorithm can be generalized to consider a more general form
of completeness statements: Compl(P) where P is a basic graph
pattern (BGP)
We evaluated completeness checking performance
We developed COOL-WD, a completeness tool for Wikidata
Managing Completeness over Web Data June 8, 2016 29 / 31

54. Ongoing Work
We plan to leverage completeness statements for checking
the soundness of queries with negation1
We plan to develop fast completeness checks for arbitrary
completeness statements1
1
Darari et al. Ensuring Soundness for SPARQL with Negation Using
Completeness Statements. Submitted to a conference.
Managing Completeness over Web Data June 8, 2016 30 / 31

55. Ongoing Work
We plan to leverage completeness statements for checking
the soundness of queries with negation1
We plan to develop fast completeness checks for arbitrary
completeness statements1
We plan to exploit the potential of natural language completeness
statements already available on the Web: 14K in Wikipedia,
24K in IMDb, 2200 in OpenStreetMap
1
Darari et al. Ensuring Soundness for SPARQL with Negation Using
Completeness Statements. Submitted to a conference.
Managing Completeness over Web Data June 8, 2016 30 / 31

56. Ongoing Work
We plan to leverage completeness statements for checking
the soundness of queries with negation1
We plan to develop fast completeness checks for arbitrary
completeness statements1
We plan to exploit the potential of natural language completeness
statements already available on the Web: 14K in Wikipedia,
24K in IMDb, 2200 in OpenStreetMap
We plan to extend COOL-WD with new cool features
Completeness analytics
Query completeness diagnostics
Linked data publication of completeness statements
Completeness gadget for tighter integration with Wikidata
1
Darari et al. Ensuring Soundness for SPARQL with Negation Using
Completeness Statements. Submitted to a conference.
Managing Completeness over Web Data June 8, 2016 30 / 31

57. Thank you!
Questions? Just drop Fariz an email: fadirra@gmail.com
Big thanks to Springer for the travel grant!
Have a look at the paper:
http://dx.doi.org/10.1007/978-3-319-38791-8_10
And finally, a completeness statement for all the slides :-)
Compl(thisSlideset, hasSlide, ?o)
Managing Completeness over Web Data June 8, 2016 31 / 31