Presentation for the final round of the Semantic Web Challenge 2012 (at ISWC2012) about Urbanopoly

1. Urbanopoly: Collection and Quality
Assessment of Geo-spatial Linked Data
via a Human Computation Game
Irene Celino, Dario Cerizza, Simone Contessa,
Marta Corubolo, Daniele Dell’Aglio, Emanuele Della Valle,
Stefano Fumeo and Federico Piccinini
Semantic Web Challenge @ ISWC 2012 - 2012/11/14

2. Citizen Science?
http://www.flickr.com/photos/mountrainiernps/6997851139/
Mount Rainier NPS
Glacier NPS http://www.flickr.com/photos/glaciernps/4427416227/
Glacier NPS http://www.flickr.com/photos/glaciernps/4427412443/
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 2 Urbanopoly

3. What about those citizens?
Gareth 1953 http://www.flickr.com/photos/gareth1953/6786545520/ paul_houle http://www.flickr.com/photos/paul_houle/3301438074/
Boris van Hoytema http://www.flickr.com/photos/borisvanhoytema/685879933/ graziano88 http://www.flickr.com/photos/8482460@N06/6884509346/
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 3 Urbanopoly

4. Citizen Computation
Human Computation Citizen Science
exploiting human capabilities exploiting volunteers
to solve computational tasks to collect scientific data or
difficult for machines to conduct experiments
"in the world"
Citizen Computation
exploiting human capabilities
to contribute to a mixed
computational system
by living "in the world"
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 4 Urbanopoly

5. Human Computation Games with a Purpose
Purpose
within
the game:
Purpose outside the game:
You help image search engines by manually tagging images
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 5 Urbanopoly

6. Citizen Computation Game with a Purpose
Purpose within Create your venues' portfolio and become
the game: the greatest landlord ever!
Purpose outside Collect and verify information about your city
the game: by playing with the neighborhood around you
http://bit.ly/urbanopoly
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 6 Urbanopoly

7. Urbanopoly – high-level view
Game purpose: check and correct geo-spatial data
from pre-existing sources + collect missing data
game to buy / sell 2
LinkedGeoData + venues with missions
players
Lombardia Open Data
data about
1 venues as
missions
bootstrap of
"venues" data
GWAP approach to
consolidate data 3
verified / improved data
+ new data
4
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 7 Urbanopoly

8. Urbanopoly Input Data
 OpenStreetMap (OSM)
http://www.openstreetmap.org/
via LinkedGeoData (LGD)
http://linkedgeodata.org/
data as linked data, described by an ontology
 Lombardia Open Data
https://dati.lombardia.it
data about "agriturismo" places as CSV converted to RDF
 Urbanopoly data bootstrap: venues are "instances" of
selected LGD "classes" with their OSM tags as features,
thus Urbanopoly data are RDF statements of the form:
<venue> <feature> <value>
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 8 Urbanopoly

9. Urbanopoly gameplay
http://bit.ly/u-video
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 9 Urbanopoly

10. Urbanopoly mini-games for Data Collection
data acquisition challenges as data validation challenges to check
contributions to an advertising campaign pre-existing data or other players’
– left: inserting a value, contribution – left: answering a quiz,
right: taking a picture right: rating a poster
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 10 Urbanopoly

11. Urbanopoly Data Consolidation
 Each statement has a confidence score:
{ <venue> <feature> <value> . } <confidence>
which indicates the probability of the statement to be true
 Each player action is taken as an evidence of the associated
knowledge and alters the confidence score
 A weighted majority voting algorithm aggregates the evidences:
 Difficulty to acquire the contribution (e.g., typing vs. check box)
 Player’s reputation (e.g., number of errors)
 Player’s distance to the venue at contribution time (as sensed by the device)
 When the confidence score overcomes a threshold, the triple
<venue> <feature> <value> gets consolidated
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 11 Urbanopoly

12. Urbanopoly Data Publication
 True statements published as linked open data
 If a statement's confidence overcomes the threshold, the statement
is asserted: <venue> <feature> <value> (as in LGD/OSM)
 But there's more interesting information to publish!
 False statements, statements' confidence, provenance info, etc.
provo:Entity
 We created a Human Computation contributionFrom aggregatedFrom Consolidated
ontology (http://swa.cefriel.it/ontologies/hc) Contribution
Information
extending the W3C PROV-O ontology
solutionTo
aggregatedBy
provo:Agent Contributor
 We published this further
knowledge as annotations to the reified Human
Computation
enabledBy Human
Computation
<venue> <feature> <value> statements solvedBy Task Algorithm
provo:Activity
 Cf. http://swa.cefriel.it/linkeddata/
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 12 Urbanopoly

13. Urbanopoly Evaluation (1/2)
 "Enjoyability" of the game (engagement potential):
 Average life play: ALP = Played Time / Active Players
 ~ 100 minutes  very good result 
 "Effectiveness" of the GWAP mechanism:
 Throughput = Solved Problems / Played Time
 ~ 287 collected evidences / hour  very good 
 ~ 5 consolidated statements / hour  can be improved 
 "Precision" of the results (measured on results' subset)
 Accuracy = ( (P – FP) + (N – FN) ) / (P + N)
 ~ 92 %  very good result 
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 13 Urbanopoly

14. Urbanopoly Evaluation (2/2)
 "Playability" of the game
 Evaluation survey at http://bit.ly/u-survey, with questions about
usability, social aspects, physical presence, motivation, etc.
 Feedbacks very encouraging 
 "Sociability" through Facebook channel
 With Facebook Insights (http://www.facebook.com/insights/),
tracking of installs, demographics,
log-ins, content sharing, etc.
 Example of published "story" on
Facebook Timeline:
 Statistics about "stories" and
"impressions":
 Interesting results, but channel to be further exploited 
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 14 Urbanopoly

15. Conclusions
 Urbanopoly is an end-user mobile application with
a multi-language attractive user interface
 Urbanopoly manages urban data at a real scale
(ca. 50,000 venues) from heterogeneous sources
 The meaning of data is core to the application and
consolidated data are published as linked open data
 Urbanopoly is aimed at geo-spatial data collection and
quality assurance, especially for dynamic data
 Our rigorous evaluation shows the high accuracy of results
and feasibility of the approach
 Urbanopoly shows a clear commercial potential: further
data collection or validation needs can be added as further
mini-games or challenges within the game
Semantic Web Challenge @ ISWC 2012 - 2012/11/14 15 Urbanopoly

16. Thanks for your attention!
Questions?
Keep on playing
Urbanopoly!
Irene Celino – CEFRIEL, ICT Institute Politecnico di Milano
email: Irene.Celino@cefriel.it – web: http://swa.cefriel.it
slides at: http://www.slideshare.net/iricelino
Semantic Web Challenge @ ISWC 2012 - 2012/11/14