Slides of COLD 2013 Paper "On-the-fly Integration of Static and Dynamic Linked Data", Andreas Harth (KIT), Craig Knoblock (USC), Steffen Stadtmüller (KIT), Rudi Studer (KIT), Pedro Szekely (USC)

1. On-the-fly Integration of Static and Dynamic Linked
Data
Andreas Harth (KIT), Craig Knoblock (USC), Steffen Stadtmüller (KIT), Rudi Studer
(KIT), Pedro Szekely (USC)
INSTITUTE OF APPLIED INFORMATICS AND FORMAL DESCRIPTION METHODS (AIFB)
KIT – University of the State of Baden-Wuerttemberg and
National Research Center of the Helmholtz Association
www.kit.edu

2. Outline
Motivation
Scenario and Overview
Modelling Sources: Karma
Accessing and Integrating Sources: Data-Fu
Demo
Conclusion
2
On-the-fly Integration of Static and Dynamic Linked Data

3. Motivation
The relevance of many types of data perishes or degrades over time
(e.g., weather information, moving objects)
Timely decision making requires
access to live data and
inclusion of new sources in a flexible manner.
Our goals
(Near) real-time access to a variety of data sources in a range of data
formats and access modalities
Rapidly integrate sources via modeling and to generate a Linked Data
interface to live sources
3
On-the-fly Integration of Static and Dynamic Linked Data

4. Static vs. Dynamic Sources
Various sources have different update intervals (from minutes to
weeks)
We treat the access to all sources in the same way via polling (HTTP
GETs)
Thus, the only distinction between „static“ and „dynamic“ sources is
how fast we refresh the query results for each source
4
On-the-fly Integration of Static and Dynamic Linked Data

5. Scenario
POIs
(Crunchbase, OS
M, Wikimapia)
5
Venues/Events
Buses/Stops
(Eventful, LastFM)
(LA Metro)
On-the-fly Integration of Static and Dynamic Linked Data
Vehicles
(Campus
Cruisers)
Marine Vessels
(AIS)

6. Architecture
6
On-the-fly Integration of Static and Dynamic Linked Data

7. Karma
Interactive tool for rapidly
extracting, cleaning, transforming, integrating, and publishing data
Tabular
Sources
Karma
Hierarchical
Sources
Database
Services
Model
…
See http://isi.edu/integration/karma/ for more info and download
7
On-the-fly Integration of Static and Dynamic Linked Data

8. Modelling Sources with Karma
Karma is a data integration tool
Linked API
Map data onto an ontology to generate Linked Data
Karma extension to enable the on-the-fly lifting of API I/O
data according to a pre-defined mapping model
Web
API
Vehicles
(Campus
Cruisers)
8
On-the-fly Integration of Static and Dynamic Linked Data

9. Linked Data Access to Event APIs
Venues/Events
LastFM API
(Eventful, LastFM)
Given a lat/lon of a location, return a list of event identifiers
http://km.aifb.kit.edu/services/lastfmwrap/geo.getevents?
lat={?lat}&long={?lon}
Given an event identifier, return details about the event
http://lastfm.rdfize.com/events/{event-id}
Eventful API
List events given a keyword search term and a date range
http://km.aifb.kit.edu/services/eventfulwrap/search?locat
ion={?loc}&date={?date}
9
On-the-fly Integration of Static and Dynamic Linked Data

10. LastFM Data-Fu Program (I)
Program at http://km.aifb.kit.edu/services/data-fu/lastfm
with input lat/lon (in RDF via HTTP POST)
Rule to search for events at given location:
{ ?p geo:long ?lon .
?p geo:lat ?lat . }
=>
{ [] http:mthd http:GET ;
http:requestURI
<http://km.aifb.kit.edu/services/lastfm
wrap/geo.getevents?lat={?lat}&long={?lo
n}> . } .
10
On-the-fly Integration of Static and Dynamic Linked Data
“For the input point with lat/long
perform an HTTP GET
at the KIT LastFM Wrapper URI
constructed with the lat/long”

11. LastFM Data-Fu Program (II)
Rule for retrieving information about the found events, including
geolocation of event:
{ ?e rdf:type lode:Event. }
=>
{ [] http:mthd http:GET ;
http:requestURI ?e . } .
“For every resource of type event
perform an HTTP GET
at the resource URI”
Query to return a table with lat/lon and label to transform to
KML/Google Earth:
:q1 qrl:select ( ?event ?place ?label ?lat ?lon ) ;
qrl:where {
?event <http://purl.org/NET/c4dm/event.owl#place> ?place .
?event rdfs:label ?label .
“Output is every entity with
?place geo:lat ?lat .
latitude, longitude and associated
?place geo:long ?lon .
label”
} .
11
On-the-fly Integration of Static and Dynamic Linked Data

12. Data Source Characteristics
12
On-the-fly Integration of Static and Dynamic Linked Data

13. Demo
Load http://people.aifb.kit.edu/aha/2013/d3/index.kml into Google Earth
Location of buses and ships are updated
13
On-the-fly Integration of Static and Dynamic Linked Data

14. Conclusion
System interoperation in distributed environments with Linked Data as
interface
Rapid integration of new sources (via Karma models and Data-Fu
scripts)
Realtime access to networked data via Data-Fu scripts/programs
http://code.google.com/p/data-fu/
Ability to rapidy integrate new sources via Karma models
http://www.isi.edu/integration/karma/
Future work
Modular organisation of programs
Manipulating resource state (Read-Write Linked Data)
Optimisations for limited bandwidth environments
14
On-the-fly Integration of Static and Dynamic Linked Data

15. Challenges
Data is provided at different places, by different owners, often over the
web (decentralised data publishing)
Data and links are provided in a many different formats/protocols
Developers have to gain a deep understanding of every API by reading
textual descriptions
Applications (user agents) are supposed to follow links as found during
runtime of the application
Developers have to define their desired interaction at design time
Developers have to write individually tailored code to consume services in
applications
15
On-the-fly Integration of Static and Dynamic Linked Data