Keynote speech at the ACM MEDES 2016 conference - 02/11/2016, Hendaye, France

1. Shared data infrastructures:
From Smart Cities to
Education
Mathieu d’Aquin (@mdaquin)
Data Science Group (@DataScienceGr)
KMi, The Open University
datahub.technology

2. A city?

3. A city?
A place

4. A city?
A place
A bunch of
people (and
organisations)

5. A city?
A place
A bunch of
people (and
organisations)
live /
reside
there

6. A city?
A place
Power
infrastructure
A bunch of
people (and
organisations)

7. A city?
A place
Power
infrastructure
Water
infrastructure
A bunch of
people (and
organisations)

8. A city?
A place
Power
infrastructure
Water
infrastructure
Transport
infrastructure
A bunch of
people (and
organisations)

9. A city?
A place
Power
infrastructure
Water
infrastructure
Transport
infrastructure
A bunch of
people (and
organisations)
use

10. A city?
A place
Power
infrastructure
Water
infrastructure
Transport
infrastructure
A bunch of
people (and
organisations)
use
under
stand
???

11. A city?
A place
Power
infrastructure
Water
infrastructure
Transport
infrastructure
A bunch of
people (and
organisations)
use
under
stand
???
effect
ive
with?
??

12. A smart
city?
A place
Power
infrastructure
Water
infrastructure
Transport
infrastructure
Data
infrastructure
A bunch of
people (and
organisations)

13. mksmart.org

14. Making Milton Keynes a Smart City

15. Making Milton Keynes a Smart City

17. Challenges
Transport
Water
Energy
By 2026, transport demand in MK is estimated to grow by
60%, with engineering solutions only likely to meet half of
this
MK is in a water stressed area and it is projected that
climate change may reduce regional water availability by
29 million litres water per day by 2025.
MK needs to transition to being a low energy city to support
sustainable economic growth. The 2013 core strategy of MK
Council aims to achieve a 22% reduction in CO2 emissions
per capita from a 2005 base by 2020.

20. data
apps

21. apps

22. apps

27. A data infrastructure
Because addressing vertically every route from data to
application in isolation is crazy!
Challenges:
- Data heterogeneity: The content of the data is not the
same
- Data diversity: The context and conditions under which
the data is available are not the same

28. Data heterogeneity
3 types of data in a city:
- Highly temporal data
- Highly spatial data
- Others...

29. Answer the question: “what do we know about x?”
where x is a place, organisation, bus route, roundabout, etc.
Data
source 1
Data
source 2
Data
source 3
Data
source 4
Data
source 5
Data
source 6
Big City Warehouse
access
ETL
process
ETL
process
ETL
process
ETL
process
ETL
process
ETL
process
Typical integration
approach

30. Answer the question: “what do we know about x?”
where x is a place, organisation, bus route, roundabout, etc.
Data
source 1
Data
source 2
Data
source 3
Data
source 4
Data
source 5
Data
source 6
Big City Warehouse
access
ETL
process
ETL
process
ETL
process
ETL
process
ETL
process
ETL
process
Typical integration
approach
Hard to maintain and
keep running at scale (i.e.
as number of datasets
grow)

31. Taking a Linked Data approach

32. Taking a Linked Data approach

33. Answer the question: “what do we know about x?”
where x is a place, organisation, bus route, roundabout, etc.
Data
source 1
Data
source 2
Data
source 3
Data
source 4
Data
source 5
Data
source 6
Query template Query template Query template Query template Query template Query template
access access access

34. Answer the question: “what do we know about x?”
where x is a place, organisation, bus route, roundabout, etc.
Data
source 1
Data
source 2
Data
source 3
Data
source 4
Data
source 5
Data
source 6
Query template Query template Query template Query template Query template Query template
access access accessCan be added and
maintained in isolation
from each other

35. Answer the question: “what do we know about x?”
where x is a place, organisation, bus route, roundabout, etc.
Data
source 1
Data
source 2
Data
source 3
Data
source 4
Data
source 5
Data
source 6
Query template Query template Query template Query template Query template Query template
access access accessCan be added and
maintained in isolation
from each other
Virtual (i.e. never fully
materialised) - no need
for maintenance

36. Result
An “Entity-API” for things in the
city, integrating hundreds of
datasets and providing
thousands of data endpoints,
each providing integrated
information about a bus stop,
an area, a restaurant, a school,
a roundabout, etc.

37. Result - Top MK - card playing with city data

38. Result - MK Insight - City data portal

39. Data Diversity
The eskimo
language has 255
different words for
“visiting linguist”
What’s the point of
integrating data if we
have to go through
every bit of it to check
if it can be used?

40. Example
Smart meter
data
Anonymisation analysisAnon data Model
prediction/
recommendation
Results

41. Example
Smart meter
data
Anonymisation
Solar panel
monitoring
Anonymisation
analysisAnon data
Anon data
Model
prediction/
recommendation
Results

42. Example
Smart meter
data
Anonymisation
Solar panel
monitoring
Anonymisation
Weather data
Location
data
Electricity
tariff data
analysisAnon data
Anon data
Model
prediction/
recommendation
Results

43. Example
Smart meter
data
Anonymisation
Solar panel
monitoring
Anonymisation
Weather data
Location
data
Electricity
tariff data
analysisAnon data
Anon data
Model
prediction/
recommendation
Data
prot.
Corp
lic. 1
Results

44. Example
Smart meter
data
Anonymisation
Solar panel
monitoring
Anonymisation
Weather data
Location
data
Electricity
tariff data
analysisAnon data
Anon data
Model
prediction/
recommendation
Data
prot.
Corp
lic. 1
Corp
lic. 2
Data
prot.
Data
prot.
Results
User
T&C
OGL
Corp
lic. 3

45. Example
Smart meter
data
Anonymisation
Solar panel
monitoring
Anonymisation
Weather data
Location
data
Electricity
tariff data
analysisAnon data
Anon data
Model
prediction/
recommendation
Data
prot.
Corp
lic. 1
Corp
lic. 2
Data
prot.
Data
prot.
Results
User
T&C
OGL
Corp
lic. 3
?

46. Example
Smart meter
data
Anonymisation
Solar panel
monitoring
Anonymisation
Weather data
Location
data
Electricity
tariff data
analysisAnon data
Anon data
Model
prediction/
recommendation
Data
prot.
Corp
lic. 1
Corp
lic. 2
Data
prot.
Data
prot.
Results
User
T&C
OGL
Corp
lic. 3
?
?
?
?

47. Semantic approach
Explicit, machine readable
representation of data policies
and licences...
… as well as of the data flows through
which they are processed

48. Handled through a sophisticated data cataloguing process

49. Handled through a sophisticated data cataloguing process

50. Result
datahub.mksmart.org
~ 400 registered users
~ 700 datasets

51. Result - Applications

52. Result - Reusable
components
datahub.technology
Open source data cataloguing
components that integrate with CMS
system
“Entity API” framework for data
integration
High Speed processing components
Data portal framework

53. What about education?

54. Learner
Platform
Analytics
VLE | Website | Library
Assessment |
Enrollment
School/University
Prediction Drop out
BI
Planning
Recommendation
Sentiment
Analysis
Collective
Intelligence
Behaviour Analysis
Collaboration
Community Support
AFEL - Analytics for Everyday Learning (afel-project.eu)

55. Learner
Platform
Analytics
VLE | Website | Library
Assessment |
Enrollment
School/University
Prediction Drop out
BI
Planning
Recommendation
Sentiment
Analysis
Collective
Intelligence
Behaviour Analysis
Collaboration
Community Support
AFEL - Analytics for Everyday Learning (afel-project.eu)
Same challenges…
… same solutions

56. Pointers and next steps
Deploying to other cities in the UK and Europe, as
well as for research projects and other types of
organisations.
Dealing with data quality and accuracy - automatic
checking based on a cross comparison with other
datasets.
Reusable, parameterizable services for analytics -
building a catalogue of pipelines and models.

57. Thank you!
@mdaquin
@DataScienceGr
http://datahub.technology