OpenGovIntelligence Workshop at NTTS2017

NTTS2017, 16 March 2017, Brussels
Hands-on Workshop on Linked
Open Statistical Data

213-17 March 2017, BrusselsNTTS 2017
Time Duration Session
18:30 – 18:40 00:10 Welcome address
Christine Kormann (Eurostat)
18:40 - 18:50 00:10 The OpenGovIntelligence Project
Evangelos Kalampokis (University of Macedonia)
18:50 – 19:05 00:15 LOD in the ESS: initiatives, enablers and challenges - a PwC study
for Eurostat
Nikolaos Loutas (PwC)
19:05 – 19:20 00:15 The use of Linked Open Statistical Data in the Flemish Government
Paul Hermans (ProXML)
19:20 – 19:30 00:10 OpenGovIntelligence Tools
Bill Roberts (Swirrl)
19:30 – 20:30 00:60 Hands-on evaluation of the tools
Agenda Overview

for Eurostat
Agenda Overview

NTTS2017, 16 March 2017, Brussels
The OpenGovIntelligence
Project

OpenGovIntelligence
§ Consortium
§ 7 European countries
§ 6 R&D partners
§ 6 pilot partners
§ Duration
§ February 2016 – January 2019

http://OpenGovIntelligence.eu
@OpenGovInt

§ Public administration publishes Open Data in an ad-hoc manner based on
existing processes, according to their mandate, and often under unclear
licenses. They also design and deliver services in a top-down manner.
§ On the other hand, society has needs and data-driven public services, not
raw data, can address these needs.
§ As a result, society should be involved in service co-production to ensure
that public services address their needs.
Motivation – Open Data

§ At the same time, the major part of OGD is of statistical nature,
meaning that consist of numeric values that are highly structured.
§ This type of data is a huge but yet unexploited resource, which could
potentially play a significant role in the creation of value for the
society.
Motivation – Open Statistical Data

§ Open Statistical Data are
fragmented
§ Searching data.gov.uk for
“unemployment” datasets:
§ 122 results (links and files)
§ These results provide access to
56 files and 610 links
§ These links lead to 18 other
portals
§ Through them to more than
2000 other files
Motivation – Fragmentation
E. Kalampokis, E. Tambouris, A. Karamanou, K. Tarabanis (2016) Open Statistics: The Rise of a new Era for Open Data?, EGOV2016, LNCS 9820, pp.31-43, Springer.

§ All these web portals provide
complementary views of the
unemployment data.
§ For example, focusing on geo
dimension:
§ Data about unemployment in
different administrative levels in the
UK.
§ ONS, NOMIS, NeSS and Open Data
Communities provide data about the
whole country.
§ Local government portals provide
data for specific areas (e.g.
Warwickshire, Cambridgeshire)
Motivation – Complementarity
Level 0 UK ONS
Level 1 Countries ONS
Level 2 Regions ONS, NOMIS, NeSS,
Level 3 Counties NOMIS
Level 4 Districts/Boroughs/Divisions ODC
Level 5 Local Enterprise Parttnership ONS, NOMIS
Level 6 Local Authorities/Communities
First Areas
ONS, NOMIS, NeSS
Level 7 Parliamentary Constituencies ONS, NOMIS
Level 8 Wards Warkwickshire,
Cambridgeshire
Level 9 Market Towns Cambridgeshire
Level 10 Super Output Area Warkwickshire
Level 11 Super Output Area Middle Layer NeSS
Level 12 Super Output Area Lower Layer NeSS
Level 13 Output Area NeSS
Level 14 Parishes Cambridgeshire

§ Linked Open Statistical Data (aka linked data cubes) have the potential to
address data interoperability and facilitate data integration on the Web.
§ Performing analytics on top of multiple datasets (previously isolated) in
order to create added value services.
Linked Open Statistical Data
http://lod-cloud.net

LOSD Innovation Ecosystem

Data Provider Public Service Provider Service Consumer

PAs Provision of Open
Government Data
Design and deliver of public
service
Provide public services
In policy making and/or
internal decision making
Businesses Business data (private) to be
used in services
Co-design and/or co-deliver
of public service
In business intelligence,
decision making etc.
Citizens/
NGOs
Citizen provided data Co-design and/or co-deliver
of public service
Information provision,
transparency etc.

OpenGovIntelligence tools
JSON QB API
Table2qb
Data Cube
Builder
Data Cube
Explorer
Data Cube
Aggregator
Cube
Visualizer
OLAP
Browser
Grafter

OpenGovIntelligence approach
JSON QB API
Custom apps
Data Cube ExplorerCube VisualizerOLAP Browser
UI
Business logic
UI
Business logic
UI
Business logic
Publishing Recommendations

OpenGovIntelligence approach
JSON QB API
Custom apps
UI
Business logic
UI
Business logic
UI
Business logic
Publishing Recommendations
User
friendliness
Data
integration

Pilots in 6 European Countries
Environmental planning in Belgium
Managing government vehicles in Greece Business planning in Lithuania
Real estate in EstoniaSearch and rescue in Ireland
Worklessness in Trafford

Open Public Review Pilot
https://ec.europa.eu/futurium/en/openpublicreview

Thank you!!!
http://OpenGovIntelligence.eu
@OpenGovInt
https://medium.com/opengovintelligence

for Eurostat
Agenda Overview

LOD in the ESS: initiatives,
enablers and challenges - a
PwC study for Eurostat
www.pwc.be
NTTS 2017 - Hands-on workshop on
Linked Open Statistical Data
Brussels, 17 March 2017
Nikolaos Loutas
PwC Data & Analytics

PwC
Outline
• Scope and approach of the study
• LOD initiatives in the ESS
• Value propositions and benefits
• Statistical LOD customer segments
• Key resources for implementing statistical LOD
• Means of dissemination
• Costs
• Enablers and good practices
• Roadblocks and challenges
2

PwC
Key
Partner
ships
Key
Activit
ies
Value
Proposi
tions
Custo
mer
Segme
nts
Cost
Structure
Revenue
Systems
Key
Resou
rces
Chan
nels
Scope and approach
Desk research
On site visits
Interviews
3
The identification and study of LOD initiatives in the ESS to identify
use cases, benefits, good practices, enablers and challenges, which
provided input to a proposal for a joint LOD strategy for the ESS.
All credits to businessmodelgeneration.com

PwC
LOD initiatives in the ESS
4
Central Statistics
Office (CSO)
Institut national
de la statistique et
des études
économiques
(INSEE)
Office for National
Statistics (ONS)
Statistics Scotland
Federal Statistical
Office (FSO)
Istituto nazionale
di statistica
(ISTAT)

PwC
• Interconnect several official statistics datasets (covering
both data and metadata) housed in different databases, data stores
and data warehouses within a NSI.
• Interconnect several official statistics datasets (covering
both data and metadata) housed in different databases, data stores
and data warehouses of different NSIs and/or Eurostat.
• Publish official statistics in a linkable, machine-readable format,
which can easily be reused and integrated with other types of data,
e.g. geospatial, weather, etc.
6
Why are NSIs using LOD
1
2
3

PwC
A selection of statistical LOD use cases
• LOD for territorial bases
(ISTAT, Italy)
• Selecting the best place to
live or to invest (Maynooth
University)
• LOD for fact-checking
• Finding data for a postcode
(ONS Geography)
• Accessing and querying census
data (CSO, Ireland)
• Evolution of Swiss communes
(FSO, Switzerland)
7
• Scottish Index of Multiple
Deprivation (Scottish
Government)
• Relate/correlate different
sources which provide
information about a specific
domain (Evangelos
Kalampokis)
• Providing catalogues of
linked metadata of open
datasets (EU Open Data
Portal and European Data
Portal)
• ModernStats - Linked Open
Metadata (UNECE)
• Integrated access to EU
and BEA data (Eurostat and
BEA)
• Digital Agenda Scoreboard
(DG CONNECT)
1 2 3
Interconnect datasets
within a NSI
Interconnect official
statistics datasets from
different NSIs and/or
Eurostat
Publish official statistics in
machine-readable, linkable
formats

PwC
LOD value propositions & benefits
National Statistical
Institutes
Having a unified view over
data, thanks to easier integration;
More flexible means of data
dissemination and wider
outreach;
Increased standardisation,
interoperability and
collaboration opportunities;
Easier to innovate and evolve;
Cost reductions, collect and
publish once, reuse many times.
Data reusers
Using the right data at the right
time in the right format;
Better understanding of the
data as the data and the model are
closely interwoven;
Increased trust, thanks to
traceability and provenance;
Easier integration with other
data from various domains;
Enhanced data exploration by
navigating the links;
Innovation.
8

PwC
Statistical LOD customer segments
9
Businesses
Public
administrations
NGOs
NSIs and
Eurostat
Data
journalists
Academia and
researchers
Citizens

PwC
Key resources for implementing statistical LOD
Technology &
Infrastructure
Building blocks frequently
used:
• Data preparation
• SPARQL endpoint
• LOD portal
• JSON-stat API
• REST API
• Data browsers
10
Web standards, such as HTTP URIs and RDF
Data standards, such as SDMX, RDF Data Cube and StatDCAT-AP.
Data &
Metadata
Skill may be available in house, outsourced or a combination of both.
• Technical skills (e.g. PHP, JAVA, data management, data quality)
• LOD knowledge (e.g. LOD principles, standards and technologies)
• Communication and promotion (e.g. people able to communicate
the why and get buy-in)
• Statistical knowledge
People &
Capabilities
Linked Data
Governance
Define overall priorities with respect to the main value proposition
Performing common analysis and on-going evaluation
Data licensing: most common licence is Creative Commons Attribution 4.0.
URI policy: to guarantee persistence, resolvability, and uniformity of Web
identifiers.

PwC
Key resources for implementing statistical LOD
Key partners
11
Industry Outsourcing technical development, consultancy
Academia
Knowledge and expertise sharing, common projects, tools
development
Key activities
Requirements Pre-implementation analysis, on-going evaluation
Development
Selection of data, creation of tools to transform, link,
publish and visualise data
Maintenance Governance, management, user support
Promotion Communication and publicity

PwC
Means of statistical LOD dissemination
Channels
12
NSIs portals Browser-based access to LOD, e.g. intuitive link navigation
Endpoint/API SPARQL, REST, URI dereferencing
Mobile Apps
Customer relationships
Contests Hackathons, app contests, prizes
Feedback Customer support, CRM

PwC
LOD cost structure
13
Development
Subcontracting creation of tools
In-house development
Maintenance
Technical maintenance, only limited as in most cases LOD is in
pilot phase
Promotion Communication and publicity, contests, prizes
Licensing Licensing for LOD tools in case open source solutions are not used

PwC
Enablers and good practices for implementing
statistical LOD
• Flexible way of integrating data, with
minimum impact on current infrastructure
• Allows to access data at different levels of
granularity – from data points to datasets
• Promotes the need for data standardisation
• Opportunities for new data-enabled services
• Ease of data navigation via browsing the
links (URIs)
• Ease of model updates because of the
flexibility of RDF
• Emerging best practice guidance
• Creates partnerships between public
administration, academia, standards
organisations and industry
• Identify clear use cases, target users and
benefits
• Start small, think big
• Use a trial and error approach with well-
defined iterations
• Look for support and knowledge from the
community and collaborate
• Rely on standards for data and metadata –
contribute to standardisation discussions
• Provide different ways of accessing the data,
from APIs to visual interfaces, to cater for
different types of users
• Provide persistent URIs and open licensing
• Measure the use and the expected benefits
14
Enablers Good practices

PwC
Roadblocks and challenges for implementing
statistical LOD
• Low awareness within the ESS of the technology and its benefits
• Insufficient promotion of success stories and implemented use cases
• Perceived lack of users’ demand for LOD
• Lack of management buy-in and support
• Organisational resistance because of changes in dissemination of official
statistics (new technology, new data formats, new data standards…)
• Perceived scarcity of necessary skills and competencies in the ESS,
combined with limited training opportunities and resources
• Proliferation of so-called standards . The ESS is not partaking in the
development of standards for LOD in statistics
• Very limited collaboration and knowledge sharing between NSIs in
statistical LOD
15

PwC
Get in touch with us to know more
16
Nikolaos Loutas
nikolaos.loutas@be.pwc.com
Daniel Brulé
daniel.brule@be.pwc.com
This publication has been prepared by PwC EU Services and is reporting on a study delivered for Eurostat under DI07171 specific contract 353.
“PwC” refers to PwC Enterprise Advisory bvba which is a member firm of PricewaterhouseCoopers International Limited, each member firm of
which is a separate legal entity.
Eurostat Project Officer: Christine.Kormann@ec.europa.eu

for Eurostat
Agenda Overview

paul@proxml.be
L(O)SD at dep. LNE,
Flemish Gov.

Context

§ Supporting decision making on environmental permits and
inspections
§ As a citizen I would like to know which emissions are happening in our/a
neighborhood.
§ As a civil servant I want to know the already reported emissions in a
neighborhood to evaluate new emission permit requests for that same area
and plan emission inspection based on previous reporting to enhance
efficiency.
§ As a company I want to compare my emission values with similar companies.
Objective

§ IMJV (integral environmental year report)
§ Air Emissions
§ Water Emissions
§ Waste
§ …
§ CBB (company registry)
§ GPBV, Integrated Pollution Prevention and Control (IPPC)
§ KBO
§ NIS/NACE
§ Supporting controlled vocabularies
Data

Observation
measure
dimension
dimension
dimension
attribute
attribute

Application

Analytics

Q&A

for Eurostat
Agenda Overview

NTTS 2017, Brussels, 16 March 2017
OpenGovΙntelligence Tools
Bill Roberts

Standards!

§For data model
§For statistical dimension values (‘code lists’)
§For API and query mechanisms

§Easier discovery
§Re-usable tools
§Combine data from different sources

OpenGovIntelligence tools
JSON qb API
Table2qb
Data Cube
Builder
Data Cube
Explorer
Data Cube
Aggregator
Cube
Visualizer
OLAP
Browser
Grafter

Publishing tools
Data Cube
Builder
Table2qb
Define and use
common publishing
practices
Data Cube

Linked Statistical Data Applications
Data Cube
Custom apps
UI
Business logic
Data access
SPARQL endpoint
UI
Business logic
Data access
UI
Business logic
Data access

JSON qb API
JSON qb API
Custom apps
DashboardsCharts, mapsData browser
and search tools
Data Cube
SPARQL endpoint
Re-use existing
JavaScript libraries
e.g. D3.js,
PivotTable.js
§ Support web
developers use
statistical data
stored in the form
of an RDF Data
Cube
§ Assume minimal
knowledge of
Linked Data

§ Enables the
exploration of an
RDF data cube by
presenting a two-
dimensional slice of
the cube as a table
§ Uses the JSON qb
API
OLAP Browser
API call: get metadata
API call: get data
API call: get cubes

Cube Visualizer
§ Presents a graphical
representation of a
one-dimensional
slice of a cube
§ Uses the JSON qb
API
API call: get data

Data Cube Explorer
§ Summarizes,
analyses and
visualizes data cubes
§ Available charts: bar
chart, Scattered
Chart, Area Chart,
HeatMap
§ Uses the JSON qb
API
API call: get data

§ For more information
§ https://github.com/OpenGovIntelligence
§ http://www.opengovintelligence.eu/
OpenGovintelligence Tools

GET dimensions
Parameters: dataset (required)
Sample result:
GET dimension-values
Parameters: dataset (required), dimension (required)
Sample result:
JSON qb API - metadata
[{"@id":"http://purl.org/linked-data/sdmx/2009/dimension#sex","label":"sex"},
{"@id":"http://example.com#timePeriod","label":"Time Period"},
{"@id":"http://example.com#refArea","label":"Reference Area”}]
{"dimension":{"URI":"http://example.com#timePeriod,"label":"Time Period”}
"values":[
{"@id":"http://example.com/concept/year2004#id", "label":"2004"},
...]}
All the dimensions
of a cube
All the values of a dimension
that appear at a specific cube

GET table
Parameters: col (required), row (required), measure (required), locked dimensions (optional)
Sample result:
JSON qb API - data
{"structure":{
"free_dimensions":{
"timePeriod":{"@id":"http://example.com#timePeriod","label": "Time Period"},
"refArea":{"@id":"http://example.com#refArea","label":"Reference Area"}},
"locked_dimensions":{
"sex":{"@id":"http://purl.org/linked-data/sdmx/2009/dimension#sex","label":"sex",
"lockedValue":{"@id":"http://purl.org/linked-data/sdmx/2009/code#sex-F","label":"sex-F"}}},
"dimension_values":{
"refArea":{
"S12000033":{"@id":"http://statistics.gov.scot/S12000033","label":"Aberdeen City“},
"S12000034":{"@id":"http://statistics.gov.scot/S12000034","label":"Aberdeenshire“}…},
"timePeriod":{
"year2004":{"@id":"http://example.com/concept/year2004#id","label":"2004"},
"year2005":{"@id":"http://example.com/concept/year2005#id","label":"2005"}…}}},
"headers":{"columns":{"refArea":["S12000033","S12000034“,..."]},
"rows":{"timePeriod":["year2004", "year2005",..."]}},
"data":[[73.4,79.6, ...], [76.6,78.8]]}
Τable representation of the cube’s
observations that match to particular criteria

for Eurostat
Agenda Overview

PublishMyData Demo

http://statistics.gov.scot

Employment vs life expectancy at birth

OpenGovIntelligence Workshop at NTTS2017

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (19)

Similaire à OpenGovIntelligence Workshop at NTTS2017

Similaire à OpenGovIntelligence Workshop at NTTS2017 (20)

Plus de OpenGovIntelligence

Plus de OpenGovIntelligence (12)

Dernier

Dernier (20)

OpenGovIntelligence Workshop at NTTS2017