Convegno Scientifico Istat Roma Aula Magna 10 giugno 2019 Ricerca scientifica,valutazione delle politiche e micro-dati Via Cesare Balbo,16

1. The European Commission’s science
and knowledge service
Joint Research Centre

2. Accesso a microdati, ricerca e miglioramento
delle politiche: esiste un valore aggiunto
europeo?
Paolo Paruolo,
Head of the Competence Centre on Microeconomic Evaluation (CC-ME)
Joint Research Centre, European Commission
Roma, 10 Giugno 2019

3. “One of the great mistakes is to
judge policies and programs by
their intentions rather than their
results”
Milton Friedman (1912–2006)
Data for policy
“Without data you are just
another person with an opinion”
W. Edwards Deming (1900–1993)
“In God we trust,
all others must bring data”

4. Main Messages
1. Linking (administrative) data is an asset
large data + naturally collected +
potential for disentangling different effects and decomposition
2. Sharing best practice has a European
dimension (subsidiarity)
Social knowledge + learning what works +
potential for cross-regional learning and cooperation
=> Both points have implications for governance

5. One basic goal of an evaluation is to learn to which degree the policy
objectives were achieved, to improve in future implementations.
Counterfactual Impact Evaluation helps to establish a causal link
between policies and their effects, relying on data available both for the
treated and the control groups.
Displacement (general equilibrium) effects must be properly
researched (lower level of aggregation)
Micro data are key
Why evaluate

6. Linking administrative data
• Microdata, basic granularity: individuals, firms, geo-units
• Already collected: limited costs
• Regularly collected: allow for time series construction
• Real time
• Non-intrusive
• As precise as it can be
• Unique identifiers and geo-referencing allow for linkages
• One can link survey data to admin data

7. Some people do it right: 1
We are all sitting on a treasure of administrative data
Collaboration between 6 Research Data Centres
from France, Germany, Netherlands and the UK to
facilitate research use of controlled access data
between these countries.
https://idan.network/
IDAN international data access network
A COLLABORATIVE PROJECT

8. Some people do it right: 2
We are all sitting on a treasure of administrative data
“Invisible yet crucial, it allows the
nation’s various public and private sector
e-service information systems to link up
and function in harmony.”
https://youtu.be/b-r6B28qVSY
X-Road is the backbone of e-Estonia
https://e-estonia.com/solutions/interoperability-services/x-road/
99% of state services are online

9. Good and properly linked microdata allow
to learn about causal effects
1. “Work experience for Graduates”
initiative, Italy (Umbria)
Eligibility ->Regression
Discontinuity Design: Don’t
rush your kid into school!
2. Education in Chile
Propensity Score Matching: Participants
in the program more likely to be employed.
Needs large dataset
3. NL childcare subsidies
Differences-in-differences:
best option is in-work benefit
for secondary workers

10. Data-Driven Policy Impact Evaluation How
Access to Microdata is Transforming Policy Design
www.springer.com/gp/book/9783319784601
Editors: Nuno Crato
Paolo Paruolo
Springer 2019 free download
at:
www.springer.com/gp/book/978
331978460Data issues:
legal and organizational framework,
data linkage, data confidentiality
Microdata access examples throughout Europe
Counterfactual studies – examples in Education, Health,
Employment… and in different countries
How studies have been used in Europe
CONTRIBUTORS: 11 European countries + Eurostat +
OECD 22 Scholars, statistical officers, administrators, and
political advisors and deciders
European perspective
Free download of the whole book (41k downloads)

11. Data-Driven Policy Impact Evaluation
How Access to Microdata is Transforming Policy Design
www.springer.com/gp/book/9783319784601
Proper methods allow to evaluate the impact of
policies
Microdata is useful for IE and available in (linked)
administrative data
There are solutions to preserve confidentiality
and to overcome other legal issues
Access could be granted across boundaries to
researchers
Editors: Nuno Crato Paolo Paruolo
Springer 2019 - free download at:
www.springer.com/gp/book/978331978460

12. Book structure
• Part I
Microdata for
Policy Research
• Part II
Microdata
Access
• Part III
Counterfactual
Studies
• Part IV
Use of
Results
• Change of culture
• GDPR and legal
aspects
• How to anonymize
data
• How to link data
probabilistically
• at ESTAT
• in Germany
• microdata linkages in
Hungary
• the Longitudinal
Internet Studies for the
Social sciences in the
Netherlands
• Labour Market in
Portugal, Italy, the
Netherlands
• CPB in the
Netherlands
• Vulnerable Youth in
Denmark
• Retirement in Italy
• Education in Chile
• Summary of evidence
on Active Labour
Market Policy in
Europe and on
Apprenticeships in
Scotland
• Pay for Performance
in Primary Health
Care, UK and Italy
• The Potential of Admin
data

13. Both spending and regulatory policies could benefit from linked
administrative data by:
1. breaking silos to better incorporate interactions across policy areas
2. allowing policy decisions and adaptation based on more robust
evidence on what works for whom
3. better measuring of impacts at individual or disaggregated group
levels, reflecting distributions across income, age, etc.
4. much more efficiently and effectively increasing citizen well-being
5. reducing evaluation lags and adjusting policies ‘On the Go’
6. reducing other lags in the policy cycle
Benefits

14. Avoiding the Pitfalls
• Data Privacy and Security: keep confidential data safe
• Dislocation from Consultative and Legislative Due Process: give
society enough time to react
• Data Accuracy: too much accuracy may harm
data providers  researchers  policy makers
Costs of data linkage are VERY SMALL
• these pitfalls can be avoided
• need for coordination:

15. Elevator pitch
• Microdata exist and can be safely organised for a
better knowledge of society and for policy research
• data-based policy assessment, monitoring, evaluation
and improvement are feasible and desirable, and
• they can be achieved in unprecedentedly fast ways
by properly linking administrative data.
• The JRC can help to bring together a community of
regions of Europe to share best practice

16. Some implications of the main messages
1. >Link Administrative data systematically
national linking agencies
cross-border collaborations
2. >Increasingly share access to
researchers to learn more
safe environments
cross-regional learning and cooperation
=> governance

17. Any questions?
You may contact us at @username & user@mail.me
Thank you
Visit us at
https://ec.europa.eu/jrc/en/microeconomic-evaluation
http://crie.jrc.ec.europa.eu
https://connected.cnect.cec.eu.int/groups/cc-me
(for European Commission services only)
ec-cc-me@ec.europa.eu (email)