1. New Challenges for the Evaluation Community:
timely responses in times of urgency
Hans Bruyninckx | European Environmental Evaluators Network | 4 November 2020
3. 2020s: a pivotal decade
“Europe stands at a critical juncture in 2020.
Its leaders have opportunities to shape future
developments that will not be available to their
successors. The coming decade will therefore be
of decisive importance in determining Europe’s
opportunities in the 21st century.”
SOER 2020
4. SOER2020
1. IPCC report on global
warming of 1.5°C
3. International Resource
Panel global outlook 2019
2. IPBES global report on biodiversity
and ecosystem services
Global context: unprecedented challenges, improved knowledge
•Urgent action needed
•Irreversibilities
•Tipping points
•Interconnected
7. A new policy context: European Green Deal
• First climate-neutral continent
• Biodiversity Strategy 2030
• New Circular Economy Action Plan
• Zero pollution strategy
• Farm to fork strategy
• Just transition
• Sustainable European Investment Plan
• Future ready economy – new industrial strategy
8. The emergence of transformative policy since 2014
Change has come very rapidly*
2014: EU 7th EAP, Horizon 2020
2015: UN Agenda 2030/SDGs, Paris Agreement
2015-2018: EU strategies: energy, mobility, circular, low-carbon, etc.
2019: European Green Deal, Digital Agenda
2020: Multiple EGD packages; 8th EAP; Horizon Europe
* But late considering 50 years of signals from science (Club of Rome; Brundlandt;
Rio 92; GEOs; IPCC; …; a vast body of knowledge of (un)sustainability )
9. The (changing) nature and role of evaluation?
What is the impact of this changing context on ‘environmental’ evaluation?
• What is the goal of the evaluation?
• What is being evaluated?
• What is the place of the evaluation in the policy process?
• When does it take place?
• Forward looking evaluation?
11. Traditional approach
- Did (a part of) the policy work?
- Revision of legislation
- Changes to a policy instrument
- Changes to the target
- Reviewing the timing
- Adjusting the budget
- Impact on the environment
- Political legitimacy
12. Evaluation for system innovation and transitions
Source: based on Geels & Schot (2007)
Small networks of actors support innovation on the basis of expectations and visions.
Learning and experiments take place
Niches
Landscape
Regime
Time
Markets and
consumer preferences
Science
Industry
Culture
Policy
Technology
New configuration breaks through, taking advantage
of ‘windows of opportunity’. Adjustments occur in
regime
The regime is dynamically stable
Landscape developments put pressure on existing regime
- System dynamics!
- Complexity
- Guidance
- Options
- Corrective action
- Learning
14. Traditional approach
- Legislation by legislation (revision)
- Institutional focus
- Ex post: efficiency, effectiveness focus (not impact)
- Ex ante: (economic) impact assessments
- Often rather detached from environmental and climate monitoring
15. Catalysing systemic change
• Recognise fundamental drivers
and system interlinkages
• Adopt transformative policy
frameworks
• Fill crucial policy gaps:
• Food
• Land and soil
• Chemicals
• Leverage the power of cities,
businesses and communities
for society-wide action
17. Policy integration largely unsuccessful: transport
EU GHG emissions in the transport sector, 1990-2017 (1990 = 100%)
International aviation
International navigation
All transport
National total GHG emissions
Other UNFCCC sectors
18. • Unsustainable agriculture still
main threat to biodiversity and
natural capital in Europe
• Pollution of soil, water, air and
food
• Over-exploitation of natural
resources
• Greening of the CAP shown to
be ineffective
Policy integration largely unsuccessful: agriculture
19. Example: climate mitigation vs air pollution
• Synergies: decarbonisation of transport
also reduces air pollution
• Trade-offs: promoting diesel vehicles and
biomass increases air pollution
Ensuring policy alignment and coherence
23. Beyond the rear view mirror
Source: based on Geels & Schot (2007)
Small networks of actors support innovation on the basis of expectations and visions.
Learning and experiments take place
Niches
Landscape
Regime
Time
Markets and
consumer preferences
Science
Industry
Culture
Policy
Technology
New configuration breaks through, taking advantage
of ‘windows of opportunity’. Adjustments occur in
regime
The regime is dynamically stable
Landscape developments put pressure on existing regime
24. Systemic change is disruptive: the ‘x-curve’
Source: Loorbach et al .
Optimisation
Destabilisation
Breakdown
Phase outExperimentation
Acceleration
Institutionalisation
Stabilisation
Conflicts
Speeding up
Scaling up
25. Investments can be “locked-in” into
unsustainable consumption and
production patterns for decades.
Lock-in risks
26. Evaluation in a context of long term change
Source: EURO-CORDEX (Jacob et al., 2014)
Precipitation
(projected change for 2071–2100)
27. Time as critical element?
Source: based on Geels & Schot (2007)
Small networks of actors support innovation on the basis of expectations and visions.
Learning and experiments take place
Niches
Landscape
Regime
Time
Markets and
consumer preferences
Science
Industry
Culture
Policy
Technology
New configuration breaks through, taking advantage
of ‘windows of opportunity’. Adjustments occur in
regime
The regime is dynamically stable
Landscape developments put pressure on existing regime
Non-problematic
Under-specified
Under-defined
28. Speeding-up in addition to scaling -up
• Seen as a consequence of scaling-up
• Thus, not an autonomous factor
29. Dangers of underestimating time
• Irreversibility
• Lower quality outcomes
• Rewarding laggards: countries and actors
• Focus on the wrong time dimension?
• Timing of reporting
• Timing of implemention
• Time dimension at appropriate level of analysis?
30. Taking time more seriously
• Explicit understanding of time dimension
• Developing approaches to understand the link between time
dimensions and the DPSIR
• Developing the necessary methodologies
• Learning and developing necessary skills, ICT systems
32. From exogenous to systemic integration of evaluative practices
• Part of transition pathways
• Iterations of evaluation
• Part of learning toolbox
• Stronger link to monitoring and reporting efforts
• Mandate for evaluators
33. Possible elements for the future
Foresight and
evaluation?
Evaluation and
Resilience
Post-normal
Evaluation
34. Conclusions
• Context is changing fundamentally
• Reflection about evaluation in this new context
necessary
• Investing in innovation
• Investing in evaluation as integral to the transition
process