By Katharina Welle, PhD candidate, University of Sussex. Prepared for the Monitoring sustainable WASH service delivery symposium, Addis Ababa, Ethiopia, 9-11 April 2013.

1. Monitoring Performance or
Performing Monitoring? Lessons on
the politics of monitoring
Monitoring for sustainable WASH services
Symposium
Addis Ababa, 10th April 2013
Katharina Welle, PhD candidate, University of Sussex

2. WHY FOCUS ON PERFORMANCE
MONITORING?

3. What is Performance Monitoring (PM)?
Feedback
Objectives Activities Outputs Outcomes ImpactsInputs
Development Results
Source: OECD-DAC (2010)
OECD-DAC Results Chain
According to the OECD-DAC Glossary of Key Terms in Development Evaluation and
Results-based Management (2010), performance monitoring refers to a continuous
process of collecting and analysing data to compare how well a project, programme or
policy is being implemented against expected results”.

4. Common assumptions on the linear relation
between monitoring and decision making
The purpose of monitoring is “to track progress
against given objectives” and “to inform
decisions, focus and orient political and policy
reforms, and to channel financial resources in
the most effective way” (UN Water, 2006: 9)

5. Common criticisms of Performance Monitoring
• Misrepresents complex
reality by reducing it to few,
measurable results,
expressed in quantitative
indicators
• Quantification suggests a
mechanical objectivity:
sanctioned methods to
produce ‘presumably
neutral facts’
• Can create ‘perverse
incentives’ and lead to
‘gaming behaviour’
Frederick Taylor (1856 – 1915)

6. Monitoring performance or performing
monitoring? The importance of actors’ framings
”narratives … employing
unifying metaphors
…that tell what needs
fixing and how it might
be fixed ….”
Rein and Schön (1996:
89)

7. INSIGHTS ON THE POLITICS OF
PERFORMANCE MONITORING

8. Rural water access trends in the Southern Nations,
Nationalities and Peoples (SNNP) Region
Official figures of water access in Ethiopia’s
Southern Region from 2000 -7
0
10
20
30
40
50
60
70
The Southern Region in Ethiopia
Source: Bureau of Water Resources 2009

9. ACTORS’ FRAMINGS OF ACCESS AND
WHICH ONES GET ELEVATED IN PM
Feedback
Objectives Activities Outputs Outcomes ImpactsInputs
Development Results

10. Parameters for defining access to rural water in
Ethiopia
Definitions of access
Joint
Monitoring
Programme
of WHO and
UNICEF
Ethiopian
Ministry of
Water and
Energy
Volume 20 litres /
person /day
15 litres /
person / day
Distance 1 km 1.5 km
Access calculation (MoWR
2009/10)
Hand
dug
well
Protect-
ed spring
Shallow
well
Deep
well
Avera
ge
benefi
ciaries
270 338 457 3,313

11. ‘Access’ results
for my case
study kebele
Scheme Average
beneficiaries
3 Hand
dug wells
810 (3 x 270)
1 Bore
hole
3,313
Total 4,123
Sche
me
Kebele
population
%
served
CSA 5,885 70%
HEW
data
~4,700 94%

12. Kebele residents’ issues with access
BH
Kebele
boundary
Water
point
Public
Tap
Public
Tap
HDW1Church
Priest
HDW2
School
Teachers
HP
HDW3

13. MONITORING DYNAMICS
Feedback
Objectives Activities Outputs Outcomes ImpactsInputs
Development Results

14. Using different calculation methods
to stage ‘performance’ in review meetings
Regional Bureau of Water
Resources
Calculation method used:
average user estimates by
scheme type
Rationale: reporting positive
access trends at federal
level
Result for case study woreda:
Access: 58% in June 2009
Woreda (district) Water, Mining
and Energy offices
Calculation method used:
users living within 1.5 km of
source
Rationale: obtaining budgets
for water supply at the
local level
Result for case study woreda:
Access: 38.1% in December
2009

15. FACTORS AFFECTING RURAL WATER
ACCESS DECISIONS
Feedback
Objectives Activities Outputs Outcomes ImpactsInputs
Development Results

16. Decisions on capital budget allocations for rural
water supply in SNNP Region, Ethiopia, 2009/10
Who Factors affecting allocations
Channel
1a
Government
block grants
Multi-sectoral ‘new budget grant formula’
Channel
1b
Sector
donors (WB,
UNICEF, etc)
Specific intervention woredas (80 out of
134 in 2009)
Channel
2
Multi-sector
programmes
(e.g.PSNP)
In case of PSNP(Productive Safety-Net
Programme ), to food-insecure woredas,
capital allocations to sectors depend on
various factors
Channel
3
NGOs No direct control by government, many
factors affect project sites of NGOs

17. Realities of “strategic planning” at woreda level
Financing modality Woreda block
grant
WASH Programme PSNP NGO
Financing channel 1a 1b 2 3
Funding period 2006-2010 2005-2012 2007-2011 2007-2011
Schemes
constructed until
2009/10
0 2 hand-dug wells
2 shallow wells
10 protected springs
8 protected
springs
9 deep wells
5 shallow wells
Construction cost
estimates based on
MoWR (2005)
Not applicable ~600,000 ETB ~300,000 ETB ~7,000,000 ETB
Sources: World Vision project document, MoFED Water Supply and Sanitation fund
utilisation sheet, MoFED PSNP Fund Utilisation Sheets & report prepared by officer for me

18. Borehole in case
study kebele
Neighbouring
kebele’s Tap

19. Multi-causality of factors affecting the repair
Inhibiting
the repair
Facilitating
the repair
Inadequate sector
support to and
oversight of WASH
committee
Cost implications of
major maintenance
not coherently
addressed
Government
logistical and
capacity
constraints
Climate of distrust
and indications of
abuse of roles among
water user
committee members
Local opinion
leader pushed
repair
Drought in 2008 amplified
water access problems of
local residents
Approach of
general elections
creates ‘window
of opportunity’
Regional inventory
results in decision to
address non-
functionality
Woreda water office
mobilised budget
sources for new
reservoirs
Woreda water
office reported
scheme
breakdown

20. Implications for performance monitoring
PM is not objective but subject to actors’ framings – in performance
monitoring, some factors are elevated while others remain
neglected
 there is a need to bring perspectives from local users higher up on the
agenda
In addition to official monitoring objectives, people involved may have
their own, private agendas, that influence the process and results
 It is important to pay attention to the practice of every day monitoring in addition
to redesigning global targets and indicators
Monitoring results are just one among many factors affecting service
delivery decisions
Can be used as a tool to legitimise actors rather than really increasing
accountability

21. Thank you
Below are additional slides that I might use