The document discusses challenges faced by third sector organizations in measuring impacts and evaluating their services and projects. It outlines that typical challenges include a lack of robust impact measurement systems, reliance on anecdotal data, difficulty measuring ultimate impacts and benefits, and a need for staff with skills in data analysis and process redesign. The document then provides explanations and examples to clarify concepts like activities, outputs, outcomes, impacts, monitoring, evaluation, and learning. It presents a monitoring and evaluation framework and discusses how different indicators and methods can be used at various levels to measure results and drive organizational learning.

1. Monitoring,
Evaluation &
Learning
Services for Third Sector organisations
(c) ISC 2019

2. The MEL Landscape - typical challenges
Impacts
Beneficiaries
Services &
Projects
• We have no way of measuring the ultimate impacts and benefits of our
interventions
• We can’t afford to design and implement robust impact measurement systems
• Most of our impact assessment information is text-based or anecdotal
• Knowledge-sharing is extremely difficult
• Donors are asking for more and more Impact data and we don’t have it
• We don’t have much more than attendance rates and “happiness sheets”
• We know how many people we have helped but not what they have been able
to achieve as a result of our support
• We don’t have staff with the skills to be able to “slice and dice” data to get
insights into what our beneficiaries want/need
• We need to find better ways to deliver our services to reach more people
• We have no idea what the average cost of an intervention is
• We have no data on problems, delays or waste in our processes
• We have lots of input data (e.g. how many services we have delivered) but not
much else
• We don’t have staff with the skills to redesign our processes to improve their
efficiency
(c) ISC 2019

3. Cutting
through the
jargon
(c) ISC 2019

4. It’s very simple, really…
Activities Outputs Beneficiaries Impacts
Third Sector organisations create assets (services, projects, programmes, interventions) and enable people
(Beneficiaries) to use those assets to make an impact for themselves and others.
• Projects
• Programmes
• Processes
• “Work”
• Products
• Deliverables
• Services
• “Stuff”
• Outcomes
• “What people
do with your
stuff”
• Benefits
• “What’s better as
a result of people
using your stuff”
(c) ISC 2019

5. The MEL Landscape
Impacts
Beneficiaries
Services &
Projects
Monitoring
Evaluation
Feedback
loops &
Learning
(c) ISC 2019

6. Monitoring vs.
Evaluation
Dimension Monitoring Evaluation
Frequency Periodic, occurs regularly Episodic
Function Tracking / oversight Assessment
Purpose Improve efficiency, provide
information for reprogramming
to improve outcomes
Improve effectiveness, impact,
value for money, future
programming, strategy and
policymaking
Focus Inputs, outputs, processes,
workplans (operational
implementation)
Effectiveness, relevance, impact,
cost-effectiveness (population
effects)
Methods Routine review of reports,
registers, administrative
databases, field observations
Scientific, rigorous research
design, complex and intensive
Information
Source
Routine or surveillance system,
field observation reports,
progress reports, rapid
assessment, programme review
meetings
Same sources for monitoring plus
population-based surveys, special
studies, qualitative methods
Cost Consistent, recurrent costs
spread across implementation
period
Episodic, often focused at the
mid-point and end of
implementation period
(c) ISC 2019

7. Services & Projects
Impacts
Beneficiaries
Services &
Projects
Monitoring
Evaluation
Monitoring & Evaluation Framework
Results-based framework
Leading Indicators:
• Inputs
• Outputs
• Process
Mostly quantitative
Monitoring tools
suited to the
results that need
to be measured
(c) ISC 2019

8. Beneficiaries
Impacts
Beneficiaries
Services &
Projects
Monitoring
Evaluation
Monitoring & Evaluation Framework
Results-based framework
Early evaluation of results
• Pre- & Post- Surveys
• Satisfaction surveys
• KAP surveys
Short-term outcomes
Quantitative and qualitative
(c) ISC 2019

9. Impacts
Impacts
Beneficiaries
Services &
Projects
Monitoring
Evaluation
Impact Assessments
Lagging indicators
Mixed methods evaluation design
SROI / VfM
Case Studies
Outcome Harvesting, Contribution Analysis,
Process Tracing, etc.
Research
Mostly qualitative
(c) ISC 2019

10. Triple-loop Learning
Performance
Results
What?How?Why?
Loop 1:
Did we do things right?
Loop 2:
Did we do the right things?
Loop 3:
How do we decide what
are the right things to do?
(c) ISC 2019

11. Learning
Impacts
Beneficiaries
Services &
Projects
Monitoring
Evaluation
Achieving the mission & vision
Impact & Sustainability
Improving delivery of the intervention
Effectiveness & Relevance
Continuous process improvement
Operational excellence
Efficiency
(c) ISC 2019

12. Ian has been consulting in the Third
Sector since 2004 and has also provided
pro bono support to many
organisations. His work has included
projects such as developing business
plans, re-designing services and
developing performance measurement
systems.
Our team
Clyde has provided computing services to
SMEs and charities for the past 15 years,
primarily in the areas of fundraising and
case management solutions, as well as
grant-making. He has helped
organisations to open-up their processes
and engage collaboratively with partners
by exploiting the internet.
Robin guides and supports Social Impact
Organisations to improve the effect they
have on the social fabric of communities
and the wellbeing of individuals and
families. Robin helps organisations
identify the social impact of their
programmes and become more effective
in delivering impact.
Clyde Williams Robin Brady Ian Seath
Clyde Williams, Ian Seath and Robin Brady bring together a unique combination of skills and experience:
 Strategy Development
 Performance Management
 Process Analysis
 Impact Management
 Monitoring and Evaluation Design, Implementation and Management
 Digital Transformation (c) ISC 2019

13. • ian.seath@improvement-skills.co.uk
• 07850 728506
• @ianjseath
• www.improvement-skills.co.uk
Ian J Seath
Improvement Skills Consulting Ltd.
Contact us
Read about our Third Sector
experience
(c) ISC 2019