These slides contain frameworks and concepts to build the capacity of NGO staff to monitor and evaluate their work. They contain some ideas that are relevant and useful for the M&E of complex systems.
Many thanks to Carlene Baugh and Scott Yetter from CHF International for sharing this material with MaFI.
M&E tools for NGO capacity building, by CHF International
1. M&E Tool for NGO Capacity Building
NGO Capacity Building Workshop
September 2010
Presented
by
Carlene Baugh
www.chfinternational.org
2. At the end of the session
participants will
• Gain awareness of:
– Framework for achieving program effects
– A tool for tracking capacity building
• Identify:
– How the tool can be applied in your NGO
capacity building program (s)
www.chfinternational.org
3. Let’s look at some
FRAMEWORKS!
www.chfinternational.org
5. Capacity
Development
Organization System Eco-System
Individual •
•
Community Based Organizations
Non-Governmental Organizations
•
•
Communities
National Health systems
• Societies
• The full gamut of
• Government departments • National level civil society individuals, organizations, networks and
•Development •
•
Social enterprises
Businesses
• Governmental Coordination Systems systems
• Cross-functional
• Cross sectoral
Professionals • Cross-hierarchical
•Civil Society
Leaders
•Youth
•Change Agents
www.chfinternational.org
6. Enabling
MARKET PLAYERS SUPPORTING
FUNCTIONS
Environment
Information
Private sector
Informing &
Government communicating
Demand CORE Supply
Setting & Business
Informal membership
enforcing rules
networks organisations
Standards Laws
RULES
Springfield Centre |
Making markets work Not-for-profit sector www.chfinternational.org
7. Traditional Program DevelopmentalSystemic
characteristics: approach
Judge success or failure Provide feedback for improvement
Measure against fixed goals New measures as goals evolve
External for objectivity Internal, integrated, interpretive
Linear cause/effect models Seek to capture system dynamics
Accountability to external Accountability to values, commitments
Accountability for control, blame Understand & respond strategically
Engender fear of failure Feed hunger for learning
Adapted from: Patton, Michael Q., 2006, “Evaluation for the Way We Work”,
The Nonprofit Quarterly, Spring.
www.chfinternational.org
9. Key Messages or
themes
Look at the bigger picture
• See yourself as a
part of a
interconnected
web of
relationships and
systems
www.chfinternational.org
10. Key Messages or
themes
Recognizing that change is…
• Continuous
• Complex
• Non-linear
• Multi-directional
• Not controllable
www.chfinternational.org
11. Key Messages or
themes
Keeping your eyes wide open
• Being attentive
along the journey
is as important as
the destination
www.chfinternational.org
12. Key Messages or
themes
Contribution not attribution
• your influence on a
better world
• you can influence
but not control
change in your
partners
www.chfinternational.org
13. What is Outcome Mapping
• An M&E tool developed by IDRC
• Shift from assessing the products of a
program TO:
Behavior change: relationships, or
actions of the people, groups, and
organizations with whom a program works
directly and those within its sphere of
influence.
www.chfinternational.org
14. What is Outcome Mapping
• Assesses the contributions of
development programs make to the
achievement of outcomes
• Learning-based and use-driven guided by
principles of participation and iterative
learning throughout the program life-cycle
• Program, project or organizational levels
www.chfinternational.org
15. What is Outcome Mapping
• Focuses on monitoring and evaluating its
results in terms of the influence of the
program on the roles these partners play
in development
www.chfinternational.org
16. Outcome Mapping
Defines the program’s Focuses on how
outcomes as changes in programs facilitate
behaviors of direct change rather that how
partners they caused change
Recognizes the Boundary Partners
complexity of
development processes Individuals, groups or
together with the organizations with whom
contexts in which they the program interacts
occur directly to effect change
www.chfinternational.org
18. Focus on direct partners
• Key concept is
« boundary partners »
• The individuals, groups,
and organizations you
work with directly and
anticipate opportunities
for influence
www.chfinternational.org
19. Examples
• Local communities (NGOs, indigenous
groups, churches, community leaders)
• Government officials and policymakers
(department, regional administration)
• Private sector (tourism, fisheries, non-
timber forest products, logging and wood
processing companies)
www.chfinternational.org
20. Example
• Outcome Challenge: The program
intends to see local communities that
recognize the importance of, and are
engaged in, the planning of resource
management activities in partnership with
other resource users in their region
www.chfinternational.org
21. progress markers =
ladder of change
Love to see
Like to see
Expect to see
www.chfinternational.org
22. Progress markers
✓ A graduated set of statements describing a
progression of changed behaviors in the
boundary partner
✓ Describe changes in actions, activities and
relationships leading to the ideal outcome
✓ Shows story of change by articulating the
complexity of the change process
✓ Can be monitored & observed
✓ Permit on-going assessment of partner’s
progress (including unintended results)
www.chfinternational.org
23. Performance M&E
• Monitoring 3 elements
– Changes in behaviors, actions, relationships,
groups (outcome journal)
– Strategies to encourage change in its
partners (strategy journal)
– Functioning of a program as an
organizational unit (performance journal)
Each of these monitoring tools builds on elements from the Intentional
Design stage, so the group should feel relatively comfortable with them.
www.chfinternational.org
24. Performance Journal mtg
– What are we doing well and what should we
continue doing?
– What are we doing “okay” and what can we
improve?
– What do we need to add to better address the
organizational practices?
– What activities do we need to modify
– Who is responsible? What are the time lines?
– Has any issue come up that we need to
evaluate in greater depth? What? When?
Why? How?
www.chfinternational.org
25. Outcome journal
To understand the change process in boundary
partners. Collects information about:
• Story of change and reasons for change
• Unexpected changes
• The actors and factors that contributed to that
change
• How we know the change occurred
• Learnings (what? how? why?)
www.chfinternational.org
26. References
• Building learning and reflection into
development programs (Sarah Earle, Fred
Carden & Terry Smutylo, 2001)
• Outcome Mapping Learning Community
http://www.outcomemapping.ca/
www.chfinternational.org
Outcome Mappingencourages a program to introduce monitoring and evaluation considerations at the planning stageand link them to the implementation and management of the program. It also unites process and outcome evaluation, making itwell-suited to the complex functioning and long-term aspects of international ddevelopment programs where outcomes areintermeshed and cannot be easily or usefully separated from each other. Focusing monitoring and evaluation around boundarypartners allows the program to measure the results it achieves within its sphere of influence, to obtain useful feedback about itsefforts to improve its performance, and to take credit for its contributions to the achievement of outcomesrather than for the outcomes themselves. The above diagram illustrates the three stages of Outcome Mapping and the twelve steps of an Outcome Mapping design workshop. The first stage, Intentional Design, helps a program establish consensus on the macro level changes it will help to bring about and plan the strategies it will use. It helps answer four questions: Why? (What is the vision to which the program wants to contribute?); Who? (Who are the program's boundary partners?); What? (What are the changes that are being sought?); and How? (How will the program contribute to the change process?). The second stage, Outcome and Performance Monitoring, provides a framework for the ongoing monitoring of the program's actions and the boundary partners' progress toward the achievement of outcomes. It is based largely on systematized self-assessment. It provides the following data collection tools for elements identified in the Intentional Design stage: an Outcome Journal" (progress markers); a Strategy Journal" (strategy maps); and a "Performance Journal" (organizational practices). The third stage, Evaluation Planning, helps the program identify evaluation priorities and develop an evaluation plan. Figure 1 (next page) illustrates the three stages of Outcome Mapping.