Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Theory of Change
Similaire à Theory of Change (20)
Dernier
Dernier (20)
Theory of Change
- 1. Theory of Change: A Blueprint for Evaluation
- 2. Presentation Overview Introduction to Theory of Change Building a Theory of Change Why is Theory of Change Important? Testing competing Theories of Change
- 3. Presentation Overview Introduction to Theory of Change Building a Theory of Change Why is Theory of Change Important? Testing competing Theories of Change
- 4. Theory of Change (ToC) Definition • Theory of change is an on-going process of reflection to explore change and how it happens – and what that means in a particular context, sector, and/or group of people. ToC thinking • Structured way of thinking about change and impact organizations would like to achieve • Integrated approach to programme design, implementation, M+E, and communication
- 5. Causal Hypothesis Q: How do I expect results to be achieved? A: If [inputs] and [activities] produce [outputs] this should lead to [outcomes] which will ultimately contribute to [goal].
- 6. Presentation Overview Introduction to Theory of Change Building a Theory of Change Why is Theory of Change Important? Testing competing Theories of Change
- 7. Theory of Change: Main Questions What is the programme? What outcomes does the programme aim to achieve? What intermediate steps lead to those outcomes? What assumptions are associated with each link in the causal chain? How can we measure outcomes?
- 8. What is the Programme? Programme Design Relevance Target Audience Social and Political Content Potential Threats and Challenges
- 9. Theory of Change: Define the Programme Incentives for Immunization Immunization Camps
- 10. I N C R E A S E D I M M U N I Z A T I O N Theory of Change: Define the Outcomes Incentives for Immunization Immunization Camps
- 11. I N C R E A S E D I M M U N I Z A T I O N Theory of Change: Intermediate Steps Incentives for Immunization Parents bring children to the camps Immunization Camps
- 12. I N C R E A S E D I M M U N I Z A T I O N Theory of Change: Assumptions Incentives for Immunization Parents bring children to the camps Immunization Camps Camp provides immunizations Parents value incentives Parents trust camps Incentives paid regularly
- 13. ToC: School-Based Malaria Intervention Malaria Intervention Assumptions Assumptions Children adhere to full med regime Higher Test Scores Test accurately measures knowledge Increased School Attendance Reduced Clinical Attacks Reduced Asymptomatic Parasitemia Reduced Anemia Improved Cognition Increased Concentratio n I N C R E A S E D K N O W L E D G E
- 14. Theory of Change Levels Inputs/ Programme Activities Outputs Intermediate outcomes Goal What we do as a part of the programme - deliver, teach, offer loans, etc. What are the resources used –funds, staff, equipment, curriculum, all materials. Tangible products or services produced as a result of the activities - usually can be counted. Short-term behavioral changes that result from the outputs - preventive health habits, usage of tablets. Long-term changes that result from outcomes – the result of the programme.
- 15. Good indicators Quantitative and qualitative Standard of comparison (i.e. baseline v. endline, defining “high-quality,” etc.) SMART • Specific - Ask (answer) one question at a time • Measurable - Quantifiable, accurate, unbiased, sensitive • Achievable - Is this impact realistic? Are the goals attainable? • Relevant - Is this the most relevant programme indicator given the needs • Time-bound - Has boundaries. When’s the deadline?
- 16. Good Indicators Does the definition adequately describe our concept? Can the variable be easily measured? Can we collect data to measure the variable?
- 17. Increased Immunization Coverage Brainstorm in groups: • Does the definition adequately describe our concept? • Can the variable be easily measured? • Can we collect data to measure the variable?
- 18. Log Frame Objectives Hierarchy Indicators Sources of Verification Assumptions / Threats Impact (Goal/ Overall objective) Increased immunization Immunization rates Household survey Adequate vaccine supply, parents do not have second thoughts Outcome (Project Objective) Parents attend the immunization camps repeatedly Follow-up attendance Household survey; Immunizatio n card Parents have the time to come Outputs Immunization camps are reliably open; Incentives are delivered Number of kg bags delivered; Camp schedules Random audits; Camp administrativ e data Nurses/assistants will show up to camp and give out incentives properly Inputs (Activities) Camps + incentives are established Camps are built, functional Random audits of camps Sufficient materials, funding, manpower Needs assessment Impact evaluation Process evaluation
- 19. What is the best time to create a theory of change? A. Before commencing an evaluation B. Before data analysis C. During the programme design D. Before data collection 25% 25% 25% 25% Before data analysis During the programme ... Before commencing an ... Before data collection
- 20. Presentation Overview Introduction to Theory of Change Building a Theory of Change Why is Theory of Change Important? Testing competing Theories of Change
- 21. Discussion WHY IS THEORY OF CHANGE IMPORTANT?
- 22. Solving the Black Box Problem Low immunization rates Needs Assessment Intervention Intervention design/Inputs Final Black Box No increase in full immunization outcome
- 23. Identifying Theory Failure vs. Implementation Failure Successful intervention Inputs Activities Outputs Outcomes Goal Implementation failure Inputs Activities Outputs Outcomes Goal Theory failure Inputs Activities Outputs Outcomes Goal
- 24. Why is Theory of Change Important For evaluators, reminds us What is it? Example Components Assumptions Conclusion to consider process For programmers, it helps us be results oriented
- 25. What is the Main Criticism of Theory of Change? A. Over-simplify the programme B. Long-term effects not considered C. Unintended consequences not considered D. Does not consider that 25% 25% 25% 25% Over-simplify the prog... programme to outcomes is not uni-directional Unintended consequenc.. Long-term effects not co... Does not consider that p...
- 26. Presentation Overview Introduction to Theory of Change Building a Theory of Change Why is Theory of Change Important? Testing competing Theories of Change
- 27. Testing Competing Theories of Change Different disciplines have different theories • Public Health • Education • Economics • Anthropology • Sociology • Political Science
- 28. Testing Competing Theories of Change Toilets in Schools Children missing school due to waterborne disease Children use toilets Increased School Attendance Less waterborne disease Girls feel more comfortable coming to school Female dropout rate decreases Girls missing school due to lack of toilets Girls use toilets
- 29. Theory of Change: Main Questions What is the programme? What outcomes does the programme aim to achieve? What intermediate steps lead to those outcomes? What assumptions are associated with each link in the causal chain? How can we measure outcomes?
Notes de l'éditeur
- Rooted in context and stakeholder analysis - Causal pathway (the arrows) - Identifies assumptions - Derived through a participatory process
- Components: Activity versus outputs: Output is the good or service delivered as a result of the programme; activities support outputs but cannot alone deliver outcomes Intermediate versus high level outcomes: Example: student knowledge has increased is IO, but a stronger economy because of higher educated workforce is a high level outcome. IO will lead to HLO
- I will cover this more tomorrow in the Measurement lecture Translating the theory of change into a log framework, which many of you are familiar with, to map the change in the project. The needs assessment underpins everything. Assessment of inputs and outputs – process evaluation, when you are looking at outcomes and impacts (changes and goals), then you switch over to impact evaluation
- **Adapted from Marc Shotland, JPAL So we’ve done our needs assessment. We have designed and implemented our intervention. But when we come back to evaluate the programme we find that there is no growth in youth employment. If we didn’t do any process monitoring or process evaluation—in other words, if we didn’t track every step of our theory of change—we might not be able to figure out why the intervention didn’t work, or where it broke down. If, as evaluators, we look only at our final outcomes or impact, we may not understand why it worked, or if it didn’t, where it broke down. Let’s peer into the black box. Were our activities implemented as planned? Were outputs produced? Were assumptions met? Did the necessary outcomes change? In this example one of our major assumptions—which is that youth were motivated to start new businesses—was not met. As a result, even though we implemented our intervention appropriately our goal was not achieved. This is why it is important to articulate a theory of change in advance and measure each step of the theory of change (through process monitoring and evaluation) as part of an effective impact evaluation.
- **Adapted from Weiss, Evaluation, 1998, p 129 The black box problem can be described in other terms. In a successful intervention that sees the expected outcome and goal change, our theory of change is realized. That is, inputs are sufficient to conduct the activities, the appropriate products and services are provided, our outcomes change in the way we hope and this contributes to our targeted goal. It is not unusual, however, that we do not see the outcome or goal change that we had hoped for. Without an articulated and measured theory of change, however, we might not know why that is. Is it because our implementation process was problematic? That is, our inputs did were not sufficient/appropriate to implement the planned activities, or somehow the activities we implemented did not lead to the right outputs. This is called implementation failure. On the other hand, maybe our implementation process was tight. Our resources were sufficient. Our activities were implemented as planned and the services provided were good. Instead, the idea behind our programme was faulty. That is, the products and services we provided didn’t actually lead to the expect knowledge or behavior change, and so our ultimate goal was not achieved. If we have breakdown at this end of the chain (outputsoutcomesimpact) this is called theory failure. The implications are really different because with implementation failure we can only know that our mechanism didn’t work. With theory failure we know that our idea doesn’t work. If we couldn’t distinguish between implementation failure and theory failure it would be very difficult to use evaluation results to shape policy and programmeming because we wouldn’t know whether it was a bad idea or whether we had just implemented the intervention so poorly that the causal chain broke down.
- You all have this theory of change diagram in handout form. On this slide it’s not important that you be able to read what is in the boxes. The idea here is that we’ve talked quite a bit about what theory of change is. Now why is it important? For programme people, it helps us keep a focus on results—eye on the prize. Programme implementation folks are usually focused on activities and outputs: How many trainings have we completed? How many beneficiaries have we reached? We should move away from interventions for interventions’ sake toward development that explicitly serves a larger purpose and goal. E.g., In the example above, that means I shouldn’t train partner NGOs just because I think it’s fun, but I should also be oriented toward the reason I am doing it. For evaluators, it reminds us to consider process Evaluators are very focused on assessing outcome change. But sometimes this is done with a blind eye to how change was achieved. We call this the black box problem (next slide). ASK: How many of you have heard of the black box problem before? Can anyone take a stab at explaining the black box problem?