Presentation by Arianna Legovini, at the Overlooked Links in the Results Chain presentation.

1. Examples from Prospective Impact Evaluations Arianna Legovini Head, Development Impact Evaluation Initiative (DIME) World Bank

2. Results chains is our working hypothesis but demand response is uncertain 2

3. Questions 3

4. Impact Evaluation Tracks mean outcomes over time in the treatment group relative to the control group Compares what DID happen with what WOULD HAVE happened (counterfactual) Identifies causal output-outcome link separately from the effect of other time-varying factors

5. Babies & Bath Water Uganda Community-Based Nutrition Failed project (HU rating) Project ran into financial difficulties Parliament negative reaction Intervention stopped …but… Strong impact evaluation results Children in treatment scored half a standard deviation better than children in the control Presidency asked to take a second look at the evaluation: saving the baby?

6. Doing BETTER 6

7. Does Financial Literacy Work? Financial literacy found not effective in changing behavior Large financial literacy pilot in Brazil shows strong results! High school students have: higher financial proficiency -- 7% increase financial autonomy -- 6% increase intention to save -- 4% increase students who save some of their income -- 6 percentage points increase Why the difference? What are we measuring? Small researcher led initiatives (half day workshops) vs Large coalition of financial sector actors working with the education sector since 2006 to develop state of the art curriculum What works?

8. What are the returns to reform? Minas Gerais, Brazil Large reform simplifying registration procedures 29 one stop shops 3 levels of government and 7 separate agencies 98% of small firms still informal How to capitalize on the reform? Experiment with various hypotheses Firm don’t know Registration and formality too costly Randomly assign communication, incentives and punitive enforcement to firms to understand If communication and incentives are enough or Tax regime and accounting requirements need to be changed Make it work!

9. What are the returns to infrastructure? It depends… Rate of return to irrigation depends on: Rate of technology adoption among farmers Use of chlorine for drinking water Institutional arrangements that have water repayment rates to pay for operations & maintenance Ethiopia, Mozambique: experimenting with technology adoption and payment systems To secure high returns to irrigation! Increase returns to investment!

10. Supply chains in Zambia Zambia suffered from high stock-out rates of essential drugs (up to 44%) Randomized pilot compared two supply chains to deliver drugs to facilities The system that seals packages at central facilities and ships them to them reduces stock out to less than 10% Scaled up projections (on malaria alone): Cure 312,014 cases and 8,433 severe cases of malaria per yr Avert 16,600 Under 5 and 2,200 adult deaths by 2015 Reduce child and adult malaria mortality by 21% and 25% Compare alternatives, improve results!

11. HIV Prevention in Senegal :Is peer-to-peer better than traditional social mobilization? National Aids control agency decided to replace social mobilization with peer-to-peer mentoring Peer-to-peer was rolled out to a random set of health districts IE found that the approaches are complementary Peer mentoring doubles who get tested, take pre-test counseling and collect test results Social mobilization is more effective addressing partners of HIV-positive individuals who get tested Policy impact Senegal now implements calls for community-based organizations to deliver prevention based on combining social mobilization and peer-to-peer counseling The new approach doubles the number of people attended

12. contacts Check results at www.worldbank.org/dime alegovini@worldbank.org