1) Innovations for Poverty Action (IPA) is a non-profit research organization that works to reduce poverty through evidence-based policy. It has conducted over 300 randomized evaluations across 50 countries.
2) IPA Malawi has grown since starting work in 2009, now employing 17 staff and overseeing a $1.6 million annual budget. It focuses on education, health, enterprise development, and agriculture.
3) Randomized controlled trials are an important tool for impact evaluation but are not fully utilized in Malawi. IPA Malawi has completed over 11 research projects in areas like agriculture, rural development, and governance.
Fighting poverty with what works - The IPA mission across the globe by Sarah de Tournemire
1. FIGHTING POVERTY WITH WHAT WORKS (EVIDENCE): The case for impact evaluations
Thomas Chataghalala Munthali, PhD
ECAMA Inaugural Research Symposium:
Golden Peacock, 10th October 2014
2. Reflection
But test everything; hold fast what is good.
1 Thessalonians 5:21 (ESV)
3. Introducing IPA
•Innovations for Poverty Action (IPA) is an international non-profit research organization that has a strong local presence through its 14 country offices serving 51 countries.
•IPA works towards the goal of reducing poverty by ensuring that policy is based on scientifically proven evidence.
4. IPA’s VISION
More evidence, less poverty
IPA’s MISSION
To discover and promote effective solutions to global poverty problems.
5. IPA works on three core areas:
•Research
•Capacity building on randomized evaluations
•Policy outreach and scaling-up
6.
7. HOW ARE WE CURRENTLY ACHIEVING OUR GOALS?
CREATE EVIDENCE
•Randomized evaluations, with a focus on understanding why programs work or not
•Partnership with academics and implementing organizations
•Creating research infrastructure around the world
TURN EVIDENCE INTO ACTION
•Dissemination through policy-briefs and conferences
•Working closely with implementation partners throughout the evaluation
•Direct role in implementing successful programs at scale
8. WHAT WE HAVE ACHIEVED
CREATE EVIDENCE
•Over 150 Principal Investigators
•14 Country Offices
•300 number of randomized evaluations completed or underway
•More than 500 staff trained in implementing randomized evaluations across 50 countries
TURN EVIDENCE INTO ACTION
•Two programs scaled across countries, leading to 40 Mn children dewormed and thousands chlorine dispensers
•Several organizations changed their program based on research results.
•Thousands of practitioners attended conferences .
To see studies done by IPA in Malawi and beyond visit: www.poverty-action.org
9. WHERE WE WORK?
14 country Offices – planning to expanding to 20 in next five years.
11. Basic History
IPA started work in Malawi in 2009 but got registered in 2010. IPA-Malawi has grown over the years and now has long-term staffing of 17 with an annual project budget of over US$1.6million.
Over 11 research projects have been completed with considerable influence on policy especially at the decentralized levels. Most of the research work has been in private sector and enterprise development as well as agriculture. Notably, most of the research has this far been largely driven by PI research interests. Currently, IPA-Malawi is developing a research agenda that reflects PI interests as well as Government’s development priorities as defined in the Malawi Growth and Development Strategy (MGDS II: 2011-2016).
12. Priority sectors
IPA-Malawi now has a five year Strategic Plan (2014-2018) that focuses on four priority sectors:
i.Education (Especially basic education and early child development),
ii.Health (especially reproductive health),
iii.Enterprise development (especially SMEs), and
iv.Agriculture (especially climate smart and smallholder commercial farming).
13. Main Challenge with evidence-based planning
•Randomized Control Trials (RCTs) not fully known/appreciated/utilised by stakeholders as an important impact evaluation tool for measuring attribution of programmes to observed impacts.
18. Components of Program Evaluation
•What is the problem?
Needs Assessment
•How, in theory, does the program fix the problem?
Program Theory Assessment
•Does the program work as planned?
Process Evaluation
•Were its goals achieved? The magnitude?
Impact Evaluation
•Given magnitude and cost, how does it compare to alternatives?
Cost Effectiveness
19. Evaluation should usually be conducted:
A. Externally and independent from the implementers of the program being evaluated
B. Externally and closely integrated with program implementers
C. Internally
D. Don’t know
20. Impact evaluation methods
1. Non- or Quasi-Experimental Methods
a. Pre-Post
b.Simple Difference
c.Differences-in-Differences
d.Multivariate Regression
e.Statistical Matching
f.Interrupted Time Series
g.Instrumental Variables
h.Regression Discontinuity
21. Ice Breaker
For every complex question there is a simple answer -- and it's wrong.
-- H.L. Mencken
22. Impact evaluation methods Contd.
2. Randomized Experiments, also known as:
–Random Assignment Studies
–Randomized Field Trials
–Social Experiments
–Randomized Controlled Trials (RCTs)
–Randomized Controlled Experiments
24. What is a randomized experiment?
Start with simple case:
•Take a sample of program applicants
•Randomly assign them to either:
Treatment Group – is offered treatment
Control Group - not allowed to receive treatment (during the evaluation period)
25. Key advantage of randomized experiments
Because members of the groups (treatment and control) do not differ systematically at the outset of the experiment,
any difference that subsequently arises between them can be attributed to the program rather than to other factors.
25
27. How to measure impact?
•What would have happened in the absence of the program?
•Take the difference between
what happened (with the program) …and - what would have happened (without the program) = IMPACT of the program
28. Some responses Contd.
Access to obstetric care and distribution of reproductive health commodities such as medication and contraceptives may be seriously hampered after disasters caused by climate change such as floods, putting women at risk for unintended pregnancies, sexually transmitted infections, including HIV.
Women and girls are generally expected to care for the sick, particularly in times of disaster and environmental stress. This limits the time they have available for income generation.
29. Some of the research (impact evaluations) done in Malawi
Active Projects:
•Finance Sector and private sector development:
–Direct Deposits and Financial Literacy : measuring the changes in household financial behavior that are caused by promoting savings in the form of financial literacy trainings and the introduction of regular or labeled savings accounts
•PIs: Dean Yang, Jessica Goldberg, Lasse Brune
–Fingerprinting to reduce risky borrowing: aims at expanding the knowledge base about the impact of fingerprinting on repayment behavior and thus costs and risks of lending that microfinance organizations generally encounter in rural markets.
•PIs: Dean Yang, Jessica Goldberg, Xavier Gine
–Business Registration and Impact Evaluation Project (BRIE): Evaluating whether or not business registration improves enterprise performance.
• PIs: Francisco Campos and Marcus Goldstein
•Agriculture
–‘Making family farmers more productive’ – Fomento project: The goal of this project is to implement and evaluate a modified version of a capital and information transfer program for extremely poor farmers .
•PIs: Susan Godlonton, Kate Ambler, Allan de Brauw
30. Overview: IPA Malawi
Completed Projects:
•Agriculture
–Making Networks Work (MNW) : The Project aims at evaluating how best to leverage communication within networks to increase the spread of two productive but under- utilized agricultural technologies in Malawi (pit planting for maize and improved crop residue management).
•PIs: Lori Beaman, Jeremy Magruder, Paul Fatch
•Rural Development:
–Access to Transport in Rural Malawi: Assessing the socio-economic impact of access to public transport on the lives of those living in remote rural areas.
•PIs: Jessica Goldberg, Dean Yang, Rebecca Thornton
•Social protection:
–Public Works Programme: aims at examining the impact of the World Bank supported public works program (PWP) as an income promoting mechanism via its impact on agricultural productivity through increased access to yield-improving inputs and any impacts on labor markets.
•PIs: Jessica Goldberg, Kathleen Beegle, Emanuela Galasso
31. Completed projects Contd.
•Enterprise Development:
–Malawi Savings Project (GLASE): to measure impact of increasing access to saving services on business creation, survival and growth, as well as on farming productivity and returns
•PIs: Dupas, Karlan, Robinson
–Borrower Responses to Fingerprinting for Loan Enforcement in Malawi: Assessing ability of fingerprinting in improving borrowers’ repayment rates
•PIs: Dean Yang, Jessica Goldberg, Xavier Gine
–Reducing Barriers to Saving in Malawi: the impact of enhanced savings products on the use of different agricultural inputs, farm output, and overall well-being in rural farming households?
•PIs: Dean Yang, Jessica Goldberg, Xavier Gine
•Governance
–Domestic Election Observers: to assess the impact of domestic election observers on detecting and preventing electoral fraud.
•PIs: George Ofosu and Daniel Posner
32. Ice Breaker
According to a new survey, women say they feel more comfortable undressing in front of men than they do undressing in front of other women. They say that women are too judgmental, where, of course, men are just grateful.
-- Jay Leno
33. For More …
Visit: www.poverty-action.org/malawi
34. You need technical support?
•Partner with Innovations for Poverty Action (IPA) for impact evaluations on your project(s)!
•Remember we are not a team of consultants – won’t charge you professional fees!
35. Sister organizations
•The Abdul Latif Jameel Poverty Action Lab (J-PAL), based at the Massachusetts Institute of Technology (MIT) and the University of Cape Town (UCT), consists of a network of leading academics who use randomized impact evaluations to test the effectiveness of alternative ways to improve welfare.
–Runs free Executive Education course on randomized evaluations
www.povertyactionlab.org
•Evidence Action responsible for taking proven impact initiatives to scale – just successfully chlorine dispensers in Zomba.
www.evidenceaction.org
36. Reflection
Therefore, I will always remind you about these things, [since] you are well- grounded in the truth that you now have.
2 Peter 1:12 (God’s Word Translation)
