What is an evidence-based approach
•Télécharger en tant que PPTX, PDF•
0 j'aime•1,356 vues
This seminar will focus on how countries can establish a policy framework to enable effective local action using an evidence-based approach, choosing between different measures when resources are scarce. The latest evidence from OECD countries on ‘what works’ and ‘what doesn’t’ will be considered, with participants sharing their own experiences from their perspectives as policy makers, researchers, practitioners and social entrepreneurs.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à What is an evidence-based approach
Similaire à What is an evidence-based approach (20)
Plus de OECD CFE
Plus de OECD CFE (20)
Dernier
Dernier (20)
What is an evidence-based approach
- 1. WHAT IS AN EVIDENCE-BASED APPROACH? Jonathan Potter
- 2. What would have happened anyway? Impact Counterfactual Policy on Policy off Impact indicator Baseline Evaluation
- 3. Disentangling the impact - six steps Monitoring Step I Take up of program Step II Recipients’ opinions Step III Recipients’ views of the differences made by the program Evaluation Step IV Comparison of the performance of ‘assisted’ with ‘typical’ ventures Step V Comparison with ‘matched’ ventures Step VI Taking account of selection bias
- 4. The most accurate impact estimate – randomised control group
- 5. Example: Growing America Through Entrepreneurship (GATE) • Scheme for unemployed to establish new businesses organised by local Labour Offices • Participants randomly assigned to groups, either given support or not • Registration in the Labour Office prevented participants registering twice • Allowed a calculation of net benefit (USD 4,500 per person)
- 6. Quasi experiments – matching and correction for bias (propensity scores)
- 7. Example: Princes Youth Business Trust • Long running programme to support youth entrepreneurship • Opinions of participants were very positive. • Comparison of assisted and typical ventures showed that assisted ventures had higher survival rates and individual had better employment prospects • A full evaluation taking account of selection bias failed to find any statistically valid impact
- 9. Qualitative stakeholder evaluation • Less legitimate for impact estimation (lack of information among respondents, opportunity for bias) • More information on processes and potential improvements and more engagement from participants
- 10. Local role in evaluation? • The techniques are not dependent on spatial scale • All actors need evidence • Bear in mind proportionality • Learn from others
Notes de l'éditeur
- The importance of the counterfactual Some things will happen even if the policy is not applied and these are part of the impact (most probably they reduce it) The approach means not just creating a baseline and performing an evaluation, but also making an estimate of the counterfactual. This is not always easy.
- This isn’t to say that only step VI is any good! But other parts do not give the full picture
- An ideal situation would be to follow the “gold standard” of medical trials – RC T This requires random allocation of subjects to treatment or not. Note that it requires an excess of demand over supply and strict delineation between the two groups. May seem difficult but can be done, particularly with pilot projects and if the evaluation is incorporated into the design of the project
- Evaluation inherent in the design Participants allowed to use other services so we have an idea of displacement effects (if the scheme is not available, how would they substitute it) This is hard evidence for design of wider schemes.
- If random assignment is not possible then we can still use statistical methods to make estimates. There are a number of different approaches: this is just one of them. We find beneficiaries and non-beneficiaries with similar characteristics and compare what happened to them. Emphasise that this is not the same as comparing beneficiaries and non-beneficiaries since it tries to correct for selection bias (fact that non-beneficiaries are qualitatively different).
- Important to note that the more statistically valid evaluation showed less impact, and that the opinions of participants are not necessarily a good guide to the genuine impact of a programme.
- Commonly used in these investigations. Eliminates the problem of differences in fixed effects.