More and more funders are only supporting evidence based models. What does this mean and how can your organization implement an evidence based model? A panel of experts has been assembled to answer any and all questions your organization has on how it can move from outputs to outcomes to impact. Brainstorm real strategies that your organization can implement and meet other organizations who are also moving towards an evidence based model.

1. Measuring Change Moving from Outputs to Outcomes to Impact 10:00am – 11:15am Facilitator: Lisa Kuzma, Richard King Mellon Foundation Speakers: Mike Bangser, MDRC / Junlei Li, Office of Child Development / Art Maxwell and Pam Meadowcroft, Meadowcroft & Associates / Tim Weidemann, Rondout Consulting

2. Share your thoughts! Text “Summit” to 57682 with your feedback and thoughts on today’s event! Share your thoughts on Twitter by adding #GPNPSummit to your tweets!

3. Measuring Change – Moving From Outputs to Outcomes to Impacts

4. The Context [We are] trying to put much more emphasis on evidence-based policy decisions here at OMB. Wherever possible, we should design new initiatives to build rigorous data about what works and then act on evidence that emerges — expanding the approaches that work best, fine tuning the ones that get mixed results, and shutting down those that are failing. -- Peter Orszag, former Director of the Office of Management and Budget 4

5. Key Assumptions Agreement on what it means to be “effective” Reliable evidence on which programs are effective, using the proper measures Reliable evidence on the core elements that contribute to effectiveness Documentation and communication of these elements Understanding of how the elements can or should be adapted to local conditions Providers have sufficient resources and capacity to replicate evidence-based practices 5

6. Key Distinctions Outputs (typically, units of service) Gross or absolute outcomes(e.g., employment, learning, or behavioral gains for those who are served) Net impacts(how the outcomes differ from what would have occurred without the intervention) 6

7. 5/11/2011 70 80 40 20 Job Program Group Control Group Outcomes vs. Impacts: Which program is more effective? 100 100 90 90 80 80 70 70 60 60 Percent Placed In A Job Percent Placed In A Job 50 50 40 40 30 30 20 20 10 10 0 0 A B 7

8. Positive Outcomes, But for Whom?

9. Key Challenges in Replication Rigorous evidence of what works The right level of model specification Proper “fit” between the replicated program model and the local population being served Other factors that may affect results (staffing, funding levels, etc.) Finding useful “leading indicators” for program staff Data issues

10. Implications for Planning and Practice Need a realistic meeting of the minds on the roles of policymakers, funders, researchers , and providers The goal is fidelity to the right things, at scale and at reasonable cost Need appropriate support for replication and organizational infrastructure (dissemination of research findings , tools and training, data systems, etc.) Beware of potential distortions in interpreting outcomes – especially the influence of participant selection Recognize that moving from outputs to outcomes to impacts is an evolutionary process

11. For More Information Michael Bangser mike.bbangser@mdrc.org 860-586-8577 www.mdrc.org

12. Thank you from the Greater Pittsburgh Nonprofit Partnership!

13. Workshop Evaluation Text “Impact” to 57682 with your answers to the following questions: Please rate the overall value of this workshop E – Excellent G – Good P – Poor F – Fair Did you learn anything that you will apply at your own organization? Y – Yes N - No Please text other comments and feedback. Submit by hitting “send!” You will receive an auto-reply from the GPNP. Sample text: “Impact E Y I’ve learned great tips today.”