Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Gates fdn measuring impact presentation nov 30 2010
Similaire à Gates fdn measuring impact presentation nov 30 2010 (20)
Plus de Center for Social Sector Leadership
Plus de Center for Social Sector Leadership (20)
Dernier
Dernier (20)
Gates fdn measuring impact presentation nov 30 2010
- 1. Prove It: How the Gates Foundation Ties Strategy to Results Schwab Charitable Philanthropy Speaker Series Center for Nonprofit and Public Leadership Haas School of Business Jodi Nelson, Interim Director Emily Parker, Sr. Strategy Officer Impact Planning and Improvement September 30, 2010
- 2. What are your questions about strategy and measurement at the Bill & Melinda Gates Foundation? © 2010 Bill & Melinda Gates Foundation | 2
- 3. Today we would like to: Collect your questions Share our aspiration: is it about “proving” it? Share what we do in practice Share our big questions Ask for your ideas from other experiences and sectors © 2010 Bill & Melinda Gates Foundation | 3
- 4. Fast Facts about the Bill & Melinda Gates Foundation (1) Number of countries in which the foundation has grantees Source: Bill & Melinda Gates Foundation Fact Sheet, September 2010, gatesfoundation.org © 2010 Bill & Melinda Gates Foundation | 4
- 6. Focus on use of limited resources
- 7. 25 strategies across 3 broad sectors: global health, development, US education
- 8. Approach to strategy maturing
- 9. Actionable Measurement beginning to take shape
- 10. in learning mode—from inside and outside
- 12. Pay out requirement hard to meet
- 13. A few strategies (i.e., high school education, US libraries)
- 14. No consistent approach to strategy development or measurement
- 15. little guidance
- 16. let a thousand flowers bloom© 2010 Bill & Melinda Gates Foundation | 5
- 17. Imagine… You lead the Agricultural Development program at the Bill & Melinda Gates Foundation. You have to decide how the foundation should invest in agriculture to help alleviate poverty in Sub-Saharan Africa and parts of Asia. What will the foundation’s goals be? How will we achieve them? How will we know if they are achieved? You have a meeting with Bill & Melinda in six months and need to tell them your plan. What do you do? © 2010 Bill & Melinda Gates Foundation | 6
- 19. We are just one actor — governments, NGOs, other donors, private citizens are also trying to improve people’s lives
- 20. We are not on the “ground” — we are strategic donors, but still just donors.
- 21. Data can be hard to come by — we may know we need it, but collecting meaningful data is hard to do.© 2010 Bill & Melinda Gates Foundation | 7
- 22. So what do we do? Use tools of strategy and measurement to plan our work despite the complexity © 2010 Bill & Melinda Gates Foundation | 8
- 23. Our aspiration is to create feedback loops so we can plan, execute, measure, learn, adjust, plan … data & experience data & experience data & experience data & experience © 2010 Bill & Melinda Gates Foundation | 9
- 24. Let’s break it down…. © 2010 Bill & Melinda Gates Foundation | 10
- 26. Financial Services for the Poor
- 27. Global Libraries
- 28. Water, Sanitation, and Hygiene
- 30. Enteric Diseases and Diarrhea
- 31. Family Planning
- 32. HIV
- 33. Malaria
- 34. Maternal, Newborn, and Child Health
- 35. Neglected and Other Infectious Diseases
- 36. Nutrition
- 37. Pneumonia
- 39. Tobacco
- 40. Tuberculosis
- 42. Early Learning
- 45. U.S. Advocacy
- 48. Measurement and learning—coupled with systematic review, reflection, and intellectual dialogue—drives improvement.unrealized strategy intended strategy deliberate strategy realized strategy emergent strategy Figure created by Henry Mintzberg, strategy theorist. © 2010 Bill & Melinda Gates Foundation | 12
- 50. Strategy review: annual update for the CEO and co-chairs on progress against an approved strategy
- 51. Strategy refresh: re-examining an approved strategy with the expectation there will be a significant change in strategic direction
- 52. Once: after a period of learning
- 53. Every strategy, every year
- 54. Every three to five years© 2010 Bill & Melinda Gates Foundation | 13
- 55. Strategy development and refresh is an iterative, phased processwith progressive analyses from broad to narrow © 2010 Bill & Melinda Gates Foundation | 14
- 58. What must the world do?
- 59. what we will do (directly or through others) to help achieve the stated goal
- 60. What must we do?
- 61. How will our actions maximize leverage?
- 63. Limited grantmaking begins Strategy developed & approved--the sky’s the limit “Learning initiative” established Endless possibilities; limited resources ??? Strategy review—strategy is adjusted © 2010 Bill & Melinda Gates Foundation | 16
- 64. What about measurement? What is it anyway? Why is it so difficult to do well? What is our philosophy and approach? 17 © 2010 Bill & Melinda Gates Foundation |
- 66. Produce objective, relevant information on program or organization performance
- 67. Developing measurable indicators that can be tracked to assess progress
- 68. What is unfortunate about the word “measurement”
- 69. Not everything is about counting or “tracking”
- 70. Under emphasizes evaluation, mixed methods, asking and answering questions about how things happen and why — not just what happens © 2010 Bill & Melinda Gates Foundation | 18
- 72. Align our data requests with those of other funders
- 74. Leverage technology for data collection and viewing results© 2010 Bill & Melinda Gates Foundation | 19
- 76. Where do we weigh in on the philosophical debates?
- 77. The “gold standard” is use © 2010 Bill & Melinda Gates Foundation | 20
- 79. Tracking execution is good for accountability
- 80. But tough to do well but can describe change over time
- 82. Impact evaluation purpose: demonstrating effectiveness of a model/approach, good for replication, informing the field, testing assumptions to check strategy
- 83. Process evaluation purpose: to improve implementation/execution at key times
- 84. Participatory evaluation purpose: to empower participants with information to act
- 85. Developmental evaluation: to identify components of a model or approach in real time Getting useful data requires choosing the right methods or approaches to measurement given what you need to do with it © 2010 Bill & Melinda Gates Foundation | 21 Example of purpose driven measurement
- 87. © 2010 Bill & Melinda Gates Foundation | What’s the essence of the approach? Measure impact (change in people’s lives) to track collective progress toward targets, not to attribute impact to the foundation’s actions or investments. Strategy Measure key output and outcome results to track progress of the foundation’s work; use evaluation to inform decision making by testing assumptions, learning what works, how and why. Initiatives Selectively track results of grants at critical points for accountability; evaluate to learn about implementation of key investments; use impact evaluation when it seeks to answer a strategic question for an initiative not as an accountability tool. Grants 23
- 88. Measuring investment types Measurement Guidelines Specific to Investment Type Types of Investment Priority for evaluation resources Research Product Development Policy and Advocacy Model Development and Demonstration of Effectiveness Systems Change Delivery at Scale Measure desired outcomes, track progress, and focus on shorter-term feedback Monitor outputs and track process Track execution and coverage/reach Measure success or failure and extent of fit with target product profile Measure attribution where it is technically feasible and ethical Use multiple measurement methods to draw conclusions © 2010 Bill & Melinda Gates Foundation | 24
- 89. Our questions… How do we identify, articulate, and evaluate alternative paths to impact? How do we develop theories of change where evidence of what works is slim? How do we keep the bar high and hold people accountable for results, while leaving room for failure and learning? data & experience data & experience What results do we aggregate? What are the most promising interventions to evaluate? data & experience data & experience How do we adjust our strategies while maintaining continuity in our partnerships? © 2010 Bill & Melinda Gates Foundation | 25
- 90. Thank You © 2010 Bill & Melinda Gates Foundation. All Rights Reserved. Bill & Melinda Gates Foundation is a registered trademark in the United States and other countries.
Notes de l'éditeur
- Strategies must be developed, tracked and measured, reviewed, and adjusted based on data. This shows the strategy business processes (measurement is implied): create the strategy, review it annually, “refresh” it periodically when a major course change and fresh look is needed. **********Development – New area of work. After period of learning. Not very common. Coming up – Urban Poverty (increase income of slumdwellers by helping them organize and develop income opportunities e.g, wastepicking).Review – annual. Every strategy every year, with the CEO, presidents, co-chairs and team. Purpose – for the team to shareWhat’s working/what’s notChallengesLessonsCourse corrections (areas of adjustment)Critical issues they’d like feedback onRefresh. Would like to drill in here. Development and refresh are closely linked – nearly the same process. How does the foundation develop a new strategy, or refresh one that already exists? What process and set of analyses do teams follow? Note this is done every 3-5 years.
- These are analytical tools to facilitate reasoning, analysis, consideration of alternative solutions, help us understand the critical role of other players and clearly articulate our role. Ultimate goal - create realistic, but ambitious strategies that can be implemented and measured. Opportunity mapTool to open the thinking, find creative ideas, challenge assumptionsProblem may be segmented by geography, end beneficiary, disease state, etc.Typically a precursor to developing a TOC2. TOCIncludes an explicit goal statementShows all the critical “levers” and causal links to achieve the goal, not just what we will doHelps ground our actions in the big picture (what others must do)Should stand on its own (i.e., be self explanatory)Right level of granularity important 3. TOAMust overlay on the TOCMade up of initiatives and sub-initiativesAs a companion to or part of the TOA, important to also state what we won’t do, and whyWhen done well, reveals risksAlso – solution leverage -- How and to what extent will the choices we make position us to achieve scale?
- Start with an anecdote – real donor terms of reference for an evaluation in Northern Uganda. Community economic development program Few years, $10 million TOR read: evaluate impact program has had on economic development in the region, conflict prevention and welfare in the target communities. Time: 18 days. Shows how measurement can be difficult for many reasons – not just the sheer act of doing it, but understanding what it is, what it can and cant do, the kind of knowledge that can be produced by collecting data, the limits to that knowledge and reality on the ground Example of no common understanding of language, no purpose in mind probably – donor in this case – were they holding the organization accountable for conflict prevention in No U?
- We use these categories to help teams within the foundation to think about what results (output, outcomes: coverage), and how (with/without attribution; shorter term feedback) they should measure above the grant level – trying to identify their strategic intent for a cluster of grants, where you can and can’t add up….
- - Putting it all back together – the theory to the practice – we still have more questions than answers. On Develop/refresh strategy: How do we identify, articulate and evaluate alternative paths to impact? On Track Progress: What results do we aggregate? What are the most promising interventions to evaluate? On Review strategy: How do we adjust our strategies while maintaining continuity in our partnerships? On Learn, Adapt, Improve: How do we keep the bar high and hold people accountable for results, while leaving room for failure and learning?