CORE Group Fall Meeting 2010. The Program Assessment Guide, Structuring Contextual Knowledge and Experience to Improve the Design, Delivery and Effectiveness of Nutrition Interventions.

1. The Program Assessment GuideStructuring Contextual Knowledge and Experience to Improve the Design, Delivery and Effectiveness of Nutrition Interventions David Pelletier Division of Nutritional Sciences Cornell University CORE Group Fall Meeting September 13, 2010 Washington, DC

2. Acknowledgments Collaborators on the MN Tool Project Allison Corsi, LesliHoey Cornell University Robin Houston, Global Nutrition Consultant Funders and Facilitators of the MN Tool Project The World Bank Micronutrient Initiative Centers for Disease Control UNICEF GAIN A2Z (USAID and Gates Foundation)

3. Genesis of This Project Micronutrient Forum (May 2009) From Evidence to Large Scale Programs The Innocenti Process (Sept 2008) Country implementers, donors, researchers Review of evidence for effectiveness at scale Identification of overarching issues needing attention

4. Innocenti: Overarching Issues Key stakeholders share common MN goals but lack the leadership to coordinate and for priority setting, advocacy and action Stakeholder groups within the MN community do not communicate effectively with one another; Stakeholders have misaligned and often competing priorities and approaches at both global and country levels. The MN community has not adequately engaged with broader nutrition, health, or development initiatives; The MN community has not harnessed the full potential of private sector resources, expertise, and delivery mechanisms (continued) Source: Micronutrient Programs: What Works and What Needs More Work? A Report of the 2008 Innocenti Process. Micronutrient Forum, Wash. DC.

5. Innocenti: Overarching Issues Country teams lack guidance and authority to assess, analyze and take actions needed to fit local contexts Weak program monitoring, evaluation and documentation hinders program effectiveness, learning, advocacy and accountability Limited funding for programs Limited funding for implementation research hinders our efforts to strengthen the design, management, implementation, evaluation, and financing of MN programs at scale

6. Relevance to the Current Interest in Nutrition-Ag Linkages Many of the overarching issues are the same There is even greater difference in disciplinary lenses, availability and standards of evidence, communication barriers, intervention complexity, implementation systems, etc. are greater The narrative ‘we know what works’ has different evidence beneath it, but we have similar pressures and incentives to pretend we do know, implement at scale and show results quickly Therefore, we have the same need to: adapt interventions, evidence and experience to new contexts question assumptions during intervention design identify and address bottlenecks during implementation assess and document progress and impacts

8. Locating Evidence, Knowledge and Experience Within the Policy Process Detailed intervention design and planning (policy implementation) Identifying problems (problem definition) Choosing solutions (policy formulation) Surveys Surveillance Reconnaissance, etc Efficacy Effectiveness Cost Knowledge of and experience in the local context * Evaluation Implementation Operations research, monitoring and QA during implementation Outcomes, impacts and related factors * Context includes epidemiology, implementation capacities, social acceptability, resources, competing priorities, policy preferences of actors, etc.

11. Shared understanding, commitment, ownership, motivation & capacity to advance the micronutrient agenda

12. Links with the broader nutrition and health agendas in the country2. Operations Research Agenda 3. Issues for Inclusion in M&E 4. Strategic Plan to Build Support, Capacity & Sustainability

13. Program Assessment Guide Laying the Groundwork Module 1: Clarifying the Problem and the Solution Module 2: Goals and Associated Values Module 3: Delivery Systems Module 4: Hard to Reach Populations Module 5: People, Roles and Responsibilities

14. Program Assessment Guide Building or Strengthening the Intervention Module 6: Needs, Inputs, Activities and System Changes Module 7: Action Planning Building Support Systems and the Enabling Environment Module 8: Monitoring, Evaluating and Quality Improvement Module 9: Organizing, Leading and Managing

15. Some Distinctive Features of the PAG Approach Link nutrition interventions to the broader health (and agriculture and other) agendas (Modules 1,2) Delivery systems as social systems (Module 3) Five universal needs and local program theory (Module 6) Clear definition of roles and responsibilities (Modules 5,7) and responsibility for follow-up (Module 9) Systematize and stress-test contextual knowledge and experience (Module 6) Build capacity for intervention planning, assessment and improvement

16. Creating Local Program Theory

17. A Common Example: Training Health Workers to Counsel Mothers The Goal: Improved Feeding Practices Train Workers Improved Counseling The Common Experience: Little Change In Feeding Practices Train Workers Little Change In Counseling Why?

18. Nutrition Example: BCC IYCF Component Improved Feeding Practices for Young Children Training of Health workers To do nutrition counseling “Desired Outcome” “Input” Building a Program Model: Specify Inputs and Desired Outcomes for Each Component

19. Caretakers Understand & Internalize the Knowledge & skills Training of Health workers To do nutrition counseling Caretakers Improve Feeding Practices Health workers Internalize the Knowledge & skills Health workers Conduct good Quality counseling “Desired Outcome” “Causal Pathway” (often assumed and not made explicit) “Input” Building a Program Model: Specify the Causal Pathway

23. Module 3: Delivery systems as social systems Significant Others (barriers and enablers) A D H J G F E B C I Household & Caregiver Functionaries (barriers and enablers) Clinic Community District Regional National

24. The PAG Approach to Creating Local Program Theory Five Universal Needs Awareness Knowledge, Information, Skill Motivation/ Commitment Resources Support from Others

25. Module 6: What does each functionary and significant other NEED in order to fulfill their role/responsibility of the Sprinkles program (on a daily basis) and what inputs, activities and system changes are required to ensure their needs are met?

26. Systematizing and stress-testing contextual knowledge and experience (Module 6) Proposers Challengers

27. Systematizing and stress-testing contextual knowledge and experience (Module 6) Outputs Proposers Challengers Actions to address barriers and enablers in the Action Plan Questions for Operations Research Issues to track in monitoring and QA

28. Experiences from Kyrgyzstan and Bolivia 1. Sponsorship, Responsibility, Commitment, Follow-up 2. Timing 3. Preparation 4. Participants 5. Contextuality (selection and sequencing) 6. Time constraints 7. Facilitation Before the workshop During the workshop

29. Adaptation to Ag-Nutrition Projects The initial focus is a multi-dimensional problem rather than a single solution Use the UNICEF Conceptual Framework as a guide Consider food, health, care simultaneously - determinants and solutions The primal need is to ensure that all three are addressed Conduct the five needs analysis on the “delivery systems” for food and health and care interventions

31. Women’s versus men’s control of income

32. Effect of crop choice on men’s versus women’s control

33. Effect of crop choice on men’s vs women’s labor in ag

34. Effect of crop choice on sales vs retention for own consumption

35. Acceptability of foods for feeding young children

36. Efficacy of foods for meeting nutritional needs of young children

37. Requirements for adopting and growing

38. Requirements for preparing, using and feeding

39. Requirements for sustaining over time (seasons and years)

41. Thank You

42. The Five Needs in Relation to the Broad Categories of Behavioral Determinants

43. Importance of an Explicit Program Model and Assumptions: Results of a Meta-Analysis Fixsen, D.L., et al., Implementation research: A synthesis of the literature. 2005, National Implementation Research Network: Tampa, FL.

44. Importance of an Explicit Program Model and Assumptions: Results of a Meta-Analysis Fixsen, D.L., et al., Implementation research: A synthesis of the literature. 2005, National Implementation Research Network: Tampa, FL.

45. Importance of an Explicit Program Model and Assumptions: Results of a Meta-Analysis Fixsen, D.L., et al., Implementation research: A synthesis of the literature. 2005, National Implementation Research Network: Tampa, FL.

46. Simple (Faulty) Program Model Lectures & Discussion & Demonstrations Teacher’s Practices More Complete (Effective) Program Model Teachers’ Knowledge Teachers’ Practices Teachers’ Skill Lectures & Discussion & Demonstrations Coaching in Classroom Practice & Feedback

48. No materials

49. Inattentive (hungry)

50. Student absenteeism

51. Teacher absenteeism

52. Teacher moraleStudent Learning

53. 3.3.2 Improved Nutritional Status Prevent undernutrition through community-based programs that increase household resilience and are built on the proven effectiveness of community workers, who are often the best change agents to deliver nutrition education. These programs should be based on an analysis of the specific social and behavioral context in each location and promote nutrition messages aimed at improving key evidence-based practices including proper maternal nutrition, exclusive breastfeeding of infants under six months, appropriate introduction of complementary foods, and promotion of adequate water, sanitation, and hygiene practices. » » Improve diet quality and diversity to prevent nutritional deficiencies and vulnerability to disease caused by low-quality diets. This can be achieved by strengthening agriculture and nutrition linkages through multiple approaches, such as increasing household production of nutritious foods, empowering women with access to income, and educating women about food utilization and nutrition as a means to improving the health and wellbeing of children in the household. We will invest in improving nutrition throughout the value chain, including research to improve the nutrient value of staple food sources and post-harvest preservation and processing techniques that increase the nutrient content of processed staple foods.

54. 3.3.2 Improved Nutritional Status » Improve delivery of nutrition services by health systems linked to community-based programs. Key nutrition services include community management of acute malnutrition (CMAM) and micronutrient supplementation. The CMAM approach ensures timely detection and referral of cases of severe acute undernutrition in the community to medical treatment by making services available at decentralized treatment points and through community outreach and mobilization. To save lives and reduce morbidity in the short- and medium-term, food-based approaches and prevention programs should be coupled with targeted micronutrient supplementation programs, including vitamin A supplementation for young children and iron folic acid supplementation for pregnant women. Our investments will include strengthening and integrating national and decentralized health and social systems responsible for planning, budgeting, and cost analysis of nutrition service delivery.

55. The preceding slides were presented at the CORE Group 2010 Fall Meeting Washington, DC To see similar presentations, please visit: www.coregroup.org/resources/meetingreports