MEASURE Evaluation Phase III Year 3 Annual Report by Sian Curtis, Project Director.

1. MEASURE Evaluation Phase III
Year 3 Annual Report
Siân Curtis
November 15 2011

2. Activity Objective
 Improved collection,
analysis, and
presentation of data to
promote better use of
data in planning,
policymaking,
managing, monitoring,
and evaluating
population, health and
nutrition programs

3. Results 1 and 6: Data Demand and Use

4. Elements of Data Demand and Use
Strategy
 Development of individual capacity to apply data
demand and use concepts and tools
 Development, application, and evaluation of data
demand and use tools
 Global leadership and collaboration to raise
awareness of the benefits of using data to
improve health programs and policies

5. Result 1 – Increased user demand for quality
information, methods and tools for decision making
Indicat Indicator Definition Year 2 Year 3
or
1.1 Instances where country organizations or programs 2 7
request and/or secure non-USAID funding for M&E or HIS
staff and/or activities as a result of MEASURE Evaluation
activities
1.2 Number of MEASURE Evaluation core-funded regional 188 155
training or workshop event participants who are funded by
sources other than MEASURE Evaluation
1.3 Number of MEASURE Evaluation core-funded regional 205 232
training or workshop event applicants who are funded by
sources other than MEASURE Evaluation
1.4 Number of MEASURE Evaluation core-funded regional 563 604
training or workshop event applicants

6. Non-USAID Funding for HIS and
M&E Activities
 Two regional M&E units funded by the Côte
D’Ivoire Ministry of Health established
 Health sector HIV data review meetings,
funded by the Nigeria federal government
and UNAIDS
 West African Monetary and Economic Union
and WAHO agree to support participants in
RHIS workshops at CESAG in 2011-2013

7. Result 6 - Increased facilitation of use
of health, population and nutrition data
Indicator Indicator Definition Year 2 Year 3
6.1 Instances of documented/reported instances in which 3 7
information produced through MEASURE Evaluation
activities in decision-making: Program, Policy and
Advocacy
6.2 Instances of the implementation of a MEASURE Evaluation 9 13
tool developed to facilitate the use of data
6.3 Instances of MEASURE Evaluation tools or strategies 1 7
designed to facilitate the use of data piloted or developed
for a specific application

8. Instances of Data Use
 Results from a 2006 PLACE study informed design of OVC
programs in Zimbabwe. A follow-up PLACE study (conducted
independently by our local partner in 2010) found reduced
attendance at high-risk sites and increased school attendance.
 Outcomes from first LQAS survey in Kenya used to target
training of community health workers on maternal health issues
and water sanitation in districts with poorer indicators.
 Nyamasheke district in Rwanda independently developed its
own strategic plan based on data gathered through the HIV
M&E system

9. Global Leadership: New Data
Demand and Use Curricula
 Data demand and use concepts and tools
 Integrating data demand and use into a M&E training
course
 Using data to improve service delivery
 Introduction to basic data analysis and interpretation for
health programs
 Data use for program managers

10. Result 2: Individual and Institutional
Capacity Building

11. Elements of Capacity Building
Strategy
 Human resources development in M&E through
training, distance learning, and communities of
practice
 Consolidation of regional training partners as
M&E centers of reference
 Capacity building through technical assistance
 Participation in global and regional TCB working
groups

12. Result 2 – Increased in-country individual and institutional
technical/managerial capacity and resources for the identification
of data needs and the collection, analysis and communication of
appropriate information to meet those needs
Indicator Indicator Definition Year 2 Year 3
2.1 Instances of regional, national or sub-national institutions 5 7
assisted in M&E/HIS strengthening by MEASURE
Evaluation that demonstrated increased capacity to
independently carry out M&E/HIS activities
2.2 Instances of Institutional or Program M&E Capacity 8 10
Building Plans developed with assistance from MEASURE
Evaluation
2.3 Number and percentage of MEASURE Evaluation core- 6 mo - 40, 95% 6 mo – 69, 96%
funded, regional training participants who are engaged in 1 year – 74, 96% 1 year – 54, 96%
M&E activities and who report using the skills learned 6
months or 1 year after training
2.4 # and % of MEASURE Evaluation core-funded, Masters 6 mo - 6, 100% 6 mo – 2, 100%
students who are engaged in M&E activities and who 1 year – 3, 100% 1 year – 3,75%
report using the skills learned 6 months or 1 year after
training
2.5 % of VLDP teams providing evidence of improved 7, 58% 17, 77%
performance at 6 months post-program which they
attribute to some degree to their VLDP experience

13. Result 2 – Increased in-country individual and institutional
technical/managerial capacity and resources for the identification
of data needs and the collection, analysis and communication of
appropriate information to meet those needs
Indicat Indicator Definition Year 2 Year 3
or
2.6 Number of training or workshop events supported by 67 119
MEASURE Evaluation
2.7 Number of participants who complete a MEASURE 1,934 4,972
Evaluation training or workshop
2.8 Number of people who complete MENTOR online courses 712 538
2.9 Instances of new MEASURE Evaluation tools, 5 1
methodologies, technical guides, curricula or indices centered
on capacity building and measure for M&E that have been
validated or otherwise finalized for general use
2.10 Instances of MEASURE Evaluation curricula, tools, 4 4
methodologies, technical guides or indices centered on
capacity building or capacity measurement for M&E that have
been piloted or developed for a specific application

14. Capacity Building Through TA
 Leadership development program with 4 ministries in
Côte d’Ivoire led to production of compendium of
harmonized HIV/AIDS indicators
 MOH in Rwanda delivers RDQA training
independently.
 Two universities in Nigeria offer M&E workshops for
the first time
 Three regional training partners offer M&E courses
independently for the first time (AAU, INSP, PHFI)

15. Application of Skills Learned
 “Currently all M&E plans for  “…Upon returning from my
the implementing partners training I made a
have been approved. The presentation to my
elements learned during the organization about
M&E training helped me preparation of M&E plans.
guide the partners in the My organization was
development of good M&E impressed and immediately
plans. “ adopted the use of M&E
plans in all its future
Participant in 2010 malaria
activities…”
M&E workshop in Ghana.
Participant, 2010 PHN M&E
workshop at AAU

16. Result 3: Coordination and Collaboration

17. Result 3 – Increased collaboration and coordination
in efforts to obtain and communicate health,
population and nutrition data in areas of mutual
interest
Indicat Indicator Definition Year 2 Year 3
or
3.1 Instances of outputs produced by international or bilateral 7 10
communities of practice or coordinating mechanisms in which
MEASURE Evaluation had a leadership role
3.2 Number of global, USG or regional coordinating mechanisms 23 45
in which MEASURE Evaluation has a high level of
participation
3.3 Number of members in MEASURE Evaluation-moderated 4,458 5,503
online communities of practice
3.4 Mean number of new threads started on a MEASURE n/a 5.2
Evaluation-moderated community of practice per quarter
3.5 Number of MEASURE Evaluation-moderated electronic n/a 18
community of practices forums in the last year

18. Collaborative Outputs
 Guidelines for Measuring Malaria Related Mortality (RBM)
 Gender and HIV indicator for post-UNGASS reporting
(USG, UN, GFATM, others).
 Operational Guidelines for M&E of HIV Programmes for
People Who Inject Drugs (UNAIDS MERG)
 Harmonized nutrition and HIV/AIDS indicators (PEPFAR
food and nutrition TWG)
 Four PEPFAR e-learning courses (USG)
 Regional ICD-10 coding training workshop (RELACSIS)

19. Result 4: Improved Information
Gathering

20. Result 4 – Improved design and implementation of
the information gathering process including tools,
methodologies and technical guidance to meet
user’s needs
Indicat Indicator Definition Year 2 Year 3
or
4.1 Instances of M&E or RHIS systems with demonstrated 6 5
improvement in quality or coverage of data
4.2 Instances of key reference documents for organizations 29 34
and/or programs developed with assistance from MEASURE
Evaluation
4.3 Instances of new MEASURE Evaluation tools, protocols, 1 0
procedures, methodologies, guides or indices designed to
improve the data gathering process with that have been
validated or otherwise finalized for general use
4.4 Instances of the implementation of a MEASURE Evaluation 57 59
tool developed to improve the information gathering process
4.5 Instances MEASURE Evaluation tools or strategies 0 6
designed to improve the data gathering process piloted or
developed for a specific application

21. Improved M&E/HIS Systems
 Scale up of redesigned HMIS led to increased reporting in
SNNPR in Ethiopia
 New HMIS registers and patient tracking systems developed
and tested in select MOF facilities in Mozambique
 Microsoft Excel reporting system improvements streamline
reporting and data aggregation for PEPFAR partners in
Mozambique
 Quality and utility of Rwanda national HIV database
(TRACNET) improved
 Standard indicators and data collection tools are established in
implemented for community-based HIV programs in Rwanda

22. Tools to Improve Data Collection Piloted of
Developed for a Specific Application
 PLACE mapping module tested in Angola
 LLIN Campaign Dashboard implemented with the
Kenya Division of Malaria Control
 PLACE Lite protocol tested in Tanzania
 Iringa Prevention Mapping Protocol developed for
Tanzania
 PEPFAR indicator database developed for Vietnam

23. Result 5: Increased Availability of Data,
Analyses, Methods, and Tools

24. Result 5 – Increased availability of
population, health and nutrition data,
analysis, methods and tools
Indicator Indicator Definition Year 2 Year 3
5.1 Instances of key actionable findings, experiences and/or 16 17
lessons learned from data analysis, methods or tools
developed by MEASURE Evaluation that are available to
decision makers and/or stakeholders
5.2 Instances of non-MEASURE Evaluation staff that explicitly n/a n/a
report knowledge gained from selected MEASURE
Evaluation products and services
5.3 Number of MEASURE Evaluation electronic and print 38+ 48
publications
5.4 Number of articles published in peer-reviewed journals 16 13
authored by MEASURE Evaluation staff about MEASURE
work
5.5 Number of MEASURE Evaluation print publications 4,499 1,196
distributed in response to requests for materials through
MEASURE Evaluation’s website

25. Result 5 – Increased availability of
population, health and nutrition data,
analysis, methods and tools
Indicator Indicator Definition Year 2 Year 3
5.6 Number of MEASURE Evaluation digital documents 185,777 261,763
downloaded from the MEASURE Evaluation website by
non-MEASURE Evaluation users
5.7 Number of external meetings, conferences and public 5+ 45
events in which MEASURE Evaluation staff present tools,
methods and findings developed by MEASURE Evaluation
5.8 Number of organizations posting MEASURE Evaluation 34 54
publications
5.9 Number of MEASURE Evaluation publications posted on n/a n/a
non-MEASURE Evaluation websites
5.10 Number of non-UNC host computers visiting MEASURE 104,751 125,954
Evaluation website
5.11 Number of new subscribers to Monitor e-newsletter over 614 4,658
the past 12 months

26. Actionable Findings
 Bangladesh Maternal Mortality and Health care Survey finds
that the maternal mortality ratio has declined by 40% since 2001.
 China PLACE-RDS comparison study finds that the proportion of
sex workers in Liuzhou with a positive syphilis test varied base
don the sampling method used with the venue-based estimate
higher (24%) than the RDS estimate (8.5%)
 Assessments of M&E systems in Nigeria and St. Vincent and
the Grenadines provide multiple findings to inform M&E system
interventions
 LAC Regional Initiative on Health Information Systems
Strengthening 2005-2012 report documents multiple lessons for
other regions

27. Top 5 Publications Downloaded
Publication Downloads
A review of constraints to using data for decision-making: 2,001
recommendations to inform the design of interventions
Fourth international RHINO workshop: Measuring and 1,757
improving RHIS performance
M&E fundamentals: A self-guided mini-course 1,569
Community-level program information reporting for HIV/AIDS 1,330
programs. Module 1: Illustrative program indicators, data
collection tools and indicator reference sheers for prevention,
HBC, and OVC programs
Zambia sexual behavior survey 2009 1,290

28. Contributing to GHI

29. GHI: Woman-Centered Programming
 Provide M&E capacity building and training to improve gender data
use
 Training module on M&E of gender and health programs developed and
tested in Nigeria, India, and Senegal.
 Improve the use of existing data and collection of new data that
effectively incorporates information on gender-related effects of
programs and policies
 The Seven Steps to Use of Routine Information to Improve HIV/AIDS
Programs includes explanation of why gender is important to include in
HIV-related routine health information and includes a case study
illustrating a gender-based analysis

30. GHI: Woman-Centered Programming
 Conduct innovative research and evaluation to improve the evidence
demonstrating the effects of gender and health programming and policy
 Analysis of the influence of gender measures on MCH in Africa finds
significant associations between gender equity measures and various MCH
outcomes in most countries (Y4)
 Collaborate in global networks and initiatives to promote and improve
the knowledge base of gender M&E
 Gender and HIV indicator included in post-UNGASS reporting
 Gender and health data and statistics annotated resource guide (Y4).
 Ongoing: compendium on harmonized gender and HIV/AIDS indicators at
national and programmatic levels.

31. GHI: Country Ownership and Sustainability
Over past 2 years:
 Strengthened national systems reported in 7
countries (indicators 2.1 and 4.1)
 Multiple examples of key reference documents to
support national RHIS/M&E systems (indicator 4.2)
 Examples of country efforts to fund RHIS/M&E
activities in 5 countries (indicator 1.1)
 Capacity building plans developed in 6 countries and
with regional training partners (Indicator 2.2)

32. The Next Two Years
Use results to date to determine
 Identify gaps: What do we need to achieve in the
rest of the project?
 Set priorities: How are we going to achieve these
results?
 Plan for evidence: How are we going to measure
that we have done what we wanted to do?

33. Concluding Remarks
 Good progress documented in all result areas
 Results used extensively in project
communications and technical presentations
 Need to focus on achieving remaining results for
next two annual reports
 Ongoing work to improve project M&E and
streamline annual report process

35. MEASURE Evaluation is funded by the U.S. Agency for
International Development and is implemented by the
Carolina Population Center at the University of North
Carolina at Chapel Hill in partnership with Futures Group
International, ICF Macro, John Snow, Inc., Management
Sciences for Health, and Tulane University. The views
expressed in this presentation do not necessarily reflect
the views of USAID or the United States Government.