Presented by Shannon Salentine, Verne Kemerer, and Jessica Fehringer at the 2016 AEA conference.

1. Measuring National M&E System
Strengthening in Nigeria:
Application of the Most Significant
Change Method
Shannon Salentine
Verne Kemerer
Jessica Fehringer
MEASURE Evaluation
October 29, 2016
American Evaluation Association
Conference

2. • The strengthening of M&E systems remains a
priority focus for national governments and
the donor community
• Progress made in developing global
indicator standards and putting in place key
system components
• Little in the published literature
• Available evidence often anecdotal,
fragmented, and difficult to find
• Limited objective, evidence-based
recommendations
Background

3. Rationale for Approach
• Describe how multiple interventions may
have together contributed to changes in
M&E systems
• Lack of baseline data, control, or
comparison groups
• Absence of a well-defined strategy that
cuts across all development partners

4. Objectives
1. Produce evidence of how M&E system
was strengthened from 2007–2012
2. Develop country-level case study to
identify M&E system progress and
strengths and identify existing needs for
strengthening M&E systems

5. Study Design
1. Topical and country-specific literature
review
2. Most significant change (MSC) workshop
with key stakeholders
3. Key informant interviews (KIIs)
4. Data analysis of secondary data sources
to compile indicators

6. MSC: Stakeholder Workshop
Steps:
1. Stakeholder workshop
to identify MSCs
• Self-assessment in
small groups using
adapted 12
Components tool
• Self-select into group
• Discussion, agree on
response, evidence
• Plenary report
Figure 1. 12 Components M&E
Systems Strengthening tool

7. MSC: KIIs and Verification
Steps (continued):
2. 18 key informant interviews with
host-country agencies to explore
their perspectives on MSCs
3. Verification workshop

8. MSC Workshop Results
• Harmonization of ART, PMTCT and HCT
indicators and data collection tools (2012)
• Improved data quality due to
harmonization and training
• States now analyze and use data
• Surveys and surveillance routinely conducted
and used for the development of M&E plan
and international reporting
• HIV National Resource Center (NACA) houses
the inventory of HIV related surveys and is
updated regularly
• 3 DHIS in use: the National Health
Management Information System
(NHMIS), PEPFAR DHIS and NACA DHIS
(DHIS 2.0)
• The integration of the paper-based
NHMIS into the eNNRIMS (2011)
• Joint bi-annual DQA exercise
using the RDQA Tool
• Research committees exist and meet
regularly
• NACA maintains up-to-date research
agenda
• Evaluation and research findings are
regularly disseminated and used

9. MSC Workshop Results:
MSCs identified in workshop:
1. Indicators are harmonized to improve reporting
processes.
2. Data quality has improved as a result of training efforts
and the harmonization process.
3. Data are now analyzed and used by states.
4. Surveys and surveillance are used in the development
of an M&E plan and for international reporting.
5. Information systems evolved from paper-based stand-
alone formats to an integrated electronic system.
6. Evaluation and research are overseen by functional
ethics committees.
Summary

10. MSC KIIs and Verification
Real change was seen in the following ways:
1. Indicators are harmonized to improve reporting
processes.
2. Data quality has improved as a result of training
efforts and the harmonization process.
3. Data are now analyzed and used by states.
4. Surveys and surveillance are used in the
development of an M&E plan and for international
reporting.
5. Information systems evolved from paper-based
stand-alone formats to an integrated electronic
system.
6. Evaluation and research are overseen by functional
ethics committees.

11. MSC Pros and Cons
• Raised interest of stakeholders at
beginning
• Defined domains of change
• Helpful to identify successes in
participatory manner
• Negative: Potential for stakeholder
bias
• Performance data useful to determine
which SCs warranted follow-up

12. Questions?
For more information:
Kemerer V. & Salentine, S. (2014). A case
study to measure national M&E system
strengthening: Nigeria. Chapel Hill, NC:
MEASURE Evaluation, University of North
Carolina.

13. This presentation was produced with the support of the United States Agency for
International Development (USAID) under the terms of MEASURE Evaluation
cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is
implemented by the Carolina Population Center, University of North Carolina at
Chapel Hill in partnership with ICF International; John Snow, Inc.; Management
Sciences for Health; Palladium; and Tulane University. Views expressed are not
necessarily those of USAID or the United States government.
www.measureevaluation.org