Overview of MEASURE Evaluation In the ECOWAS Region
1. Overview of MEASURE
Evaluation In the ECOWAS
Region
Jason B. Smith
ECOWAS Annual Forum of Technical and
Financial Partners in the Area of Health
Information Systems
Dakar, Senegal, October 21-22, 2013
2. Presentation Overview
Brief history of MEASURE Evaluation
The BIG picture – vision, mission, objectives
What we do – Phase 3
How we’re organized
Where we work
Work in ECOWAS Countries
Where do we go from here?
3. The MEASURE Projects
MEASURE DHS
MEASURE Evaluation
MEASURE CDC – Division of Reproductive
Health
MEASURE BUCEN – International Programs
Division
5. Past Achievements (pre Phase 3)
Development of comprehensive series of tools and
methods to address different elements of the health
information cycle and system
M&E/SI systems established and/or strengthened in
multiple countries
Highly respected regional M&E training programs
have trained hundreds of individuals
High demand for MEASURE Evaluation products and
active role in global initiatives confirm global
leadership position in M&E/HIS
6. Phase 3: Building on Past Success
Consolidate and evaluate tools developed in Phase II
through field application.
Increased emphasis on improving data quality and data
use in field SI systems
Increased emphasis on organizational development
Consolidate regional training programs with increasing
responsibility passed to training partners
Continued participation in global initiatives
Comprehensive knowledge management strategy
7. Phase III Activity Objective
Improved collection, analysis, and presentation
of data to promote better use of data in planning,
policymaking, managing, monitoring, and
evaluating population, health and nutrition
programs
8. Phase III Vision
Institutions and people empowered to
identify, collect, analyze, and use technically
sound information to improve global health
and well being.
11. What We Do
Technical leadership
Collaboration and Coordination
Sustainable capacity development
Innovation: creation & use of state of the art tools and
methods
Knowledge sharing and information dissemination
Use of best practices to support evidence-based
decision making.
12. Core Technical Areas
Element Technical Areas
HIV/AIDS; Malaria; TB; MCH; AI, NUT, WASH
Cross-cutting Technical Areas
CBT; DDU; KM; Gender; RHIS; Vital Events;
Evaluation; GIS
14. Presence in ECOWAS Countries
MEASURE Evaluation Phase 3
COUNTRY
Nature of Presence
BURKINA
FASO
Malaria Training
COTE
D'IVOIRE
Country Office
GHANA
Training Partner University of Ghana, Malaria Training, MPH
development, Research
LIBERIA
Technical Assistance - LQAS
MALI
Country Office
NIGERIA
Country Office, Training Partners - ABU, OAU
SENEGAL
Training Partner – CESAG
15. Special Mention - CESAG
Workshops in PHN, HIV and RHIS
ToT Activities
IMEA Course
Expansion to country-level training
TA in DQA and M&E assessments
Involvement in WCA TWG
Founding member of GEMNet-Health
16. Individuals Trained by MEASURE Evaluation
Regional Training Partners, July 1998 – June 2013
COUNTRY
7/986/13
7/086/13
COUNTRY
7/986/13
7/086/13
BENIN
24
11
LIBERIA
22
14
BURKINA
FASO
33
18
MALI
61
28
CAPE VERDE
2
0
NIGER
8
3
COTE
D'IVOIRE
58
21
NIGERIA
68
37
GAMBIA
4
1
SENEGAL
88
34
GHANA
35
21
SIERRE
LEONE
6
4
GUINEA
20
5
TOGO
10
6
GUINEABISSAU
9
7
TOTAL
448
210
17. # of Trainees and # of Training Events by MEASURE Evaluation
All Classroom (Non-Virtual) Trainings, July 2008– June 2013
COUNTRY
# of
# of
Trainees Trainings
COUNTRY
# of
# of
Trainees Trainings
BENIN
12
0
LIBERIA
204
7
BURKINA
FASO
18
3
MALI
646
10
CAPE VERDE
0
0
NIGER
5
0
COTE
D'IVOIRE
855
35
NIGERIA
1,987
60
GAMBIA
1
0
SENEGAL
90
18
GHANA
40
5
SIERRE
LEONE
7
0
GUINEA
5
0
TOGO
9
0
GUINEABISSAU
7
0
TOTAL
3,886
138
18. Online Certificate Course Completions,
M&E Learning Center,2007-present
COUNTRY
# of
Completions
COUNTRY
# of
Completions
BENIN
21
LIBERIA
73
BURKINA
FASO
51
MALI
10
CAPE VERDE
0
NIGER
8
COTE
D'IVOIRE
39
NIGERIA
793
GAMBIA
7
SENEGAL
24
GHANA
109
SIERRE
LEONE
44
GUINEA
5
TOGO
19
GUINEABISSAU
3
TOTAL
1,206
19. Individuals Trained via Virtual Programs,
M&E Learning Center,2007-present
COUNTRY
# of
Completions
COUNTRY
# of
Completions
BENIN
0
LIBERIA
0
BURKINA
FASO
0
MALI
19
CAPE VERDE
0
NIGER
0
COTE
D'IVOIRE
24
NIGERIA
89
GAMBIA
0
SENEGAL
9
GHANA
32
SIERRE
LEONE
0
GUINEA
0
TOGO
0
GUINEABISSAU
11
TOTAL
184
20. Memorandum of Understanding
WAHO – MEASURE Evaluation
Through this MoU, MEval and WAHO propose to
establish a close relationship between partners
based on cooperation and mutual respect in
order to promote coordination and harmonization
of their activities in Health Management
Information System (HMIS) Strengthening.
21. Areas of Collaboration
Coordination and harmonization of policies
Routine Health Information Systems (RHIS)
Development of evaluation research
Use of organization development approaches to improve the work
environments for M&E activities
Promotion/dissemination of best practices in M&E
M&E human resource development, including leadership strengthening
Diversification of health financing mechanisms
WAHO institutional development and capacity building
Monitoring & evaluation practice
22.
23. PRISM
The PRISM toolkit includes :
Performance Diagnostic Tool - the primary component
Overview and Facility/Office Checklist - examines
technical determinants
Management Assessment Tool – assesses
management practices
Organizational and Behavioral Assessment Tool identifies behavioral and organizational factors affect
RHIS performance
24.
25. Back to the Big Picture:
Emerging Areas
Increased emphasis on evaluation – increase knowledge
of what works in public health
Health systems strengthening (HSS)
Health system interventions in global programs
HIS as a component of HSS
Increasing integration and/or linkages between vertical M&E
systems and national systems
Institutional capacity building, sustainability and ownership
Integration cross-cutting areas like gender and geospatial
methods into project activities
26. The Future in the ECOWAS Region
Depends somewhat on the MEASURE Evaluation
Phase 4 procurement but….
Continue work with RTPs
Continue work at country level
Expand Collaboration & Coordination with WAHO
27. MEASURE Evaluation is funded by the U.S. Agency for
International Development (USAID) through Cooperative
Agreement GHA-A-00-08-00003-00 and is
implemented by the Carolina Population Center at the
University of North Carolina at Chapel Hill, in partnership
With Futures Group International, John Snow, Inc., Macro
International Inc., Management Sciences for Health, and
Tulane University. The views expressed in this presentation
do not necessarily reflect the views of USAID or the United
States government.
Hi my name is Jason Smith
Associate Professor of Public Health at UNC-CH
Deputy Director of CBT for MEASURE Evaluation
Asked to introduce MEASURE Evaluation so that the ECOWAS forum can know more about who we are, what we do and how we might be able to improve our collaboration in the region.
Carolina Population Center, UNC
Project leadership; applied research; impact evaluation; CBT; M&E; KM
Futures Group International (FGI)
Data demand and use; Policy & advocacy; M&E; OVC; IT
John Snow Inc. (JSI)
HSS; RHIS; M&E; CB; IT
ICF Macro
Surveys, research, M&E (esp. malaria, AI); IT
Management Sciences for Health (MSH)
Organizational development; management and leadership
Tulane University
Applied research (esp. HSS, malaria); CBT
Where We Work
Changes all the time. Not completely accurate. Doesn’t account for RTPs or Associate Awards.
BURKINA Le projet MEASURE Evaluation, financé par l’USAID, en partenariat avec le Centre de Recherche en Santé de Nouna (CRSN), le Centre de Recherche Internationale pour la Santé de l’Université de Ouagadougou (CRIS/UO) et l’Unité de Formation et de Recherche en Sciences de la Santé de l’Université de Ouagadougou (UFR/SDS) a le plaisir d’annoncer l’organisation d’un atelier de formation en suivi et évaluation des programmes de lutte contre le paludisme.
COTE D’IVOIRE MEASURE Evaluation collaborates with three government ministries in CI:
The Ministry of Health and Fight Against Aids,The Ministry of Education, andThe Ministry of Woman, Family and Infant
Strengthening the HMIS Reporting System
Working with the Ministry of Health and Fight Against AIDS
Improving OVC Data with the Ministry of Woman, Family and Infant
Assessing Information Systems in the Ministry of Education
GHANA MEASURE Evaluation works with the Ghana AIDS Commission to provide technical assistance in carrying out components of the national HIV and AIDS strategy
LIBERIA – MOH LQAS
MALI MEASURE Evaluation has maintained an office in Mali. The bulk of work during Phase III has been in support of the country’s National Malaria Control Program, with additional technical assistance to USAID’s non-malaria monitoring and evaluation (M&E) activities.
NIGERIA “We work in a lot of different health areas, collaborating with several ministries and a variety of partners.”
National Level Strengthening of HIS and M&E Capacity
Strengthening State Level Health Information Systems
DDU – Hi Impact Research WSs , OVC, Impact of Health Education, PMTCT, Assessment of Primary Care Facilities, Malaria Research , GIS
Proud of our collaborative work with all of our Training Partners including Nigeria, Ghana, Burkins in the ECOWAS region, but CESAG is one of our oldest training collaborations and also one of the most fully articulated so because we are here in Dakar, I wanted to give them some special mention.
Centre Africain d’Etudes Superieures en Gestion
Collaborating with MEASURE Evaluation since 1998
Serves participants from all over Africa
Maps to countries where we have a presence ether RTP or Country Office
Workshops: M&E Fundamentals, M&E for Population, Health and Nutrition programs, M&E for HIV/AIDS programs, M&E of Malaria programs, Use of TB Program Data, M&E for RHIS Performance Improvement, M&E and GIS.
70% overall Francophone, the vast majority of these served by CESAG
Evaluation, DDU, RHIS, Gender, GIS, KM
a footnote: this is a cumulative total of successful completions all (9) courses that we host on the M&E Learning Center. The lion’s share of these will be for the M&E Fundamentals-English course that was originally launched in 2007.
Undercount in two ways:
Only completions
Doesn’t count other places we link to or provide content for such as USAID’s Global E-Learning platform – authored or co-authored 11 M&E-related courses here
Blended approaches to team training. New in Phase 3 but something we hope to do more of in the future.
Another significant development that occurred just this year was the signing of a MOU between WAHO and MEASURE Evaluation. We had been cooperating and coordinating on some aspects of M&E capacity-building in an informal way over the years but decided this year to formalize our relationship.
Coordination and harmonization of policies
Indicators, BPs in Monitoring and Evaluation (M&E) implementation
Routine Health Information Systems (RHIS)
RHIS Assessment
Improve data quality via introduction of Routine Data Quality Audit (RDQA) and Data Quality Assessment (DQA)approaches
ID data use barriers and develop action plans to overcome them.
Establish a regional training centre of excellence in health information
Development of evaluation research
Use of organization development approaches to improve the work environment under which M&E activities take place
Promotion and dissemination of best practices in M&E
Development of human resources for health, including leadership strengthening
Diversification of health financing mechanisms
M&E financing----> business planning
WAHO institutional development and capacity building
M&E Curriculum development
Monitoring & evaluation practice
I want to highlight the notion of doing more work in the RHIS area.
At MEASURE we often approach improvement of HIS by using a set of tools we’ve developed called PRISM.
The Performance of Routine Information System Management (PRISM) framework consists of tools to assess Routine Health Information System (RHIS) performance, identify technical, behavioral and organizational factors that affect RHIS, aid in designing priority interventions to improve performance, and improve quality and use of routine health data.
Need to do additional prism assessments
I believe two have been done them to date. Liberia and CI
Mali HIS coming soon (December USAID FUNDS). Possibly work in Niger in early 2014 TA but no $ yet.
The beta version of the PRISM toolkit includes the following questionnaires, data collection methods, and processes for assessing RHIS performance, processes, and determinants:
Performance Diagnostic Tool - As the primary component in the PRISM toolkit, this tool determines the overall level of RHIS performance to identify strengths and weaknesses;
The other three tools identify the underlying technical, organizational, and behavioral reasons for those strengths and weaknesses.
Overview and Facility/Office Checklist - This tool examines technical determinants, such as the structure and design of existing information systems in the health sector, information flows, and interaction between different information systems. It allows users to understand the availability and status of RHIS resources and procedures used at health facilities.
Management Assessment Tool - This tool is designed to take rapid stock of the RHIS management practices and aid in developing recommendations for better management.
Organizational and Behavioral Assessment Tool - This tool identifies behavioral and organizational factors affect RHIS performance, including data demand, motivation, confidence level, task competence, and problem-solving skills.