Presentation_Jurczynska - Catalyzing Investments in RMNCAH at the Community L...
CSHGP MNH Lunch Roundtable_Koblinsky_0.11.12
1. CSHGP Program Learning
Agenda: Maternal and Newborn
Health
Marge Koblinsky
Senior Maternal Health Advisor, USAID
CORE Group Fall Meeting
October 11, 2012
1
2. Presentation Outline
Purpose of consultancy
Methods
Findings from project reviews
Conclusions and recommendations
2
3. Purpose of Consultancy
Devise a prospective learning strategy to link
NGO efforts in maternal/newborn health to
global/national policy and strategy discourse
Highlight patterns of learning —the what and
how -- guided by stakeholder needs, literature
and capacity
3
4. Methods
Stakeholder Interests
TAG—25 people
Stakeholder interviews —10 people
Project doc review--purpose, avail data,
conclusions
Past (closed) projects — ―intrapartum care pkg‖
• Reviewed 17 projects>40% LOE MNCH; 5 in-depth
Current (active) projects (36)
• Reviewed 4 with 100%MNC, OR grant (16>40% LOE MNCH)
Global literature review – community MNH
4
7. Stakeholders’ main priorities
process of working with costs of these
TBAs; effectiveness interventions packages
effect of specific and approaches
community approaches for CB-HMIS and its use in
improving implementation decision-making and costs
and access for MNH of implementation
interventions NGO contributions to
decision making re the strengthening referral
selection of MNH systems
intervention packages and Postpartum and postnatal
the implementation care: access and
approaches implementation
7
8. Findings: Stakeholder interests
& availability of data
Topic Stakeholder question Indicators Project
info/comments
TBAs -Process of working with TBAs % trained -TBA not
delivery distinguished by
-New roles attendant indicator from any
(Other Std MNC trained delivery
-Best practice for TBAs indicator) attendant;
not key word;
-How they link to big picture
-Because of MOH
-How effective are TBAS? What policies TBA info not
do they do? incorporated into
project reports shared
with MOH
- May have qual data
8 and know # trained
10. Proj purpose and data available
5 completed & 4 OR projects
Project info Data collected Comments
Purpose Enhance demand &/or Vulnerable not generally
quality of govt program- defined
vulnerable
Study Design Pre-post ¾ OR –comparison area
Intervention package DIP- planned; Typically not stratified to
Complex interventions test cpts of complex
intervention; outputs not
collected
Outcome data YES! (next slides)
No cost data
Impact data Deaths not reported OR projects: may collect
death data-VAs
10
11. Example of project inputs
Aim: Enhance demand and/or quality of govt
programs (e.g., AKF Pak implementing CMW program
of GOP; Nepal HealthRight—CB-NCP program of GON)
Interventions:
Many known effective MNH interventions
Delivered through trained, supervised skilled works in
setting with QI system
Access: village comm; grp meeting, CHWs , TTBAs,
savings groups, emergency transport plans
Awareness—comm mob, BCC, mass media
11
12. Women delivered in a health facility in selected
CSHGP projects (13), base/endline surveys (%)
Latin
Africa Asia America
100
80
Percent (%)
60
40
20
0
Baseline Endline
15. Limitations of review
Sample projects reviewed may not be
representative of all MNH projects in the
portfolio
Copious project documentation restricted in-
depth review
Variation in response to reporting guidance
15
16. General Conclusions
Project information simply not easily
accessible—spread over 500 or more pages
• Needs to be captured in one relatively brief document
with objectives, methods, intervention description,
results, discussion/interpretation, conclusion
Study design appropriate for question
• Effectiveness studies—quasi exp design
• OR—aim to improve implementation of a known
effective intervention
16
17. Conclusions: Stakeholder interests
Effectiveness and cost effectiveness
Standard projects not designed to address
questions of effectiveness/cost-effectiveness
Outputs not reported so don’t know adequacy
of implementation
• ―Use data‖ e.g., of delivery kits, BPCR, not widely
available; relationship with use of SBA or facility
not available
17
18. Learning to-date and
Recommendations
1. Cross-cutting themes (e.g, Community
case management)
2. Monitoring and evaluation of on-going
projects
3. Operations research
18
19. Conclusions: Project Reviews
Learning to-date
Cross-cutting themes (e.g, Community case
management)
• External person surveys the projects in place/reviews
project reports
• Limited number of such efforts
• Potential topics:
– What factors most affect the sustainability (or integration)
of community based approaches/interventions (e.g., CHW
or TTBA outreach, women’s groups?
– What mechanisms are available for integrating such
community approaches into the formal health system?
19
20. Conclusions: Project Reviews
Learning to-date
Monitoring & evaluation of on-going projects
• Pre-post surveys, specified indicators
• Across project learning possible—and there are
increases in coverage noted (above)—but in individual
project cannot state it was due to project inputs
• Potential M&E:
– Are inputs and outputs on target to achieve the desired
outcomes: Specify at a population level
– Who is the recipient of the interventions (equity)?
– What contextual factors impact the projects? (eg. Road
density, transport available; supply availability) (see CI)
– Country case studies
20
21. Conclusions: Project Reviews
Learning to-date
Operations Research
Excellent beginning toward contributing to
global discourse
Improve with more specificity, less complex
intervention packages and approaches
Potential OR:
• Are community or professional providers more
appropriate for counseling messages? Where?
• Can new technologies (mobiles) improve
community worker performance?
21
22. Recommendations
OR projects
What is of interest to local policy/program managers?
Collect right indicators at right levels—linking inputs,
activities, outputs, outcomes
• Look beyond mortality at morbidity esp for MH
• Develop quality of care indicators (eg., disrespect/abuse)
Develop succinct reports accessible to outside groups
that follow normal journal requirements—and publish!
22
23. General Recommendation
Decrease amount of time on descriptive
info, increase time for analytical and
interpretive information/project
Project development stage -- formative
research/ analysis /writing
Midterm–adequacy surveys that measure
inputs and outputs
Final – evaluation survey plus 12 months to
analyze what happened/why, how to
communicate results!
23
24. Many thanks to Jennifer
Yourkevitch, Kirsten Unfried, Leo
Ryan, Nazo Kureshy
Thank you!
wwww.mchip.net
Follow us on:
Notes de l'éditeur
Objective of this exercise:Learning strategy aimed at showing value added (otherwise program could be run out of missions?) Strengthen program leaning potentialshow how info may be analyzed and organized across projects through systematizing efforts to harness learning for future evals and Developed further through upcoming M&E and OR plans .Need framework and roadmap for next 5 years (ck CCM) Input to a short publishable paper –frontiers of learning with NGOs for Jl Implementation & Research (new USAID jl??)
For project review, 40% LOE in MNH was used as threshold.Within the OR portfolio there are 16 active MNH projects with 40% or more LOE in MNC. With the aim of learning about the MNH objectives, research questions and means used to answer these questions, four projects with 100% LOE in MNC, an operations research grant, DIPs and mid-term evaluation reports, were selected for in-depth analysis. They include AKF Pakistan (begun in 2008 and will complete in 2013), HealthRight Nepal (2009-2013), CRWRC Bangladesh (2009-2014) and CHS Ecuador (2009-2014). In sub-Saharan Africa, only five projects had 60% or more LOE in MNC and four began in 2010 or 2011. 17 closed projects (since 2000) met criteria. Of these, 5 were examined in more depth to garner qualitative data related to interventions.
From past, chose 21 projects reviewed in some depth;Haiti (3 closed), Peru (2 closed), Kenya (2 closed), Nepal (2 – 1 active, 2 closed), Bangladesh (2 – 1 active, 1 closed)Other active – Ecuador & Pakistan Other closed – Honduras, Nicaragua, Senegal, Malawi, India, Vietnam, East Timor
Output data: measure of engagement with the interventions depending on approach used. Not required indicators (eg, % mothers who report a PP visit within 2 days of delivery (home or facility)May collect “capacity indicators”—plans of the organization Input data—population levels of inputs not specified (may have # women groups, and # their meetings)Contextual information may be in DIP but not carried forward –eg., SES, demog, health services—available pub/priv, fac/pop ratio etc, road density, travel time,
Projects not stratified nor
AMREF 2005-2010Health Right 2006-2010Kenya DHS 2003 & 2008-09 -------Plan & CARE 2007-2010Nepal DHS 2006 & 2011-------Future 2005-2009INMED 2006-2010Peru DHS 2004-06 & 2010 (both continuous)