Overview of the SC4CCM project and end-line evaluation questions focused on scalability and sustainability. Methodological approaches including case selection strategies, mixed method approaches, within-case and cross-case analysis processes. (Sangeeta Mookherji, GWU)

2. Background to the SC4CCM project
•
In 2009, SC4CCM set out to improve community-level availability of key products for sick child management in 3 countries: Malawi, Rwanda, Ethiopia.
•
Tasked with demonstrating that tested innovations could be scaled and sustained after project end.
•
Midline evaluations (2012-13) focused on measuring effects; findings kicked off scale-up process.
•
Endline evaluation (2014) focused on learning about scalability and sustainability of SC4CCM innovations.

3. Over-arching Endline Evaluation Questions
•
To what extent (geographic breadth and institutional depth) have the SC4CCM innovations to support commodity availability at the community level, specific to each country, been scaled up?
•
To what extent have the program effects of SC4CCM observed at midline been maintained at endline?
•
To what extent have the country- specific innovations been institutionalized at endline?
•
What aspects of the SC4CCM design, implementation, and overall project approach contribute to scalability and sustainability of the particular innovation supported in a country?

4. Case selection strategy
•
Compare original with scale-up districts
•
Learn from extremes: best and worst performing with SCM/PA
•
Management component focus: choose near and far CHWs; high and low QI
•
Get multiple perspectives – district, HC, CHW

5. Multiple, Mixed methods design
Qualitative case studies
•
Purposively selected cases
•
In-depth interviews (district, HC, CHWs)
•
Structured observations of tool use (HC and CHW)
•
RSP tool documentation review (HC, CHW)
•
QI team documentation review (district, HC)
•
Standard field manual
Endline surveys
•
Representative selection of HCs and CHWs
•
Comparable to midline LIAT
•
Structured survey interviews
•
Physical inventory of key products
•
Assessment of knowledge and competency using RSPs

6. Within-case (country) analysis process
•
Team discussion and analysis notes after each district; feedback from evaluation advisor
•
One week focused analysis immediately after data collection – whole team, with project management and evaluation advisor participation
•
Writing of country-specific endline report, integrating LIAT data
•
Presentation of endline findings to country stakeholders
Focused analysis steps:
•
District-by-district analysis, using notes, original program theory, evaluation questions; ongoing identification of contextual factors
•
“Deep dives” – RSP procedures: improvements, benefits, challenges; QI teams: functionality, benefits, challenges; PA: perceived improvements, other changes, challenges
•
Analysis of contextual factors – contributions and barriers; likelihood of affecting scale-up or sustainability; can it be changed?
•
“Validation” and revision of program theory based on the data

7. Cross-case synthesis process
•
Common themes identified from program theory and country-specific analyses
•
Codebook developed
•
Qualitative data coded in atlas.ti – to make the data more manageable, and allow for focused cross- theme analysis to understand relationships and non-linearity
•
One week focused synthesis analysis with whole team participation
Synthesis analysis steps:
•
Data integration - PA: used qual to explain survey findings; RSP: qual- survey validation; QI: used qual findings as lead
•
Differences and similarities between countries; full range of benefits and challenges (RSP and QI); explaining PA; context
•
Scale: compared how it is happening across countries; RSP v. QI; partner implementation
•
Sustainability: Data visibility and use comparisons and mechanisms; evidence of political commitment
•
Project inputs and implementation strategy contributions

8. Data-based country program theories

9. Pilot-to-program theory

10. Acknowledgements
JSI Research & Training Institute, Inc. (JSI) implemented the Supply Chains for Community Case Management (SC4CCM) project with funding from the Bill & Melinda Gates Foundation.
JSI partnered with the Milken Institute School of Public Health at The George Washington University for the endline evaluation.
•GWU team: Sangeeta Mookherji, Jillian Dunning, Teemar Fisseha, Anisa Saleh
•SC4CCM team: Yasmin Chandani, Sarah Andersson, Alexis Heaton, Megan Noel, Barbara Felling, Savitha Subramaniam, Mildred Shieshia