Overview of the SC4CCM project and end-line evaluation questions focused on scalability and sustainability. Methodological approaches including case selection strategies, mixed method approaches, within-case and cross-case analysis processes. (Sangeeta Mookherji, GWU)

2. Background to the SC4CCM project
• In 2009, SC4CCM set out to improve community-level
availability of key products for sick child management
in 3 countries: Malawi, Rwanda, Ethiopia.
• Tasked with demonstrating that tested innovations
could be scaled and sustained after project end.
• Midline evaluations (2012-13) focused on measuring
effects; findings kicked off scale-up process.
• Endline evaluation (2014) focused on learning about
scalability and sustainability of SC4CCM innovations.

3. Over-arching Endline Evaluation
Questions
• To what extent (geographic breadth
and institutional depth) have the
SC4CCM innovations to support
commodity availability at the
community level, specific to each
country, been scaled up?
• To what extent have the program
effects of SC4CCM observed at
midline been maintained at
endline?
• To what extent have the country-
specific innovations been
institutionalized at endline?
• What aspects of the SC4CCM
design, implementation, and overall
project approach contribute to
scalability and sustainability of the
particular innovation supported in a
country?

4. Case selection
strategy
• Compare original
with scale-up
districts
• Learn from
extremes: best and
worst performing
with SCM/PA
• Management
component focus:
choose near and far
CHWs; high and
low QI
• Get multiple
perspectives –
district, HC, CHW

5. Multiple, Mixed methods design
Qualitative case studies
• Purposively selected cases
• In-depth interviews (district,
HC, CHWs)
• Structured observations of
tool use (HC and CHW)
• RSP tool documentation
review (HC, CHW)
• QI team documentation
review (district, HC)
• Standard field manual
Endline surveys
• Representative selection of
HCs and CHWs
• Comparable to midline LIAT
• Structured survey
interviews
• Physical inventory of key
products
• Assessment of knowledge
and competency using RSPs

6. Within-case (country) analysis process
• Team discussion and analysis
notes after each district;
feedback from evaluation
advisor
• One week focused analysis
immediately after data
collection – whole team, with
project management and
evaluation advisor
participation
• Writing of country-specific
endline report, integrating
LIAT data
• Presentation of endline
findings to country
stakeholders
Focused analysis steps:
• District-by-district analysis, using
notes, original program theory,
evaluation questions; ongoing
identification of contextual factors
• “Deep dives” – RSP procedures:
improvements, benefits, challenges; QI
teams: functionality, benefits,
challenges; PA: perceived
improvements, other changes,
challenges
• Analysis of contextual factors –
contributions and barriers; likelihood
of affecting scale-up or sustainability;
can it be changed?
• “Validation” and revision of program
theory based on the data

7. Cross-case synthesis process
• Common themes identified
from program theory and
country-specific analyses
• Codebook developed
• Qualitative data coded in
atlas.ti – to make the data
more manageable, and
allow for focused cross-
theme analysis to
understand relationships
and non-linearity
• One week focused
synthesis analysis with
whole team participation
Synthesis analysis steps:
• Data integration - PA: used qual to
explain survey findings; RSP: qual-
survey validation; QI: used qual
findings as lead
• Differences and similarities
between countries; full range of
benefits and challenges (RSP and
QI); explaining PA; context
• Scale: compared how it is
happening across countries; RSP v.
QI; partner implementation
• Sustainability: Data visibility and
use comparisons and mechanisms;
evidence of political commitment
• Project inputs and implementation
strategy contributions

8. Data-based country program theories

9. Pilot-to-program theory

10. Acknowledgements
JSI Research & Training Institute, Inc. (JSI) implemented the
Supply Chains for Community Case Management (SC4CCM)
project with funding from the Bill & Melinda Gates
Foundation.
JSI partnered with the Milken Institute School of Public
Health at The George Washington University for the endline
evaluation.
• GWU team: Sangeeta Mookherji, Jillian Dunning, Teemar
Fisseha, Anisa Saleh
• SC4CCM team: Yasmin Chandani, Sarah Andersson,
Alexis Heaton, Megan Noel, Barbara Felling, Savitha
Subramaniam, Mildred Shieshia