Proposing an Evaluation Plan: ASPIRE

Girls’ Empowerment through Education and Health Activity (ASPIRE)
Proposing an Evaluation Plan
Ashley Day
December 18, 2015

ASHLEY DAY | 1
Introduction
Malawian women and girls are subject to vast gender disparities. While women account
for 51.4% of the country’s labor force, they struggle to gain access to, and rights over, resources
and opportunities.1
They also face difficulties in achieving the education necessary to find
gainful employment, which would allow them to escape this battle for resources and livelihood
activities. Cultural norms do not promote girls’ education, and attrition rates reflect these beliefs.
According to the 2010 Demographic and Health Survey (DHS) in Malawi, only 38% of women
aged 20-24 had completed primary school, and only 28% entered secondary school, with a mere
12% completing it.2
In fact, a Malawian girl is 50 times more likely to be married under the age
of 18 than she is to enter university, and this prevalence of early marriage has caused USAID to
rank Malawi as one of their top 35 priority countries for addressing gender-based violence.3
The Girls’ Empowerment through Education and Health Activity (ASPIRE) is one of
many USAID programs that aim to improve the status and well-being of these disadvantaged
Malawian girls. This paper discusses the goals and context of the ASPIRE program, and
proposes an evaluation plan to monitor the project’s progress in achieving its early outcomes.
Girls’ Empowerment through Education and Health Activity (ASPIRE)
The Girls’ Empowerment through Education and Health Activity (ASPIRE) is a project
launched by USAID/Malawi and implemented by Save the Children Federation, Inc.4
Adolescents in Malawi face many barriers in attempting to receive an education, and with a
lower cultural status, girls suffer the most. As described above, the 2010 DHS revealed low
levels of female education, but it also shows the gender disparity in Malawian schools. The
Gender Parity Index (GPI) – a measurement that assesses differences in school attendance rates
between males and females, and is calculated by dividing the gross attendance ratio (GAR) for
females by the GAR for males – shows that discrepancies increase in higher levels of schooling.5
While the average GPI for primary school is 0.95 (a GPI of less than one reveals a disparity in
favor of males), the average GPI for secondary school is 0.77, and as low as 0.52 in the lowest
wealth quintile.6
While poor education outcomes and work prospects are significant problems in and of
themselves, there is also evidence that health outcomes improve for women and their
communities with additional years of schooling.7
This is of critical importance in Malawi, where
health expenditures per capita are a mere $26 USD, and life expectancy is 55 years (placing
Malawi among the 20 lowest countries by both metrics).8,9
Attrition of students from primary
and secondary schools is contributing to poor knowledge of health topics, including HIV/AIDS
and other communicable diseases, sexual and reproductive health, family planning, and how to
access health services.
The ASPIRE project aims to improve education and health outcomes for 125,000
adolescent girls in the Balaka and Machinga districts of Malawi.10
In order to accomplish this,
the project activities are designed to achieve four long-term outcomes: 1) Improved reading
skills for girls in upper primary school, 2) Decreased structural barriers to girls’ education, 3)
Decreased cultural barriers to girls’ education, and 4) Increased adoption of positive sexual,
reproductive, and health care seeking behaviors.11
These outcomes will allow girls to be more
successful in school, empowering them to continue with their education, while equipping them
with the knowledge they need to promote good health for themselves and their communities.
Key activities include: health education for girls; teacher training for improved reading
instruction; the creation of a safe learning environment, free from bullying and sexual

ASHLEY DAY | 2
harassment; improved Water, Sanitation, and Hygiene (WASH) interventions in schools; and
teacher, parent, and community sensitization on how to support girls’ education.12
Evaluation Context
The ASPIRE program launched in December of 2014, and is contracted to run until
December of 2018. The project is part of a larger collaboration between USAID/Malawi and the
Ministry of Education, Science and Technology (MoEST) called the Early Grade Reading
Project (EGRP).13
All USAID funded activities support the country’s National Reading Strategy
(NRS), and aim to “strengthen teaching methodologies, develop appropriate learning materials,
increase parental and communal support for reading, and improve the policy environment for
reading to help government, schools, and communities build a stronger foundation for learning
for all students.”14
While all EGRP programs endeavor to increase the quality of, and access to, education,
the ASPIRE project is the most targeted intervention for adolescent girls. By improving sexual
health knowledge, and decreasing some of the barriers that keep girls out of school (including
cultural norms around girls’ education, teenage pregnancy, and insufficient sanitation facilities
and sanitary products), ASPIRE promotes improved retention for female students. Furthermore,
by improving the reading skills of girls in upper primary schools, these students will be better
equipped for higher learning and more likely to succeed in school and beyond.
The ASPIRE program is contracted at $10.4 million over four years, and a Monitoring &
Evaluation (M&E) Specialist is included in the program’s key full-time staff. This M&E
Specialist was responsible for conducting baseline surveys and assessments prior to the onset of
program activities, so baseline data is available to use in assessing progress at this one-year
mark. In the long-term, the program will conduct a longitudinal study with a comparison group
and post-ex collection, and thus baseline information is also available for comparison schools in
the targeted districts.
Evaluation Needs and Uses
As the ASPIRE program is now completing its first of four years, Save the Children is
seeking an evaluation to assess the project’s progress toward its short-term outcomes, and to
evaluate whether the program is being implemented appropriately. The findings of this
evaluation are needed to assess current activities and overall strategy, and to give the project
further direction toward achieving its goals.
The evaluation will focus on: the satisfaction of key project stakeholders (namely
ASPIRE students and teachers) with the activities that have been introduced; the effectiveness of
program activities thus far; strengths and weakness of the current model; and potential for
growth into more schools in Balaka and Machinga.
Key Stakeholders
Adolescent girls in the Balaka and Machinga districts of Malawi are the primary
beneficiaries of this intervention, and thus they are also the stakeholders of highest importance.
Additionally, many organizations (governmental and non-governmental), associated parties
(parents, teachers, etc.), and various community entities (schools, clinics, and general community
bodies) will also have a strong stake in the ASPIRE program. The matrix below illustrates
ASPIRE’s key stakeholders, arranged according to their influence and importance as pertains to
this program.

ASHLEY DAY | 3
Intended Audience
The primary intended audience for this evaluation is Save the Children. Now at the one-
year mark of the project, this interim evaluation will assess the organization’s implementation of
the program and progress toward its short-term goals. This evaluation will be improvement-
oriented, and the findings will be used to adjust program activities for greatest impact. As Save
the Children plans to spread the ASPIRE program to more schools in the Balaka and Machinga
districts, this evaluation will aid in decision-making in order for the project to be scaled
successfully. Save the Children will be able to use the results of this evaluation to improve upon
the current model, and implement a tested and revised version of the project to further schools in
the target districts.
Malawi’s Ministries of Education and Health will serve as a secondary audience for this
evaluation, as well as USAID/Malawi. This partnership is involved in program design,
implementation, and oversight, and thus they are primary stakeholders in improving and
expanding this program. This audience will likely receive a higher-level report of findings, while
the exhaustive information will be used by Save the Children for internal review and redesign
prior to proposed action.
Additional donors will be a third audience, as this evaluation will illustrate progress thus
far and attract further funding for program operations. This evaluation may also be considered as
an accountability tool, as Save the Children can use the findings to discuss areas of weakness and
how the project intends to address these needs.
Figure 1: Key stakeholders of the ASPIRE project, mapped according to importance
and influence.

ASHLEY DAY | 4
Finally, schools and teachers may be included as a third audience, in order to gauge
community response to the evaluation. By involving teachers in the decision-making process,
Save the Children will be able to design the most effective, contextually appropriate program
activities to address participants’ needs.
Evaluation Purpose
In the interest of optimizing the ASPIRE project’s effectiveness, this interim evaluation
will be formative in nature, and the findings will guide program improvements. As ASPIRE is
now completing its first full year of activities, this evaluation will focus on identifying the
strengths and weaknesses of the current model in order to improve upon program design. This
will include gathering information on participants’ satisfaction with the project thus far, and
monitoring progress on early outcomes. The ultimate goal of the ASPIRE project is to improve
the education and health outcomes of 125,000 adolescent girls in the Balaka and Machinga
districts of Malawi. In order to reach this goal, an evaluation of the subset of schools where
activities have been implemented is necessary before scaling the project to other target schools in
these areas. This evaluation will provide specific, actionable recommendations for improving the
quality and impact of the ASPIRE project.
Evaluation Questions
As described above, this evaluation will be formative and improvement-oriented. It will
inform how the current model is adjusted in implementation schools and scaled to further schools
in Balaka and Machinga. Assessing progress toward short-term outcomes at this one-year mark –
as well as documenting participants’ opinions on activities – will help Save the Children to
evaluate which elements of the program are working well and which may need adjustments. As
such, key questions for evaluation are as follows:
1. What are the early/short-term outcomes for teacher and student participants?
As ASPIRE finishes its first year, it will also be completing the first quarter of its
work given the lifespan of the project (four years in total). As such, this evaluation will
be important in assessing early progress toward short-term outcomes, in order to guide
program improvements in the second year. Have teachers been trained in teaching skills
to improve reading, as well as how to create a safe learning environment? Is there
evidence that they have begun to adopt and implement these practices in the classroom?
Have girls experienced any early benefits of these interventions? Do girls show a greater
understanding of sexual and reproductive health, and the importance of accessing health
services? Have WASH activities been implemented, and thus do students have better
access to clean water and secure sanitary facilities? Do communities show progress in
supporting girls’ education? At this stage, it will be critical for Save the Children to
evaluate progress on these outcomes in order to assess early effectiveness and inform any
necessary revisions to program activities and implementation.
2. Are there any unintended outcomes, either positive or negative?
ASPIRE’s program theory was designed in such a way as to achieve various
short-, medium-, and long-term positive outcomes. While the first question will evaluate
progress toward the project’s specified short-term goals, this second question will look at
any possible unintended outcomes. Are there any side effects or spillovers of the project,

ASHLEY DAY | 5
leading to other positive or negative outcomes? Do the effects of the ASPIRE program
spillover into the communities, affecting parents and other non-primary targets? Are there
stakeholders that were not initially included in the analysis? And if so, how are they
helped or harmed by the program activities? For example, how are male students in
ASPIRE classrooms affected by changes in reading instruction and classroom
management? An evaluation guided by this question will reveal further pertinent
information for Save the Children to use in program redesign and scale. If there are, in
fact, unintended outcomes, how can they be bolstered or avoided (depending on the
positive or negative nature) as the project is scaled?
3. What do students and teachers like and dislike about the program?
Aside from evaluating specific metrics to measure progress toward early
outcomes, this evaluation will gather qualitative information on participants’ opinions to
further inform program improvements. How do students feel about the health education
that they are receiving? Do they find that information is useful and actionable? What do
teachers think about the training they have received on new teaching methods and
classroom management? Do they feel that these learnings can be applied to their
classrooms and lessons? Are there any complaints about program activities thus far – and
if so, do participants have suggested improvements? While evaluating progress toward
early outcomes will be extremely valuable in assessing the project’s strengths, this
qualitative information will provide Save the Children with further critical data to inform
whether or not these activities are the right ones.
4. What implementation issues have emerged and how are they being addressed?
ASPIRE’s ultimate goal is to improve the education and health outcomes for
125,000 Malawian girls in Balaka and Machinga districts. Reaching this intended
beneficiary population will take time, and as such, Save the Children has not yet
implemented ASPIRE activities in all target schools. After a full year of operations, it
will be useful for Save the Children to document implementation challenges in order to
address these issues when they scale to other schools. Were participants willing and able
to join the proposed activities? If not, what can be done to make adoption easier in further
schools? Is the community supportive of the program? If not, what kind of education and
sensitization might help bolster community support? Were there any other
implementation issues that can be analyzed and addressed as the project scales? Asking
these questions will allow Save the Children to be more prepared as they launch activities
in the next group of communities and schools.
By seeking an evaluation along these questions, Save the Children will be equipped with vital
information to inform program improvements and scaling activities.
Program Theory
As mentioned above, ASPIRE strives to improve the education and health outcomes of
125,000 Malawian girls by way of four long-term outcomes: 1) Improving the reading skills of
girls in upper primary school, 2) Decreasing structural barriers to girls’ education, 3) Decreasing
cultural barriers to girls’ education, and 4) Increasing the adoption of positive sexual,
reproductive, and health care seeking behaviors. In order to achieve these long-term outcomes

ASHLEY DAY | 6
and ASPIRE’s ultimate program impact, Save the Children has implemented key education,
training, and community sensitization activities, as well as infrastructure improvements.
Under outcome one, teacher trainings in the areas of improved reading instruction and
classroom management will allow upper primary teachers to foster a safe and supportive learning
environment for ASPIRE students. Adoption and implementation of these new skills and
techniques will encourage girls to practice their reading at school, leading to improved reading
skills over time. Improved reading skills will be of great benefit to girls both inside and outside
of the classroom, as literacy will allow for higher education and empower women to seek
opportunities once they are out of school. Outcome two will be supported by improvements to
school infrastructure and the availability of important sanitary products. By providing girls with
clean water and secure sanitary facilities at school, and ensuring they have access to feminine
hygiene products, they will be more able to learn in this safe and hygienic environment, thereby
decreasing key structural barriers to their education. Under outcome three, community
sensitization and training in how to support girls’ education will lead to greater encouragement
and backing from teachers, parents, and communities at large. This community support will help
break down the cultural barriers that impede girls’ education. Finally, extensive health education
for girls will support outcome four, which will include information on HIV/AIDS, family
planning, safe sex practices, and the importance of seeking health care services. Adoption of
these learnings will lead to positive behavior change in sexual and health practices. It will also
lead to a decrease in the rate of teenage pregnancy, which not only supports outcome four’s
positive health behavior change, but also feeds into outcome three by reducing another key
cultural barrier that keeps girls out of school. For a complete overview of ASPIRE’s program
activities and theory of change, refer to the project’s outcome map in Appendix A.
Evaluating ASPIRE’s activities against proven best practices will further inform program
strengths and needed revisions. A review of various studies on the impact of education on
HIV/AIDS prevalence in east Africa reveals ambiguous outcomes at first examination. As
Hargraves and Boler detail in “Girl power: The impact of girls’ education on HIV and sexual
behavior,” some studies suggest that there is a negative impact of education on HIV/AIDS
(meaning more highly educated women are at a higher risk of contracting the virus) while others
suggest a positive impact or no impact whatsoever.15
Hargraves and Boler looked across this
body of work – with studies from Malawi, Tanzania, Uganda, Zambia, and other countries in the
region – and found an important distinction in the data sets when looking at them on a timeline.
As the authors explain, during the earlier years of the epidemic (prior to 1995), there was
evidence that higher education did in fact lead to greater risk for HIV/AIDS, as education also
led to higher economic status, urban living, and greater mobility and sexual prospects.16
However, as the virus began to spread more rapidly – particularly in sub-Saharan Africa – and
experts gained further understanding about its spread and prevention, this relationship has
changed. Data from more recent studies indicates that, “more highly educated girls and women
are better able to negotiate safer sex and reduce HIV rates.…Across all the countries reviewed,
girls who had completed secondary education had a lower risk of HIV infection and practised
safer sex than girls who had only finished primary education.”17
Among the youth population in
sub-Saharan Africa, 75% of the 15-24 year-olds who are infected with HIV are women and
girls.18
ASPIRE’s education component, which will equip female participants with greater
knowledge about HIV/AIDS as well as sexual and reproductive health, is a proven best practice
in Malawi and neighboring countries. This activity will have positive impacts on the health
outcomes of this highly vulnerable population.

ASHLEY DAY | 7
In terms of the impact of teacher training, according to a baseline study of the Malawi
Teacher Professional Development Support program (MTPDS), professional development
among primary school teachers is quite low, and Malawian teachers have little to no knowledge
of specific methodologies to use for reading instruction.19
The authors hypothesize that this lack
of teaching skills is contributing greatly to poor literacy attainments, with 97.1% of Standard 2
(grade 2) students and 69.3% of Standard 4 students unable to correctly answer a single reading
comprehension problem.20
The MTPDS intervention focused on Continuing Professional
Development (CPD) training in literacy, including regular coaching for participant teachers, and
the provision of structured, carefully sequenced lesson plans for instruction.21
In an endline
evaluation of MTPDS, students in intervention classrooms showed statistically significant
literacy gains against students in control schools, and teachers who received ongoing coaching
showed even further gains in their students’ outcomes compared against teachers using only the
structured lesson plans.22
These findings show the significance of teacher training and enhanced
teacher performance in improving reading outcomes.
Furthermore, an evaluation of Malawi’s Early Grade Reading Assessment (EGRA) –
another component of USAID’s umbrella Early Grade Reading Project (of which ASPIRE is a
part) – found a similar strength in the coaching portion of their teacher training program.23
While
they note that other factors (such as training, reading materials, and structured lesson plans) also
Figure 2: “Changing association between education and risk of HIV infection over time from
serial cross-sectional data collected in Uganda, Zambia Tanzania, and Malawi.”18

ASHLEY DAY | 8
impacted learning outcomes, and that coaching would have little impact on its own, there is a
strong correlation between coaching and improved reading outcomes when combined with these
other intervention strategies. Save the Children should take specific activities – most importantly
ongoing teacher support – under advisement as they evaluate their own strategies in this area and
consider possible revisions for greater impact.
In “Letting girls learn: Promising approaches in primary and secondary education,”
authors Herz et al. looked at research across Africa, Asia, Latin America, and the Middle East to
identify trends and recommend actionable interventions to support girls’ education. Many of the
findings of this study support ASPIRE’s activities, particularly in term of the positive effects of
eliminating various cultural and structural barriers to girls’ education. “Protect[ing] girls’ privacy
in coeducational settings, by providing separate facilities such as lavatories” was a key finding in
terms of “what works,” supporting ASPIRE’s various WASH interventions and aim to reduce
structural barriers.24
Furthermore, Herz et al. suggest that, “Deepen[ing] community involvement
in the design and implementation of educational policies and local schooling” can help with
enrollment rates for both boys and girls, but may favor girls as parents are often more reluctant to
send their daughters to school.25
This cultural barrier posed by community beliefs is not the only
norm that is hindering girls’ education, as “Research from Malawi shows that school-aged girls
spend more time than boys on household chores, and spend less time the first hour after school
on studying or relaxing.”26
These specific points should be included in Save the Children’s
community trainings, in order to promote actionable change in decreasing cultural barriers to
girls’ education. By involving communities in the discussion and decision-making, the ASPIRE
Figure 3: Correlation between teacher coaching visits and students’ oral reading fluency.23

ASHLEY DAY | 9
program will be able to foster a more nurturing and supportive environment for girls to receive
an education and to be successful in their studies.
This body of literature shows that ASPIRE’s program activities are well aligned with best
practices to achieve the project’s four main long-term outcomes. Furthermore, ASPIRE’s theory
of change promotes synergies between these outcome streams. Training teachers in how to
eliminate bullying and sexual harassment from the classroom, for example, will combine with
WASH program activities in creating a safe learning environment for girls, decreasing structural
barriers and encouraging girls to practice reading at school. Literacy, too, will contribute to
ASPIRE’s ultimate impact goal in various ways. While it is evident that increased reading skills
will improve the educational outcomes of girls, studies show that literacy affects health
outcomes, as well. In a study of literacy and health in Malawi, author Smith-Greenaway finds
that literacy is a key determinant of young adults’ health, and that “each unit increase in literacy
skills is associated with 15.8 percent higher odds of experiencing better self-rated health and 24.9
percent lower odds of reporting a prolonged sickness.”27
By investing in teachers to promote
better reading instruction, Save the Children will support its goal to improve both educational
and health outcomes for ASPIRE’s participating female students.
While these studies affirm the validity of ASPIRE’s broad program activities, Save the
Children should use these findings – in conjunction with the results of the evaluation – to
improve specific strategies where possible or necessary. While teacher trainings and community
sensitization programming have proven effective in improving the educational outcomes of girls,
there may be ways for Save the Children to incorporate the learnings of these studies to improve
upon their current model and achieve greater impact through their activities.
Evaluation Framework
As detailed above, this evaluation will be guided by four key questions. For all
quantitative data, the evaluation will rely on SMART indicators whenever possible, or indicators
that are Specific, Measureable, Attainable, Relevant, and Time-bound.28
In addition to this,
qualitative indicators will be incorporated under each outcome in order to gain detailed
information on participants’ opinions of various program activities. A careful analysis of the data
collected along these indicators will provide meaningful, data-driven answers to the evaluation’s
four guiding questions. Again, these questions are as follows:
1. What are the early/short-term outcomes for teacher and student participants?
2. Are there any unintended outcomes, either positive or negative?
3. What do students and teachers like and dislike about the program?
4. What implementation issues have emerged and how are they being addressed?
For a detailed description of all outcomes, indicators, and data collection methods, refer to the
Evaluation Matrix in Appendix B. The matrix details specific methods of data collection for each
outcome and indicator, including surveys, interviews, observations, focus groups, exams, and
attendance records. It also lists key sources of information for this data, including program
participants (students and teachers), Save the Children, ASPIRE schools and clinics, and
surrounding communities. It also provides a general timeline for evaluation activities (following
the one-year mark of program implementation), and explains the sequencing of data collection
(qualitative interviews following quantitative methods). Key information pertaining to outcomes

ASHLEY DAY | 10
and indicators is also detailed below, and further information on methodology and evaluation
methods can be found in the following section.
Ø Outcome 1: Teachers gain teaching skills and techniques to improve students’ reading.
Indicators:
• Number of teachers trained in reading instruction.
• Percent of trained teachers implementing new techniques.
• Percent increase in students’ reading comprehension test scores.
• Teacher satisfaction with new teaching methods.
• Student satisfaction with teaching approach.
Comments: Evaluation of outcome one will rely on five metrics, three quantitative and two
qualitative: 1) the raw number of teachers trained, as recorded by program staff; 2) the
percentage of these trained teachers who have adopted and implemented new techniques, as
evaluated by teacher surveys and direct classroom observations (spot checks); 3) the percent
increase in reading comprehension test scores, as evaluated by a brief exam, similar in design
and difficulty to one administered at the outset of the program, allowing for a comparative
analysis against baseline data; 4) teacher satisfaction with new teaching methods, as assessed
through surveys and interviews; and 5) student satisfaction with teaching approach, also
evaluated through surveys and interviews. Indicators 1-3 will gather quantitative information to
evaluate progress toward the implementation and adoption of ASPIRE’s activities. The
information gathered for indicators 4 and 5, through surveys and key informant interviews with
students and teachers, will provide more detailed information about the activities strengths and
weaknesses. Qualitative interviews will follow all other quantitative and qualitative assessment
(as surveys will involve both), and current teacher and student participants will be the key
sources of information, along with Save the Children.
Ø Outcome 2: Teachers gain skills in how to create a safe learning environment for
students.
Indicators:
• Number of teachers trained in how to eliminate bullying and sexual
harassment from the classroom.
• Percent of trained teachers implementing new techniques.
• Percent change in student satisfaction with learning environment.
• Teacher satisfaction with approach to creating a safe environment.
• Student satisfaction with learning environment.
Comments: Evaluation of outcome two will follow a similar structure to the assessment approach
in outcome one, and will utilize three quantitative and two qualitative metrics: 1) the raw number
of teachers trained in creating a safe learning environment, as recorded by program staff; 2) the
percentage of these trained teachers who have adopted and implemented these techniques, as
evaluated by surveys and classroom observations; 3) the percent change in student satisfaction
with the learning environment, as evaluated through survey data and compared against baseline
data; 4) teacher satisfaction with techniques, assessed through surveys and interviews; and 5)

ASHLEY DAY | 11
student satisfaction with learning environment, also assessed through surveys and interviews.
Once again, indicators 1-3 will assess progress toward outcome 2, while qualitative information
gathered on indicators 4 and 5 will inform improvements on program activities by providing
richer information on specific challenges. Qualitative interviews will again follow surveys and
other quantitative data collection, as analysis along quantitative measures may highlight
important questions to ask in these key informant interviews. Teachers and students will be the
key sources of information, along with data provided by Save the Children.
Ø Outcome 3: Girls have access to clean water.
Indicators:
• Percent increase in schools with clean water source.
• Girls’ self-reported change in well-being due to clean water access.
Comments: Evaluation of outcome three will look quantitatively at the percent increase in
schools equipped with clean sources of water. This will be assessed through observations, as well
as a comparison of Save the Children’s baseline data with current data from participating
schools. Evaluation of this outcome will also utilize surveys and key informant interviews to
look qualitatively at how girls report having access to an improved water source has changed
their well-being at school. Surveys and interviews will follow quantitative measurements, and
ASPIRE schools, Save the Children, and female students will be key sources of information.
Ø Outcome 4: Girls have access to safe and sanitary toilet facilities.
Indicators:
• Percent increase in schools with secure sanitation facilities.
• Girls’ self-reported change in well-being due to safe facility access.
Comments: Evaluation of outcome four will follow the structure of the assessment for outcome
three. It will rely on observations and comparative data analysis to measure the percent increase
in schools with secure sanitation facilities, and surveys and key informant interviews to assess
how girls view their well-being as a result of this change in condition. Surveys and interviews
will follow quantitative measurements, and ASPIRE schools, Save the Children, and female
students will be key sources of information.
Ø Outcome 5: Girls have access to feminine hygiene products.
Indicators:
• Percent increase in schools and clinics with available stock of sanitary
products.
• Girls self-reported change in well-being due to availability of hygiene
products.
Comments: Evaluation of outcome five will follow the same structure as outlined above for
outcomes three and four. Observations and comparative data analysis will measure increases in
stock availability, though for outcome five, health clinics will be assessed in addition to schools.

ASHLEY DAY | 12
Surveys and interviews with female students will again provide rich qualitative information on
how this change in condition has affected their well-being. Surveys and interviews will again
follow quantitative measurements, and ASPIRE schools, surrounding clinics, Save the Children,
and female students will be key sources of information.
Ø Outcome 6: Teachers, parents, and communities have greater understanding of how to
support girls’ education.
Indicators:
• Number of attendants at community trainings.
• Attendants’ self-reported takeaways from community trainings.
• Attendants’ self-reported actions based on community trainings.
Comments: Evaluation of outcome six will rely on Save the Children’s records of community
trainings in order to measure the number of attendants, but the most important information to
come out of the evaluation on this outcome will be the qualitative findings regarding
participants’ self-reported takeaways and actions. As described above, research shows that
involving communities in the discussion and decision-making around education will promote
community support for girls’ education. Focus groups and key informant interviews will reveal if
ASPIRE trainings are making progress toward achieving this goal, and if communities show
support for girls’ education. Qualitative baseline data on community beliefs will be used to
assess changes in opinion and action, and interviews and focus groups will again follow
quantitative evaluation. Teacher, parent, and community attendants will be the key sources of
information, along with records and baseline data gathered by with Save the Children.
Ø Outcome 7: Girls have greater knowledge about HIV/AIDS, family planning, and safe sex
practices.
Indicators:
• Number of girls who have received education on HIV/AIDS, family
planning, and safe sex.
• Percent increase in scores on sexual health knowledge.
• Percent of educated girls who report acting on new knowledge.
• Student satisfaction with sexual health education.
Comments: Evaluation of outcome seven will rely on three quantitative indicators and one
qualitative: 1) the number of girls who have received education on sexual health, as recorded by
program staff; 2) the percent increase in scores on sexual health, as evaluated by a brief health
quiz, similar in design and content to one administered at the outset of the program and evaluated
against baseline data; 3) the percent of educated girls who report acting on new knowledge, as
evaluated by surveys of female participants; and 4) student satisfaction with sexual health
education, as assessed by surveys and key informant interviews. Here again, qualitative data on
will be gathered after all quantitative measures, and ASPIRE female students will be the key
sources of information. Save the Children will also provide relevant data for comparison and
further evaluation.

ASHLEY DAY | 13
Ø Outcome 8: Girls have better understanding of when, why, and how to seek health care.
Indicators:
• Number of girls who have received health care education.
• Percent of educated girls who report acting on new knowledge.
• Student satisfaction with health care education.
Comments: Evaluation of outcome eight will follow the structure of outcome seven, here with
only two quantitative indicators and one qualitative: 1) the number of girls who have received
health care education, as recorded by program staff; 2) the percent of educated girls who report
acting on new knowledge, as assessed by surveys and observations; and 3) student satisfaction
with health care education, as evaluated by surveys and interviews. Key informant interviews
and observations will follow all other quantitative and qualitative (survey) data collection, and
ASPIRE female students will be the key sources of information. Save the Children will also
provide relevant data on activity participants.
Data Collection
Approach
As this evaluation is formative in nature, and meant to inform program improvements, it
will follow a non-experimental approach. The ASPIRE program is being implemented in a
phase-in/pipeline design. All of ASPIRE’s target schools were identified prior to project onset,
but interventions were only initiated in a subset of these schools, with the remaining schools
serving as a comparison group. For the purposes of this evaluation, we will use baseline data
collected on intervention schools to assess progress, but we will not utilize information gathered
on comparison schools. Comparison data will be used at a later date when evaluating project
success in achieving midterm and long-term outcomes.
As mentioned above, in the long-term, Save the Children will use a quasi-experimental
approach in conducting a longitudinal study with a comparison group and post-ex collection.
This will allow the organization to see the impact of the ASPIRE program when compared
against non-intervention schools. For the purposes of this end of year evaluation, however, such
rigor is not necessary in order to assess early progress and identify potential improvements. This
evaluation is context-based in design, and findings will be used to enhance program activities
within the evaluated context. As such, a non-experimental approach will suffice in gathering rich
data to guide project improvements.
Methodology
We will utilize a mixed methods methodology in conducting this review, relying on both
quantitative and qualitative data. We will conduct quantitative data collection first, along with
any qualitative information assessed through surveys. Then, based on this information and our
initial statistical analysis, we will return to participants for key informant interviews in order to
gather rich, supporting qualitative information.
As this evaluation is intended to improve upon the current model of the ASPIRE project,
quantitative data will be useful in assessing progress toward early outcomes thus far. However,
without supporting qualitative information, it would be difficult to ascertain how the program
could be most effectively improved, and thus quantitative data findings might not be actionable.

ASHLEY DAY | 14
By utilizing mixed methods in our data collection, we will be able to supplement key quantitative
findings with specific information on why a program component may or may not be working.
This qualitative information will allow Save the Children to enhance the quality of the ASPIRE
program by incorporating the feedback of key stakeholders into redesign activities, and
designing activities in ways that these stakeholders identify as needed and beneficial.
Methods
We will use a variety of methods in order to gather data to assess indicators. Surveys –
which will be conducted with ASPIRE’s teachers and students – will gather quantitative data on
the project’s progress when compared against baseline data. Surveys will also highlight key
qualitative findings that can be further explored through interviews. These surveys will be quite
useful in this evaluation, as content will include questions on multiple outcomes and indicators,
providing quick and comprehensive information. For a draft survey tool created for ASPIRE
students, see Appendix C.
Observations (or “spot checks”) will also be used in order to confirm information
reported through basic records, surveys, and interviews. Classroom observations, for example,
will evaluate the true rate of application of new teaching techniques against reported teacher
uptake. Brief tests will evaluate early gains in specific intervention areas, namely literacy and
health education. Key informant interviews – again of both teachers and students – will provide
detailed information to guide this formative evaluation in making recommendations for program
adjustment. Finally, focus groups in target communities will begin to assess communal opinions,
and changes in support levels for girls’ education.
Sources of Information
The target population of the ASPIRE program is adolescent girls. They will be a primary
source of information in this evaluation, as will ASPIRE teachers, as many of the interventions
to benefit these girls involve teacher training and the adoption of new teaching techniques.
Schools and clinics will also be information sources, as well as parents and community members.
Finally, Save the Children will be a source for baseline and more recent data, in order to draw
comparisons with the data collected for this evaluation. Most of this information will be gathered
at ASPIRE schools, though visits into nearby clinics and communities will also be necessary in
order to gather quality data for specific indicators.
For some of the established indicators, it will be possible to gather data across all of
ASPIRE’s current intervention schools. For example, along with observations, monitoring data
gathered by Save the Children will allow for a holistic assessment of the percentage of schools
with improved water sources and sanitation facilities. However, given that this evaluation will be
non-experimental in its approach, we will draw upon a convenience sample for many of our
evaluation methods, including surveys, classroom observations, exams, and focus groups. Key
informant interviews will draw upon a purposive sample, based on information gathered via
other data collection. As the purpose of this evaluation is to highlight program
strengths/weaknesses and progress toward early outcomes, interviews will be conducted in order
to dig deeper into survey responses of interest, and gather rich information to answer overarching
evaluation questions. We will draw upon a purposive sample for these interviews, targeting
students and teachers with reported survey information that we wish to explore further.

ASHLEY DAY | 15
Conclusion
Through the ASPIRE project, Save the Children will have the ability to improve the
health and education outcomes of 125,000 Malawian girls. This will not only impact their ability
to succeed in primary and secondary school, but also promote higher learning, greater
opportunities, and healthier families and communities for these girls. However, in order to be
effective in achieving this goal, it will be crucial for Save the Children to use the findings of this
evaluation to inform program improvements. As a review of relevant and contextual literature
has shown, current project activities are aligned with best practices in ASPIRE’s intervention
areas. Nevertheless, quantitative and qualitative data from this evaluation should be used to
ensure these activities are the right ones for this specific target population. By conducting this
formative evaluation at the one-year mark, Save the Children will be equipped with vital
information on early progress and the satisfaction of participants, empowering them to improve
upon the current model and scale an effective and impactful program.

ASHLEY DAY | 15
Appendix A: Outcome Map

ASHLEY DAY | 16
OUTCOME INDICATORS WHO WHAT WHEN
1. Teachers gain
teaching skills and
techniques to improve
students’ reading
w Number of teachers
trained in reading
instruction
w Percent of trained
teachers implementing
new techniques
w Percent increase in
reading comprehension
test scores.
w Teacher satisfaction
with new teaching
methods
w Student satisfaction
with teaching approach
w ASIPRE’s
participating upper
primary school teachers
w Female students in the
classrooms of ASPIRE’s
participant teachers
w Save the Children
w Records of teacher
attendance at trainings to
date
w Teacher surveys
w Classroom
observations (spot
checks)
w Student surveys
w Reading
comprehension test
w Key informant
interviews (both students
and teachers)
w Post one-year mark of
program activities
w Classroom
observations and key
informant interviews to
follow surveys and
quantitative data
collection
2. Teachers gain skills in
how to create a safe
learning environment for
students
w Number of teachers
trained in how to
eliminate bullying and
sexual harassment from
the classroom
w Percent of trained
teachers implementing
new techniques
w Percent change in
student satisfaction with
learning environment
w Teacher satisfaction
with approach to
creating a safe
environment
w. Student satisfaction
with learning
environment
w ASIPRE’s
participating upper
primary school teachers
w Female students in the
classrooms of ASPIRE’s
participant teachers
w Save the Children
w Records of teacher
date
w Teacher surveys
w Classroom
observations (spot
checks)
w Student surveys
w Key informant
interviews (both students
and teachers)
program activities
w Classroom
observations and key
follow surveys and
quantitative data
collection
Appendix B: Evaluation Matrix

ASHLEY DAY | 17
3. Girls have access to
clean water
schools with clean water
source
w Girls’ self-reported
change in well-being
due to clean water
access
w ASIPRE intervention
schools
w ASPIRE female
students
w Save the Children
w Records of water
sources
w Observations
w Student surveys
w Key informant
interviews with female
students
program activities
w Surveys and key
follow quantitative data
collection
safe and sanitary toilet
facilities
schools with secure
sanitation facilities
w Girls’ self-reported
due to safe facility
access
schools
w ASPIRE female
students
w Save the Children
w Records of sanitation
facilities
w Observations
w Student surveys
w Key informant
students
program activities
w Surveys and key
collection
feminine hygiene
products
schools and clinics with
available stock of
sanitary products
w Girls self-reported
due to availability of
hygiene products
schools
w Health clinics
surrounding ASIPRE
intervention schools
w ASPIRE female
students
w Save the Children
w Records of availability
of hygiene products
w Observations
w Student surveys
w Key informant
students
program activities
w Surveys and key
collection
6. Teachers, parents, and
communities have
greater understanding of
how to support girls’
education
w Number of attendants
at community trainings
w Attendants’ self-
reported takeaways from
community trainings
w Attendants’ self-
reported actions based
on community trainings
w Save the Children
w Teacher, parent, and
community participants
w Records of community
date
w Key informant
interviews with teacher,
parent, and community
participants
w Focus group
discussions
program activities
w Key informant
interviews and focus
groups to follow
quantitative data
collection

ASHLEY DAY | 18
7. Girls have greater
knowledge about
HIV/AIDS, family
planning, and safe sex
practices
w Number of girls who
have received education
on HIV/AIDS, family
planning, and safe sex
scores on sexual health
knowledge
w Percent of educated
girls who report acting
on new knowledge
with sexual health
education
w ASPIRE female
students
w Save the Children
w Records of student
attendance at sexual
health trainings to date
w Student surveys
w Sexual health test
w Key informant
students
program activities
w Key informant
interviews to follow
surveys and quantitative
data collection
8. Girls have better
understanding of when,
why, and how to seek
health care
w Number of girls who
have received health
care education
w Percent of educated
girls who report acting
on new knowledge
with health care
education
w ASPIRE female
students
w Save the Children
w Records of student
attendance at health
trainings to date
w Student surveys
w Observations
w Key informant
students
program activities
w Observations and key
follow surveys and
quantitative data
collection

ASHLEY DAY | 19
Appendix C: Instrument
Survey Tool for Female ASPIRE Students
Name of participant: _______________________________________
School of participant: ______________________________________
Participants’ English teacher: ________________________________
Year in school: ________________
1. Which reading activities do you like best? Which reading activities do you dislike?
2. Does your teacher encourage you to practice reading? How does he or she do this?
3. Do you feel comfortable asking and answering questions in class?
4. Are other students ever mean to you inside of the classroom? Yes No
5. If so, what do they do?
6. Do you feel safe when you are at school? Yes No
7. Explain why you feel safe or why you do not feel safe at school.
8. Does your school have a source of clean water? Yes No
9. If so, how does having clean water affect you at school?
10. What would be different if you didn’t have clean water at school?
11. Does your school have clean, private toilets? Yes No
12. If so, how does having clean, private toilets affect you at school?
13. What would be different if you didn’t have clean, private toilets at school?
14. Can you get feminine products at school or a nearby clinic? Yes No

ASHLEY DAY | 20
15. If so, how does being able to get feminine products affect you at school?
16. What would be different if you could not get feminine products at school or a clinic?
17. Did you like your sexual health class?
18. Did you learn new things in your sexual health class? Yes No
19. If so, what did you learn?
20. What else would you like to learn about in your sexual health class?
21. Will you act on these new things that you learned in your sexual health class? Yes No
22. If so, what will you do differently now that you know these things?
23. Did you like your health care class?
24. Did you learn new things in your health care class? Yes No
25. If so, what did you learn?
26. What else would you like to learn in your health care class?
27. Will you act on these new things that you learned in your health care class? Yes No
28. If so, what will you do differently now that you know these things?
29. What do you think would help you to do better in school?

ASHLEY DAY | 21

1
“Labor force, female (% of total labor force).” World Bank, n.d. Web. Nov. 2015.
2
Ministry of Health, Malawi. (2011, Sept.). Demographic and Household Survey 2010
3
USAID. (2015, Nov. 6). Gender Equality and Women’s Empowerment: USAID/Malawi’s Activities: Gender
Based Violence, Women’s Empowerment, and Adolescent Girls. Retrieved from https://usaid.gov/gender-
equality-and-womens-empowerment
4
USAID. (2015, Sept. 29). USAID Malawi Education Fact Sheet. Retrieved from
https://www.usaid.gov/malawi/fact-sheets/usaid-malawi-education-fact-sheet-2012-13
5
Ministry of Health, Malawi. (2011, Sept.). Demographic and Household Survey 2010
6
Ibid.
7
USAID. (2015, Nov. 6). Gender Equality and Women’s Empowerment: USAID/Malawi’s Activities: Gender
Based Violence, Women’s Empowerment, and Adolescent Girls. Retrieved from https://usaid.gov/gender-
equality-and-womens-empowerment
8
“Health expenditure per capita (current US$).” World Bank, n.d. Web. Nov. 2015.
9
“Life expectancy at birth, female (years).” World Bank, n.d. Web. Nov. 2015.
10
USAID. (2015, July 21). Country Profiles: Malawi: Planning for the Future. Retrieved from
https://www.usaid.gov/letgirlslearn/country-profiles
11
Federal Grants. (2014, June 19). The Girls Empowerment through Education and Health Activity. Retrieved
from http://www.federalgrants.com/The-Girls-Empowerment-through-Education-and-Health-Activity-
ASPIRE-46665.html
12
USAID. (2015, July 21). Country Profiles: Malawi: Planning for the Future. Retrieved from
https://www.usaid.gov/letgirlslearn/country-profiles
13
USAID. (2015, Sept. 29). USAID Malawi Education Fact Sheet. Retrieved from
https://www.usaid.gov/malawi/fact-sheets/usaid-malawi-education-fact-sheet-2012-13
14
Ibid.
15
Hargreaves, J. and T. Boler. (2006). Girl power: the impact of girls’ education on HIV and sexual
behaviour. ActionAid International.
16
Ibid.
17
Ibid.
18
Ibid.
19
Randolph, E. and S. Harvey. (2011, August). Malawi Teacher Professional Development Support
(MTPDS): Baseline Data Report. USAID/Malawi, Creative Associates International, RTI International,
and Seward Inc. Retrieved from http://pdf.usaid.gov/pdf_docs/PA00HVSP.pdf
20
Ibid.
21
Randolph, E., M. Nkhoma, and S. Backman. (2013, June). ABE/LINK Malawi Teacher Professional
Development Support (MTPDS): M&E Report. USAID/Malawi, Creative Associates International, RTI
International, and Seward Inc. Retrieved from http://pdf.usaid.gov/pdf_docs/PA00JB9W.pdf
22
Ibid.
23
Pouezevara, S., M. Costello, and O. Banda. (2012). ABE/LINK Malawi Teacher Professional
Development Support (MTPDS): Malawi Reading Intervention Early Grade Reading Assessment
(EGRA) Final Assessment. USAID/Malawi, Creative Associates International, RTI International, and
Seward Inc. Retrieved from http://pdf.usaid.gov/pdf_docs/PA00JB9Q.pdf
24
Herz, B., Subbarao, K., Habib, M., and Raney, L. (1991, Sept.) Letting girls learn: Promising
approaches in primary and secondary education. World Bank.
25
Ibid.
26
Ibid.
27
Smith-Greenaway, E. (2015, February). Are literacy skills associated with young adults' health in
Africa? Evidence from Malawi. Social Science & Medicine(27), p. 124-133.
doi:10.1016/j.socscimed.2014.07.036
28
Rossi et al., 2004. Evaluation: A Systematic Approach (7th Ed.). Sage Publication, Inc: New York, NY,
Chapter Seven: Measuring and Monitoring Program Evaluation

Proposing an Evaluation Plan: ASPIRE

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Proposing an Evaluation Plan: ASPIRE

Similaire à Proposing an Evaluation Plan: ASPIRE (20)

Dernier

Dernier (20)

Proposing an Evaluation Plan: ASPIRE