Making communications land - Are they received and understood as intended? we...
Mapping evidence-based programmes for schools and the community using CAYT [March 2017]
1. Jamila Boughelaf
Head of Programmes
Mapping evidence-based
programmes for schools and
the community
The Centre for Analysis of Youth Transitions
2. Our approach
A holistic, life-course, systemic approach to prevention:
Developing life skills that build resilience to risk.
Throughout a young person’s lifetime
CMOannualreport:2011‘Onthestateofthepublic’shealth’
3. How do we improve alcohol and
drug education?
Evidence
•Accountability
•Confidence
•Learning and
improvement
•Evaluation of
impact
4. What is evidence?
• Rigorous evaluation assessing impact
and outcomes
• Replicability across multiple settings
• Sustained effects over time
“A well supported evidence-based intervention
programme is usually comprised of two
components: a strong magnitude of impact,
along with a fair and rigorous methodological
approach” (Nation et al., 2003)
5. How does evidence inform
practice?
Standards for
curriculum based
education
Evidence-based
off the shelf
programmes
6. Building our evidence base
Mentor-ADEPIS is publicly acknowledged as
the leading source of evidence-based resources
for alcohol and drug education and prevention for
schools.
7. CAYT database
Aims to draw together evidence on ‘what
works’ in terms of policies and
interventions designed to assist young
people in their transitions from education
to work, as well as reducing engagement
in risky behaviours.
cayt.mentor-adepis.org
8.
9. Target audience
• Commissioners willing to use the database
to obtain evidence of what really works to
improve outcomes
• Practitioners or service providers willing to
have their evaluation studies assessed for
impact and quality and then deposited in the
repository so that users can understand
whether their service has genuine impact.
10. Our focus
Early interventions and youth programmes
aimed at:
• Children, adolescents, teenagers and young
adults
• Preventing risky behaviour/promoting good
behaviour
• Ensuring better school attendance or
attainment for youth
• Ensuring better labour market outcomes for
youth
11.
12. Criteria for inclusion
• Clear and independent evaluation
• Clear theory of change with clear
outcomes
• Programme development based on
evidence
Inclusion of information on best and
‘emerging’ practice in the prevention field
13. Impact grades
High
Medium
Low
None
Level of evidence grades
(type of study)
Various
RCTs
RCT
Statistical
modelling
Use of comparison
groups
Before and after
evaluation
Statistical correlation
Descriptive
Basic study
Our approach
In today’s complex world, young people need life skills that help them negotiate challenging situations and build their resilience to a range of negative risks.
We believe the best strategy for supporting the immediate and long-term well-being of children and young people is through a holistic life-course approach to intervention and prevention. This means targeting a variety of settings, including schools, communities and families, as well as ensuring interventions take place early and often, rather than acting in the later years when problematic behaviours are more difficult to address and change.
dispelling some more linear approaches, e.g.:
· Focus on providing drugs harm information alone does not work
· Awareness campaigns don’t work
· Only reaching YP via school doesn’t work
· D&A focus alone fails to recognise broader risks
· Focus on enforcement legislation (e.g. NPS) alone doesn’t work
Using solid evidence can help us measure impact, better inform policy guidance and improve professional service delivery. Evidence should be used in the design, delivery and evaluation of programmes targeting young people.
Here are some important principles about using evidence:
Evidence is support, not proof or truth, of an assertion
Evidence provides a deeper understanding and insight into the impact of our work
Evidence gives us opportunities for reflection and improvement
Evidence is for anyone, adding value to all those involved in the delivery of, or benefitting from, services
Generally, evidence is intended as any argument or “proof” in support of a conclusion or assertion. It can be seen as information that helps us prove or demonstrate truth – or disprove something that is false.
Prevention science has been expanding in recent years and it is now used to address practitioners and policymakers when selecting and designing interventions. Drawing on the previous criteria, prevention scientists produced International standards to assess prevention programmes.
"Evidence-based practice" – meaning "best practice" or "with well-supported evidence" – is a crucial element in policy development and the implementation of programmes in the prevention field. When selecting prevention programmes for young people, policy makers, practitioners and health and education professionals need easy access to reliable and independently validated information.
Mentor is working for an effective, comprehensive and national prevention strategy, through families, schools and communities. No magic bullet can prevent a young person experimenting with alcohol or drugs but we want to create an ecosystem of prevention in the UK which increases protective factors and reduces risks.
Mentor advocates for programmes that have been proven by hard evidence to change young people’s attitudes and behaviour to alcohol and drugs and to (re)engage them in education, training volunteering and work.
GBG – Largest RCT of its kind currently running
Quality Assessment framework and quality mark for schools
Building our evidence base
Evidence is crucial to our work, as it helps ensure programmes are effective at protecting children and young people from the harms of drugs and alcohol.
The Alcohol and Drug Education and Prevention Information Service is a platform for sharing information and resources aimed at schools and practitioners working in drug and alcohol prevention.
The resources we have already produced draw on eight years of work with the Drug Education Forum, which supported local authorities and schools to implement best practice in drug education.
In April 2015, Mentor was granted additional joint government funding to manage the Centre for Analysis of Youth Transitions (CAYT) and integrate its repository of evidence-based impact studies into Mentor-ADEPIS.
CAYT promotes and shares all those interventions based on “what works” in terms of evidence. It serves as a repository of evidence based prevention programs making available information on best and promising practice in the prevention field.
In doing this CAYT differentiates from other repositories 1) targeting an audience made of practitioners, teachers, NGOs etc rather than academics and researchers 2) including a wide range of programmes even the “emerging” ones that would otherwise be considered as lacking strong evidence
3) Making such resources and information available to everyone
Criteria for inclusion reflect this vision and are
Clear and independent evaluation: We tend to accept those programmes that provide an unbiased evaluation, normally conducted by an independent researcher –this regardless the level of evidence attached-
Clear theory of change and outcomes: We want to look at those programmes whose outcomes and theory of change can contribute to CAYT’s goals and vision
Programmes based on evidence and/or promoting the use of evidence in the process of programme design