Presentation from "The challenges of measuring informal science learning" at the 2013 Science Communication Conference organised by the British Science Association - slides by Steph Sinclair

1. Informal Science Learning
Stephanie Sinclair

2. Today’s Talk
• Reminder what the Review set out to achieve
• Highlight key findings from the Informal Learning Review
• Examine what this means for the sector

3. Aims of the Review
Practical outcomes of the study:
• A better understanding of the scope of informal learning, its
theoretical base and the types of change it can bring about in
people’s understanding, behaviour and attitudes to science
• Better understanding of how to evaluate the impact of informal
science learning
• Best practice in reaching deprived learners schools and families
• Best practice in linking informal and formal learning.

4. 2 Reports Commissioned
• From UK-based GHK Consulting – report examines who is doing what,
who are they reaching and how are they evaluating.
• From Stanford - Oregon State University- examining the science
education ‘ecosystem’ and the role of informal learning within it.
• Plus a commentary piece from John Holman and Clare Matterson

5. Fieldwork
Autumn 2011 – Spring 2012
Interviews: 60 stakeholders of informal learning
Surveys: online surveys of informal learning providers
Case studies: 10 exemplar organisations
Family studies: family choices for their leisure time
Literature review: academic and ‘grey’ literature
Ecological mapping: to describe the science learning eco-system

6. Findings – the Informal Learning Landscape
• Diversity: ‘…. As far as we can tell, such diversity is not matched
elsewhere’
• Strong sense of mission
• Complex interactions…

7. Quantity of interactions among UK science education sectors – greatest
interactivity in the middle
Stanford-Oregon: Analysing the UK Science Education Community: The contribution of

8. Findings – Evaluation
Evaluation activity is widespread but unsophisticated:
- user surveys are most common methodology used
- mostly done by providers themselves, not by external evaluators
‘Overall, this is a community eager to find out what its users think of its
activities, but less inclined to measure long-term impact’

9. What best describes your evaluation activity?
Statement Responses Percentage
Formative evaluation 160 91%
Summative evaluation 29 15%
Evaluate when required by funders 69 39%
Planned rolling evaluation
programme in place 37
21%
Evaluate if concerns identified 32 18%
Evaluation undertaken on an ad hoc
basis 24 14%

10. Barriers to evaluation

11. Findings – Research, practice and profession

12. How well known is the literature?
• Modal value for how many individuals had read most cited publications =
0
• Modal value for how many recognised but had not read each publication
= 2
• Most read article read by < 50%
So what do they read?
- Policy documents, evaluations and online resources (eg > 50% read the
ASDC newsletter)

13. So what do these findings mean for the sector?
- How we evaluate
- How we increase research base and research capacity
- Better linking research and practice

14. Moving towards a more sophisticated evaluation model
• greater sharing of results
• identification of a common set of related proximal and distal indicators of
impact
• working with other sectors to test alternative methods

15. Linking research and practice
What Wellcome Trust are doing
•Exploring potential programme of research grants with National Science
Foundation
What we can all do
-Ensure we are using research to underpin our own practice