Presented at the HEA Annual Conference, 3 July 2014

1. Analytics for profiling and promoting
learners’ epistemologies (APPLE)
PHASE 1: DEVELOPING THE SURVEY TOOL
Bethany Alden Rivers, The University of Northampton
John Richardson, The Open University

2. Epistemic beliefs
are our ideas about
‘knowledge and
knowing’.
(Hofer, 2002, p. 3)

3. ‘…how students themselves think
about knowledge, learning and
teaching is a primary factor
influencing their experience of
higher education itself.’
(Richardson, 2013, p. 192).

4. Universities need to care more
about developing learners’
epistemologies.
(Lucas & Tan, 2013)

5. The problem…
Researchers need better ways of understanding epistemic
beliefs (Schraw, et al., 2002).
‘Well-validated quantitative instruments that could be used
to measure epistemological development in large samples
of students are still lacking’ (Richardson, 2013, p. 201).

6. Epistemic Beliefs Inventory (EBI)
Measure of Epistemological Review
(Baxter Magolda, 1992)
Argumentative Reasoning
(Kuhn, 1991)
Reasoning Knowing
+

7. Argumentative reasoning (Kuhn, 1991)
Used ill-structured questions to determine:
Absolutist believe that experts could know or find specific
causes of problems.
Multiplist deny any certainty between experts and believe their
own view is just as plausible as an expert’s.
Evaluativist knowledge is uncertain but experts do exist
Realist pre-absolutist, with an absent epistemology (Kuhn &
Weinstock , 2002)

8. Measure of epistemological reflection (Baxter Magolda, 1996)
Role of the learner
Role of the peers
Role of the instructor
Evaluation
Nature of knowledge
Absolute knowing Transitional knowing Independent knowing Contextual knowing
Ways of Knowing
5 Domains

9. Theoretical framework
Reasoning
(Kuhn & Weinstock, 2002)
Knowing
(Baxter Magolda, 1996)
Realist
Absolutist
Multiplist Transitional Knowing
Independent Knowing
Contextual Knowing
Absolute Knowing
Evaluativist

10. Creating the Epistemic Beliefs Inventory
• Converting qualitative interview protocol into
quantitative ‘statements’.
• 5-point Likert scale
• 3 demographic questions: age, gender, educational
background
• 12 questions related to Reasoning
• 20 questions related to Knowing

11. Examples of EBI statements for
Reasoning
• Experts know what causes prisoners to return to crime.
• I know what causes prisoners to return to crime.
• An expert would probably share my views on what causes
prisoners to return to crime.
• It is possible to have more than one point of view as to what
causes prisoners to return to crime.
• Compared to the average person, I know a lot about what causes
prisoners to return to crime.

12. Examples of EBI statements for Knowing
• The goal of learning is to come up with my own perspective on things.
(Role of the Learner)
• My classmates play a very important role in my own learning. (Role of
Peers)
• The main goal of an instructor should be to help students think for
themselves. (Role of the Instructor)
• I think my instructor should reward me when I think for myself.
(Evaluation)
• Not everything my instructor says is true because we all have our own
beliefs about things. (Nature of Knowledge)

14. Research questions
1. Are survey items 1-12 a reliable measure of one’s capacity for reasoning?
2. Are survey items 13-32 a reliable measure of one’s capacity for knowing?
3. Is there a correlation between reasoning and knowing?
4. Is there a correlation between demographics, reasoning and knowing?

15. The sample
• Web surveys were sent to students at the University of Northampton via an
e-newsletter.
• Completed (anonymous) surveys were returned by 77 students.
• Most of the participants reported having ‘some university experience’, A-
Levels, or GCSEs.
• 77% of the participants were female.
• The mean age was 31 years.

16. 1. Are survey items 1-12 a reliable measure of
one’s capacity for reasoning?
• Cronbach’s alpha (all variables) = 0.83, which shows good inter-reliability
• Exploratory factor analysis showed two possible factors, and this was
confirmed by comparison with the results of a parallel analysis of 1000
random variables.
• Factor 1 explained 41.14% of variance with salient loadings on 8 variables.
• Factor 2 explained 19.51% of variance with salient loadings on 4 variables.
• Cronbach’s alpha (Factor 1) = 0.90, which shows excellent inter-reliability.
• Cronbach’s alpha (Factor 2) = 0.75, which is also good.

17. 2 possible constructs for Reasoning?
• Factor 2 showed salient loadings on 4 variables.
• Mean ranking on Factor 2 variables was very low (M = 1.47).
• Reviewing the ‘statements’ related to these variables, it is clear
that they are worded in a way where a low response (i.e.
Definitely Agree) is optimal.
• E.g. ‘It is possible to have more than one point of view as to why
prisoners return to crime.’
• For the next iteration, these statements may need to be
reworded and another factor analysis be carried out.

18. 2. Are survey items 13-32 a reliable measure of
one’s capacity for knowing?
• Exploratory factor analysis showed four possible factors, and this was
confirmed by comparison with the results of a parallel analysis of 1000
random variables.
• Factor 1 explained 20.93% of variance with salient loadings on 6 variables.
• Factor 2 explained 13.20% of variance with salient loadings on 4 variables.
• Factor 3 explained 9.88% of variance with salient loadings on 3 variables.
• Factor 4 explained 7.96% of variance with salient loadings on 4 variables.

19. 4 possible constructs for Knowing?
Factor 1 (statements):
• An instructor should work with each student to set goals for learning.
• Students and instructors should critique each other.
• The main goal of an instructor should be to help students think for themselves.
• It is important for instructors to help students apply knowledge to their own lives.
• I think my instructor should reward me when I think for myself.
• My instructor should have a way to measure how much I understand the newly
acquired knowledge.
• DOMAINS 3 and 4 = THE ROLE OF INSTRUCTOR AND ASSESSMENT

20. 4 possible constructs for Knowing?
Factor 2 (statements):
• I share my views with my classmates because I often gain a different perspective on
things. Students and instructors should critique each other.
• I like to express my opinions to my classmates so we can talk about different views.
It is important for instructors to help students apply knowledge to their own lives.
• My classmates play a very important role in my own learning.
• Even though I could learn just fine without my classmates, it is sometimes nice to
have them around to talk to about what we're learning.
• DOMAIN 2: THE ROLE OF PEERS

21. 4 possible constructs for Knowing?
Factor 3 (statements):
• What my instructor teaches me is true.
• The role of an instructor is to transmit knowledge to their students.
• Not everything my instructor says is true because we all have our own
beliefs about things
DOMAIN 5: NATURE OF KNOWLEDGE

22. 4 possible constructs for Knowing?
Factor 4 (statements):
• Learning is about obtaining knowledge from someone who knows more
than I do.
• It is necessary for me to interpret the information I receive from my
instructor in order to understand it.
• The goal of learning is to come up with my own perspective on things.
• Learning is about using knowledge to think through problems.
DOMAIN 1: ROLE OF THE LEARNER

23. Four Constructs for Knowing
1. The Role of the Instructor and Assessment
2. The Role of Peers
3. Nature of Knowledge
4. Role of the Learner

24. 3. Is there a correlation between reasoning and
knowing?
• 2-tailed test was carried out to test correlations across all factors
• Factor 1 (from Reasoning) and Factor 1 (from Knowing) was the
only relationship that was statistically significant. (p < 0.05)
• This indicates that a student’s capacity for reasoning is
significantly correlated to his or her view of the role that the
instructor and assessment plays in learning.

25. 4. Is there a correlation between
demographics, reasoning and knowing?
• Most of the participants reported having ‘some university
experience’, A-Levels, or GCSEs.
• 77% of the participants were female.
• The mean age was 31 years.
• There was a statistically significant relationship between Education
and Factor 1 (Role of Instructor and Assessment). Lower mean
rankings reported by students with ‘no previous education’.
• There was a statistically significant pattern involving Education *
Gender and Factor 3 (Nature of Knowledge). Lower mean rankings
reported by male participants.

26. What’s next?
Take stock of what we’ve learned so far.
Rework the EBI to address problematic components.
Second iteration of testing September to November 2014
Use of visualisations to research a case for enhancing practice…

27. What can an epistemological profile
allow us to do?

28. A basis for development
A bespoke personal development plan
Personalised content
Curriculum mapping

29. The bigger problem?
How do we
profile and
promote
learners’
epistemic beliefs
in large online
communities of
practice, such as
MOOCs?

30. References
Baxter Magolda, M. (1992) Knowing and Reasoning in College: Gender-Related Patterns in
Students’ Intellectual Development, San Francisco, Jossey-Bass.
Hofer, B. K. (2002) ‘Personal epistemology as a psychological and educational construct: an
introduction’, in Hofer, B. K. and Pintrich, P. R. (eds.) Personal Epistemology: The Psychology of
Beliefs about Knowledge and Knowing, Mahwah, NJ, Erlbaum.
Lucas, U. and Tan, P. L. (2013) ‘Developing a capacity to engage in critical reflection: students’
‘ways of knowing’ within an undergraduate business and accounting programme’, Studies in
Higher Education, vol. 38, no. 1, pp. 104-123.
Kuhn, D. (1991) The Skills of Argument, Cambridge, Cambridge University Press.
Richardson, J. T. E. (2013) Epistemological Development, Educational Research Review, vol. 9, pp.
191-206.
Schraw, G., Bendixen, L. D., & Dunkle, M. E. (2002). ‘Development and validation of the Epistemic
Belief Inventory (EBI)’. In B. K. Hofer & P. R. Pintrich (Eds.),Personal epistemology: The psychology
of beliefs about knowledge and knowing (pp. 261–275). Mahwah, NJ: Erlbaum.

31. Bethany Alden Rivers
Bethany.Alden@northampton.ac.uk
Project site:
www.aldenrivers.com/apple
Twitter: @Alden_Rivers