This document summarizes research from Andy J. Saltarelli on investigating the effects of social context on educational outcomes. It discusses four studies that took diverse approaches: 1) A phenomenological study of the Lost Boys of Sudan that found they developed a cultural hybrid and demonstrated educational resilience. 2) An experiment showing that satisfying belongingness needs can promote online cooperation. 3) An intervention study that closed global participation gaps in MOOCs by addressing social belonging and identity threats. 4) A qualitative study of MOOC faculty sensemaking strategies in open online initiatives. The document advocates for mixed methods and diverse approaches to research on learning in social contexts.

1. Andy J. Saltarelli, PhD
andysaltarelli.com
A Walk Around Pasteur’s
Quadrant:
Diverse Approaches to
Investigating the Effects of Social
Context on Educational
Outcomes

2. Background

3. Human
Development
Educational
Psychology
Educational
Technology

4. “It seems clear to me that we are
well past the quantitative–
qualitative debate and more
concerned with issues of providing
good, valid, and reliable evidence
to support our inferences and
conceptual models, regardless of
the nature of the general
methodology.” ~ Pintrich, 2000, p.
223

5. Goal of Usefulness
Goal of
Scientific
Understanding
YesNo
Yes
No
Basic
Research
(Bohr)
Use-inspired
basic
research
(Pasteur)
Pure applied
research
(Edison)
Pasteur’s
Quadrant
(Pintrich, 2000; Stokes, 1997)

6. Goal of Usefulness
Goal of
Scientific
Understanding
YesNo
Yes
No
Lost Boys
Online
Cooperatio
n
Multimedia
Learning
MOOC
Faculty
MOOC
Learners

7. Research

8. Research
Lost Boys of Sudan

9. Existential-phenomenological inquiry
(Gilgun, 2005; Giorgi & Giorgi, 2003)
Followed 19 youth over 7 years after being
resettled in US
Multiple semi-structured interviews with
youth, foster parents, and social workers
Lost Boys of Sudan

10. Cultural creation combining the good
parts of American culture with the
good parts of native Sudanese culture
“I’ve become like a hybrid between
here, two cultures you know and these
two cultures make me, I’m making
good thing out of it.”
Lost Boys of Sudan
(Qin, Saltarelli, et al., 2014, Journal of Adolescent
Research)
“Cultural appropriation”, integrative
adaptation

11. 19 HS diplomas, 7 bachelor’s, 10
community college, 4 master’s
“Every single one of us has to go to
college because we need to go
back and help.”
“Education is my mother and education
is my father” (Chanoff, 2005)
Lost Boys of Sudan
(Rana…Saltarelli, 2011, Teachers College
Record)
Educational resilience

12. Research
Effects of Belongingness and
Synchronicity on Face-to-face
and Online Cooperative
Learning

13. Does adapting face-to-face (FTF)
pedagogies to online settings raise
‘boundary questions’ about whether
the same pedagogy stimulates
different psychological processes
under FTF and online conditions?

14. Social Interdependence
Theory
(Deutsch 1949; Lewin, 1948; D. W. Johnson & Johnson, 1989,
2005)
Interdependent Goal
Structures (Positive
Interdependence)
Promotive Interaction
Goal Achievement
+Motivation, +Achievement,
+Well-being, +Relationships

15. Constructive Controversy
(Deutsch 1949; Lewin, 1948; Johnson & Johnson, 1998;
2009)
Argue incompatible views within a cooperative context
Seek agreement integrating the best evidence and
reasoning from both positions
5-step Procedure:

16. Constructive Controversy
40 Years of Research — Meta-Analysis
(Johnson & Johnson, 2009)
(ES = Mean Effect
Sizes)
Constructive Controversy
v. Debate
Constructive
Controversy v.
Individualistic
Achievement .62 ES .76 ES
Perspective
Taking .97 ES .59 ES
Motivation .73 ES .65 ES
Self-esteem .56 ES .85 ES
In face-to-face
settings

17. MEDIARICHNESS
SYNCHRONICITY
Face-To-Face
VideoAudioText
Synchronous Asynchronous
Online Constructive Controversy
Test Constructive Controversy
1 FTF x 2 Synchronicity (Sync, Async) x 3 Media (Audio, Video, Text)
(Roseth, Saltarelli, & Glass 2011, Journal of Education Psychology).

18. Previous Results
(Roseth, Saltarelli, & Glass, 2011; Journal of Educational Psychology)
In Asynchronous CMC
Achievement↓ Motivation↓ Belongingness↓
Theory: What are the mechanisms by which asynchronous
CMC affects constructive controversy?
Practice: Can satisfying belongingness needs ameliorate the
negative effects of asynchronous CMC?

19. Belongingness
(Baumeister & Leary, 1995; Deci & Ryan, 2000; Walton et al., 2012)
Innate
Needs
Competence
Belongingness
Autonomy
Self-Regulation
Motivation

20. SYNCHRONICITY
Face-To-Face Synchronous Asynchronous
Belongingness
Test Constructive Controversy
3 Synchronicity (FTF, Sync, Async) x 3 Belongingness (Acceptance, Control, Mild
Rejection)
(Saltarelli & Roseth, 2014, Journal of Education Psychology).

21. Belongingness Manipulation
(Romero-Canyas et al., 2010)
Complete personality
profile

22. Belongingness Manipulation
Rank potential partners based on profile

23. Belongingness Manipulation
Partner pairing

24. Synchronous Scaffold
Synchronous CMC Scaffold:
- WordPress, Google DocsTM
- Integrated text-based chat
Procedure:
- Complete initial
belongingness activity
- Dyads complete activity
over 70 min. class period

25. Asynchronous Scaffold
Asynchronous CMC
Scaffold:
- Custom WordPress,
BuddyPress
Procedure:
- Complete initial
belongingness activity
- Dyads complete activity
over 6 days

26. Dependent Variables
Operationalization
1. Time Time spent? (1-item), Time preferred?(1-item)
2. Social
Interdependence
Cooperation (7-items, α=.89), Competition (7-items, α=.93),
Individualism (7-items, α=.86
3. Conflict
Regulation
Relational Regulation (3-items, α=.80), Epistemic Regulation (3-
items, α=.82)
4. Motivation
Relatedness (8-items, α=.88), Interest (7-items, α=.92), Value (7-
items, α=.93)
5. Achievement
Multiple-choice questions (4-items, α=.41), Integrative statement: #
of arguments (κ=.95), use of evidence (κ=.90), integrative (κ=.87)
6. Perceptions of
Technology
Technology Acceptance (4-items, α=.90), Task-technology Fit (2-
items, α=.94)
DV

27. Summary of Findings
Async
CMC
FTF and Sync
CMC
▲ Cooperative perceptions
▲ Epistemic conflict
Led to…
▲ Motiv
▲
Achievement
▲ Competitive perceptions
▲ Relational conflict
Led to…
▼Motivation

28. Summary of Findings
Belongingnes
s
Met
▲Cooperative perceptions
▲ Epistemic regulation
▲ Intrinsic motivation
▲ Perceptions of
technology
Buffers but does not offset
the deleterious effects of
asynchronous CMC

29. Implications for Practice
▲ Satisfying belongingness needs can promote
cooperation and motivation in online contexts
▲ Instructors may be able to monitor and enhance
cooperative perceptions and epistemic regulation
▲ Varying synchronicity to match task demands may
maximize affordances and minimize constraints

30. Looking Forward & DBR

31. Looking Forward & DBR

32. Looking Forward & DBR

33. Research
Psychological Interventions
Close Global Participation Gaps
in Online Learning

34. Reasoning
together
http://lytics.stanford.edu/
Computer
Science
Learning
Science
EDM
LAK L@S
Sociology
Social
Psychology
Sociology
Design
“We engage in use-driven research and data-driven de

35. Data does not speak for itself. Rather,
people must actively make meaning of
the data… interpretation is a central part
of the data use process…noticing,
interpreting, and constructing
implications for action— are shaped by
individual beliefs, knowledge, and
motivation and are influenced by the
nature and patterns of social interaction

36. MOCs may eye the world market, but
does the world want them?
Global Participation Gap
16 MOOCs (N~ 67,000):
- 50% less persistence (Kizilcec & Halawa, in press )
Previous run of course (N= 41,186):
- 50% less video lectures watched
- 40% less likely to take assessments
Current self-paced version (N = 60,000 enrolled, 4,562 did intervention)
- 75% less video lectures watched
- 82% less likely to take assessments
Why?
- Language Barriers
- Access to Technology
(Kizilcec, Saltarelli, & Cohen, Under Review,
PNAS)
Western (Europe, Oceania, and Northern America) v. Non-
Western (Africa, Asia, and Latin America)

37. MOOCs may eye the world market, but
does the world want them?

38. Social Belonging & Identity
Threat to belongingness (Walton et al., 2014)
Threat to identity (Cohen et al., 2006, 2009, 2014; Steele, 1988)
Cultural mismatch (Harackiewicz et al., 2014; Stephens et al.,

39. Social Belonging & Identity
"I didn’t go to a very good university, and I
worried that my previous courses had not
prepared me well for this online course.
Honestly, when I first enrolled, I thought the
instructor was a bit scary. I thought the
grading was critical and hard, and I worried
about whether other students would
respect me. I was nervous about writing on
the discussion forum and I didn’t want to
ask people for help with quizzes. ~ Tom

40. Psychological Interventions
Intervention embedded in course survey
N = 4,562
Study Tips Self-Affirmation Social Belonging
1. Read Quotes
2. Write reflection
3. Write letter
1. Choose key values
2. Write reflection
3. Write letter
1. Read Quotes
2. Write reflection
3. Write letter
Perceived Beneficiary
Self Other Self Other Self Other

41. Results
Gaps completely closed by…
- Belongingness intervention if peer recipient
▲ 73% videos watched (gap: z = 0.98, P = 0.33)
- Self-affirmation intervention if self recipient
▲ 55% videos watch (gap: z = 0.85, P = 0.39)
▲ 100% assessments taken (gap: z = -0.72, P = 0.47)
No statistically significant effect on Western
learners

42. Research
“My Goal Is To Surf It, Not Just
Stand There”: Professors’
Sensemaking Strategies in
University Open Online
Learning Initiatives

43. Method
▲ Three-part semi-structured interviews with 16 MOOC
faculty
▲ Grounded theory (Corbin & Strauss, 1994, 2007,
2014)
▲Emotion and metaphorical language codes (Plutchik,
2011)
Sensemaking explores the social contexts in which
meaning is constructed; it is grounded in multiple
dimensions of a person’s identity; it emphasizes
retrospective meaning-making of events, choices, and

44. Preliminary Results
▲ Incredible diversity of motivations, 13 distinct codes
▲ Role soft infrastructure – institutional values,
affirmations, ethos of intellectual generosity
▲ Expressions of agency and joy, cf. rhetoric of
disruption and unbunlding (Carey, 2015; Christensen &
Weise, 2014)
“And that is so empowering. It's such a great feeling to
be in a place where you can have a new idea and
somebody will support you in the exploration of that

45. ▲ Lost Boys – give voice to and understand lived
experience, qualitative phenomenology
▲ Online Cooperation – theory testing and design-
based research in authentic setting, quantitative
experimental-control
▲MOOC Learners/Gaps – leverage big-er data,
experimental-control, scale “wise” interventions
▲MOOC Faculty – give voice to, action research,
program evaluation, qualitative grounded theory
Summary