Myself and two colleagues present on the basics of Just-in-Time Teaching as well as the preliminary results of our research on the effectiveness of JiTT in different disciplines and for different types of students (as measured by the BIg Five personality traits).

1. Just-in-Time Teaching:
Tilting Classes Across the Academy
Jeff Loats, Arlene Sgoutas, & Randi Smith
(Physics) (Women’s Studies) (Psychology)
Colorado Learning and Teaching
with Technology
August, 2016

2. Presentation Overview
• What is it?
Introduction to Just-in-Time Teaching
(JiTT)
• Does it work?
Data from our courses
• How do I get started?
Recommendations for implementing JiTT

3. In your teaching do you have a method for holding
students accountable for preparing for class?
A) I don’t, but I ask/threaten really well.
B) I use a paper method (quiz, journal, others?)
C) I use a digital method (clickers, others?)
D) I have some other method.
3
18%
51%
15%
17%
(~250 others)

4. Consider a typical day in your class. What fraction
of students did their preparatory work before
coming to class?
A) 0% - 20%
B) 20% - 40%
C) 40% - 60%
D) 60% - 80%
E) 80% - 100%
4
29%
34%
29%
13%
5%
(~260 others)

5. Student Preparation Research
Quotes from Sappington, Kinsey, & Munsayac (2002)
"72% of Connor-Greene’s (2000) sample reported
that they rarely or never read their assignments by
the due date.”
"Burchfield and Sappington (2000): On any
given day, less than a third of students in
this population had adequately prepared
for class."
Recent USPIRG survey: 70% of students
admit that they sometimes don’t even
obtain required textbooks.

6. JUST-IN-TIME TEACHING
Online pre-class assignments
called WarmUps
First half - Students
• Conceptual questions, answered in sentences
• Graded on thoughtful effort
Second half - Instructor
• Responses are read “just in time”
• Instructor modifies that day’s plan accordingly.
• Aggregate and individual (anonymous) responses
are displayed in class.
Learne
r
Teacher

7. JUST-IN-TIME TEACHING
A different student role:
• Actively prepare for class
(not just reading/watching)
• Actively engage in class
• Compare your progress & plan accordingly
A different instructor role:
• Actively prepare for class with these humans
(not just going over last year’s notes )
• Modify class accordingly
• Create interactive engagement opportunities
Learne
r
Teacher

8. TOUTED BENEFITS OF JITT
• Student and instructor preparation
• Supports development of metacognitive skills
• Brings student voices into the classroom
• Promotes engagement with “higher-level”
questions (cf. Bloom’s taxonomy)
• Consistent with other research-based
instructional strategies
(learner-centered, universal design, etc.)

9. Example Questions &
Responses

10. Suppose you are interested
in researching why and how
some women fake orgasms.
How would you design a
study to answer those
questions?
Perhaps I’d tell a random group of women to fake
an orgasm. Additionally, a control group that is not
told to fake an orgasm should be included. After
both groups’ ensuing sexual encounters, ask very
specific and pointed questions about sexual arousal,
climax, or the lack thereof, and the experience
overall.

11. Identify and define the three different
conceptualizations of gender and sex that have
characterized the study of gender in recent years.
1.) Gender as equivalent to sex: gender is "rooted
in biological characteristics," or each animal is
assigned a gender according to their sex; females
are feminine, males are masculine.
2.)Gender as separate from sex: sex is the physical
characteristics (reproductive roles), and gender is
"personality roles" constructed from socialization.
3.) Gender as the assignment of meaning to bodies:
That gender and sex are both social constructs.

12. Sounds good in theory,
but does it actually work?
Convince me it’s worth my time…

13. Our Data: Set
• We examined data from over 20 courses across
5 disciplines, over 500 students total
–Upper- and lower-division courses
–Larger and smaller courses
–General studies, required major courses,
elective courses
• Our implementations of JiTT vary
–Weekly vs. biweekly
–5-20% of final course grade

14. Our Data: Participation Rates
• Response rates (averaged by class) vary from
~50% to ~80% across our two-dozen courses.
• Recall: The average preparation rate reported
by Burchfield & Sappington (2000) was
~33%.

15. 0%
20%
40%
60%
80%
100%
WMS 390Y (N = 19)
Avg. = 77%
0%
20%
40%
60%
80%
100%
PSY 2850 (N = 23)
Avg. = 68%
~Daily WarmUps
~Weekly WarmUps
0%
20%
40%
60%
80%
100%
Day 2 4 6 8 10 12 14 16 18 20 22 24 26 28
Physics 2711 (N = 24)
Avg. = 72%
0%
20%
40%
60%
80%
100%
Day 2 4 6 8 10 12 14 16 18 20 22 24 26 28
Physics 1000 (N = 22)
Avg. = 63%

16. Our Data: WarmUp
Correlations
• Bulk correlation of Total Grade ↔ WarmUps
– 𝒓 ranges from 𝒓 = 𝟎. 𝟏𝟑𝟕 to 𝒓 = 𝟎. 𝟖𝟒𝟐
–Combined* correlation: 𝒓 = 𝟎. 𝟓𝟕 (N = 581)
• Hattie (2009) noted that effect sizes for most
pedagogical techniques are small to medium
(see, e.g.,). From Cohen (1988):
𝒓 = 𝟎. 𝟑𝟎 is medium effect
𝒓 = 𝟎. 𝟓𝟎 is large effect

17. Our Data: Correlation Spread
*Correlations
were converted
to Fisher Zs,
averaged and
converted back
to Pearson r.
0.000
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
Total Grade vs. WarmUp
Correlations by Class
r = 0.57
Combined* correlation

18. Our Data: Testing Personality
Effects
• In trying to tease out causality, we are looking
to control for personality traits.
–The Big Five: openness, conscientiousness,
extraversion, agreeableness, and neuroticism.
–For example: Does the WarmUp ↔ Total
Grade correlation just come from
conscientiousness?
• So we cajoled 77 students into an extensive
personality assessment.

19. Our Data: Personality Effect
Results
From our 77 students we have
WarmUp ↔ Total Grade: 𝒓 = 𝟎. 𝟐𝟏𝟑 (puzzling)
Variable
Statistical
Significance
Extraversion 0.460
Agreeableness 0.258
Conscientiousness 0.227
Neuroticism 0.483
Openness 0.034
WarmUp Grade 0.032

20. PRELIMINARY RESEARCH
SUMMARY
 Participation rates appear to be quite stable
across disciplines, levels, course size, etc.
 The importance of WarmUps to learning is not
explained by student conscientiousness.
 Is JiTT especially effective in scaffolded
topics?
 Are students that completed the personality
assessment a skewed sample? Why is their
WarmUp ↔ Total Grade correlation so low?

21. Wow—you guys are so convincing.
I’m totally sold on this JiTT stuff.
Now what?
First… pause for questions?

22. A POSSIBLE PLAN
Choose one course you will teach next term.
A. Write two questions for each class meeting:
1. One lower-level (maybe multi-choice?).
One higher-level (sentences).
2. Give yourself 10 minutes to write each one
B. Write a standard (1st) metacognitive question
(What was most interesting or confusing …?)
C. Discuss one question at the top of class, and
one in the middle. Use the metacognitive
responses as break points or highlights.

23. OUR SUMMARY
JiTT may be among the easiest research-based
instructional strategies that you can consistently
integrate into your teaching.
JiTT is a research-based instructional strategy
that addresses often-neglected areas.
JiTT brings other benefits to students and
instructors beyond those measured in most
studies.

24. Strongly
Disagree
Strongly
Agree
1 2 3 4 5
Completing the warm-ups in this
class helps to prepare me for class
meetings.
I do more of my assigned reading
for this class than I would if there
weren't warm-ups assigned.
The warm-ups are worth the time
and effort required to complete
them because they help me learn.
I find it easier to learn the material
in this class because of the
warmups.
The warm-ups feel like a waste of
my time. 2.19
3.63
3.89
3.58
3.95
STUDENT SURVEY RESPONSES I

25. Strongly
Disagree
Strongly
Agree
1 2 3 4 5
The warm-ups are worth enough
points/credit toward my final grade
in this class, given the time and
effort they require.
Overall, I find I have to spend less
time studying for exams because of
the learning I've done for the
warmups.
I'm better able to determine what I
understand well and what I need to
study more because of the warmups.
My instructor uses the warm-ups in
a meaningful way to tailor class
meetings based on our needs.
The warm-ups are a nuisance to
complete.
2.58
4.09
3.28
2.74
3.59
STUDENT SURVEY RESPONSES
II

26. YOUR SUMMARY
If you want to implement JiTT, what is your next
concrete action?
Jeff Loats: jeff.loats@gmail.com, @JeffLoats
Arlene Sgoutas: sgoutasg@msudenver.edu
Randi Smith: rsmit216@msudenver.edu
Slides: www.slideshare.net/JeffLoats
Thanks for your attention!

28. JITT REFERENCES & RESOURCES
Second Book (interdisciplinary):
Simkins, Scott and Maier, Mark (Eds.) (2010) Just inTimeTeaching:Across the Disciplines, Across the Academy, Stylus Publishing.
Original Book (physics examples):
Gregor M. Novak, AndrewGavrin,Wolfgang Christian, Evelyn Patterson (1999) Just-in-TimeTeaching: BlendingActive Learning with
WebTechnology. Prentice Hall. Upper Saddle River NJ.
K.A. Marrs, and G. Novak. (2004). Just-in-TimeTeaching in Biology: Creating an Active LearnerClassroom Using the Internet.Cell
Biology Education, v. 3, p. 49-61.
Jay R. Howard (2004). Just-in-TimeTeaching in Sociology or How I Convinced My Students toActually Read the Assignment. Teaching
Sociology,Vol. 32 (No. 4 ). pp. 385-390. Published by:American SociologicalAssociation
StableURL: http://www.jstor.org/stable/3649666
S. Linneman,T. Plake (2006). Searching for the Difference:A ControlledTest of Just-in-TimeTeaching for Large-Enrollment
IntroductoryGeologyCourses. Journal of Geoscience Education,Vol. 54 (No. 1)
Stable URL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18
Sappington J, Kinsey K and Munsayac K (2002)Two studies of reading compliance among college students.Teaching of Psychology
29(4): 272–274.
http://orgs.bloomu.edu/tale/documents/reading_sappington_twostudies.pdf
Louis Deslauriers, Ellen Schelew and CarlWieman (2011). Improved Learning in a Large-Enrollment Physics Class. Science,Vol. 332 no.
6031 pp. 862-864 DOI: 10.1126/science.1201783
https://www.sciencemag.org/content/332/6031/862.abstract
Freeman S, et al. (2014) Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci
USA 111:8410–8415.
http://www.pnas.org/content/111/23/8410.abstract
ON-DEMAND SLIDES