Slides from a 40-minutes panel presentation discussing the effectiveness of Just-in-Time Teaching across many disciplines, levels of courses and course-types. Presenters: Arlene Sgoutas Jeff Loats Randi Smith Courtney Rocheleau

1. Just-in-Time Teaching
A Strategy For Success
Arlene Sgoutas, Jeff Loats,
Courtney Rocheleau, & Randi Smith
Teaching and Learning
with Technology
Symposium

2. Presentation Overview
• What is it?
Introduction to Just-in-Time Teaching
(JiTT)
• Does it work?
Data from our courses
• How do I get started?
Recommendations for implementing JiTT

3. In your teaching do you have a method for
holding students accountable for preparing for
class?
A) I don’t, but I ask/threaten really well.
B) I use a paper method (quiz, journal,
others?)
C) I use a digital method (clickers, others?)
D) I have some other method.
3
17%
51%
11%
17%
(~230 others)

4. Consider a typical day in your class. What
fraction of students did their preparatory work
before coming to class?
A) 0% - 20%
B) 20% - 40%
C) 40% - 60%
D) 60% - 80%
E) 80% - 100%
4
28%
34%
20%
14%
5%
(~260 others)

5. Student Preparation
Quotes from Sappington, Kinsey, & Munsayac (2002)
"72% of Connor-Greene’s (2000) sample
reported that they rarely or never read their
assignments by the due date.”
"Burchfield and Sappington (2000): On
any given day, less than a third of students
in this population had adequately
prepared for class."
Recent USPIRG survey: 70% of students
admit that they sometimes don’t even
obtain required textbooks.

6. JUST-IN-TIME TEACHING
Online pre-class assignments
called WarmUps
First half - Students
• Conceptual questions, answered in sentences
• Graded on thoughtful effort
Second half - Instructor
• Responses are read “just in time”
• Instructor modifies that day’s plan accordingly.
• Aggregate and individual (anonymous) responses
are displayed in class.
Learne
r
Teacher

7. JUST-IN-TIME TEACHING
A different student role:
• Actively prepare for class
(not just reading/watching)
• Actively engage in class
• Compare your progress & plan accordingly
A different instructor role:
• Actively prepare for class with these humans
(not just going over last year’s notes )
• Modify class accordingly
• Create interactive engagement opportunities
Learne
r
Teacher

8. BENEFITS OF JITT
• Student and instructor preparation
• Supports development of metacognitive skills
• Brings student voices into the classroom
• Promotes engagement with “higher-level”
questions (cf. Bloom’s taxonomy)
• Consistent with other research-based
instructional strategies
(learner-centered, universal design, etc.)

9. Example Questions &
Responses

10. Suppose you are
interested in
researching why and
how some women fake
orgasms. How would
you design a study to
answer those
questions?
Perhaps I’d tell a random group of women
to fake an orgasm. Additionally, a control
group that is not told to fake an orgasm
should be included. After both groups’
ensuing sexual encounters, ask very
specific and pointed questions about
sexual arousal, climax, or the lack thereof,

11. Give an example of a time you conformed due
to normative influence. Give an example of a
time you conformed due to informational
influence.
I gave into normative social influence when I
bought a pair of Uggs because I felt obligated
to follow the norm. I have given into
informational social influence when I moved to
a new school and had to ask students around
here how to act in certain situations because it
was not like my other school.

12. Sounds good in theory, but does it
actually work?
Convince me it’s worth my time…

13. Our Studies
• Have examined data from 18 courses
across 5 disciplines
– Upper- and lower-division courses
– Larger and smaller courses
– General studies, required major courses,
elective courses
• Vary in implementation of JiTT
– Weekly vs. biweekly
– 5-20% of final course grade

14. Data from Our Courses
Overall Response Rates:
46% - 74% average response rate across
the entire 15-week semester.
(mean of averages across 18 classes =
59%)
• Recall: ~33% in Burchfield & Sappington
(2000)

15. 0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14
PSY2850: Smith, N = 33
PSY2410: Rocheleau, N = 36 WMS 390Z: Sgoutas, N = 18
0
20
40
60
80
100
1 2 3 4 5 7 8 9 10 11 14 15 16 17 18 19 22 23 24 25 26 27
PHY2010: Loats, N = 78
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13
PSY1001: Smith, N = 22
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
PHY2020: Loats, N = 73
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

16. Course Performance
Effect sizes for most pedagogical techniques
are small to medium (see, e.g., Hattie, 2009)
Correlations with total grade (w/warm-ups
partialed out) across 18 classes range from
r = 0.38 – 0.84; Mr = .60 (r2 = .36)
• Per Cohen (1988),
r = .30 is medium effect,
r = .50 is large effect

17. Limitations
• Could be alternative
explanations/confounds that are
uncontrolled in these analyses; e.g.,
– Academic motivation
– Quality of answer
– Conscientiousness
– Interest in course
• But pattern repeated across wide variety
of courses

18. Wow—you guys are so convincing.
I’m totally sold on this JiTT stuff.
Now what?
Questions?

19. A POSSIBLE PLAN
Choose one course you will teach next term.
A. Write two questions for each class meeting:
1. One lower-level (maybe multi-choice?).
One higher-level (sentences).
2. Give yourself 10 minutes to write each one
B. Write a standard (1st) metacognitive question
(What was most interesting or confusing …?)
C. Discuss one question at the top of class, and
one in the middle. Use the metacognitive
responses as break points or highlights.

20. OUR SUMMARY
JiTT may be among the easiest research-based
instructional strategies that you can consistently
integrate into your teaching.
From an evidence-based perspective, JiTT
addresses often-neglected areas.

21. YOUR SUMMARY
If you want to implement JiTT, what is your next
concrete action?
Randi Smith: rsmit216@msudenver.edu
Jeff Loats: jeff.loats@gmail.com, @JeffLoats
Courtney Rocheleau:
crochel1@msudenver.edu
Arlene Sgoutas: sgoutasg@msudenver.edu
Slides: www.slideshare.net/JeffLoats
Thanks for your attention!

22. JITT REFERENCES & RESOURCES
Second Book (interdisciplinary):
Simkins, Scott and Maier, Mark (Eds.) (2010) Just inTimeTeaching:Across the Disciplines, Across the Academy, Stylus Publishing.
Original Book (physics examples):
Gregor M. Novak, AndrewGavrin,Wolfgang Christian, Evelyn Patterson (1999) Just-in-TimeTeaching: BlendingActive Learning with
WebTechnology. Prentice Hall. Upper Saddle River NJ.
K.A. Marrs, and G. Novak. (2004). Just-in-TimeTeaching in Biology: Creating an Active LearnerClassroom Using the Internet. Cell
Biology Education, v. 3, p. 49-61.
Jay R. Howard (2004). Just-in-TimeTeaching in Sociology or How I Convinced My Students toActually Read the Assignment. Teaching
Sociology,Vol. 32 (No. 4 ). pp. 385-390. Published by:American SociologicalAssociation
StableURL: http://www.jstor.org/stable/3649666
S. Linneman,T. Plake (2006). Searching for the Difference:A ControlledTest of Just-in-TimeTeaching for Large-Enrollment
IntroductoryGeologyCourses. Journal of Geoscience Education,Vol. 54 (No. 1)
StableURL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18
Sappington J, Kinsey K and Munsayac K (2002)Two studies of reading compliance among college students.Teaching of Psychology
29(4): 272–274.
http://orgs.bloomu.edu/tale/documents/reading_sappington_twostudies.pdf
Louis Deslauriers, Ellen Schelew and CarlWieman (2011). Improved Learning in a Large-Enrollment Physics Class. Science,Vol. 332 no.
6031 pp. 862-864 DOI: 10.1126/science.1201783
https://www.sciencemag.org/content/332/6031/862.abstract
Freeman S, et al. (2014) Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci
USA 111:8410–8415.
http://www.pnas.org/content/111/23/8410.abstract
ON-DEMAND SLIDES