0
Understanding In-Video Dropouts
and Interaction Peaks in
Online Lecture Videos
Juho Kim (MIT CSAIL)
Philip J. Guo (MIT CSA...
Video Lectures in MOOCs
Classrooms: rich, natural interaction data

Maria Fleischmann / Worldbank on Flickr | CC by-nc-nd
Love Krittaya | public d...
liquidnight on Flickr | CC by-nc-sa
How do learners use videos?

Data-Driven Approach:
Analyze learners’ interaction
with the video player
Why does data matter?
• detailed understanding of video usage
• design implications for
– Instructors
– Video editors
– Pl...
How do learners use videos?
• Watch sequentially
• Pause
• Re-watch
• Skip / Skim
Collective Interaction Traces
Student #7888
Student #7887
......
Student #4
Student #3
Student #2
Student #1
video time
Collective Interaction Traces
into Interaction Patterns
interaction
events

video time

second-by-second activity tracking
~40M video interaction events
from 4 edX courses

Learners
127,839

Mean Video Processed
Videos
Length
Events
862
7:46
39....
Analyzing Clickstream
• Events: play / pause
• In-video time and absolute time
• Learner ID: first-time or re-watching
Cli...
Collective Interaction Patterns
1. In-video dropout
viewership

video time

2. Interaction peaks
interaction
events

video...
1. In-video dropout
viewership

video time

2. Interaction peaks
interaction
events

video time
In-video Dropout
% watching sessions that
end before the video finishes.

viewership

video time
36%: dropouts withinrate few seconds
55%: overall dropout first
viewership

36%
19%

55%

video time

Why?
• Auto-play pla...
Longer videos lead to more dropouts.
Re-watching sessions lead to more dropouts
than first-time sessions.
Tutorial videos lead to more dropouts
than lecture videos.
Lecture
• introduction to concepts
• continuous flow

Tutorial
...
1. In-video dropout
viewership

video time

2. Interaction peaks
interaction
events

video time
Interaction Peaks
Temporal peaks in the number of interaction
events, where a significant number of learners
show similar ...
Analytic Workflow
Bin data into
per-second
segments

Apply a
kernel
smoother

Detect peaks
Re-watching sessions show stronger and
more peaks than first-time sessions.
Tutorial videos show stronger and
more peaks than lecture videos.
Lectures

Tutorials
What causes an interaction peak?
Observation: Visual transitions in the
video often coincide with a peak.
lecture
video

number of
re-watching
sessions
Analytic Workflow
Step 1. Visual Analysis
• second-by-second pixel differences
between adjacent frames

head
slide

head
s...
Analytic Workflow
Step 2. Peak Categorization
• Manually inspected 80 videos
• Interaction peak <-> Visual transition
Five Explanations for
an Interaction Peak
Type 1. Beginning of new material
Type 2. Returning to content
Type 3. Tutorial ...
Type 1. Beginning of new material

interaction
Type 2. Returning to content

interaction
Type 3. Tutorial step

interaction
Type 4. Replaying a segment

interaction
Type 5. Non-visual explanation

interaction
61% of interaction peaks
involved a visual transition.
Peak Category

Frequency

Type 1. Beginning of new material

25%

T...
Can interaction data improve
the video learning experience?
Lessons for instructors, video editors,
and platform designers
1. Make shorter videos.
2. Add informative titles and easy ...
Lessons for instructors, video editors,
and platform designers
4. Make interaction peaks more accessible.

5. Enable one-c...
Next Steps: More Data Streams
• What would transcript / text add to the
analysis? How about acoustic data?
Next Steps: Scalability
• Reliably & automatically detect peak types?
• How much data is needed until we see patterns?
vie...
For instructors & editors: Video Analytics
For learners: Data-Driven Video UI
Contributions
• A first MOOC-scale in-video dropout rate
analysis
• A first MOOC-scale in-video interaction peak
analysis
...
Understanding In-Video Dropouts and
Interaction Peaks in Online Lecture Videos

Juho Kim
MIT CSAIL
juhokim@mit.edu
juhokim...
Backup slides
Domain
educational videos

Theory
interactive learning

How-to videos
MOOC videos

Role of interactivity
Learner control
S...
Vision in learnersourcing
• Feedback loop between
– Learners: natural, pedagogically useful activities
– System: improve i...
How can we design
online video learning platforms
that are as effective as
in-person classrooms?
Prochain SlideShare
Chargement dans... 5
×

Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos

1,457

Published on

Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos

Juho Kim, Philip J. Guo, Daniel T. Seaton, Piotr Mitros, Krzysztof Z. Gajos, Robert C. Miller

Presented at ACM Learning at Scale 2014, March 4-5 2014, Atlanta, GA, USA.

Published in: Éducation
0 commentaires
6 mentions J'aime
Statistiques
Remarques
  • Soyez le premier à commenter

Aucun téléchargement
Vues
Total des vues
1,457
Sur Slideshare
0
À partir des ajouts
0
Nombre d'ajouts
7
Actions
Partages
0
Téléchargements
17
Commentaires
0
J'aime
6
Ajouts 0
No embeds

No notes for slide

Transcript of "Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos"

  1. 1. Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos Juho Kim (MIT CSAIL) Philip J. Guo (MIT CSAIL, U of Rochester) Daniel T. Seaton (MIT Office of Digital Learning) Piotr Mitros (edX) Krzysztof Z. Gajos (Harvard EECS) Robert C. Miller (MIT CSAIL) 2014.03.04 Learning at Scale
  2. 2. Video Lectures in MOOCs
  3. 3. Classrooms: rich, natural interaction data Maria Fleischmann / Worldbank on Flickr | CC by-nc-nd Love Krittaya | public domain armgov on Flickr | CC by-nc-sa unknown author | from pc4all.co.kr
  4. 4. liquidnight on Flickr | CC by-nc-sa
  5. 5. How do learners use videos? Data-Driven Approach: Analyze learners’ interaction with the video player
  6. 6. Why does data matter? • detailed understanding of video usage • design implications for – Instructors – Video editors – Platform designers • new video interfaces and formats Improved video learning experience
  7. 7. How do learners use videos? • Watch sequentially • Pause • Re-watch • Skip / Skim
  8. 8. Collective Interaction Traces Student #7888 Student #7887 ...... Student #4 Student #3 Student #2 Student #1 video time
  9. 9. Collective Interaction Traces into Interaction Patterns interaction events video time second-by-second activity tracking
  10. 10. ~40M video interaction events from 4 edX courses Learners 127,839 Mean Video Processed Videos Length Events 862 7:46 39.3M Courses: Computer science, Statistics, Chemistry
  11. 11. Analyzing Clickstream • Events: play / pause • In-video time and absolute time • Learner ID: first-time or re-watching Clickstream interaction log Per-learner watching segments learner xxx learner xxx 0:00 play 0: 34 pause 0:57 play 1:47 pause Segment 1 - 0:00-0:34 Segment 2 - 0:57-1:47 Per-second stats for views, re-watches, plays, & pauses
  12. 12. Collective Interaction Patterns 1. In-video dropout viewership video time 2. Interaction peaks interaction events video time
  13. 13. 1. In-video dropout viewership video time 2. Interaction peaks interaction events video time
  14. 14. In-video Dropout % watching sessions that end before the video finishes. viewership video time
  15. 15. 36%: dropouts withinrate few seconds 55%: overall dropout first viewership 36% 19% 55% video time Why? • Auto-play playing unwanted videos • Misleading video titles / interfaces
  16. 16. Longer videos lead to more dropouts.
  17. 17. Re-watching sessions lead to more dropouts than first-time sessions.
  18. 18. Tutorial videos lead to more dropouts than lecture videos. Lecture • introduction to concepts • continuous flow Tutorial • supplementary examples • step-by-step demos
  19. 19. 1. In-video dropout viewership video time 2. Interaction peaks interaction events video time
  20. 20. Interaction Peaks Temporal peaks in the number of interaction events, where a significant number of learners show similar interaction patterns interaction events video time
  21. 21. Analytic Workflow Bin data into per-second segments Apply a kernel smoother Detect peaks
  22. 22. Re-watching sessions show stronger and more peaks than first-time sessions.
  23. 23. Tutorial videos show stronger and more peaks than lecture videos. Lectures Tutorials
  24. 24. What causes an interaction peak?
  25. 25. Observation: Visual transitions in the video often coincide with a peak.
  26. 26. lecture video number of re-watching sessions
  27. 27. Analytic Workflow Step 1. Visual Analysis • second-by-second pixel differences between adjacent frames head slide head slide head slide
  28. 28. Analytic Workflow Step 2. Peak Categorization • Manually inspected 80 videos • Interaction peak <-> Visual transition
  29. 29. Five Explanations for an Interaction Peak Type 1. Beginning of new material Type 2. Returning to content Type 3. Tutorial step Type 4. Replaying a segment Type 5. Non-visual explanation
  30. 30. Type 1. Beginning of new material interaction
  31. 31. Type 2. Returning to content interaction
  32. 32. Type 3. Tutorial step interaction
  33. 33. Type 4. Replaying a segment interaction
  34. 34. Type 5. Non-visual explanation interaction
  35. 35. 61% of interaction peaks involved a visual transition. Peak Category Frequency Type 1. Beginning of new material 25% Type 2. Returning to content 23% Type 3. Tutorial step 7% Type 4. Replaying a segment 6% Type 5. Non-visual explanation 39% 61%
  36. 36. Can interaction data improve the video learning experience?
  37. 37. Lessons for instructors, video editors, and platform designers 1. Make shorter videos. 2. Add informative titles and easy navigation. 3. Avoid abrupt visual transitions.
  38. 38. Lessons for instructors, video editors, and platform designers 4. Make interaction peaks more accessible. 5. Enable one-click access for tutorial steps.
  39. 39. Next Steps: More Data Streams • What would transcript / text add to the analysis? How about acoustic data?
  40. 40. Next Steps: Scalability • Reliably & automatically detect peak types? • How much data is needed until we see patterns? viewership 5 learners video time viewership 5,000 learners video time
  41. 41. For instructors & editors: Video Analytics
  42. 42. For learners: Data-Driven Video UI
  43. 43. Contributions • A first MOOC-scale in-video dropout rate analysis • A first MOOC-scale in-video interaction peak analysis • Categorization of learner activities responsible for an interaction peak • Data-driven design implications for video authoring, editing, and interface design
  44. 44. Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos Juho Kim MIT CSAIL juhokim@mit.edu juhokim.com
  45. 45. Backup slides
  46. 46. Domain educational videos Theory interactive learning How-to videos MOOC videos Role of interactivity Learner control Subgoal labeling Method scalable data collection to realize theory Crowdsourcing Learnersourcing
  47. 47. Vision in learnersourcing • Feedback loop between – Learners: natural, pedagogically useful activities – System: improve interaction using learner data • Visualize and analyze large-scale video learning activities • Use data to inform learning platform design
  48. 48. How can we design online video learning platforms that are as effective as in-person classrooms?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×