5. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
5
Introduction (1)
A. Core of the Thesis
Evaluation Manual Editing vs. Automated Editing
Can time be saved and quality preserved?
6. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
6
Introduction (1)
A. Core of the Thesis
Evaluation Manual Editing vs. Automated Editing
Can time be saved and quality preserved?
Survey and workflow time tracking
9. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
9
Introduction (2)
B. Learning Videos
Specific Type “Frontal Lecture / Studio Recording”
Recording with teleprompter (screen) text
10. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
10
Introduction (2)
B. Learning Videos
Specific Type “Frontal Lecture / Studio Recording”
Recording with teleprompter (screen) text
11. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
11
Introduction (2)
B. Learning Videos
Specific Type “Frontal Lecture / Studio Recording”
Recording with teleprompter (screen) text
… Die Schülerinnen und Schüler können
diese also nicht nur im Gegenstand
Informatik beziehungsweise
Digitale Grundbildung erarbeiten, sondern
auch in den Fächern Bewegung und Sport,
Bildnerische Erziehung …
Screen Text:
14. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
14
Introduction (3)
C. Video Editing
Part of Postproduction
Take the best (parts) and leave the rest
15. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
15
Introduction (3)
C. Video Editing
Part of Postproduction
Take the best (parts) and leave the rest
Concatenate parts in a way that viewers are not
distracted
16. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
16
Introduction (4)
C. Video Editing
… Die Schülerinnen und Schüler können
diese also nicht nur im Gegenstand
Informatik beziehungsweise
Digitale Grundbildung erarbeiten, sondern
auch in den Fächern Bewegung und Sport,
Bildnerische Erziehung …
17. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
17
Introduction (4)
C. Video Editing
… Die Schülerinnen und Schüler können
diese also nicht nur im Gegenstand
Informatik beziehungsweise
Digitale Grundbildung erarbeiten, sondern
auch in den Fächern Bewegung und Sport,
Bildnerische Erziehung …
18. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
18
Introduction (4)
C. Video Editing
… Die Schülerinnen und Schüler können
diese also nicht nur im Gegenstand
Informatik beziehungsweise
Digitale Grundbildung erarbeiten, sondern
auch in den Fächern Bewegung und Sport,
Bildnerische Erziehung …
19. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
19
Introduction (4)
C. Video Editing
… Die Schülerinnen und Schüler können
diese also nicht nur im Gegenstand
Informatik beziehungsweise
Digitale Grundbildung erarbeiten, sondern
auch in den Fächern Bewegung und Sport,
Bildnerische Erziehung …
20. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
20
Introduction (4)
C. Video Editing
… Die Schülerinnen und Schüler können
diese also nicht nur im Gegenstand
Informatik beziehungsweise
Digitale Grundbildung erarbeiten, sondern
auch in den Fächern Bewegung und Sport,
Bildnerische Erziehung …
21. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
21
Introduction (4)
C. Video Editing
… Die Schülerinnen und Schüler können
diese also nicht nur im Gegenstand
Informatik beziehungsweise
Digitale Grundbildung erarbeiten, sondern
auch in den Fächern Bewegung und Sport,
Bildnerische Erziehung …
22. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
22
Introduction (4)
C. Video Editing
… Die Schülerinnen und Schüler können
diese also nicht nur im Gegenstand
Informatik beziehungsweise
Digitale Grundbildung erarbeiten, sondern
auch in den Fächern Bewegung und Sport,
Bildnerische Erziehung …
28. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
28
Introduction (5)
D. Video Quality
QoS: Quality of Service
QoE: Quality of Experience
QoP: Quality of Perception
31. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
31
Introduction (6)
E. „Good” Learning Videos should
Visualize content (Mayer, 2002).
Avoid unnecessary audio noise (Richardson, 1998)
32. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
32
Introduction (6)
E. „Good” Learning Videos should
Visualize content (Mayer, 2002).
Avoid unnecessary audio noise (Richardson, 1998)
Maintain a consistent (sound) volume (Robinson et al., 2003)
33. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
33
Introduction (6)
E. „Good” Learning Videos should
Visualize content (Mayer, 2002).
Avoid unnecessary audio noise (Richardson, 1998)
Maintain a consistent (sound) volume (Robinson et al., 2003)
Implement as discreet cuts as possible (Lima et al., 2012)
36. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
36
Methodology (1)
A. Reference Videos
Ten Raw Recordings with belonging Screen Text
Length between 50s and 4min
37. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
37
Methodology (1)
A. Reference Videos
Ten Raw Recordings with belonging Screen Text
Length between 50s and 4min
One to Five Takes
38. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
38
Methodology (1)
A. Reference Videos
Ten Raw Recordings with belonging Screen Text
Length between 50s and 4min
One to Five Takes
German / English and Male / Female Split
41. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Measure Time
Manual Workflow performed by Video Editors
tracked with Stopwatch
41
Methodology (2)
42. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Measure Time
Manual Workflow performed by Video Editors
tracked with Stopwatch
Corresponding Steps tracked with Process Time in
Automated Workflow
42
Methodology (2)
43. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Measure Time
Manual Workflow performed by Video Editors
tracked with Stopwatch
Corresponding Steps tracked with Process Time in
Automated Workflow
Direct Time Consumption Comparison
43
Methodology (2)
48. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
Two videos and two versions
Embedded in an Online Survey (LimeSurvey)
48
Methodology (3)
49. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
Two videos and two versions
Embedded in an Online Survey (LimeSurvey)
Rating Questions about Quality (QoP and QoE)
49
Methodology (3)
50. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
Two videos and two versions
Embedded in an Online Survey (LimeSurvey)
Rating Questions about Quality (QoP and QoE)
Open Question
50
Methodology (3)
51. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
Two videos and two versions
Embedded in an Online Survey (LimeSurvey)
Rating Questions about Quality (QoP and QoE)
Open Question
t-Test for Significance
51
Methodology (3)
53. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
53
Methodology (3)
Video 2 Video 3
manually | automatically manually | automatically
54. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
54
Methodology (3)
Video 2 Video 3
manually | automatically manually | automatically
Group A Evaluation
55. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
55
Methodology (3)
Video 2 Video 3
manually | automatically manually | automatically
Group B Evaluation
Group A Evaluation
56. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
1. The words were pronounced clearly and distinctly.
2. I can learn well with this learning video.
3. Generally I like this video.
4. The content of the video matches the subtitles.
5. Image Quality is good in my opinion.
6. Sound Quality is good in my opinion.
56
Methodology (3)
57. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
1. The words were pronounced clearly and distinctly.
2. I can learn well with this learning video.
3. Generally I like this video.
4. The content of the video matches the subtitles.
5. Image Quality is good in my opinion.
6. Sound Quality is good in my opinion.
57
Methodology (3)
QoP
58. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
1. The words were pronounced clearly and distinctly.
2. I can learn well with this learning video.
3. Generally I like this video.
4. The content of the video matches the subtitles.
5. Image Quality is good in my opinion.
6. Sound Quality is good in my opinion.
58
Methodology (3)
QoP
QoE
59. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
C. Measure Quality
1. The words were pronounced clearly and distinctly.
2. I can learn well with this learning video.
3. Generally I like this video.
4. The content of the video matches the subtitles.
5. Image Quality is good in my opinion.
6. Sound Quality is good in my opinion.
59
Methodology (3)
QoP
QoE
Open Question: What was particularly good or bad?
61. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
A. Time Consumption
61
Results (1)
Video
Time: Manual
Workflow
Time: Automated
Workflow
Time Saved in %
1
2
3
4
5
6
7
8
9
10
62. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
A. Time Consumption
62
Results (1)
Video
Time: Manual
Workflow
Time: Automated
Workflow
Time Saved in %
1 193s
2 250s
3 141s
4 101s
5 91s
6 80s
7 60s
8 261s
9 171s
10 273s
63. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
A. Time Consumption
63
Results (1)
Video
Time: Manual
Workflow
Time: Automated
Workflow
Time Saved in %
1 193s 44s
2 250s 23s
3 141s 35s
4 101s 19s
5 91s 23s
6 80s 22s
7 60s 19s
8 261s 73s
9 171s 40s
10 273s 48s
64. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
A. Time Consumption
64
Results (1)
Video
Time: Manual
Workflow
Time: Automated
Workflow
Time Saved in %
1 193s 44s 77 %
2 250s 23s 90 %
3 141s 35s 75 %
4 101s 19s 81 %
5 91s 23s 74 %
6 80s 22s 72 %
7 60s 19s 68 %
8 261s 73s 72 %
9 171s 40s 76 %
10 273s 48s 82 %
65. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
A. Time Consumption
65
Results (1)
Video
Time: Manual
Workflow
Time: Automated
Workflow
Time Saved in %
1 193s 44s 77 %
2 250s 23s 90 %
3 141s 35s 75 %
4 101s 19s 81 %
5 91s 23s 74 %
6 80s 22s 72 %
7 60s 19s 68 %
8 261s 73s 72 %
9 171s 40s 76 %
10 273s 48s 82 %
Automated Workflow on average 76% faster (SD 6%)
68. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
129 Participants in Online Survey
Group A: 74 Participants
Group B: 55 Participants
68
Results (2)
69. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
129 Participants in Online Survey
Group A: 74 Participants (V2 M - V3 A)
Group B: 55 Participants (V2 A - V3 M)
69
Results (2)
70. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
129 Participants in Online Survey
Group A: 74 Participants (V2 M - V3 A)
Group B: 55 Participants (V2 A - V3 M)
85 men and 44 women
70
Results (2)
71. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
129 Participants in Online Survey
Group A: 74 Participants (V2 M - V3 A)
Group B: 55 Participants (V2 A - V3 M)
85 men and 44 women
Content watched mostly on Smartphones with
built-in Speakers (44.2% / 30.2%)
71
Results (2)
72. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
129 Participants in Online Survey
Group A: 74 Participants (V2 M - V3 A)
Group B: 55 Participants (V2 A - V3 M)
85 men and 44 women
Content watched mostly on Smartphones with
built-in Speakers (44.2% / 30.2%)
Participants learn with videos at least a few times
a month or more often(61.3%)
72
Results (2)
73. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
73
Results (3)
Question
Video 2 Video 3
Manual Automated t-Test Manual Automated t-Test
The words were pronounced
clearly and distinctly.
I can learn well with
this learning video.
Generally I like this
video.
The content of the
video matches the subtitles.
Image quality is good
in my opinion.
Sound quality is good
in my opinion.
74. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
74
Results (3)
Question
Video 2 Video 3
Manual Automated t-Test Manual Automated t-Test
The words were pronounced
clearly and distinctly. 3.6 3.9 0.35
I can learn well with
this learning video. 3.1 2.6 0.03
Generally I like this
video. 3.2 3.0 0.47
The content of the
video matches the subtitles. 3.7 4.0 0.22
Image quality is good
in my opinion. 4.0 4.0 0.58
Sound quality is good
in my opinion. 3.8 4.0 0.34
75. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
75
Results (3)
Question
Video 2 Video 3
Manual Automated t-Test Manual Automated t-Test
The words were pronounced
clearly and distinctly. 3.6 3.9 0.35
I can learn well with
this learning video. 3.1 2.6 0.03
Generally I like this
video. 3.2 3.0 0.47
The content of the
video matches the subtitles. 3.7 4.0 0.22
Image quality is good
in my opinion. 4.0 4.0 0.58
Sound quality is good
in my opinion. 3.8 4.0 0.34
76. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
76
Results (3)
Question
Video 2 Video 3
Manual Automated t-Test Manual Automated t-Test
The words were pronounced
clearly and distinctly. 3.6 3.9 0.35 4.2 4.1 0.58
I can learn well with
this learning video. 3.1 2.6 0.03 3.1 3.3 0.53
Generally I like this
video. 3.2 3.0 0.47 3.6 3.6 0.98
The content of the
video matches the subtitles. 3.7 4.0 0.22 4.0 3.9 0.63
Image quality is good
in my opinion. 4.0 4.0 0.58 4.2 4.1 0.64
Sound quality is good
in my opinion. 3.8 4.0 0.34 4.2 4.0 0.43
79. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
79
Results (4) What did you find particularly bad about the learning
Video?
80. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
80
Results (4)
Video 2
Manually Edited
10 %
12 %
45 %
32 %
Greenscreen is distracting
Missing visualization of the content
Audio Quality Lacking
Other (General Comments)
What did you find particularly bad about the learning
Video?
81. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
81
Results (4)
Video 2
Manually Edited
10 %
12 %
45 %
32 %
Greenscreen is distracting
Missing visualization of the content
Audio Quality Lacking
Other (General Comments)
Video 2
Automatically
Edited
7 %
65 %
28 %
What did you find particularly bad about the learning
Video?
82. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
Only one quality question has a statistically
significant difference (automated rated worse than
manually)
82
Results (5)
83. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
B. Preserving Quality
Only one quality question has a statistically
significant difference (automated rated worse than
manually)
Other quality factors are not influenced
83
Results (5)
87. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
87
Conclusion
1. Time can be saved drastically
2. Quality can almost be preserved
3. Not following principles of multimedia content creation can
affect quality
88. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
88
Conclusion
1. Time can be saved drastically
2. Quality can almost be preserved
3. Not following principles of multimedia content creation can
affect quality
4. Indiscreet Cuts can distract viewers from content
89. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
89
Conclusion
1. Time can be saved drastically
2. Quality can almost be preserved
3. Not following principles of multimedia content creation can
affect quality
4. Indiscreet Cuts can distract viewers from content
5. Bad Audio Quality affects the viewers concentration
90. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
90
Conclusion
1. Time can be saved drastically
2. Quality can almost be preserved
3. Not following principles of multimedia content creation can
affect quality
4. Indiscreet Cuts can distract viewers from content
5. Bad Audio Quality affects the viewers concentration
6. Greenscreen should be avoided
92. www.tugraz.at ■
28.04.2022
David Nußbaumer
Empirical Analysis of Automated Editing of
Raw Learning Video Footage
Cho, Sunghyun, Jue Wang, and Seungyong Lee (Aug. 2012). Video deblurring
for hand-held cameras using patch-based synthesis". In: ACM Transactions
on Graphics 31.4, pp. 1{9. doi: 10.1145/2185520.2185560.
Lima, Edirlei Soares de et al. (July 2012). Automatic Video Editing for Video-
Based Interactive Storytelling". In: 2012 IEEE International Conference on
Multimedia and Expo. IEEE. doi: 10.1109/icme.2012.83.
Mayer, Richard E. (2002). Multimedia Learning". In: The Annual Report of Educational
Psychology in Japan. Vol. 41, pp. 27-29.
Richardson, Craig H. (1998). Improving Audio Quality in Distance Learning Applications."
In: Distance Learning '98. Proceedings of the AnnualConference
on Distance Teaching and Learning (14th, Madison,WI, August 5-7, 1998).
Robinson, Charles Q., Steve R. Lyman, and Je rey Riedmiller (2003). Intelligent
Program Loudness Measurement and Control: What Satis
fi
es Listeners?"
92
References