A Survey of Annotation Tools for Lecture Materials

A Survey of Annotation Tools
For Lecture Materials

‣ Kai Michael Höver
‣ Guido Rößling
Department of Computer Science
Technische Universität Darmstadt
Germany

Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver |

Freitag, 22. Oktober 2010

Introduction

 Lecture (Recording) Systems
 present learning material
 annotate learning material

Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 2


Active annotating & interaction

 Studies have shown the importance of active learning rather than
passively receiving information
 Making annotations can support students in structuring information and to
enhance them with their own ideas and knowledge
 Interaction is also an important element of teaching & learning
 Interaction can be achieved by exchanging these annotations
 Intentions:
 highlighting and directing attention
 sharing information
 asking questions ...
 Stimulus for discussions
Hake, R., 1998, Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses, American Journal of Physics, 66(1), pp. 64-74.
Koivunen, M. R. & Swick, R. (2001). Metadata based annotation infrastructure offers flexibility and extensibility for collaborative applications and beyond. In Proceedings of the first international
conference on knowledge capture (KCAP 2001), victoria, BC, canada
McConnell, J. J. (1996). Active learning and its use in computer science. In ITiCSE '96: Proceedings of the 1st conference on integrating technology into computer science education. ACM.
Prince, M. (2004). Does active learning work? A review of the research. A Review of the Research. Journal of Engineering Education, 93(3), 223-231.



Many systems exist!
Which to choose? What‘s missing?
 AWS  StuPad
 AOF  Ubiquitous
 Classroom Presenter
Presenter  WeLCoME
 CoScribe  WriteOn
 DEBBIE  ...
 DyKnow
 eMargo
 Livenotes
 MRAS
 PaperCP



What‘s coming next?

 Short presentation of 14 systems (no claims of being complete,
but interesting :-)
 We cover
 Systems that enable students to annotate learning material
 scientific papers available
 MS PowerPoint, OpenOffice, Keynote
 Annotea



Taxonomy

 Scenarios
 Annotation Material
 Annotation Classes



Taxonomy: Scenarios

 Educator can annotate during/after a lecture
 Student can annotate during/after a lecture
 Student can access educator‘s annotations during/after a lecture
 Student can access fellow student‘s annotations during/after lecture
 Educator can access student‘s annotations during/after a lecture

 Educator can select among student‘s annotations, add comments,
and present both to other students
 Student presents an application from his/her laptop on a public
screen; the educator or students can add annotations



Taxonomy: Annotation Material

 Slide (electronic or printed)
 Whiteboard or blank sheet
 Background (templates like graph paper, sheet of music)
 Lecture notes, text document
 Video
 Software application
 Annotation



Taxonomy: Annotation Classes

 Text notes (with/without hyperlink support)
 Digital ink / ink
 Audio
 Tags / Bookmarks



eClass (Classroom 2000) + StuPad

 eClass integrates captured audio
and video with slides and
instructor‘s annotations
 StuPad‘s motivation: desire of
students to add private
annotations
➡ blank notebook (not sharable)

Abowd, G. D. (1999). Classroom 2000: An experiment with the instrumentation of a living educational environment. IBM Syst. J., 38(4), 508-530.
Truong, K. N. & Abowd, G. D. (1999). Stupad: Integrating student notes with class lectures. In CHI '99 extended abstracts on human factors in computing systems. ACM.



Authoring On the Fly (AOF)

 lecture recording system
 educator can annotate the slides
 transmits both slides and educator‘s annotations to students‘ PCs
 students can add their own annotations on a different layer

!
Müller, R. & Ottmann, T. (2000). The "authoring on the fly" system for automated recording and replay of (tele)presentations. Multimedia Systems, 8(3), 158-176.



!"##$%&'()*'
+!#'(+/3!4(3*!3*+!)+05+06!7/! (#$/)!$0!*(,,+/!"'3$2+3*+0!A+6268!(?!3*+.!#$5+0!3$$!%:#*!$?!3*+!
AOF+AWS
+! 30"/)%(33+,! 5("! 3*+!;7<=!
/3! 3$! 3*+! <//$3"3($/! @+-!
$0(2(/"'!,$#:%+/3D6!
@(3*! 3*+)+! 34$! 5(+4)8! (3! ()! &$))(-'+! 3$! ?$''$4! 3*+!
+0+! "0+! /+4! "//$3"3($/)! $/! ,()#:))($/! 3*0+",! /$3+! -.! /$3+8! $0! 3$! ?$''$4! (3! ?0$%! 3*+!
!3$!3*+!#'(+/36! &+0)&+#3(5+! $?! 3*+! ,$#:%+/3! "/,! 5(+4! 3*+! /$3+)! "/#*$0+,! "3!
+! ).)3+%! #"/! 0+30(+5+! "''! )&+#(?(#!0+2($/)!$?!3*+!"03(?"#36!O$3*!5(+4)!"0+!(/3+0'(/R+,T!-.!
 adding text notes to lecture recordings 5(+48! "! :)+0! #"/!
4(3*! 3*+(0! #:00+/3! '+#3:0+! ,$:-'+H#'(#R(/2! "/! "//$3"3($/! (/! 3*+! 3*0+",!
3*+%6! F*+.! #"/! 3*+/! #0+"3+! /"5(2"3+! ,(0+#3'.! 3$! 3*+! 0+)&+#3(5+! '$#"3($/! $?! 3*+! ,$#:%+/3!
 text notes can 4*+0+!3*"3!/$3+!()!"/#*$0+,!"/,8!?$0!+G"%&'+8!)3"03!0+&'".!?0$%!
! 0+&'.(/2! 3$! +G()3(/2! /$3+)D! be threaded
!3*+(0!#$H'+"0/+0)6!! 3*+0+6!
➡ discussions !
!!!!K+#$0,(/2)! !!!

@@@!

QX!

Y$#"'!
0+&$)(3$0.!

!
! BEIJKL!S!!
<NQM7KLX!XE;QJ;;E7N!7B!<!YLQFJKL!KLQ7KXENI!@EFM!<7BZ;[NQ!
Lauer, T., Trahasch, S., & Zupancic, B. (2005). Anchored discussions of multimedia lecture recordings. In Frontiers in education, 2005. FIE '05. Proceedings 35th annual conference.
MFF=8!BF=
!
Oct 22, 2010 | Department of Computer Science | 3$! 3*+! 34$! 5(+4)8! Michael Höver | Slide 12 34$!
</"'$2$:)! Telecooperation | Kai 3*+! :)+0! *")! "')$!
+0! &$))(-('(3(+)! 3$! #0+"3+! /+4! /$3+)6! <! /+4! "//$3"3($/! #"/! -+!
! Freitag, 22. Oktober 2010 #0+"3+,! ,(0+#3'.! "3! "/.! &$)(3($/! $/! "! )'(,+! "3! "/.! 3(%+! (/! 3*+!

Classroom Presenter
C. Liao et al.
Classroom/Ubiquitous Presenter PaperCP

d Tablet PCs. Instructor

the digital solu-
vantages in data
and archiving,
drawbacks are
d with these
a degraded
and writing ex-
due to limited Students

ize and screen Fig. 1. (Left) The original Tablet PC interface of a digital Active
Public display Learning infrastructure. (Right) The new equivalent interface,
n, the cost of the
PaperCP, based on Anoto technology, which consists of Blue-
Anderson, R. & Linnell, N. (2009). Promoting interaction in distance education. Elearn, 2009(8).

andownthe limita- shown here, the instructor presents a slide with anprintouts.
Figure 1. Classroom Presenter.The system runs on instructor, student, and public displays, each
Wilkerson, M., Griswold, W. G., & Simon, B. (2005). Ubiquitous presenter: Increasing student access and control in a digital lecturing environment. In SIGCSE '05. ACM.
with its user interface. In the scenario
tooth digital pens and
Liao, C. et al. (2007). Papercp: Exploring the integration of physical and digital affordances for active learning. In Human-Computer interaction – INTERACT 2007.
activity.The students write solutions to the activity on their Tablet PCs and submit their
posed thethe instructor view) and thenComputerstudent solutions in aon the public | Kai Michael Höver | Slide 13
answers to by battery can preview the
instructor.The instructor
Oct 22, 2010 | Department of
on the right of
ﬁlm strip (shown
Science | Telecooperation
selectively show student answers
easefor class discussion.
display the tension

!"#$%&&'$$ $$()(*+,-$*./01234516$7$"869$,:;28<;$

!"#$%&'()*+, -"%, ./#'0, !%)%, (, )%1()2(&/%, *%3#4%, 5'), $"%#),

Livenotes
$#1%, 67889:;, &<$, !%)%, /#1#$%*, #=, $%)10, '5, >)'4%00#=?,
!
4(>(&#/#$@, 67A9, BCD, EFG., .HI:, (=*, <0(&#/#$@, 64)(1>%*,
!
2%@&'()*;, /#1#$%*, 04)%%=, 0%=0#$#3#$@, $', >%=, #=><$;, (=*, (,
!
>)%00<)%J0%=0#$#3%,04)%%=,$"($,$)%($%*,(44#*%=$(/,"(=*,4'=$(4$,
!#$",$"%,04)%%=,(0,>%=,#=><$:+,,
!
!
!
 supports real-time conversation !
within small groups of students ! !
during lectures !
!
 shared whiteboard for students !
!
 instructor can load PPT file into the !
whiteboard‘s background !
!
!
!
! ,
! !"#$%&'()'*+&',"-&./0&1'$1&%'".0&%234&'2/%'0+&'&56&%"7&.0'
! 8&14%"9&8'".'0+"1'636&%:';"0+'0+&'"76/%0&8'934<#%/$.8'1="8&'
/44$6>".#'0+&'0/6'=&20?+3.8'3%&3'/2'0+&';+"0&9/3%8)'
!
G%4'=*;,$"%,#=$%)5(4%,!(0,'3%)"(</%*,#=,KLLK,$',#=4')>')($%,
!
=%!, ?)'<>, (!()%=%00, (=*, $', %=(&/%, /%4$<)%, 0/#*%0, $', &%,
!
Kam, M., Wang, J., Iles, A., Tse, E., Chiu, J., Glaser, D., et al. (2005). Livenotes: A system for cooperative and augmented note-taking in lectures. '5, $"%, ./#'0, (=*, 1(*%, #$,
#1>')$%*+, -"#0, $(M%*, $"%, 4(>(4#$@,
In CHI '05: Proceedings of the SIGCHI conference on human factors in computing systems. ACM !
#1>'00#&/%, $', #=$)'*<4%, N#3%='$%0, #=$', (=, <=*%)?)(*<($%,
!
4/(00,'=,(,='=J$)#3#(/,04(/%,!#$"'<$,*#0)<>$#'=+,O%,$"%)%5')%,
!
1#?)($%*, N#3%='$%0, $', /(>$'>0, (=*, 5'<=*, $"($, #$, !')2%*,
%55%4$#3%/@, !#$", 0#?=#5#4(=$/@, ?)%($%), 4'1><$#=?, >'!%)+,
!
Freitag, 22. Oktober 2010 -"#)*;, $"%, #=$%)5(4%, !(0, 4"(=?%*, #=, KLLKJKLLP, $', #=4/<*%,

DEBBIE, DyKnow
L6:;&)--0A--M*&6%;+-!%,'),'-D8=)+-H@*67*>7)-6,-B8C,%?
3,-'()-%'()&-(*,45-6.-'()-')*/()&-%&*778-4).6,)4-'&))-')&96,%7%:8-+;/(-
 DePauw Electronic Black Board for Interactive
<=*&),'<5-*,4-<+6>76,:5<-'()-+';4),'+-96:('-/(%%+)-'%-?&6')-%&-'8=)-'()+)-4).6,
'()6&-=&6@*')-,%')+-9*&:6,A-!"#$%&'('+(%?+-'()-B8C,%?-6,')&.*/)A-D()-'(6/E-6
Education
Note annotation in electronic classrooms 87

7).'F(*,4-=*,)-?*+-4&*?,->8-'()-')*/()&-*,4-'()-'(6,,)&-6,E-?*+-*44)4->8-*-+';4
&6:('-=*,)-+(%?+-'()-+';4),'G+-=&6@*')-,%')-*&)*A-H+-)I=7*6,)4-9%&)-.;778-6,-+)/
*'-'()-),4-%.-/7*++-+';4),'+-/*,-=&6,'-%&-+*@)-'()6&-,%')>%%E-.%&-7*')&-&)@6)?-*,
!!"!#$%&'()*+')&,-!%,.)&),/)

Figure 8. Screen dump of participant’s version of DEBBIE.

L6:;&)--0A--M*&6%;+-!%,'),'-D8=)+-H@*67*>7)-6,-B8C,%? 6. User experiences, conclusions, and future work
3,-'()-%'()&-(*,45-6.-'()-')*/()&-%&*778-4).6,)4-'&))-')&96,%7%:8-+;/(-*+-<&%%'5 to the system has been positive, although evidence to date is only
Reaction
),'<5-*,4-<+6>76,:5<-'()-+';4),'+-96:('-/(%%+)-'%-?&6')-%&-'8=)-'()+)-4).6,6'6%,+-6,
anecdotal. For example, one of the authors is currently teaching a ‘Theory
&-=&6@*')-,%')+-9*&:6,A-!"#$%&'('+(%?+-'()-B8C,%?-6,')&.*/)A-D()-'(6/E-6,E-6,-'()
of Computation’ class which is being completely delivered using the system.
The course has an enrollment of 13 students, and each student has access to
(*,4-=*,)-?*+-4&*?,->8-'()-')*/()&-*,4-'()-'(6,,)&-6,E-?*+-*44)4->8-*-+';4),'A--D()
'-=*,)-+(%?+-'()-+';4),'G+-=&6@*')-,%')-*&)*A-H+-)I=7*6,)4-9%&)-.;778-6,-+)/'6%,-0AJ5 tablet to use large lecture classes usingbeen structured so that each ICLS'08: Proceedings of the 8th international
a video
in
during class. The course has
L6:;&)-NA-B8C,%?-!%77*>%&*'6@)-$%')-D*E6,:-O,')&.*/)
Johri, A. & Lohani, V. K. (2008). Creating a participatory learning environmentan ‘engagement grade’ which is based on the computing. In
pen-based quality of the
student receives
)-),4-%.-/7*++-+';4),'+-/*,-=&6,'-%&-+*@)-'()6&-,%')>%%E-.%&-7*')&-&)@6)?-*,4-+';48A- Learning Sciences.
conference for the learning sciences. International Society of the annotations which are added to the electronic notebooks, and on the quality
Berque, D. (2006). An evaluation of a broad deployment of dyknow software to support note taking and interaction distributed through computers. J. Comput. Small Coll., 21(6), 204-216.
of the answers given to short in-class quizzes which are using pen-based
B8C,%?-')*/()&+-%.'),-+(*&)-=&)=*&)4-/%,'),'-?6'(-+';4),'+-.%77%?6,:-*
Berque, D., Bonebright, T., & Whitesell, M. (2004). Using pen-based computers across approximately every 10 to 15 minutes. SIGCSE Bull., 36(1), 61-65.
the electronic notebooks the computer science curriculum. The idea of
=&*/'6/)5- *,4- 46+/7%+)<- =*'')&,- '(*'- ?)- /*77- =&%:&)++6@)- 46+/7%+;&)A- - D(6+- ')
these quizzes is to require the students to be engaged with the material in an
ongoing way. When asked to provide anonymous written assessment of the use
Oct 22, 2010 | Department of Computer ),/%;&*:)+-+';4),'+-'%-/%,'&6>;')-'%-'()-+(*&)4-*&'6.*/'-,%')>%%E5-'()&)>8-()7=
Science DEBBIE in this class, students generally reacted favourably. One student,
of | Telecooperation | Kai Michael Höver | Slide 15
referring to the frequent electronic quizzes, wrote ‘. . .giving us time to work and
'%-'(6,E-*>%;'-/%,'),'-*+-6'-6+->)6,:-=&)+),')4A--L%&-)I*9=7)5-*-')*/()&-96:('-=&%
think about topics is the only way we are going to understand. . .’ while another
wrote ‘. . .it is nice to be able to take notes on top of your notes for the class and
Freitag, 22. Oktober 2010 /7*++-?6'(-*-B8C,%?-=*:)-'(*'-=%+)+-*-+(%&'-=&%:&*996,:-=&%>7)9A---D()-')*/(
I realize that I am getting the main points of the lecture as opposed to picking

WriteOn 1.0

 Instructor can annotate running
Windows applications
 broadcast of defined area &
annotations to students‘ PCs
 Students can add ink notes
 Lecturer can add annotations to
the recording

!

Chandrasekar, S., Tront, J. G., & Prey, J. C. (2009). Writeon1.0: A tablet pc-based tool for effective classroom instruction. In Iticse '09: Proceedings of the 14th annual ACM SIGCSE
conference on innovation and technology in computer science education. ACM.



MRAS

 Microsoft Research Annotation System
 annotating lecture videos
 personal notes & discussions
 text or audio
 linking of Web resources (URLs)

Figure 2: Web-based MRAS interface.

Bargeron, D., Grudin, J., Gupta, A., Sanocki, E., Li, F., & Leetiernan, S. (2002). Asynchronous collaboration around and Functions
4.4 Annotation System Features multimedia applied to on-demand education.
Journal of Management Information Systems, 18(4), 117-145. The video in the upper left of the browser window in Figure 2 is displayed with a standard media player. The
slides on the right are synchronized with the video. In the lower left is an annotation set: comments, questions, and
replies made by previous viewers of the video. Each annotation is linked to the specific point in the video at which
Oct 22, 2010 | Department of Computer Science | Telecooperation |paused when the video; the reply|below itThealso highlighted. To the annotation thatthe annotation preview
the video
Kai Michael Höver created. is red arrow points to the lower right is was created closest to
the current position of
the annotation was
Slide 17
window, which automatically shows text associated with the currently highlighted annotation.
As the video plays, annotation titles scroll by. A viewer can click on an annotation and read it in the preview
Freitag, 22. Oktober 2010 frame, right-click to seek to the point in the video where an annotation was created, or reply to it, forming a threaded

We-LCoME

 Wiki e-Learning Compound
Multimedia Environment
 combines Wiki-functionality
(DokuWiki) with enriching Lecture
Recording
 Users add additional information
(e.g. captions) in the Wiki (special
syntax)
 Additional information are aggregate
with the recording to a SMIL video
lecture

Ferretti, S., Mirri, S., Muratori, L. A., Roccetti, M., & Salomoni, P. (2008). E-Learning 2.0: You are we-lcome!.
W4A '08: Proceedings of the 2008 international cross-disciplinary conference on web accessibility (W4A). ACM.



eMargo

 digital margin for slides and lecture
notes/text documents
 highlighting
 discussion support
 tagging

Sesink, W., Göller, S., Rößling, G., & Hofmann, D. (2007). Emargo: Eine digitale randspalte zum selbststudium (nicht nur) der informatik. In Proceedings der pre-conference
workshops der 5. E-Learning fachtagung informatik (delfi 2007). Logos Verlag Berlin.
Rößling, G., Joy, M., Moreno, A., Radenski, A., Malmi, L., Kerren, A., et al. (2008). Enhancing learning management systems to better support computer science education.
SIGCSE Bull., 40(4), 142-166



CoScribe

 annotation system of handwritten
notes for both printed and digital
slides
 synchronization from printed
slides to digital slides
 annotations can be categorized
(e.g. important, question, ...)
 annotations can be shared

!

Steimle, J., Brdiczka, O., & Mühlhäuser, M. (2009). Coscribe: Integrating paper and digital documents for collaborative knowledge work. IEEE Trans. Learn. Technol., 2(3), 174-188.



Taxonomy: Scenarios (revisited)
 Educator can annotate during/after a lecture: 6/4, 3 both, 1 not
 Student can annotate during/after a lecture: 8/4, 2 both
 Student can access educator‘s annotations during/after a lecture:
 4/4 , 5 both, 1 not
 Student can access fellow student‘s annotations during/after lecture:
 1/4, 2 both, 7 not
 Educator can access student‘s annotations during/after a lecture:
 3/6, 2 both, 3 not
 Educator can select among student‘s annotations, comment, and present them
to other students: Yes = 3
 Student presents an application from his/her laptop on a public screen; the
educator or students can add annotations: Yes = 0



Taxonomy: Annotation Material
(revisited)
 Slide (electronic or printed)
 4 no slides, 8 electronic slides, 1 printed slides, 1 both
 Whiteboard or blank sheet: 7 yes, 7 no
 Background (templates like graph paper, sheet of music)
 2 yes, 12 no
 Lecture notes, text documents: 2 yes, 12 no
 Video: 2 yes, 12 no
 Application: 2 yes, 12 no
 Annotations: 3 yes, 11 no



Taxonomy: Annotation Classes
(revisited)
 Text notes (with/without hyperlink support)
 10 yes (2 supporting hyperlinks)
 Digital ink / ink
 9 digital ink, 2 pen (Anoto)
 Audio: 1
 Tag / Bookmark
 1 tags, 2 bookmarks



Summary

 Annotation material
 Slides only?
 text documents? Web sites?
 Annotation classes
 digital ink & text annotations
 integration of Web resources? Web 2.0 content?
 Focus on one specific period of time
 during/after a lecture
 sharing of annotations only during a lecture
 less support of student interaction



Thank you
for listening!

Questions?



A Survey of Annotation Tools for Lecture Materials

Recommandé

Recommandé

Contenu connexe

Similaire à A Survey of Annotation Tools for Lecture Materials

Similaire à A Survey of Annotation Tools for Lecture Materials (20)

Dernier

Dernier (20)

A Survey of Annotation Tools for Lecture Materials