Presentation at ITiCSE 2011. Related paper can be retrieved from http://dx.doi.org/10.1145/1999747.1999774

1. Evalua&ng
how
students
would
use
a
Collabora&ve
Linked
Learning
Space
Kai
Michael
Höver
Michael
Hartle
Guido
Rößling
Max
Mühlhäuser
Technische
Universität
Darmstadt

2. Outline
1)
Mo@va@on
for
CLLS
(Collabora@ve
Linked
Learning
Space)
2)
Demo
of
CLLS
3)
User
study
results
Telecoopera@on 2

3. Students‘
usage
of
addi&onal
learning
resources

Previously
conducted
survey
with
104
CS
students
[1]

Students
use
the
Web
for
learning,
especially

Wikipedia

Google
to
find
learning
resources

Forum
discussions
[1] Höver, K.M., Rößling, G. & Mühlhäuser, M., 2010, DeLFI 2010: Die 8. e-Learning Fachtagung Informatik der Gesellschaft für Informatik
e.V, Studierende, das Web und Vorlesungsaufzeichnungen. GI
Telecoopera@on 3

4. Students
use
Google:
Google
trends
sta&s&cs
for
„Quicksort“
Start
of
End
of
Start
of
summer
summer
winter
term term term
Telecoopera@on 4

5. Example
contradicts explains
Diagram Wikipedia article
illustrates Q&A
Forum discussion
Lecture Material
Diagram
Telecoopera@on 5

6. Problem

Students
use
a
broad
variety
of
different
learning
resources.

However,
these
learning
resources
are
not
explicitly
interlinked
with
each
other
(may
be
only
in
a
student‘s
mind)
➡
they
cannot
be
shared
➡
can
hardly
be
retrieved
Telecoopera@on 6

8. Focus
of
the
survey

Ques@ons
for
students:

How
do
students
assess
the
opportunity
to
link
learning
resources
with
other
material
including
a
seman@c
descrip@on?

How
do
students
assess
the
usability
of
CLLS?

Do
students
see
a
benefit
in
using
CLLS
and
how
would
they
use
it?

Ques@ons
for
educators:

How
do
educators
assess
the
opportunity
to
link
learning
resources
with
each
other
and
how
would
they
use
this?

Would
educators
use
such
a
system
for
their
courses?

Which
pros
and
cons
do
they
see?
Telecoopera@on 8

9. Survey
Methodology

Semi-­‐structured
interviews
(ø
1.5h
each)

14
CS
students

6
CS
educators

Assessment
ques@ons

5-­‐point
Likert
scale

1
=
strongly
disagree
/
not
helpful

5
=
strongly
agree
/
very
helpful

Computa@on
of
arithme@c
mean
(AM)
and
standard
devia@on
(SD)

One-­‐sample
t-­‐tests
on
the
0.05
level
Telecoopera@on 9

10. Survey
Structure
A)
Usability
B)
Linking
learning
resources
C)
Sharing
learning
resources
D)
Filtering
shared
learning
resources
E)
Visualiza@on
F)
Archiving
linked
learning
resources
Telecoopera@on 10

11. Usability

System
Usability
Scale
test
[1]

Average
score
=
88
/
100

Standard
devia@on
=
4

Students
would
use
such
a
system
mainly
with

laptops

tablets
such
as
iPad

(mobile
devices
if
UI
is
appropriate)

Need
for
suppor@ng
different
OSs
and
devices
[1] Brooke, J. (1996). SUS - A quick and dirty usability scale. Usability Evaluation in Industry, 189-194.
Telecoopera@on 11

12. Linking
learning
resources

Possibility
of
linking
learning
resources

Students:
AM=4.71,
SD=0.47;

Educators:
contradicts

for
themselves:
AM=4.5,
SD=0.84

for
students:
AM=4.83,
SD=0.41

Possibility
of
describing
the
seman@c
rela@on

Students:
AM=4.57,
SD=0.65
 Educators:

for
themselves:
AM=3.67,
SD=0.52
explains

for
students:
AM=4.20,
SD=0.84
Telecoopera@on 12

13. Linking
learning
resources
-­‐
Assessment
of
link
descrip6ons
Students Educators
explains 4,86
4,83
example for 4,57
5
illustrates 4,14
4 also
important:
3,86 -­‐
prerequisite
for
extends -­‐
ques0on
to
4,6
contradicts 3,79
3,4
agrees 2,57
2,8
0 1 3 4 5
Telecoopera@on 13

14. Linking
learning
resources
-­‐
Assessment
of
types
of
learning
resources
Students Educators
5
Slide 5
4,93
PDF document 4,8
4,57
Website 4,8
4,5
Figure, diagram 4,4
Wikipedia article 4,07
4,6
Furthermore:
Forum post 3,64 -­‐ Anima2on
5
Video 3,57 -­‐ Source
Code
4,8
3,5
Word proc. doc. 3,2
3,36
digital pen note 4
3,14
Spreadsheet doc. 3,2
2,79
Chat message 3,8
2,64
Blog post 4,6
0 1 3 4 5
Telecoopera@on 14

15. Linking
learning
resources
-­‐
detail
level
of
anchors

A
more
fine-­‐grained
level
than
on
the
document
level
is
important

Need
for
referencing

paragraphs,
sentences,
words

@mestamps,
periods
of
@me

code
snippets
contradicts
Telecoopera@on 15

16. Sharing
learning
resources

Students
already
share
created
or
found
learning
resources
with
fellow
students
by

eMail

Messenger

Web
storage
Exchanging learning
resources with CLLS Private links should
is helpful be supported
AM=4,71 (0,47) AM=2,93 (1,49)
AM=4,67 (0,82)
Telecoopera@on 16

17. Filtering
learning
resources
Educator 5
Type 4,5
Chain of filters
Semantic 4,36 is helpful
∅ Rating 4,14 AM=4.36 (0.93)
Group/Person 3,21
Period of time 3
0 1 3 4 5
Telecoopera@on 17

18. Visualiza&on

A
graphical
representa@on
of
a
knowledge
graph
is
helpful

Students:
 AM
=
4.0
(SD
=
0.88)

Educators:

for
themselves:
AM
=
3.8
(SD
=
0.84)

for
students:
AM
=
4.0
(SD
=
1.0)

Visualiza@on
approaches:

Different
shape
and
colors
for
different
kinds
of
resources

Different
sizes
and
distances
regarding
their
importance
Telecoopera@on 18

19. Archiving
linked
learning
resources

Making
a
snapshot
is
helpful

Students:
AM
=
3.79
(SD
=
1.19)

Educators:
AM
=
4.6
(SD
=
0.55)

Making
copies
of
Web
resources
prevents
them
from
being
deleted
or
modified.
Telecoopera@on 19

20. Pros
&
Cons

Students:

Ac@ve
augmenta@on
of
learning
material

Everything
is
in
one
place
and
integrated

Exchange
of
learning
resources

Educators:

Possible
to
observe
students‘
ac@vi@es
outside
the
classroom

Recognize
possible
comprehension
difficul@es
of
students

Alterna@ve
or
addi@onal
representa@ons
of
the
lecture
material

Ac@ve
working
with
the
learning
material

Social
presence

Students
&
Educator
Cons:

online
only

spam
Telecoopera@on 20

21. Future
work

Long
term
study

Integra@on
of
collabora@on
tools
and
social
networks

Support
of
fine-­‐grained
anchors
of
different
media
types

Visualiza@on

Archiving
the
learning
space
Telecoopera@on 21

22. Thank
you
for
listening!
Ques&ons?
Telecoopera@on 22