Workshop given as part of NTU's Innovations in Teaching Seminar , September 2015

1. student
generated
content
for
enhanced
engagement
and
learning
Simon
Bates
simon.bates@ubc.ca
The
University
of
Bri7sh
Columbia

2. Overview
for
the
session
1)
About
PeerWise
2)
Hands-­‐on
session
3) Research
highlights
4) Q&A
student
generated
content
for
enhanced
engagement
and
learning

3. 1)
About
PeerWise
student
generated
content
for
enhanced
engagement
and
learning

4. a
web-­‐based
MCQ
repository
created
by
students

5. InsFtuFons
signing
up
per
year:
2009:
22
2010:
66
2011:
204
2012:
266
2013
(Jan-­‐Jun):
214
Growing
content
repository:
Courses:
2,500
Logins/month:
75,000
Ques7ons:
600,000
Answers:
12,000,000

6. 0
5,000,000
10,000,000
15,000,000
20,000,000
25,000,000
30,000,000
Jun 09
Dec 09
Jun 10
Dec 10
Jun 11
Dec 11
Jun 12
Dec 12
Jun 13
Dec 13
Jun 14
Dec 14
May 15
Total answers at end of date
Submitted Answers (2009‐2015)

7. Student
ownership
over
learning
resource
Student
familiarity
with
social
soPware
Leveraging
student
energy
and
creaFvity

8. Why
student
authored
ques7ons?
• A
student
contribu7ng
a
new
ques7on
would
develop
the
ques7on
stem:
QuesFon
Student
authored
ques7on
HUBS192,
2010
University
of
Otago

9. Why
student
authored
ques7ons?
AlternaFves
• And
a
set
of
plausible
alterna7ves
• Of
course,
the
correct
answer
must
be
indicated

10. Why
student
authored
ques7ons?
AlternaFves
• And
a
set
of
plausible
alterna7ves
• Of
course,
the
correct
answer
must
be
indicated

11. Why
student
authored
ques7ons?
ExplanaFon
• And
an
explana7on,
in
their
own
words
• Useful
for
students
who
answer
incorrectly

12. “You
don't
really
understand
how
much
or
how
li5le
you
know
about
a
concept
un9l
you
try
to
devise
a
good,
original
ques9on
about
it”
“The
aspect
I
found
truly
useful
was
the
crea9on
of
ques9ons,
which
reinforced
much
of
[my]
understanding
while
also
ac9vely
making
me
clarify
and
solidify
my
thought
processes
(especially
the
explana9on
parts)”
Why
student
authored
ques7ons?
Student
feedback

14. Badges
Points
Leaderboards

15. 2)
Hands-­‐on
session
student
generated
content
for
enhanced
engagement
and
learning
Your
turn:
open
a
new
window
in
your
browser

16. Google:
peerwise
Start
typing:
Bri5sh
Columbia…...
hVp://bit.ly/pwntu
Or
go
direct
to
the
URL:

18. Registra7on

19. Step
1
–
choose
a
name
Our
recommendaFon:
please
use
your
first
iniFal
and
your
surname
(e.g.
sbates)

20. Step
2
–
enter
a
password

21. Step
3
–
enter
the
Course
ID
11498

22. Step
4
–
enter
your
Iden7fier
Any
3-­‐digit
number
between
100
and
500

23. That’s
it!

24. now
log
in…

25. Choose
the
“NTU2015
example”
course

26. 1)
Your
ques7ons:
“view”
2)
Create
new
ques7on
3)
Preview
4)
“Make
changes”
or
“Save
ques7on”
CreaFng
a
quesFon

27. 1)
Unanswered
ques7ons:
“view”
2)
Choose
ques7on
3)
Select
answer
Answering
quesFons

28. 3)
Research
highlights
student
generated
content
for
enhanced
engagement
and
learning

29. Research
highlights
• Rela7onship
between
ac7vity
and
performance
• The
quality
of
student-­‐authored
ques7ons
• Influencing
student
behaviour
with
rewards

30. Ac7vity
and
learning
• Generally,
students:
– Par7cipate
beyond
minimum
requirements
– Engage
in
community
learning,
correc7ng
errors
– Create
problems,
not
exercises
– Provide
posi7ve
feedback

31. Correla7on
with
end
of
course
outcomes

32. Correla7on
with
end
of
course
outcomes

33. Correla7on
with
end
of
course
outcomes

34. Ques7on
quality
Comprehensive
categorisa7on
of
>50%
of
repository
for
two
successive
academic
years
Principal
measures
to
define
a
‘high
quality
ques7on’
-­‐
cogni7ve
level
of
ques7on
-­‐
explana7on
quality
-­‐
other
binary
criteria

35. Category Description
6 Create (synthesise ideas)
5 Assess
4 Analyse (multi-step)
3 Apply (1-step calcs.)
2 Understand
1 Remember
Cogni7ve
level
of
ques7on

36. 0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
1 2 3 4 5 6
Taxonomic Category
PercentageofSubmittedQuestions
First semester N = 350
Second semester N = 252
Results: Question level Physics 1A / 1B 2011

37. • Badge
/
achievement
system
• Randomised
controlled
trial
(n
>
1000)
Authoring
ques7ons
Evalua7ng
ques7ons
Answering
ques7ons
Other
Do
rewards
work?

38. No
difference
between
groups
with
respect
to
answer
correctness
+
13%
+
22%
DistribuFon
of
days
of
acFvity
differs
significantly
between
groups:
p
<
0.001
DistribuFon
of
answers
per
student
differs
significantly
between
groups:
p
<
0.001

39. Implica7ons
• Badges
can
have
a
measurable
posi7ve
impact
on
some
student
ac7ons
• No
nega7ve
effects
on
ac7vity
or
percep7ons:
a
low-­‐risk
proposi7on

40. Please
join
us!
hVp://www.peerwise-­‐community.org/
These
slides
hVp://bit.ly/batestalks

41. 4)
Q
&
A
student
generated
content
for
enhanced
engagement
and
learning

42. Publica7ons
“Student-­‐generated
content:
using
PeerWise
to
enhance
engagement
and
outcomes
in
introductory
physics
courses”
(S.
P.
Bates,
R.
K.
Galloway,
K.
L.
McBride),
In
2011
Physics
Educa7on
Research
Conference
(NS
Rebello,
PV
Engelhardt,
C
Singh,
eds.),
Amer.
Inst.
Physics,
volume
1413,
2012.
(Physics
Educa7on
Research
Conference,
Omaha,
NE,
2011)
“Student-­‐generated
content:
Enhancing
learning
through
sharing
mulFple-­‐choice
quesFons”
(J.
A.
Hardy,
S.
P.
Bates,
M.
M.
Casey,
K.
W.
Galloway,
R.
K.
Galloway,
A.
E.
Kay,
P.
Kirsop,
&
H.
A.
McQueen)
Interna7onal
Journal
of
Science
Educa7on
(2014)
1-­‐15
“Assessing
the
quality
of
a
student-­‐generated
quesFon
repository”
(S.
P.
Bates,
R.
K.
Galloway,
J.
Riise,
and
D.
Homer)
Physical
Review
Special
Topics:PER
(2014)
10,
020105
“The
Effect
of
Virtual
Achievements
on
Student
Engagement”
(P.
Denny),
In
Proceedings
of
the
SIGCHI
Conference
on
Human
Factors
in
Compu7ng
Systems
(CHI
'13),
pp
763-­‐772.
ACM,
2013.

43. Acknowledgements
These
slides
were
based
on
an
online
workshop
at
the
Western
Conference
on
Science
Educa7on,
London
Ontario
in
Summer
2013.