1. Lenandlar Singh
Department of Computer Science
University of Guyana
lenandlar.singh@uog.edu.gy
Peerwise and Students’ Contribution: Experiences from
the Field
CO13 – Connecting Online Conference
February 03, 2013

2. Outline
 Contributing Student Pedagogy (CSP)
 PeerWise and CSP
 PeerWise – the Tool
 Brief Literature Review
 Experiment
 Results
 Observations/Discussion
 Future Work
 References

3. Contributing Student Pedagogy(CSP)
 “…a pedagogy that encourages students to contribute to the
learning of others and to value the contributions of others”
 “CSP in formal education is anticipatory of learning processes
found in industry and research, in which the roles and
responsibilities of ‘teacher’ and ‘student’ are fluid”
 “Preparing students for this shift is one motivation for use of CSP”
 “CSP approaches are linked to constructivist and community
theories of learning, and provide opportunities to engage students
more deeply in subject material”
[Hamer et al., 2008]

4. Contributing Student Pedagogy(CSP)
 Traditionally, students are much more familiar with answering questions set by their
course instructor than they are with authoring questions of their own [Sykes et al.,
2011]
 benefits of the latter is well documented [Nicol, 2007]
 “By requiring students to engage in new roles, deeper and more substantial learning
may emerge” [Hamer et al., 2008]
 “…professional computing practice has largely diverged from hierarchical structures
and now operates in a style markedly different from traditional formal education in
which an authority disseminates textbook knowledge” [Hamer et al., 2008]
 Students now produce much more context than before – Youtube Videos, Social
Networks, etc… [Denny et al., 2010a]
 use of technology makes it possible for students to efficiently share their authored
questions with one another.

5. PeerWise and CSP
 PeerWise is a web-based system that allows for the development
multiple choice question banks entirely by students [Denny, 2008]
 Students Create MC questions, provide answers and explanations
 Students can answer MC questions, provide comments, rate questions
 Give Students a rating score for contribution
 Developed in 2007 at University of Auckland, NZ by Paul Denny [PhD
Research]

6. Why PeerWise?
 Many tools available for creating questions – MC,
Open-response, etc [Sykes et al., 2011]
 PeerWise “attempts to exploit the familiarity that
students have with social software and the
phenomenon of Web 2.0” [Denny, 2010]
 “…familiarity of students with other social networking
tools, and PeerWise’s intuitive interface, facilitates
ease of use by students with a range of computing
experience” [Sykes et al., 2011]

7. Literature Review
 Student Perceive PeerWise to be useful and Enjoyable
[Denny et al., 2008]
 external motivators are needed only for question
generation; exam performance is correlated with
participation in on-line discussions; and, despite
student enthusiasm, drill-and-practice use does not
contribute to exam success[Denny et al., 2008b]
 Measure benefits to student Learning [Denny et al.,
2010b]

8. Literature Review
 Student who used PeerWise performed significantly better in Final
Exam compared to low activity users; was easy to set up by instructor,
and students used it to study for final exam [Denny et al., 2010c]
 Student reported positively about PeerWise with explicit recognition
of its value to learning [Sykes et al., 2011]
 Student contributed beyond what was expected and performance on
peerwise was correlated significantly with exam scores[Sykes et al.,
2011]
 Students are eager participants, rate PeerWise highly, and produce
large repository of relevant, good quality questions [Bottomley &
Denny, 2011]

9. Literature Review – Subject Domains
 Computer Science/Programming - [Denny, 2008]
 Engineering [Denny et al., 2009]
 Biomedical Engineering [Denny et al., 2010a]
 Information Systems [Smith et al., 2011]
 Veterinary Medicine [Sykes et al., 2011]
 Biomedical Science [Bottomley & Denny, 2011]
 Introductory Physics [Bates et al., 2011]
 Introductory Cell Biology [Smith, 2011]

10. PeerWise – The Tool

11. PeerWise

12. Social PeerWise

13. Social PeerWise

14. PeerWise Uptake
UG

15. PeerWise at UG (Dept. of CS)

16. PeerWise at UG (Dept. of CS)

17. Experiment (2011 & 2012)
 CSI 312 (2011) & CSI 3102 (2012) – Introduction to Object Oriented
Programming
 Approximately:
 25 3rd year BSc in Comp Sci students (2011)
 30 3rd year BSc in Comp Sci students (2012)
 Peerwise used for Approximately 8 weeks in both years
 Students were asked to created 3-5 MCQs, and answer, comment, and
participate as much otherwise
 2% of Courswork Marks allocated (2011) & 3% (2012)
 No training on creating MC questions was conducted

18. Administrator’s Interface

19. Results – Contribution Summary

20. Results

21. Results – Contribution Breakdown
View
Details

22. Results – Individual Contribution

23. Results – Individual Contribution

24. Results – Individual Contribution

25. Results – Sample Comments
Question
Details

26. Performance Analysis
Metric Total Average/Student
MCQs Written 126 7.4
Answered 894 52
Comments 342 20
Answered Correctly 693(78%) 41
Answered Incorrectly 201(22%) 12
Metric Total Average/Student
MCQs Written 115 4.0
Answered 666 23
Comments 61 2
Answered Correctly 415(62%) 15
Answered Incorrectly 251(38%) 9

27. Usage Pattern – CSI 312/2011

28. Usage Pattern – CSI 3102/2012

29. PeerWise Contribtion vs Final Exam Performance
 Correlations
Answered Commented AnsCorrect Final Exam
Marks
Created 0.4
0.3
0.63
0.41
0.4
0.33
-0.86
(0.25)
Answered -0.167
0.05
Commented 0.410
0.60
0.47
0.67
-.507
0.31
AnsCorrect 0.98
0.95
-.188
0.15

30. Student Survey
 Students were asked a number of questions via an
online survey about their experience using Peerwise:
 Ease of use
 Impacted their learning
 Motivation to use tool and contribute
 If believe tool is useful for students
Students Response
“Peerwise was an awesome tool once I got acquainted
with it (not that it took much time to get acquainted
with).”

31. Student Survey
Students Response cont’d
“In choosing question content, it caused me to do
research on particular areas before I posted a question. I
had to be sure about something before I posted it”
“ Answering and reviewing other person's questions also
taught me alot as it was important to research and be
certain before presenting suggestions and reviews to
questions.”

32. Student Survey
Students Response cont’d
“I believe the interface was easy to use and presented a
useful way of keeping my interest(the badges and leader
board). In fact, once I got started, seeing my position
rise on the leader board and getting new badges was
addictive.”
“I definitely think the tool would be useful for students.
My only regret is that more students didn't post and
critique questions posted.”

33. Student Survey
Students Response cont’d
“My recommendation is that even more use of this tool
be advocated for the next batch of students. Also, I
wish more emphasis could be placed on them getting
started earlier. I found it fun and think that persons
will enjoy it once they get started.”
“I would also advocate its use for other subject areas,
it's definitely a great learning tool.”

34. Student Survey
Students Response cont’d
“Setting up questions was easy enough.”
“…the ability to add explanations and comments for
questions was a big help in instances where I may have
answered a question incorrectly”
“ a contributor should be notified maybe by email when a
question that they supplied has comments especially in
cases where the comments may be critical of the answer
the contributor would have chosen”

35. Student Survey
Students Response cont’d
“ I actually used the tool in preparation for my exams, i
wasn't able to get through all of the questions however. I
think it can be useful to students as a quick reference
and a quick crash study before exams in order to
sharpen up before.” “
“On the flip side questions that are incorrect need to
be fixed so as to ensure students don't end up taking
the wrong answers and thinking those are correct.”

36. Student Survey
Students Response cont’d
“it was useful because it encompasses a new way of
learning”
“its a very useful tool for student because it has a new
method of learning whereas the student post questions,
also provide answer and explanation.”
“The idea of class participation motivated me whereas
everyone is involved. Also both contributing and
answering question motivated me.”
“It was very easy, no complication at all”

37. Student Survey
Students Response cont’d
“Well to my knowledge Ive learn a lot whereas the class is
involved and everyone will try to compete to post "hard"
questions, by doing that i have leaned a lot.”
“I think it should be implemented for other courses
especially in Computer Science”

38. Observation/Discussion
 Students contributed more than expected
 Some evidence that ‘low-achieving’ students were
more active in creating and answering questions
 Negative/low correlation between number of
questions created and performance on final exam
 Negative/low correlation between all other variables
and Final Exam performance

39. Conclusion
 Did it work?
 Yes, for 2%/3% most definitely
 Students enjoyed working with the tool

40. Future Work
 Further Analysis on Existing Dataset
 Use Taxonomy to categorize/rate/rank questions
 Develop/Use Taxonomy to Evaluate Questions
 Use PeerWise extensively – Difference
Groups/Courses

41. References
 Bates S.P., Galloway R.K., McBride K.L., (2011) Student generated content: using PeerWise to enhance engagement
and outcomes in introductory physics courses. Proceedings of the 2011 Physics Education Research Conference
 Bottomley, S., & Denny, P. (2011). A Participatory Learning Approach to Biochemistry Using student Authored
and Evaluated Multiple-choice Questions. Bi0chemistry and Molecular Biology Education, 39(5), 352-361.
 Denny,P., Hamer, J., Luxton-Reilly, A., & Purchase, H. (2008). PeerWise. Koli Calling ’08 November 13-16, 2008,
Koli, Finland
 Denny, P., Luxton-Reilly,A., & Hamer, J.(2008b). Student Use of the PeerWise System. ITiCSE’08, June 30–July 2,
2008, Madrid, Spain.
 Denny, P., Hamer, J., & Luxton-Reilly,A.(2009). Students sharing and evaluating MCQs in a large first year
Engineering course . 20th Australasian Association for Engineering Education Conference , Adelaide, Australia.
 Denny, P. (2010). Motivating Online Collaborative Learning. ITiCSE’10, June 26–30, 2010, Bikent, Ankara, Turkey
 Denny, P., Micou, M., & Simon, B.(2010a). Evaluation of PeerWise as an Educational Tool for Bioengineers.
American Society for Engineering Education.

42. References Denny, P., Hanks, B., & Simon, B., (2010b). Peerwise: replication study of a student-collaborative self-testing
web service in a U.S. setting. SIGCSE 2010. ACM, USA, pp.421-425.
 Denny, P., et al. (2010c). Evaluation of PeerWise as an Educational Tool for Bioengineers. American Society
for Engineering Education, 2010
 Hamer,J. et al. (2008). Contributing Student Pedagogy. Inroads -Sigcse Bulletin, 40(4), 194-212.
 Hooper, A.S.C., Park, S.J., & Gerondis, G. (2011). Promoting student participation and collaborative
learning in a large INFO 101 class: Student perceptions of PeerWise Web 2.0 technology. Proceedings
of Higher Education Research and Development Society of Australasia International Conference
(HERDSA 2011).
 Nicol, D., 2007. E-assessment by design: using multiple-choice tests to good effect. Journal of Further and
Higher Education, 31 (1) pp.53-64.
 Smith, K.M. (2011). Peerwise Participation –supporting students in a multiple choice environment.
Proceedings of the Effective Learning in the Biosciences Conference, Edinburgh, UK.
 Sykes, A., et al. (2011). PeerWise - The Marmite of Veterinary Student Learning