Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Peer review in an economics writing course: the Pen is still mightier than the SWoRD,
Similar to Peer review in an economics writing course: the Pen is still mightier than the SWoRD, (20)
More from Course Design Institute and pICT SDSU
More from Course Design Institute and pICT SDSU (20)
Recently uploaded
Recently uploaded (20)
Peer review in an economics writing course: the Pen is still mightier than the SWoRD,
- 1. Peer review in an economics writing course: the Pen is still mightier than the SWoRD Jennifer Imazeki Department of Economics jimazeki@gmail.com http://economicsforteachers.blogspot.com
- 2. SWoRD Scaffolded Writing and Reviewing in the Disciplines Students submit papers Students review papers of 4-6 peers Comments and 7-point numeric rating Students ‘back evaluate’ reviews they receive 5-point numeric rating
- 3. What makes SWoRD different? Student review and backevaluation scores grades for both writing and reviewing Reviewing grade: “Accuracy” + “Helpfulness” Accuracy: Consistency compared to other reviewers Helpfulness: Back evaluation ratings Writing grade: adjusted average of numeric ratings from reviewers Scoring algorithm to remove bias
- 4. Econ 449W: Economic Literacy “The goal of this course is to make you better economists by making you better writers… This course will focus on how to apply and use [economic] knowledge, by incorporating economic material and the economic way of thinking into writing for a more general audience.”
- 5. Econ 449W: Structure 2010: 6 peer-reviewed papers, exchanged in class 1 reviewer for each paper, plus instructor feedback No feedback for reviewers, participation points only
- 6. Potential benefits of SWoRD(vs. other peer review tools) Students read multiple papers, receive feedback from multiple reviewers Back evaluations provide incentive to take reviews seriously, give students feedback for improving reviews (without requiring instructor intervention) Rubric scores converted to grades with adjustment for bias
- 7. Econ 449W: 2010 vs. 2011 2010: 6 peer-reviewed papers, exchanged in class 1 reviewer for each paper, plus instructor feedback No feedback for reviewers, participation points only 2011: 4 peer reviewed papers, all online submission 4-5 reviewers for each paper, no/less instructor feedback on draft 1 Writers back-evaluate reviewers, reviewing score based 50% on those back evaluations
- 8. Assignment timeline 5 days 5 days 5 days 9pm Monday 9pm Sunday 9pm Friday 9pm Wednesday First Draft Final draft Final reviews (5) Reviews (5) Back reviews by Thurs Back reviews by Wednesday
- 9. Issues Lack of flexibility Lack of transparency Scoring algorithm questionable
- 12. Peer review in an economics writing course: the Pen is still mightier than the SWoRD Jennifer Imazeki Department of Economics jimazeki@gmail.com http://economicsforteachers.blogspot.com
Editor's Notes
- http://www.lrdc.pitt.edu/schunn/sword/index.html
- Review process itself similar to other options (e.g., PeerMark, doing it manually)
- Key point here is that the focus is not JUST writing but incorporating economic ideas into writing (so they have to know some econ)
- - Explain process (half class writing/half reviewing)- I already had rubrics and peer review instructions for each assignment, so fixed costs low
- This was the main reason I wanted to try SWoRD – already did some peer review but SWoRD a) allows for more feedback overall and b) potentially less grading for me
- - the biggest change in the structurewas that because you need at least 4 or more reviews in SWoRD to get meaningful composite scores, I reduced the number of writing assignments and stretched out the timeline for each paperI had hoped to give less feedback on the first draft but after first assignment, realized that wasn’t going to happen – partly a problem with my comment and rating prompts, partly students don’t know how to give good feedback, partly that students don’t feel ‘qualified’ to give feedback (since emphasis in this course is as much on content as writing) These are challenges for peer review in general (not necessarily SWoRD): students need training in how to give good comments; prompts and ratings need to be well-designed, not vague, get students to recognize their own ability to be ‘experts’But the problems students have with giving good reviewing feedback is even more important with if student grades are going to be based on those reviews and that feedbackBack evaluations did not provide good assessment or sufficient feedback for improving comments
- One thing I want to point out is that my class met on Mondays and Wednesdays – issues with SWoRD deadlines
- Interface overall is not very flexible (deadlines biggest problem but also admin)Explain about how SWoRD requires curve but not transparent (show screenshots)Can see raw scores, final scores, but can’t see scores without curve applied
- Stats for each draft
- Draft grades (cannot change order of presentation in SWoRD)
- So can SWoRD replace instructor grading? No