Achievement Network Open Response Scoring Workshop given at Cleveland Elementary School on Tuesday, January 12, 2010

1. ConstructedResponse Palooza Winter 2010

2. Think, pair, share What are some of the challenges you face when grading constructed response questions?

4. Provide an opportunity to examine and discuss an anchor set for a given writing prompt.

5. Provide teachers with the opportunity to examine student samples from an administered constructed response item and norm along the ANet rubric.OVERARCHING GOAL: To leave with a better understanding of using rubrics to effectively score constructed responses and to take this back to your schools to improve your school’s overall use of constructed response data.

6. agenda I. Rubrics 101 15 min. Different types of rubrics Different types of questions/prompts Scoring II. Norming around an Anchor Set 45 min. Review of passage, item, and rubric Model of Scoring Paired Scoring and Debrief II. Practice Norming around ANet Responses 1 hour Groups use rubrics to score student responses Debrief IV. Next Steps 10 min.

9. A starting point to initiate a student’s own thoughts on the topic, supported by their experiences and ideas

10. No “correct” answer (but must address topic specified by the prompt)

11. Style and organization are important to the score

12. Uses Analytic Rubric and scored in two domains: Topic Development and Conventions

14. Student must answer the question asked and answer must be supported with details from the text

15. Answers found in reading the passage or can be inferred from information in the passage

16. The explanation and support from the text matter much more than style and organization

17. Uses Holistic Rubric, conventions are not scored

19. Analytic Rubric Topic/Idea Development

20. Analytic Rubric Standard English Conventions

21. Sample Constructed Response Question Based on the article, describe what people have learned from studying slave quilts. Support your answer with important information from the article. Example is question 35 from the 2008 6th grade ELA MCAS Passage “Pattern for Freedom: Women’s Quilt as Art”

22. Holistic Rubric

23. Scoring Bias in Constructed Response items It’s important for validity not to let writing skills impinge on reading comprehension scores. Things to look past (to avoid scorer bias) Penmanship Neatness Mechanical errors Spelling Length

24. Process for scoring student responses to Constructed Response Prompts Read the passage and the accompanying prompt. Review the rubric for the prompt and highlight key words. Create a list of scoring notes. Begin to build an anchor set. Score remaining student responses using rubric and anchor set.

26. Review the rubric

27. highlight key words in the rubric

31. An anchor set will include examples of student responses at each score point at the rubric.

33. Student Response #2 Score: 1 Why a 1? There is minimal attempt to explain that slave quilts contain meaningful designs. There are no supporting details from the article included. Why not a 0? The information provided was not incorrect. The response, while lacking details from the passage, does show understanding of quilts helping slaves escape from slavery and that designs had different meanings.

34. Student Response #3 Score: 2 Why a 2? The response is a general description of what people have learned from studying slave quilts. The focus of the discussion is on the symbolism of the colors and patterns used in slave quilts. Why not a 3? The response is partially complete in that it lacks any reference to the use of quilts for passing down history. Details from the story are general.

35. Student Response #4 Score: 3 Why a 3? The response clearly, completely, and accurately explains that the study of slave quilts has taught people about the personal history of slaves, aspects of African culture, and symbols used in the Underground railroad. Relevant quotes from the article are included to support each idea. The introduction and conclusion do not contribute any additional information to this response. Why not a 2? The essay is clear and complete. Specific details, including quotes from the article, are used in the response. No inaccurate details are included.

36. Think, Pair, Score Read student samples #5 and #6. Score each sample using the rubric. Justify your score. Share your thoughts with your neighbors.

37. Student Responses #5 and #6 Student Response #5 Score: 3 Why a 3? The response is a clear, complete and accurate description of what people have learned from studying slave quilts. Details are included to explain the meaning of several historical symbols as well as quilt patterns designed to guide runaway slaves. The symbolic meaning of different colors are also discussed. Student Response #6 Score: 2 Why a 2? This response provides a partial explanation of what people have learned from studying slave quilts. Information is provided about the symbolism of the color black and the significance of certain patterns, but overall details are limited.

38. Scoring an ANET prompt Read the passage and the accompanying prompt. Review the rubric for the prompt and highlight key words. Create a list of scoring notes. Begin to build an anchor set. Score remaining student responses using rubric and anchor set.

39. ANet Writing Prompt Read the passage: Take a moment to read the passage from Lost in the Grand Canyon by Deborah Hopkinson. Read the prompt According to the article, Powell was “a born leader with courage and ambition.” Based on the article do you agree or disagree with this statement? Why? Use evidence from the article to support your answer.

40. Review the Rubric

41. Highlight Key words in the rubric

42. Scoring Notes (things to look for) Powell is a born leader Willing to search last unknown area (Grand Canyon). Fought in the Civil War- lost arm, unable to row, but still leads journey. Shouts orders and warnings when river turns wild. Races downstream searching for the boat. Comes up with plan to ‘run down a chute’. Shoots off rifles and waits for the men who left- before continuing on the journey. Powell and crew make it safely down the Colorado River. Powell is not a born leader Places blame on others when things go wrong. Orders crew members around. Powell is unprepared- doesn’t know how long trip will take and there isn’t enough food. Tries to persuade crew members to stay, but is unsuccessful.

43. Model Scoring- Response A Please refer to Response A Score: 0 Why a 0? All details/information provided in response are incorrect. Why not a 1? Student does not include any relevant and correct details in response to support a position.

44. Model Scoring- Response B Please refer to Response B Score 2: Why a 2? Student provides a partial explanation of why Powell was a born leader. Details are included about Powell attempting to save men after the boat went over the waterfall and that he did not allow his men to starve. The overall details are limited and the response is not complete. Why not a 3? The response includes some relevant details from the passage to support the position, however response is not complete.

45. Begin to Build an Anchor set You will be receiving a set of papers for this prompt. Unfortunately, each scoring point on the rubric does not have a student sample. Independently and then in pairs take some time to review each of the papers in light of the rubric and provide a score for each one. Please make notes for each example as to why you believe it deserves that score.

46. Anchor Set Response C Score: 2 Most examples contained in response were relevant and supported the position of Powell being a born leader. Response was partially complete. Response D Score: 1 Response is a minimal description of Powell being a born leader. Supporting details from article are limited and there is little attempt to explain how or why Powell should be viewed as a born leader. Response E Score: 1 This response includes a minimal description of Powell being a born leader. Some details are incorrect or irrelevant.

47. Anchor Set Response F Score: 0 Response only restates the prompt. No information from passage is included. Response demonstrates no description of whether Powell is a born leader. Response G Score: 2 Response includes multiple relevant details to support Powell being a born leader. Specific details were included to support thoughts presented. Response was not complete. Response H Score: 2 This response includes relevant details. Final section seems to include inaccuracies and is incomplete.

48. Takeaways

49. Best practices

51. Please take the anchor set, passages, rubrics and student responses with you.

52. Please contact us with any questions you might have.