Staff training clinic on alignment of test items and state standards using both Revised Bloom's Taxonomy and Depth of Knowledge levels, May 2009

1. Practical Alignment for Item / Test Developers: Revised Bloom’s Taxonomy (RBT) and Depth of Knowledge Levels (DOK) Staff Trainings, 2009 Jerrie W. Brown, Content Lead for Social Studies Assessments North Carolina State University Center for Urban & Community Services Technical Outreach for Public Schools (TOPS)

3. What is alignment? The No Child Left Behind Act of 2001 (NCLB) requires that each state align assessments to content standards. Once the enacted (instruction), intended (state standards), and assessed (state tests) curricula have been measured, questions can be asked about the extent to which content is similar across them. To the extent content is the same, they are said to be aligned. J. Brown, NCSU, 2009

7. Revised Bloom’s Taxonomy (RBT) Table for _____________________________ (Subject / Grade Level) Adapted from Anderson, Lorin W. and David R. Krathwohl, et al., (2001). J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) B. Conceptual Knowledge (0%) C. Procedural Knowledge (0%) D. Meta- Cognitive Knowledge (0%)

8. Depth of Knowledge (DOK) Levels for _____________________________ (Subject / Grade Level) Adapted from Norm Webb, (2002). J. Brown, NCSU, 2009 Content Categories Depth of Knowledge Levels Categorical Concurrence Yes / No 1. Recall 2. Skill/Concept 3. Strategic Thinking 4. Extended Thinking A. Topic/Strand __________ (Specify) B. Topic/Strand __________ (Specify) C. Topic/Strand __________ (Specify)

9. RBT’s Knowledge Dimension: J. Brown, NCSU, 2009 Major Types Definitions A Factual The basic elements students must know to be acquainted with a discipline or solve problems in it B Conceptual The interrelationships among the basic elements within a larger structure that enable them to function together C Procedural How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods D Metacognitive Knowledge of cognition in general as well as awareness and knowledge of one’s own cognition

10. RBT’s Knowledge Dimension: J. Brown, NCSU, 2009 Major Types Subtypes A Factual AA. Knowledge of terminology AB. Knowledge of specific details and elements B Conceptual BA. Knowledge of classifications and categories BB. Knowledge of principles and generalizations BC. Knowledge of theories, models, and structures C Procedural CA. Knowledge of subject-specific skills and algorithms CB. Knowledge of subject-specific techniques and methods CC. Knowledge of criteria for determining when to use appropriate procedures D Metacognitive DA. Strategic knowledge DB. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge DC. Self-knowledge

11. RBT’s Cognitive Process Dimension: J. Brown, NCSU, 2009 Cognitive Process Categories Definitions 1 Remember Retrieve relevant knowledge from long-term memory 2 Understand Construct meaning from instructional messages, including oral, written, and graphic communication 3 Apply Carry out or use a procedure in a given situation 4 Analyze Break material down into its constituent parts and determine how the parts relate to one another and to an overall structure or purpose 5 Evaluate Make judgments based on criteria and standards 6 Create Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure

12. RBT’s Cognitive Process Dimension: J. Brown, NCSU, 2009 Cognitive Process Categories Cognitive Processes 1 Remember 1.1 Recognizing 1.2 Recalling 2 Understand 2.1 Interpreting 2.2 Exemplifying 2.3 Classifying 2.4 Summarizing 2.5 Inferring 2.6 Comparing 2.7 Explaining 3 Apply 3.1 Executing 3.2 Implementing 4 Analyze 4.1 Differentiating 4.2 Organizing 4.3 Attributing 5 Evaluate 5.1 Checking 5.2 Critiquing 6 Create 6.1 Generating 6.2 Planning 6.3 Producing

15. Subject-Specific DOKs J. Brown, NCSU, 2009 Science Social Studies Math Level 1 Recall and Reproduction Recall of Information Recall Level 2 Skills and Concepts Basic Reasoning Skill/ Concept Level 3 Strategic Thinking Complex Reasoning Strategic Thinking Level 4 Extended Thinking Extended Reasoning Extended Thinking

18. Rating WA State Standards J. Brown, NCSU, 2009

19. Plotting Standards J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) 10 1, 5 9 B. Conceptual Knowledge (0%) 2, 3 8 6 C. Procedural Knowledge (0%) 7 4 D. Meta- Cognitive Knowledge (0%)

20. Rating ‘WASL’ Test Items J. Brown, NCSU, 2009

21. Plotting Test Items J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) 10 1, 5 9 B. Conceptual Knowledge (0%) 2, 3 8 6 C. Procedural Knowledge (0%) 7 4 D. Meta- Cognitive Knowledge (0%)

22. Individual and Group Results

23. Thank You for Participating! FOR MORE INFORMATION : Mr. Jerrie W. Brown, Consultant North Carolina State University Center for Urban and Community Services Technical Outreach for Public Schools 1500 Blue Ridge Road Raleigh, N. Carolina 27607 Tel: (919) 515-1125 Email: [email_address] Website: http://www.cuacs.ncsu.edu