1. Linking Learning to Assessments
BJDSOP Faculty Retreat
5/16/2012

2. Linking Learning to Assessment
1. Writing learning objectives
2. Relating objectives to assessment
3. Writing assessment questions
4. How to analyze knowledge tests for
discrimination

3. Learning objectives
1. Define an objective.
2. How/ when to author
3. How to align objectives with
assessments.

4. Characteristics of
objectives
• Specific and focused
• Targets performance
• Realistic to achieve
• Can be measured and validated
• Time-bound with a deadline

5. Learning objectives…
• … describe the intended
result of instruction.
LWBAT

6. Outcomes
• recall, identify, choose,
• solve,
• calculate
• apply therapeutic concepts to
clinical scenarios

7. Iterative process
Lecture/lab
Objectives Assessments
LWBAT WTAT?

8. “Begin with the end
in mind”
Stephen Covey
The Seven Habits of Highly Effective People

9. • We need to
know what
we want the
end resultto
be before we
plan.

10. Write your test
questions first
then
write your
objectives

11. Questions already
written?
• Rewrite objectives to align
with what is required of
the students on the
assessments

12. Rationale
1. Alignment
2. Focuses attention on what is
most important*
3. Promotes continuous
improvement

13. Consider these….

14. Example 1
• Question
– Which statement is correct…
• Objective:
–Student will be able to identify..

15. Example 2
• Refer to the case above.
• What is the most appropriate
therapy at this time?
• Objective:
–Student will be able to select/…

16. Work smart
• What (are) the KSAs, being
assessed by your test questions?
• Restate those as your objectives
in LWBAT terms…..

17. Objectives RECAP
1. What is a learning objective?
2. When do we right them?
3. Why?
4. How do we align objectives
with assessments?

18. What is a good question?
1. Good form

19. Bad format DO NOT COPY

20. Good format – please copy

21. Factual recall format

22. Better

23. Basic Science examples

24. More BSexamples

25. Vignettes NBME
• “..we believe vignette items are generally
more appropriate ..”
–test application of knowledge to
patient situations
–pose appropriate clinical challenges

26. Non vignette

27. Short Vignette

28. Long Vignette

29. What is a good question?
1. Good form
2.Best practices

31. www.nbme.org /

32. NBME Guidelines

33. Best Practices
• No T F
• MC with 4-5 good options
• 1 correct answer preferred

34. TestwisenessBAD
• Grammatical cues
• Absolute terms
• Long correct answer
• Word repeats

35. Irrelevant Difficulty
BAD
• Options are long, complicated
• Numeric data not stated
consistently
• Vague – “rarely”
• Avoid all/none of the above
• Hinged responses

36. K- type questions
• Avoid
• Re

38. Good questions
• Big stems
• ~ Same length distractors (short)
• Avoid absolutes / vague terms
• Avoid negatively phrased items

39. What is a good question?
1.Good form
2.Best practices
3.Performs well

43. Performance

47. A good question?
1. % correct
2. Item discrimination
3. Distractor performance

48. Item Analysis

51. A good question?
1. % correct
2. Item discrimination
3. Distractor performance

52. Test matrix
2010 2011 %
Change
CK
13 11 (15.3%)
APP
3 5 66.6%
Correct
86 90.5 4.9%
response CK
Correct APP
62 65.8 6.1%

53. Bloom’s Taxonomy
Level
Less 1. Knowledge
complex
2. Comprehension
3. Application
4. Analysis
5. Synthesis
6. Evaluation
More
complex

54. VERBS
Bloom’s Level Verbs
1. Knowledge match, recognize, select, compute, define,
label, name, describe
2. Comprehension restate, elaborate, identify, explain, paraphrase,
summarize
3. Application Apply knowledge, solve problems
4. Analysis outline, draw a diagram, illustrate, discriminate,
subdivide
5. Synthesis compare, contrast, organize, generate, design,
formulate
6. Evaluation support, interpret, criticize, judge, critique,
appraise

55. Linking Learning to Assessment
1. Writing learning objectives
2. Relating objectives to assessment
3. Writing assessment questions
4. How to analyze knowledge tests for
discrimination

56. References
1. Case SM, Swanson DB, Becker DF. Verbosity, window
dressing, and red herrings: do they make a better test item?
Academic Medicine. 1996;71:528-530.
2. NBME Constructing Written Test Questions for the Basic and
Clinical Sciences
3. Haladyna, T. M., & Downing, S. M. (1989a). A taxonomy of
multiple-choice item-writing rules. Applied Measurement in
Education, 2(1), 37-50.
4. Frisbie, D. A. (1990, April). The evolution of the multiple true-
false item format. Paper presented attheAnual Meeting of
the National Council on Measurement in Education, Boston.

57. • Sands (2002), the “basic precept of course-
planning [is]: What do [you] want students to
be able to do at the end of the semester?” In
other words, course goals and objectives
should guide the design of your course rather
than technology (Aycock, Garnham, &Kaleta,
2002). Sands’ first principle for developing a
blended course is to “work backward from the
final course goal…to avoid a
counterproductive focus on technology.”