education

1. Writing MCQs for
for TBLs and POGILs
USING THE GUIDELINES FOR NBME QUESTION WRITING

2. Why use multiple choice questions?
Advantages
 Versatile
 Efficient
 Accurate
 Reliable
 Controllable
 Reduces guessing
Disadvantages
 Difficult to write well
 Tendency of author to write recall
items only
 Influenced by test-savvy
responders
 Subject to clueing

3. Terminology used in MCQs
Stem
The question or statement that precedes answer
choices
Options
Response choices
Key
The single best correct answer
Distractors
Incorrect options (not the best choices)

4. Basic Rules for NBME Question
Writing that Apply to TBL and POGIL
Rule # 1 - Each item should focus on an important
concept rather than something trivial, tricky, or overly
complicated
 Options should be free of fine distinctions
 Avoid options too close to differentiate
 Ask, does it pass the “who cares” test?

5. Basic Rules for NBME Question
Writing that Apply to TBL and POGIL
Rule # 2 - The stem should pose a clear question
 Stem can be long; options should be short, and relatively even in
length
 To determine if the question is focused, cover up the options and
see if the responder can answer the question based only on the
stem; if not, rewrite the question
 Avoid complete the question or fill in the blank type questions.

6. Basic Rules for NBME Question
Writing that Apply to TBL and POGIL
Rule # 3 - All distractors should be homogeneous
 They should fall into the same category as the correct answer
(i.e, all tests, treatments, diagnoses, prognoses, etc).
 All distractors should be plausible, grammatically consistent,
logically compatible, and of relatively the same length as the
correct answer

7. Basic Rules for NBME Question
Writing that Apply to TBL and POGIL
Rule # 4 - Avoid technical flaws that provide
benefit to test savvy students or trainees
 Do NOT write any questions of the form, “which of the following
statements are correct?” or Each of the following are correct
EXCEPT ….” These are unfocused and have heterogeneous
options.

8. In general,
 Strive for positively worded questions
 Negative stems are usually flawed questions
 Avoid the use of not, least, except
 Negative items are also easily misread
 Negative items are equally misread by high and low scoring students
 Negative items do not discriminate well between those that know the
content and those that do not
 Be precise, and avoid indefinite or vague terminology

9. Frequently used Stems, after setting up the
scenario for a question
 What is the most likely mechanism of action?
 Which of the following is the most likely explanation for these findings?
 Which of the following anatomic structures is most likely injured?
 Which of the following is most likely to be found on physical exam?
 Which of the following laboratory findings will most likely be found?
 What is the most likely diagnosis (cause)?
 Which of the following diagnostic studies would best confirm the diagnosis?
 What complication is likely to occur?

10. Options should represent the continuum of
possible choices
B D C E A
Least correct Most correct

11. Use Superlatives
 Gives strength to the item
 Provide only one best answer
 Incorrect options not necessarily "wrong“ but
are less desirable

12. Options Should be Plausible and
Homogeneous
 Logical and believable
 Equally likely to be chosen by a student who does not know the
correct answer
 Parallel in form, length, amount of detail, and content
 Avoid absolutes (there are no absolutes in medicine!)
 Similar in content (diagnoses, treatments, physical findings)
 Avoid “all of the above” or “none of the above” options

13. Brain tumors can be detected on imaging:
A. Because of mass effect
B. Due to their intensity and density
differences from the brain
C. The presence of edema
D. The uptake of IV contrast by the tumor
E. All of the above
Example of a flawed question
Options are not all parallel in grammatical structure or homogeneous, all of
above should not be used, and there is no question in the stem

14. B12 deficiency may be associated with:
A
.
Loss of deep tendon reflexes
B
.
Babinski signs
C
.
Megaloblastic anemia
D
.
All of the above
E
.
A & B only
Example of a flawed question

15. When reviewing MCQs…….
 Swap with a colleague to check for:
 Gold standard test – cover the options
 Check for relevant, applied knowledge
 Are all options feasible, plausible?
 Is the answer detectable without needing to
apply knowledge? If so, too easy to guess

16. Reference
 Online Item writing tutorial is available from the National
Board of Medical Examiners at
http://download.usmle.org/IWTutorial/intro.htm