2. Selected-response item An item that requires a student to select a response from available alternatives. Examples of these are : Multiple choice True – False Matching items
4. Multiple -choice Item Multiple-choice items take the general form of a question or an incomplete statement with a set of possible answers , one which is correct. Stem – the part of the item that is either a question or an incomplete statement Alternatives – the possible answers Answers – the correct alternative Distracters – the incorrect alternatives
5. Direct-Question Format Which river is the largest in the United States of America? Mississippi < Missouri Ohio Rio Grande Most writers prefer the direct- question format because they feel it presents the problem in a clearer manner.
6. Incomplete-Sentence Format The largest river in the United States of America is the ________________. Mississippi < Missouri Ohio Rio Grande The advantage of the incomplete –sentence format is that it may present the problem in a more concise manner.
7. Best Answer Format Which variable is generally thought to be the most important when buying a house ? Cost Builder Design Location
8. Guidelines for Developing Multiple-Choice Items Use a Printed Format That Makes the Item as Clear as Possible. Few recommendations : Provide brief but clear directions. Directions should include how the selected alternative should be marked. The item stem should be numbered for easy identification, while the alternatives are indented and identified with letters. Either capital or lower case letters followed by a period or parenthesis can be used for the alternatives. If a scoring sheet is used, make the alternative letters on the scoring sheet and the test as similar as possible.
9. There is no need to capitalize the beginning of alternatives unless they begin with a proper name. When the item stem is a complete sentence, there should not be a period at the end of the alternatives ( see Example 4) When the stem is in the form of an incomplete statement with the missing phrase at the end of the sentence, alternatives should end with a period . (see Example 5) Keep the alternatives in a vertical list instead of placing them side by side because it is easier for students to scan a vertical list quickly. Use correct grammar and formal language structure in writing items. All items should be written so that the entire question appears on the page.
10. Example 4 Directions : read each question carefully and select the best answer . Circle the letter of the answer you have selected. Which type of validity study involves a substantial time interval between the test is administered and when the criterion is measured? delayed study content study factorial study predictive study <
11. Example 5 2. The type of validity study that involves a substantial time interval between when the test is administered and when the criterion is measured is a ____________________. delayed study content study factorial study predictive study <
12. Have the Item Stem Contain all the Information Necessary to Understand the Problem or Question Example 6 Poor Item – Inadequate Stem Absolute zero point. interval scale nominal scale ordinal scale ratio scale <
13. Example 7 Better Item – Adequate Stem Which scale of measurement incorporates a true or absolute zero point? interval scale nominal scale ordinal scale ratio scale <
14. Example 8 Poor Item – Unnecessary content There are several different scales of measurement used in educational settings. Which scale of measurement incorporates a true or absolute zero point? interval scale nominal scale ordinal scale ratio scale <
15. Provide between Three and Five Alternatives Certain research even suggests that items with three alternatives can be as effective as items with four or five alternatives ( e.g. Costin, 1970; Grier, 1975; Sidick, Barrett, & Doverspike, 1994)
16. Keep the Alternative Brief and Arrange Them in an Order That Promotes Efficient Scanning. Example 9 Poor Item – Inadequate Stem and Lengthy Alternatives Andrew Jackson _____________________________ was born in Virginia did not fight in the American Revolution due to a childhood illness. was the 7th president of the United States < served three terms as president of the United States
17. Example 10 Better Item – Adequate Stem and Brief Alternatives. 2. Who was the 7th president of the United States of America? Andrew Jackson < James Monroe John Adams Martin Van Buren
18. Example 11 Poor Item – Illogical Arrangement of Alternatives What year did the Spanish-American War occur ? 1912 1890 1908 1898 < 1902
19. Example 12 Better Item – Logical Arrangement of Alternatives What year did the Spanish-American War occur ? 1890 1898 < 1902 1908 1912
20. Avoid Negatively Stated stems in Most Situations Example 13 Poor Item – Negatively Stated Stem Which state does not have a coastline on the Gulf of Mexico ? Alabama Florida Tennessee < Texas
21. Example 14 Better Item – Negative Term Highlighted Which state does NOT have a coastline on the Gulf of Mexico ? Alabama Florida Tennessee < Texas
22.
23. Example 16 Better Item – Multiple True-False Item Which of the following states have a coastline on the Gulf of Mexico? Underscore the T if the state has a Gulf coastline or the F if the state does not have a Gulf coastline Alabama T F Florida T F Tennessee T F Texas T F
24.
25. Example 18 Better Item – Cues Avoided Which type of validity study involves a substantial time interval between when the test is administered and when the criterion is measured? interval study content study factorial study predictive study <
26. Example 19 Poor Item – Unequal Length and Complexity of Alternatives Ecology is the study of ______________ genesis organisms and their relationships to the environment < internal balances evolution
27. Example 20 Better Item – Alternatives similar in Length and Complexity Ecology is the study of ______________ The genetic and molecular basis of organisms Organisms and their relationship to the environment < How organisms maintain their delicate internal balance How organisms have slowly evolved over the last million years
28. Example 21 Poor Item – Alternative Contains a Visual Cue to the Correct Answer
29. Example 21 – Poor Item – Alternative Contains The correlation between two measures is 0.90. What is the coefficient of determination? 0.1 0.3 0.81 < 0.9
30. Example 22 Better Item – Cues Avoided The correlation between two measures is 0.90. What is the coefficient of determination? 0.10 0.30 0.81 < 0.99
31. Make Sure All Alternatives Are Grammatically Correct Relative to the Stem. Example 23 Poor Item – Grammatical Cue Present Which individuals are credited with making the first successful flights in a heavier-than-air aircraft that was both powered and controlled ? Octave Chanute Otto Lilienthal Samuel Langley Wilbur and Orville Wright <
32. Example 24 – Better Item – Grammatical Cue Avoided Which individuals are credited with making the first successful flights in a heavier than-air aircraft that was both powered and controlled? Octave Chanute and Sir George Cayley Otto Lilienthal and Francis Herbert Wenham Samuel Langley and Alphonse Penaud Wilbur and Orville Wright
33. Another common error is inattention to the articles a and an . See the following. Example 25 Poor Item – Grammatical Cue Present A coherent and unifying explanation for a class of phenomenon is a _______________ analysis experiment observation theory <
34. Example 26 Poor Item – Grammatical Cue avoided A coherent and unifying explanation for a class of phenomenon is a _______________ conjecture. hyphotesis. prediction. theory <
35. Example 27 Poor Item – Grammatical Cue avoided A coherent and unifying explanation for a class of phenomenon is a(n) _______________ experiment. hyphotesis. observation. theory <
36. Make Sure No Item Reveals the Answer to Another Item. Have All Distracters Appear Plausible. Use Alternative Positions in a Random Manner for the Correct Answer. Minimize the Use of “None of the Above” and Avoid Using “ All of the Above”. Avoid Artificially Inflating the Reading level. Limit the Use of Always and Never in the Alternatives.
37. Avoid Using the Exact Phrasing from the Text. Organize the Test in a Logical Manner. Give Careful Consideration to the Number of Items on Your Test. Be Flexible When Applying These Guidelines.
38. Checklist for the Development of Multiple-Choice Items Are the items clear and easy to read? Does the item clearly state the problem or question? Are there between three and five alternatives? Are the alternatives brief and arranged in an order that promotes efficient scanning? Have you avoided negatively stated items? Is there only one alternative that is correct or represents the best answer?
39. Have you checked for cues that accidentally identify the correct answer? Are all alternatives grammatically correct relative to the stem? Have you checked to make sure no item reveals the answer to another item? Do all distracters appear plausible? Did you use alternative positions in a random manner for the correct answer?
40. Did you minimize the use of “none of the above” and avoid using “all of the above”? Is the reading level appropriate? Did you limit the use of always and never in the alternatives? Did you avoid using the exact phrasing form the text? Is the test organized in a logical manner? Can the test be completed in the allotted time period?
41. Strengths of Multiple-Choice Items Multiple-Choice Items Are Versatile Multiple-Choice Items Can Be Scored in an Objective Manner Multiple-Choice Items Are Not Unduly Subject to Guessing Multiple-Choice Items Are Not Significantly Influenced by Response Sets. Multiple-Choice Items Are an Efficient Way of Sampling the Content Domain. Multiple-Choice Items Provide Information about the Type of Errors That Students Are Making
42. Weaknesses of Multiple-Choice Items Multiple-choice items are not effective for measuring all educational objectives. Multiple-choice items are not easy to write.
43. Corrections for Guessing Corrected Score = Right – Wrong / (n-1) Where right = number of items answered correctly Wrong = number of items answered incorrectly n = number of alternatives or potential answers For true-false items it is simply calculated as : Corrected Score = Right − Wrong
44. True – False Items It involve a statement or question that the student marks as true or false , agree or disagree, yes or no and so on.
45. Example 29 True – False with Directions Directions : Carefully read each of the following statements . If the statement is true, underscore the T. If the statement is false, underscore the F. T F In recent years, malaria has been eliminated worldwide. T F The ozone layer protects us from harmful ultraviolet radiation.
46. Example 30 Correct – Incorrect Item with Directions Directions : Carefully read each of the following sentences. If the sentence is grammatically correct , underscore C. If the sentence contains a grammatical error , underscore I for the incorrect. C I He set the book on the table. C I She set on the couch.
