Overall Plan Content definition Test specifications Item development Test design and assembly Test production

1. Advance
Assessment and
Evaluation
PRESENTED BY
MUHAMMAD MUNSIF
MPHIL EDUCATION
(EVENING)
MUNSIFSAIL@GM
AIL.COM

2. TEST
CONSTRUCTION
• STEPS INVOLVING IN
TEST CONSTRUCTION
• PLANNING THE
CLASSROOM TEST
• DEVELOPING TEST
SPECIFICATION
• RULES FOR
CONSTRUCTING TEST
ITEMS

3. Steps Involving in Test Construction
Effective test development/construction
requires Twelve Steps
1. Overall Plan
2. Content definition
3. Test specifications
4. Item development
5. Test design and assembly
6. Test production
7. Test administration
8. Scoring test responses
9. Passing scores
10. Reporting test results
11. Item banking
12. Test technical report

4. Step 1: Overall Plan
• What construct is to be measured?
• What score interpretations are desired?
• What test format or combination of formats (selected
response or constructed response/performance) is most
appropriate for the planned assessment?
• What test administration modality will be used (paper and
pencil or computer based)?

5. Step 2: Content Definition
•What content is to be tested?
•Content defining methods vary in rigor, depending on the
purpose of the test, the consequences of decisions made from
the resulting test scores, and the amount of defensibility
required for any decisions resulting from test scores.
•Content defining methods may be very simple and
straightforward (Lower-stakes achievement tests ).
•Content defining methods must be systematic, comprehensive,
and defensible (High-stakes achievement tests ).

6. Step 3: Test Specifications
• The test specifications or test blueprint identifies the objectives
and skills which are to be tested and the relative weight on the
subject/test given to each.
Test specifications must describe:
 The type of testing format to be used (selected response or
constructed response/performance)
 The total number of test items to be created or selected for the
test, as well as the type or format of test items

7. Step 3: Test Specifications
 The cognitive classification system to be used (e.g., modified
Bloom’s taxonomy)
 Whether or not the test items or performance prompts will contain
visual stimuli (e.g., photographs, graphs, charts)
 The expected item scoring rules (e.g., 1 point for correct, 0 points
for incorrect)
 How test scores will be interpreted (e.g., norm or criterion
referenced)
 The time limit for each item.

8. Step 4: Item Development
•Item development can proceed only when a clearly agreed
upon set of objectives is available. The form chosen depends
on decisions made in the test plan (e.g., purpose, audience,
method of administration, scoring)
•Methods used to systematically develop selected response
items.
Creating effective test items (appropriate cognitive level)
What test item formats to use for the proposed
examination

9. Step 5: Test Design and
Assembly
• Assembling is a collection of test items into a test or test form is a critical
step in test development.
• The specific method and process of assembling test items into final test
forms depends on the mode of examination delivery.
Test Assembly Rules for Essay Tests.
1. All examinees must take the same items. Do not give them a chance to
choose which items they want to answer.
2. Meaningful comparisons normally can be made only if all examinees
take the same test.

10. Step 5: Test Design and
Assembly
Test Assembly Rules for Multiple-Choice Tests.
1. Set the number of items so that at least 95 percent of the examinees
can answer all items..
2. The correct choice should appear about an equal number of times in
each response position.
3. Directions to examinees should be written on the test to indicate
whether guessing is permitted or not.

11. Step 6: Test Production
•The production, printing, or publication of examinations in test
development is often overlooked with respect to its validity
aspects.
•Test production activities and their validity implications apply
equally to performance tests and selected-response tests.
•Test production security standards and policies must be
developed, implemented.
•Printers usually can provide some type of off-press copy for
final review by test development staff.

12. Step 7: Test
Administration
•The administration of tests is the most public and visible aspect of testing.
•Major validity issues associated with test administration because much of the
standardization of testing conditions relates to the quality of test administration.
•Security is a major concern for test administration. For most examinations (e.g.,
large-scale, high-stakes tests), the entire test development process is highly
secure, with extremely limited access to test materials.
•For paper-and-pencil test administration, proctoring is one of the most critical
issues.
•For large-scale computer-based tests, proctoring is generally delegated to the
agency providing the computer test administration network.

13. Step 8: Scoring Test
Responses
•Test scoring is the process of applying a scoring key to examinee responses to the test
stimuli.
•Many fundamental validity issues associated with test scoring. The most obvious issue
relates to accuracy of scoring.
•Scoring can be extremely simple or very complex, depending on the type of test item or
stimuli.
•The responses to single-best-answer ,multiple-choice items are easily scored by
computer software, whereas responses to complex computer simulation problems can
be more challenging to score reliably.
•Final scoring of the examinee responses follows the preliminary scoring and key
validation procedures. The final answer key must be carefully proofread, and quality
controlled for absolute accuracy.

14. Step 9: Passing Scores
•Tests require some type of cut score (passing score) or
performance standard.
•The methods and procedures used to establish cut scores are a
major source of validity evidence and are fundamental part of the
test development process.
•Methods and procedures used to establish passing scores may
take place at several different stages of test development,
depending on the methods used and the overarching philosophy of
standard setting adopted by the test developers and users.

15. Step 10: Reporting Test
Results
•Score reporting is an important, often complex, step in test
development.
•Examinees have a right to an accurate, timely, meaningful, and
useful report of their test performance.
•The reporting of test scores to examinees is an extremely
important step for all types of test development projects.

16. Step 11: Item Banking
•The process of securely storing test items for potential future use is typically
referred to as item banking.
•Effective test questions are so difficult to develop and so many resources
must be used to produce useful test items that perform well, it is sensible to
store such questions, together with all their relevant performance data, to
reuse such questions on some future form of the examination.
•Item banks can be as simple as a secure file cabinet using paper copies of
examination questions, with the appropriate identifying information and test
usage data (such as item difficulty, item discrimination).

17. Step 12: Test Technical Report
•Testing program with meaningful consequences for examinees should be
systematically documented and summarized in a technical report describing all
important aspects of test development, administration, scoring, reporting, and
test analyses and evaluation.
•Test developers are often unwilling to fully document their testing programs.
•Time and effort spent in such documentation activities are rewarded by the
ease of retrieval of important validity data and by their support of research
efforts on tests.
•Technical reports preserve essential validity evidence for the historical record,
including any recommendations for future testing program improvement

18. Planning the Classroom Test
Defining Targets
Aims, Goals and Objectives
Taxonomies of Instructional Objectives
Bloom’s Taxonomy
SOLO Taxonomy

19. Defining Targets
Target specifies and unpacks
the objective and spells out what students will
be able to do during and after the lesson or
lesson series.

20. Aims, Goals and Objectives
•Aims: general statements that provide direction to
educational action.
•Goals: generalized statement about what is to be
learned.
•Objectives: focuses on educational outcomes.

21. Taxonomies of Instructional
Objectives
TAX ONOMY
Arrangements Groups

22. Bloom’s Taxonomy
•Bloom's Taxonomy was created in 1956 under the
leadership of educational psychologist Dr. Benjamin
Bloom in order to promote higher order thinking in
education, rather than just remembering facts (rote
learning). It is most often used when designing
educational, training, and learning processes.

23. Bloom’s Taxonomy (Cognitive
Domain)
•Bloom’s taxonomy is a classification system for defining and
distinguishing different levels of human cognition.
•The committee identified three domains of educational activities
or learning (Bloom, et al. 1956):
Cognitive Domain: mental skills (knowledge) (Bloom’s
Taxonomy)
Affective Domain: growth in feelings or emotional areas (attitude
or self) (Krathwohl’s Taxonomy)
Psychomotor Domain: manual or physical skills (skills) (Harrow’s
Taxonomy)

25. SOLO Taxonomy
•SOLO stands for “Structure of Observed Learning
Outcome”.
•Developed by John Biggs and Kevin Collis in 1982.
•It describes the level of increasing complexity in
student’s understanding of a subject through five
stages/levels. Not all students get through all five
stages which shows differentiation.

26. Level of SOLO Taxonomy
1. Pre-structural : incompetence.
2. Uni-structural : one relevant aspect.
3. Multi-structural : several relevant and independent
aspects.
4. Relational : integrated into a structure.
5. Extended abstract : generalized to new domain.

28. BLOOM’S TAXONOMY
LEVELS
SOLO TAXONOMY LEVELS
Knowledge Uni-structural
Comprehensive Multi-
Application structural
Analysis
Synthesis Relational
Evaluation Extended Abstract

29. Developing Test Specification

30. Developing Test Specification

31. Developing Test Specification
•The first column lists the content areas to be assessed.
•The rest of the columns are devoted to levels of the objectives
assessed by individual items according to Bloom’s categories.
•The body of the table consists of the number of each item in
every category.
•A final column lists the totals of items in each content area.
•This table can then be converted into percentages, which
some experts prefer.

32. Developing Test Specification
 Table of specifications should help you to ensure you cover the
curriculum content and have items of varying complexity.
 After designing the test, you can determine how you will interpret
the scores:
Norm-referenced scores= The performance of a student is
relative to how other students performed (e.g. percentile ranks and
standard scores)
Criterion-referenced scores= Absolute, and represent how well
the individual student has mastered the content (e.g. percent
correct and mastery/non-mastery scores)

33. Rules/Guidelines for Constructing Test
Items
 Fixed Choice items (MCQ’s & Alternative
Response).
 Supply Type (Restricted Response, Essay
Response and Extended Response)

34. Fixed Choice items (MCQ’s &
Alternative Response)
This is the most common objective-type item. Multiple-choice item is a test question
which has numerous alternative choices from which the examinee is to select the
correct answer. It is generally recommended that one use 4 or 5 choices per question,
whenever possible. Using fewer alternatives often results in items with inferior
characteristics. The item choices are typically identified on the test copy by the letters A
through E.
Stem: This is the part of the item in which the problem is stated for the examinee. It
can be a question, a set of directions or a statement with an embedded blank.
Options/Alternatives: These are the choices given for the item.
Key: This is the correct choice for the item.
Distractors: These are the incorrect choices for the item.

35. Rules/Guidelines for Fixed
Choice items
The general rules used for writing multiple-choice items are described below.
Recognize that these are general rules; not all rules will be applicable to all types of
testing.
1. The stem should contain the problem and any qualifications. The entire stem must
always precede the alternatives.
2. Each item should be as short and verbally uncomplicated as possible. Give as much
context as is necessary to answer the question, but do not include unnecessary
information.
3. Avoid negatively stated items. If you have to use this kind of item, emphasize the fact
by underlining the negative part, putting it in capital letters or using italics.
4. If one or more alternatives are partially correct, ask for the "best" answer.

36. Rules/Guidelines for Fixed
Choice items
5. Try to test a different point in each question. If creating item clones, sufficiently change
the context, vocabulary, and order of alternatives, so that students cannot recognize the
two items as clones.
6. If an omission occurs in the stem, it should appear near the end of the stem and not at the
beginning.
7. Use a logical sequence for alternatives. If two alternatives are very similar (cognitively or
visually), they should be placed next to one another to allow students to compare them
more easily.
8. Make all incorrect alternatives (i.e., distractors) plausible and attractive. It is often useful
to use popular misconceptions and frequent mistakes as distractors.
9. All alternatives should be homogeneous in content, form and grammatical structure.

37. Rules/Guidelines for Fixed
Choice items
10. Make all alternatives grammatically consistent with the stem.
11. Use 4or 5 alternatives in each item.
12. Avoid repeating words between the stem and key. It can be done, however, to make
distractors more attractive.
13. Alternatives should not overlap in meaning or be synonymous with one another.
14. Avoid terms such as "always" or "never," as they generally signal incorrect choices.
15. Do not use "none of the above" as a last option when the correct answer is simply the
best answer among the choices offered.
16. Try to avoid "all of the above" as a last option. If an examinee can eliminate any of the
other choices, this choice can be automatically eliminated as well.

38. Supply Type (Essay Response)
•Essay items are useful when examinees have to show how they arrived at an
answer.
•Writing ability is a good example of the kind of test that should be given in an
essay response format.
•It is difficult to score reliably and can require a significant amount of time to be
graded.
•Grading is often affected by the verbal fluency in the answer, handwriting,
presence or lack of spelling errors, grammar used and the subjective
judgements of the grader.
•Training of graders can require a large amount of time and needs to be
repeated at frequent intervals throughout the grading.

39. Rules/Guidelines for Supply Type
Items
The following rules may be useful in developing and grading essay questions:
1. The shorter the answer required for a given essay item, generally the better.
2. More objectives can be tested in the same period, and factors such as verbal fluency, spelling,
etc., have less of an opportunity to influence the grader. Help the examinees focus their
answers by giving them a starting sentence for their essay.
3. Make sure questions are sharply focused on a single issue.
4. Avoid humorous items. Classroom testing is very important and humorous items may cause
students to either not take the exam seriously or become confused or anxious.
5. Items should measure only the construct of interest, not one s knowledge of the item context.
6. Write items to measure what students know, not what they do not know.