2. Test Bias,
Essay Test,
more...
Accuracy
Accuracy in writing involves the process of editing and revising writing samples. One important
element of working toward accuracy in writing includes oral reading-response exercises, where
students read their writing out loud to the class and receive constructive feedback. This
collaborative exercise is considered more helpful to students that written comments because it
offers personal contact.
Glossary of Education
Home > Glossary > Reliability
Reliability
Capacity of an assessment method to perform in a consistent, stable fashion during successive
uses. Reliability is a prerequisite for validity. An unreliable indicator cannot produce trustworthy
results. Also, the extent to which the same result is achieved when a measure is repeatedly
applied to the same group.
Related terms:
Data,
Accuracy,
Construct Validity,
Test Construction,
Measurement Techniques,
Correlation
Not what you're looking for?
Validity
Validity is the extent to which a test appropriately measures the intended testing area. For
example, a valid math test is focused on the numbers and concepts without distracting students
with difficult or unfamiliar language or ideas.
Related terms:
Evaluation,
Achievement Test,
Competency Test,
Reliability,
Accountability,
Correlation,
3. CAA Centre >>Resources>>Objective Test Design
What are Objective Test Questions?
Objective tests require a user to choose or provide a response to
a question whose correct answer is predetermined. Such a
question might require a student to :
select a solution from a set of choices (MCQ, true-false,
matching)
identify an object or position (graphical hotspot) or
supply brief numeric or text responses (text input)
Because the correct answers to objective test questions are pre-
determined, they are well suited to the many forms of CAA that
involve automated marking. The electronic marking of the
responses is completely non-subjective as no judgement has to
be made on the correctness or otherwise of an answer at the
time of marking. However, it is worth noting that in terms of
in-built bias, an objective test is only as objective as the test's
designer makes it. The tutorial below offers an introduction to a
selection of question types in popular use with advice on
construction and best practice. As the question type most
commonly associated with CAA is multiple choice, particular
emphasis has been given to this and should be viewed first. The
principles detailed within that section should however, be of
use when considering other types.
1. Multiple Choice Items
o Anatomy of a question
o Writing Stems
o Writing Distracters
2. Extending Multiple Choice Items
3. Assertion-Reason
4. 4. Multiple Response
5. Matching
6. True/False
7. Text/Numerical
8. Ranking
9. Graphical hotspot
10. Sore Finger
11. Field Simulation
12. Sequencing
13. References
Objectivity in testing is essential. Studies have shown that test
results can be directly impacted by a Tester's failure to be
objective:
The task involved administering the Wechsler Intelligence
Scale for Children to twelve youngsters. Each child was tested
by two experimenters, one of whom administered the odd-
numbered questions, the other the even-numbered questions.
With each child, one of the experimenters was told beforehand
that the subject was above average in intelligence. The other
was told that the subject was below average.
The results? The youngsters scored an average of seven and a
half points higher on the half of the test administered by the
experimenter who had been told they were bright.
[Rosenthal, R. and Rosnow, R. L. Artifact in Behavioral
Research. New York: Academic Press, 1969. (Emphasis
added.)]
It is widely accepted within the Quality Assurance industry that
in order for the Quality Assurance process to be effective,
Testers must anticipate that the code will fail. A Tester should
ask, "How can I make this code fail?" instead of asking, "How
can I verify this fix?"
Testing that occurs without proper objectivity may ultimately
demonstrate that the Development Team was successful in
creating code with no errors, instead of finding the errors that
unquestionably exist. Testing that occurs without proper
objectivity will most likely result in the identification of fewer
defects.
5. Computer Science Objective Tests
October 27, 1999
Version: Provisional
What are objective tests?
Objective tests as we use them are multiple choice: questions are presented along with a number of options, or
choices, for you to choose from. Only one of these options is the correct answer.
There is no subjective judgement exercised by the examiner when assessing the mark given for the answers in an
objective test. The answers are either right or
wrong. There are several formats for objective tests but you will be given only tests of the multiple choice type.
Contrast this with traditional subjective tests. Typically subjective tests are composed of several multipart
questions, a selection of which must be attempted by
you, the test candidate. These tests usually require you to do a fair amount of writing.
There are several advantages and disadvantages to subjective tests:
Disadvantages
Dependence on presentation.
If you have bad handwriting the examiner may be unable/unwilling to struggle in order to
understand your answer.
Conversely, you may present a well laid out and planned answer that contains a lot of "waffle" or
hazy knowledge, yet receives a good mark.
Facility in writing.
"Those who write fluently, speedily and whose flow of thought comes easily, have an
advantage over those who think more slowly (but possibly more
deeply) and those who write with less facility (but possibly with better quality content)."
p.29, "Objective Testing in Education and Training", W. Bonney Rust
Question evasion.
If there are several questions in a subjective test, only some of which must be answered,
then it is possible for the candidate to avoid questions in areas of
the curriculum in which they are weak.
6. It also follows that as candidates are free to choose a subset of the test questions, it
becomes difficult to compare candidates as they are effectively taking different tests.
Question Spotting.
A good strategy for preparing to take a subjective test is to look at past papers. Candidates soon
spot recurring questions on specific topic areas so that it
becomes possible to only revise some of the syllabus and still obtain a respectable grade.
Variation in marking.
This is possibly the greatest disadvantage to a subjective test: inconsistency in subjective
assessment.
Research* has shown that experienced examiners award widely varying marks not only on the
same piece of work marked by other examiners, but on their
own marked scripts re-marked after a passage of time.
* "An Examination of Examinations", Hartog and Rhodes
Advantages
Unlike objective tests, subjective tests can demonstrate the candidates ability to marshall
material and present it in a logical order.
Subjective tests allow a candidate to express originality of thought.
Subjective tests allow the candidate to demonstrate their ability to develop an argument.
In addition, subjective tests allow the examiner to assess the candidate's quality of written
expression.
Summary
One the criticisms made of objective tests is is the assertion that they simply test your ability in factual, or memory
recall. However you will be asked to answer
multiple choice questions covering all the material in the syllabus and covering the whole range of intellectual
skills. To know more about this you should consult
Bloom's Taxonomy.
Objective tests have many more individual questions than a subjective test. This means that the whole of the
course syllabus will be covered and you, and we, will
7. know if you or the class as a whole are having any difficulties in any particular topics covered by the course.
Objective Testing: Bloom's taxonomy
In his book, Taxonomy of Educational Objectives Professor Bloom derived a list of intellectual skills that
could be assessed by carefully constructed questions.
Knowledge
Recall the facts and concepts.
Comprehension
Understand what the facts and concepts mean.
Application
Apply the understanding of facts and concepts in a given situation.
Analysis
Extract from a context the facts you need to know.
Synthesis
Combine facts and concepts you understand to achieve a specified goal.
Evaluation
Assess a situation where knowledge is partial and ambiquous.