validity and reliability of research methods

1. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
Research Methodology
Dr. Ayat Ismail, PhD
Urban design & planning dept.
Ain Shams University
ayat.ismail@eng.asu.edu.eg
Validity and reliability of research methods

2. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
Topics
• Qualities of measurement devices/tools
• What is Validity?
• Kinds of Validity
• What is Reliability?
• Kinds of Reliability
• The relationship between validity and reliability

3. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
Qualities of measurement devices/tools
1. Validity
Does it measure what it is supposed to measure?
2. Reliability
How representative is the measurement?
3. Objectivity
Do independent scorers agree?
4. Practicality
Is it easy to construct, administer, score and interpret?

4. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
1. Validity
• Validity refers to whether or not a test measures what it
intends to measure.
• A test with high validity has items closely linked to the
test’s intended focus.
• A test with poor validity does not measure the content
and competencies it ought to.
Examples? The ruler and the thermometer

5. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
1. Validity
Kinds of Validity
• “Content”: related to objectives and their sampling. Content
validity refers to the connections between the test items and
the subject-related tasks. The test should evaluate only the
content related to the field of study in a manner sufficiently
representative, relevant, and comprehensible.

6. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
1. Validity
Kinds of Validity
• “Construct”: referring to the theory underlying the target. It
implies using the construct (concepts, ideas, notions) in
accordance to the state of the art in the field. Construct
validity seeks agreement between updated subject-matter
theories and the specific measuring components of the test.
Example? a test of intelligence nowadays must include
measures of multiple intelligences, rather than just logical-
mathematical and linguistic ability measures.

7. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
1. Validity
Kinds of Validity
• “Criterion”: related to concrete criteria in the real world.
Also referred to as instrumental validity, it is used to
demonstrate the accuracy of a measure or procedure by
comparing it with another process or method which has
been demonstrated to be valid.
Example? imagine a hands-on driving test has been proved to
be an accurate test of driving skills. How does written test can
be validated in comparison to hand-on driving test?
It can be concurrent or predictive.

8. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
1. Validity
Kinds of Validity
• “Criterion”: It can be concurrent or predictive.
 “Concurrent”: correlating high with another measure
already validated. Concurrent validity uses statistical
methods of correlation to other measures.
 “Predictive”: Capable of anticipating some later measure.
Predictive validity considers the question, "How well does
the test predict examinees' future

9. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
1. Validity
Kinds of Validity
• “Face”: related to the test overall appearance. Face validity is
determined by a review of the items and not through the use
of statistical analyses. Unlike content validity, face validity is
not investigated through formal procedures. Instead, anyone
who looks over the test, including examinees, may develop
an informal opinion as to whether or not the test is
measuring what it is supposed to measure.

10. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
2. Reliability
• Reliability is the extent to which an experiment, test, or
any measuring procedure shows the same result on
repeated trials.

11. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
2. Reliability
Kinds of Validity
• “Equivalency”: related to the co-occurrence of two items.
Equivalency reliability is the extent to which two items
measure identical concepts at an identical level of
difficulty. Equivalency reliability is determined by relating
two sets of test scores to one another to highlight the
degree of relationship or association

12. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
2. Reliability
Kinds of Validity
• “Stability”: related to time consistency. (sometimes called
test, re-test reliability) is the agreement of measuring
instruments over time. To determine stability, a measure
or test is repeated on the same subjects at a future date.
Results are compared and correlated with the initial test
to give a measure of stability.
Example? Instruments with a high stability reliability are
thermometers, compasses, measuring cups, etc.

13. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
2. Reliability
Kinds of Validity
• “Internal”: related to the instruments. Internal
consistency is the extent to which tests or procedures
assess the same characteristic, skill or quality. It is a
measure of the precision between the measuring
instruments used in a study. This type of reliability often
helps researchers interpret data and predict the value of
scores and the limits of the relationship among variables

14. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
2. Reliability
Kinds of Validity
• “Interrater”: related to the examiners’ criterion.
Interraterreliability is the extent to which two or more
individuals (coders or raters) agree.
Example? when two or more teachers use a rating scale with
which they are rating the students’ oral responses in an
interview (1 being most negative, 5 being most positive). If
one researcher gives a "1" to a student response, while
another researcher gives a "5," obviously the interrater
reliability would be inconsistent

15. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
Sources of error
• Examinee (is a human being)
• Examiner (is a human being)
• Examination (is designed by and for human beings)

16. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
Relationship between validity & reliability
• Validity and reliability are closely related
• A test cannot be considered valid unless the measurements
resulting from it are reliable.
• Likewise, results from a test can be reliable and not
necessarily valid.

17. MSc
Integrated urbanism
& Sustainable Design
(IUSD)
Thank You