Inter item reliability with surveys

1. Inter-item Reliability
1

2. Inter-item Reliability
With inter-item reliability or consistency we are
trying to determine the degree to which
responses to the items follow consistent
patterns.
2

3. For example
Let’s imagine that two items are designed to assess
how “teachable” someone is. Item 1 is a difficult item
for someone to agree with. So on a scale of 1 – 5 with
1 being “Strongly Disagree” and 5 being “Strongly
Agree” here is what the response pattern might look
like:
3

4. For example
4
Respondent Item 1
Bob 1
Carla 2
Don 1
Elsa 3
Fiona 1
Gordon 2
Etc.

5. For example
Now take a look at item 2. It is much easier to
agree with:
5
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.

6. For example
6
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
Notice the
consistency
in the item
response
patterns

7. For example
7
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
Bob & Fiona
are less likely
to agree with
these items
targeting
teachableness.

8. For example
8
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
But even
though they
are both less
likely to agree
they
consistently
agree more
with item 2
than 1.

9. For example
9
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
This shows a
consistent
pattern across
items.

10. For example
10
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
Notice that this
pattern holds
across all of
the other
respondents.
Item 2 is easier
to agree with
than Item 1

11. For example
11
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
Notice also
that Elsa is the
most agreeable
across both
items.

12. For example
12
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
And Bob &
Fiona are
consistently
the least
agreeable
across both
items.

13. For example
13
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
The item responses to this instrument,
therefore, demonstrate high inter-item
reliability or consistency across persons
with different levels of the trait.

14. For example
14
Respondent Item 1 Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
This type of reliability is measured
principally by a statistic called
Cronbach's Alpha and ranges
traditionally from 0.00 (no consistency)
to +1.00 (completely consistent).