2. Naturalistic Evaluation
An approach to assessment that evolved from
the work of researchers at the Indiana Center
for Evaluation
Wolf
Tymitz
Guba
Lincoln
Has been used in numerous applications
including schools, social programs, museums,
and health care
4. Two Streams
Naturalistic Evaluation has evolved from two parallel
streams
‘Responsive evaluation’ from the evaluation field
Naturalistic methodologies from the area of qualitative
‘inquiry’ used by anthropologists and sociologists
practicing ‘ethnography’
Combining the two makes Naturalistic Evaluation an
intensive endeavor
6. Purpose
Generally used to discover what is wrong
(generally) and how to fix it (in a manner
that will last), or how something is working
(generally) and why (so you know how to
keep doing it, or for possible use by others)
This is more rigorous than finding out what
is wrong (according to the audience) and
deciding what to do (according to the
audience)
8. Major Role of Evaluation
Guba and Lincoln believe it is to respond to audience
requirements for information in ways that take account
of the different value perspectives of its members
Naturalistic Evaluation puts the evaluator in the role of
learner – the informants teach
However, criteria for scientific rigor still apply!
9. General Phases
Familiarization phase
Gaining entry
Understanding the Circumstance and the players
Three C’s
Collection of Data
Classification and Analysis of Data
Confirmation of Propositions
Synthesis
Presentation of findings and debriefing
10. Naturalistic Evaluation
Pros
When done correctly, it can provide a very complete
picture of a program’s context, stakeholders,
operations, effects (qualitative and quantitative),
and systems of relationship
A participant-oriented ‘gold standard’
Cons
When done correctly, can
Take a prohibitively long time
Cost a lot
Be very labor intensive
When done incorrectly, can give a false sense of
knowledge
12. Background of Naturalistic
Evaluation Approach
O The Naturalistic approach gives the
evaluator freedom to choose the methods
used to collect, analyze, and interpret their
data.
14. Responsive Evaluation
O Stake (175) is one of the
major developers of the
naturalistic evaluation
approaches.
O Stake was concerned that
conventional approaches
were not sufficiently receptive
to the needs of the evaluation
client.
O Stake believed that
evaluators should use
whatever data-gathering
schemes seem appropriate
relying heavily on human
observers and judges.
15. Robert Stake
O First evaluation theorist to introduce Participant
Oriented Evaluation into the field of education.
O Introduced the theory of Responsive Evaluation
which focuses on “re-directing data gathering and
interpretative efforts around emerging issues of
importance to program practitioners in the
evaluation setting” (Abma, 2005)
O Created the Countenance Framework in 1967. The
model refers to the two faces of evaluation:
description & judgment.
O Description includes the evaluator’s observation
and list of benchmarks for the activities being
evaluated. Judgment is the evaluator’s overall
rating of merit.