Summary of course on Curriculum Evaluation

1. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
1
MPF 1483: CURRICULUM
EVALUATION
Prepared by
DR. SHAFEEQ HUSSAIN
VAZHATHODI AL-HUDAWI (2013),
Senior Lecturer, Faculty of Education,
UTM Johor, MALAYSIA
I. DEFINITIONS
Various definitions:
1. Tyler (1950): the process of
determining to what extend
educational objectives are being
attained
2. Borg & Gall (1983): the process of
making judgment about the merit,
value or worth of educational
programmes, projects, materials
and techniques
3. Smith & Glass (1987): the process
of establishing value judgments
based on evidence abt a program /
product
4. Stufflebeam et al. (1971): the
process of delineating, obtaining &
providing useful info for judging
decision alternatives
5. Provus (1971): the comparison of
performance to some standards to
determine whether discrepancies
existed
6. The process of collecting data on a
programme to determine its value
or worth with the aim of deciding
whether to adopt, reject, or revise
the programme.
7. Joint Committee on Standards
for Educational Evaluation
(2003) process of systematic and
focussed determination of a
curriculum’s value. “Evaluation is
the systematic assessment of the
worth or merit of an object”
(1994, p. 3)
II. A BRIEF SKETCH ON
EVALUATION
-Evaluation is closely associated with
assessing achievement of students
against behavioural objectives or
conducting norm-reference testing.
-From 1970, emphasis was given to
professional judgment, i.e., collection
and analysis of quality of information
for judging and deciding on a
curriculum.
In short, Evaluation is an integral part
of the curriculum development
process
III. TYPES OF EVALUATION
Acccording to Scriven (1967), there are
two types of evaluation in curriculum:
1) Formative evaluation
– an on-going program
- it is a program improvement
– provide data about educational
program
 to assist developer in improving the
program
For example, results of formative
evaluation may help in
1. Selection of programme
components
2. Modification of programme
elements
2) Summative evaluation
– it is done at the completion of a
program
- it concerned with overall
effectiveness of the program
- it provides data to determine the
merits and demerits of the programme
For example, results of formative
evaluation may help in
1. Determining what were the
effects of the curriculum, and
evaluate them whether or not,
they were intended
2. Noticing whether something
was overlooked by that
everyone else

2. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
2
IV. GOALS OF EVALUATION
IN CURRICULUM
DEVELOPMENT
• Enables curriculum makers
reviewing and modifying – to
cater the current and future needs
• Curriculum should be continuously
reviewed & reviewed
WHY??
• To MAINTAIN, MODIFY
- Quality
- Relevancy
- Adequacy
• OR ELIMINATE
QUESTIONS SHOULD BE ASKED:
1) Does the program (curriculum)
meet existing or expected
needs?
2) Does the program contain
extraneous and outdated
materials?
3) Are the students able to perform
adequately once they finish
their study?
Wentling (1980): EVALUATION
MUST DO MORE THAN :
• just analyzing the extent to
which a program had adhered to
an original plan
• OR attained its primary goals
and objectives
Therefore, curriculum evaluation needs
to go beyond the assessment of student
behavior (thus, Tyler’s definition of CE
is inadequate)
• It should include the overall
effect on students, teachers and
society.
BECAUSE
Curriculum is inclusive of
• INTENTS (Aims, Goals and
Objectives)
• CONTENTS (Various
subject-matter contents
included, their Scope and
sequence, balance;
Teaching-learning materials,
resources and experiences,
i.e, syllabus, content outline,
textbooks)
• PROCESS(Pedagogy,
Learning strategies)
• PRODUCT or OUTCOME
• Curriculum is the link between
society and school
• Through Curriculum we meet
Educational goal of the nation
• The task of evaluating the
curriculum involves a
COMPLEX PROCESS
• Hence, CE is about evaluating
EVERYTHING about
curriculum.
CE should have strategies to
determine whether decision about
all the elements of curriculum
mentioned above is optimal
V. THE WORKING
DEFINITION OF
‘CURRICULUM
EVALUATION’
Evaluation is the provision of
information at…
• A: STAGES of determining of
aims, planning, tryout, field
trail, implementation and
quality control of program
development, concerning…
• B: ENTITY of Teachers’
guide, -study material, -
instructional tools, -the whole
package, from the point of view
of…
• C: CRITERIA of -Fit to
standards, -eliciting processes,
-yielding outcomes
on the basis of
• D: DATA such as -judgment, -
observation, -examination of
product
summarized in
• E: MODE OF SUMMARY of:
Quantitative, Qualitative,
Mixed Methods

3. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
3
for sake of making decision
about
• F: ROLE: -modifying, -
revising the use of the
program.
VI. MODELS OF
‘CURRICULUM
EVALUATION’
CIPP model aims to evaluate how
CORE VALUES are put in
1. GOALS (Context Evaluation)
2. PLANS (Input Evaluation)
3. ACTIONS (Process Evaluation)
4. OUTCOME (Product
Evaluation)
See the FIGURE in the appendix
DETAILS ON
CIPP MODEL (Finch & Bjorquist,
1977)
1) Context evaluation
 Whether or not to offer a
curriculum
 If so, what its parameters will be –
including goals and objectives
 It includes- environment, students’
b/ground, school climate, goals and
objective of the curriculum
 It involves analysis of goals and
needs in a specific education setting
 Goals should be reviewed and
revised periodically in relation to:
1) the reasons for offering the
program
2) the intended target group
3) the content
As curriculum developers:
 Establish conducive learning
environment to promote a positive
learning climate
 Monitor current trends and issues in
education
 Responsive to latest innovation and
development in education
EXAMPLE:
Context: Examine the environment
in which technology is used in
teaching and learning
2) Input evaluation
• Deciding what resources and
strategies that will be used to
achieve curriculum goals and
objectives
• This includes – the quality of the
curriculum and syllabus, students,
teachers, staff, facilities and
infrastructure
EXAMPLE:
Input: Examine what resources are
put into technology integration
3) Process evaluation
• Focuses on decisions associated
with curriculum effects on students
• Whether the content is learned by
students
• Involves the data collection
• Data is collected over a period of
time to detect strengths and
weaknesses of the program
• Process evaluation includes:
teachers’ methods of teaching,
students’ performance /
achievement, the facilities used
• Conduct professional development
courses to upgrade (teaching) skills
of teachers
• Review students assessment
EXAMPLE:
Process: Did the technology
integration work smoothly?
4) Product evaluation
• To determine the extent to which
the goals of the programs have been
achieved
• It is also involves data collection 
through feedback from former

4. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
4
students or graduates, employer and
society
• Data collected - to make decisions
about the program
• Whether to retain or modify the
existence program
• It concerns on the number and
quality of students produced
• Can get from teachers and students
• Info – valuable in determining
adequacy and appropriateness of
the curriculum
EXAMPLE
Product: Did the learners learn using
technology? How do you know?
MODEL TWO: CONGRUENCE-
CONTINGENCY MODEL
by Robert Stakes
Here evaluation is done through
Formal means of collecting for
extensive data by
• Observing dynamics among
people involved in curriculum
process
• Allowing them greater
participation in judging
programs
• Taking position regarding a
program’s worth
By considering
1. Antecedents: Conditions prior to
teaching and learning that may
influence outcomes
2. Transactions: The Process of
interactions among
 Students and teachers
 Students and Students
 Students and curriculum
materials
which are affected by time factors,
space arrangements,
communication flow, and teaching
process
3. Outcomes: the Immediate and
long-range program results of
 Student achievement,
 teachers’ perception of their
competence and their
influence on administrative
policies
Here we look at the CONSIDERED
and ACTUAL Antecedents,
Transactions and Outcomes.
SEE the diagram in the Appendix
VII. APPROACHES TO
CURRICULUM
EVALUATION
• Approaches to curriculum
evaluation mean the beliefs and
assumptions (often termed
'philosophies' or 'perspectives') of
designers of the curriculum on
‘what should be evaluated’?
• The important approaches are:
• Traditional: What are the
important aspects of our cultural
heritage that should be
preserved,
• What are the important aspects
of our cultural heritage that
should be preserved i.e.,
the students can be manipulated
in order to bring about desired
results’. THEREFORE, ‘the
need for efficiency as seen in
the manufacturing sector’
(Hopkins, 1994)
• Experiential: What experience
will lead to the healthy growth
of the individual?,
• What experience will lead to the
healthy growth of the
individual? i.e.,
learning occurred in a cycle of
a) experience
b) reflection
c) action
• Structure of Discipline: What is
the structure of the disciplines of
knowledge?
• What is the structure of the
disciplines of knowledge?, i.e.,
in order for members of society
to function well in the wake of

5. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
5
the global economy, there is a
need to acquire high levels of
literacy and numeric in subject
areas
• Behavioural: At the completion of
the curriculum, what should the
learner be able to do?
• What is the structure of the
disciplines of knowledge?, i.e.,
learning occurs when behavior
indicates that a given stimulus
was effective in producing the
desired outcome
• Constructivist: How can people
learn to make sense of world and to
think more productively and
creatively?
• How can people learn to make
sense of world and to think
more productively and
creatively?, i.e.,
learning takes place in such a
manner that the recipients of
information build up on those
skills and knowledge that they
receive from the environment’.
THEREFORE, ‘most of the
time, there is a need to teach
ideas in holistic manner’
VIII. ECLECTIC APPROACH TO
CE
Utilizing various scientific, humanistic
or scientific-humanistic approaches
simultaneously in order to overcome
the focus of CE model on
Some particular feature of
evaluation
Some unique functions / roles
For they prescribe
Specific patterns for evaluation
activities
Hence, they are not ALTERNATIVES,
but COMPLEMENTARY to each
other, and therefore ECLECTIC
approach, so as to
 Analyze various political, social
and moral reals as to their
significance in Curriculum
IX. IMPORTANT CONCEPTS,
TO DERIVING GENERAL
AIMS AND SPECIFIC
MAJOR OBJECTIVES
a. Critical changes in Society
a. Most Important changes,
their implications on
objectives
b. Nature of changes:
economic, social, health,
development
c. Employment Patterns of
the time
d. Requirement in Health,
Welfare, Political and
Social Affairs
e. Students’ concerns of
learning needs
f. Development in the
content of the Subject-
matter
b. Relevance and Balance
c. The Language of Instruction
d. New Forms of Education
e. Minimum Learning
Requirements
• A comprehensive evaluation
framework or model is
necessary to achieve a
systematic, effective and
efficient evaluation
X. SUMMARY ON SIX
STAGES OF CURRICULUM
EVALATION
See the APPENDIX
XI. STAGE ONE: Determination
of General Educational Aims
Curriculum evaluation is NOT’ just
determining whether educational
objectives are achieved, but rather it
is about determining the worth and
merits of objectives themselves

6. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
6
XII. EVALUTION CRITERIA &
PROCEDURE at the
PLANNING STAGE
At the PLANNING STAGE, we look
how well we have selected,
conceptualized and formulated
1A. Instructional Objectives
1B. Scope and Sequence of the content
1C. Teaching Learning Strategies
1D.Instructional Materials
1Aa. Instructional Objectives:
CRITERIA: Are they
• RELATED to the OBJECTIVE OF
THE PROGRAM
• CLEARLY STATED
• APPROPRIATE FOR, AND
ATTAINABLE by TARGET
LEARNERS (based on level and
mental development).
• IMPORTANT ENOUGH to
encourage further learning
1Ab. Instructional Objectives:
PROCEDURE (How):
• Seeking and Analyzing relevant
opinions and judgments by
panel of experts (Curriculum
specialists, Psychologists, etc).
1Ba: Evaluation of the Scope &
Sequence of the Content of
Instruction: CRITERIA
• Relevance to Instructional
Objectives
• Up-to-datedness of Content
• Relevance to the child and his
environment
• Content Balance
• Balance in the content in
terms of students’ activities
envisaged.
• Balance between various
subdivision of the subject
matter
• Organizational Structure of the
content
1Bb: Evaluation of the Scope &
Sequence of The Content of
Instruction: PROCEDURE (How)
• Seeking and Analyzing relevant
opinions and judgments by
panel of
• Subject Specialist
• Educational
Psychologists
• Teacher-Educators
• Experienced Teachers
1Ca. Teaching-Learning Strategies:
CRITERIA:
• Effectiveness & Economy
• Feasibility in Classroom
• Teacher Competence
1Ca. Teaching-Learning Strategies:
PROCEDURE (How):
 In-House Analysis
 Experts’ Panel
 Simulation at Center
 Limited Tryout in Schools
1Da. Instructional Materials:
CRITERIA:
• INSTRUCTIONAL
CHARACTERISTICS: Contents
are
academically
• related to instructional
objectives
• valid and reliable
• logically Organized
psychologically
• relevant
• appropriately Sequenced
• Hierarchically structured
and sequence
• Reinforcing
• Promoting Understanding
and Critical Thinking
Linguistically
• Accuracy and suitability
• PRACTICAL
CHARACTERISTICS

7. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
7
• Cost Range
• Flexibility to adapt the
programs to TL condition
• Easiness to use
1Da. Instructional Materials
Strategies: PROCEDURE (How):
• Seeking and Analyzing relevant
opinions and judgments by
panel of
• Subject Specialist
• Educational
Psychologists
• Teacher-Educators
• Experienced Teachers
• Production Experts
These FOUR elements of curriculum,
i.e., 1A, 1B, 1C, 1D should be analyzed
based on the CRITERIA (1Aa,1Ba…),
following the PROCEDURE (1Ab,
1Bb,….).
XIII. 5 QESTIONS OF VALUES
IN CURRICULUM
EVALUATION
These ‘5 Q’s’ address
‘What ‘evaluation’ can do?’. The
5Q’s are The Questions of:
a. INTRINSIC VALUE
This related to:
i. Goodness, up-to-datedness &
Appropriateness of ‘planned’ and
‘enacted curriculum’ at philosophical
and psychological, sociological and
subject-areas levels, and
ii. Their Congruence with
NATIONAL EDUCATIONAL
PHILOSOPHY & NATIONAL
EDUCATIONAL CONCERNS
How to do this (Procedure):
i. Getting ‘specialists’ and
‘experts’ to analyze the
curriculum document
b. INSTRUMENTAL VALUE
decides the ROLE of Evaluation at the
sociological or pedagogical context, i.e.
What is the CURRICULUM Good
for?, i.e.,
Will what is planned in the curriculum
be attained?
• Objective of the program:
‘planned curriculum’ versus
‘enacted curriculum’
To what extent?
• With the suggested ‘contents,
materials, methods’ can the
‘philosophical and
psychological’ orientations be
achieved
By which students?
c. COMPARATIVE VALUE
Is the NEW program Better than the
OLD one in terms of
• Skill training
• Students advancement
• Delivery and other costs
d. IDEALIZATION VALUE
How to have the BEST possible
CURRICULUM? in order to
HIGHTEN Students achievement, for
optimal benefits.
This is a continual concern to fine-tune
curriculum entities: Alternatives: to
involve students fully into learning?
f. DECISION VALUE
Making QUALITY decision on
whether to RETAIN, MODIFY OR
DISCARD the program based on the
above FOUR considerations AND
evidences and
Here it is important to consider
‘WHAT IS THE VALUE OF THESE
DECISIONS?
XIV. NORM & CRITERION-
BASED EVALUATIONS
Norm-Referenced Evaluation
 Evaluating students’
performance relative to the
performance of other students
• The performance of current
students or of previous students

8. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
8
can be compared, according to
standards of group, where
students are grouped for
particular courses
Criterion-Referenced Evaluation
 Measuring students’ actual
performance and comparing it
with the objectives of
instruction identified in the
syllabus, or
 on a specific criterion (e.g.,
Evaluation of mastery learning
schemes)
XV. EVLAUTION OF THE
EFFECTIVENESS OF
TEACHER TRAINING
At the LARGE SCALE
IMPLEMENTATION, it is necessary
to provide INTRODUCTORY AND
SUPPORTING COURSES
(TEACHER TRAINING)
a. Training related to activities
and background information
b. Training related to program
objectives
c. Training related to teachers’
role and classroom
management
d. Training related to criteria
for monitoring students’
achievement.
Furthermore, the EVALUATION OF
THE EFFECTIVENESS OF SUCH
TEACHER TRAINING COURSES are
very significant.
CRUCIAL influence of teacher in
effectiveness of implementation is
acknowledged, but NOT REFECTED
in evaluating the effectiveness of such
courses. Such course can be of various
types, namely:
 Concentrated workshop type
 Preparatory course spread over
weeks
 Extended courses with
specialists
 Action-research
 Development-Cluster groups
All these types of course need to be
evaluated for their effectiveness in
terms of
 Teachers’ knowledge of
o the subject matter
o activities
o aims and objectives
o acceptance of classroom
activities with are
congruent to aims
 changes in Teacher’s attitude to
the new program course and
students behavior
HOW that can be done:
 Interviews
 Checklist
 Questionnaires
 Follow-up observations
 Informal discussion
 Involving Teachers in devising
new methods, than they receive
information
XVI. THE NEED FOR QUALITY
CONTROL OF THE
IMPLEMENTED
PROGRAM
At the large-scale implementation
stage, curriculum tends to
DETERIORATE, by being less
effective.
Loss of effectiveness of particular
 curricula,
 methods or procedures
to
 entire student population,
 some schools,
 subgroup of students.
By decrease in
 Achievement Level
 student interest
 in higher mental process
(application, analysis, synthesis)
Therefore, the EVALUATOR should
CONTROL the QUALITY in terms of
1. Need
2. Cause
3. Corrective measures
By seeing

9. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
9
1. Problem
2. Process
3. Data
STUDY the Appendix on Schemata of
Quality Control
XVII. CONTINUITY IN THE
PROCESS OF QUALITY
CONTROL OF THE
IMPLEMENTED
CURRICULUM
Evaluation itself is a ‘quality control’
of
 Curriculum
 Instructional methods
 Procedures
That is, it is a continuous process of
 Collecting information on
problem, process,
 Taking CORRECTIVE
measures.
It should be CONTINUED, because
EDUCATION is SIGNIFICANT
public enterprise, involving
MILLIONS of children.
 Quality control require effort,
time & money but vital to detect
problem, Hence we need a
curriculum center to undertake
quality control,
CONTINUALLY and
SYSTEMATICALY to make
curriculum
 RELEVANT
 RESPOSIVE to the needs of
community and nation

10. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
10
APPENDIX: Different stages of program development and utilisation: different methods, strategies, personS
Determination of
General Aims
Decision about: general aims, school
structure
Studies on:
Expected changes, Cultural Values, Social forces, Present
level of achievement, feasibility of programs
Planning Writing Outline: Preparing
instructional material
Examining adequacy of objective, contents, strategies
Judgement of material
Tryout Monitoring teaching in tryout
classes, Modifying material
Collect evidence through observation, judgement,
discussion with teachers, students
Student products
Field-Trail Slightly modify program ,
Determine optimal conditions of
program use
Collect evidence about the efficiency of program under
various conditions
Implementation Links with supervisors, examination
system, teacher training
Examining final form
Evidence on efficiency of teacher training
Quality Control Implement recommendation, plan
‘second generation’ programs
Examining quality of implementation
Studying reason for changes in efficiency
Suggesting remedies if needed

11. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
11
APPENDIX: Schemata on Quality Control
Need for Quality Control Cause of deterioration Corrective measures and
effectiveness
Problem Does implemented curriculum remain
effective
Why and how deterioration originated -Appropriate quality control measures
-Is particular measure effective
Process Compare student achievement data in
the current term with those in the
previous terms or years
Survey how curriculum is
implemented, under what conditions,
to what students groups
-Analyze and compare a series of
formative tests
-derive hypothesis which explain why
the effectiveness of the curriculum
decreasing
-Appropriate quality control measures
-Is particular measure effective under
small-scale experimental situations
-apply the verified quality control
measures to the target population
Data -Summative achievement programs or
summative achievement tests
administered every year
-end of course examinations
-standardized achievement tests
-expert and teacher judgment
-attitude and interest survey
-Questionnaires
-Other unobtrusive measures
-formative tests
-school survey data
- Interview and questionnaires
-expert and teacher judgment
-classroom observation
-formative and summative tests
-attitude and interest survey
-Other unobtrusive measures

12. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
12

13. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
13
In CE, the worth could be gauged only after assessment of the graduates needs of a program. And needs of the following types
Concept Definition Example
Need Something that is
necessary or useful for
fulfilling a defensible
purpose
Competent, effect
instruction in the basic
skill areas
Defensible purpose A desired end that has
been legitimated
consistent with a guiding
philosophy, set of
professional standards,
institutional mission,
mandated curriculum,
national constitution or
public policies, etc
Students’ development of
basic academic skills
Treatment need A certain service,
competent service
provider, or other helping
agent
Competent instructors in
twelfth-grade courses in
math, science and
language arts
Outcome need Achievement or outcome
required to meet a
defensible purpose
Students’ demonstration of
proficiency in specified
areas, such as twelfth-
grade math, science, and
language arts
Assessment A systematic assessment
of the extent to which
treatment or outcome
needs are being met
Examination of students’
scores on national tests
and evaluation of the
involved teachers

14. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL!
14
APPENDIX: Robert Stakes’s Congruence and contegency model
Curriculum Development Curriculum Evaluation
Considered antecedents Actual antecedents involved
Considered Transactions Evaluation of Transactions
Identified Outcomes Attained Outcomes
Empirical contingency
Empirical contingency
Logical contingency
Logical contingency
Congruence
Congruence
Congruence