Current Trends

1. Introduction toIntroduction to
AssessmentAssessment
ESL Materials and TestingESL Materials and Testing
Week 8Week 8

2. What is assessment?What is assessment?
 Not the same as testing!Not the same as testing!
 An ongoing process to ensure that theAn ongoing process to ensure that the
course/class objectives and goals are met.course/class objectives and goals are met.
 A process, not a product.A process, not a product.
 A test is a form of assessment. (Brown, 2004,A test is a form of assessment. (Brown, 2004,
p. 5)p. 5)

3. Informal and Formal AssessmentInformal and Formal Assessment
 Informal assessment can take a number ofInformal assessment can take a number of
forms:forms:
 unplanned comments, verbal feedback to students,unplanned comments, verbal feedback to students,
observing students perform a task or work in smallobserving students perform a task or work in small
groups, and so on.groups, and so on.
 Formal assessment are exercises or proceduresFormal assessment are exercises or procedures
which are:which are:
 systematicsystematic
 give students and teachers angive students and teachers an appraisalappraisal of students’of students’
achievement such as tests.achievement such as tests.

4. Traditional AssessmentTraditional Assessment
 Multiple-choiceMultiple-choice
 True-falseTrue-false
 MatchingMatching
 Norm-referenced and criterion referenced testsNorm-referenced and criterion referenced tests

5. Norm and Criterion-referenced testsNorm and Criterion-referenced tests
 Norm-referenced testNorm-referenced test
 standardized tests (college board, TOEFL, GRE)standardized tests (college board, TOEFL, GRE)
 Place test-takers on a mathematical continuum inPlace test-takers on a mathematical continuum in
rank orderrank order
 Criterion-referenced testsCriterion-referenced tests
 give test-takers feedback on specific objectivesgive test-takers feedback on specific objectives
(“criterea”)(“criterea”)
 test objectives of a coursetest objectives of a course
 known as “instructional value”known as “instructional value”

6. Authentic AssessmentAuthentic Assessment
 Authentic assessmentAuthentic assessment
 reflects student learning, achievement, motivation,reflects student learning, achievement, motivation,
and attitudes on instructionally relevant classroomand attitudes on instructionally relevant classroom
activities (O’Malley & Valdez, 1996).activities (O’Malley & Valdez, 1996).
 Examples:Examples:
 performance assessmentperformance assessment
 portfoliosportfolios
 self-assessmentself-assessment

7. Purposes for AssessmentPurposes for Assessment
 Diagnose students strengths and needsDiagnose students strengths and needs
 Provide feedback on student learningProvide feedback on student learning
 Provide a basis for instructional placementProvide a basis for instructional placement
 Inform and guide instructionInform and guide instruction
 Communicate learning expectationsCommunicate learning expectations
 Motivate and focus students’ attention andMotivate and focus students’ attention and
efforteffort
 Provide practice applying knowledge and skillsProvide practice applying knowledge and skills

8. Purposes continuedPurposes continued
 Provide a basis for evaluation for the purposeProvide a basis for evaluation for the purpose
of:of:
 GradingGrading
 Promotion/graduationPromotion/graduation
 Program admission/selectionProgram admission/selection
 AccountabilityAccountability
 Gauge program effectivenessGauge program effectiveness

9. Assessment InstrumentsAssessment Instruments
Pre-assessmentPre-assessment
(diagnostic)(diagnostic)
Formative (ongoing)Formative (ongoing) Summative (final)Summative (final)
PretestsPretests QuizzesQuizzes Teacher-made testTeacher-made test
ObservationsObservations DiscussionsDiscussions PortfoliosPortfolios
Journals/logsJournals/logs AssignmentsAssignments ProjectsProjects
DiscussionsDiscussions ProjectsProjects Standardized testsStandardized tests
QuestionnairesQuestionnaires ObservationsObservations
InterviewsInterviews PortfoliosPortfolios
Journal logsJournal logs
Standardized testsStandardized tests

10. DiscussionDiscussion
 How would you document a studentHow would you document a student
performance during a discussion?performance during a discussion?
 Which types of assessments noted in the chartWhich types of assessments noted in the chart
could be considered authentic assessment?could be considered authentic assessment?

11. Principles of Language AssessmentPrinciples of Language Assessment
 PracticalityPracticality
 ReliabilityReliability
 ValidityValidity
 AuthenticityAuthenticity
 WashbackWashback

12. PracticalityPracticality
 An effective test is practicalAn effective test is practical
 Is not excessively expensiveIs not excessively expensive
 Stays within appropriate time constraintsStays within appropriate time constraints
 Is relatively easy to administerIs relatively easy to administer
 Has a scoring/evaluation procedure that is specificHas a scoring/evaluation procedure that is specific
and time-efficientand time-efficient

13. ReliabilityReliability
 A reliable test is consistent and dependable. IfA reliable test is consistent and dependable. If
you give the same test to the same students inyou give the same test to the same students in
two different occasions, the test should yieldtwo different occasions, the test should yield
similar results.similar results.
 Student-related reliabilityStudent-related reliability
 Rater reliabilityRater reliability
 Test administration reliabilityTest administration reliability
 Test reliabilityTest reliability

14. Student Related ReliabilityStudent Related Reliability
 The most common issue in student relatedThe most common issue in student related
reliability is caused by temporary illness, fatigue,reliability is caused by temporary illness, fatigue,
a bad day, anxiety, and other physical anda bad day, anxiety, and other physical and
psychological factors which may make anpsychological factors which may make an
“observed” score deviate from a “true” score.“observed” score deviate from a “true” score.

15. Rater ReliabilityRater Reliability
 Human error, subjectivity, and bias may enterHuman error, subjectivity, and bias may enter
into the scoring process.into the scoring process.
 Inter-rater reliability occurs when two or moreInter-rater reliability occurs when two or more
scorers yield inconsistent scores of the same test,scorers yield inconsistent scores of the same test,
possibly for lack of attention to scoring criteria,possibly for lack of attention to scoring criteria,
inexperience, inattention, or even preconceivedinexperience, inattention, or even preconceived
bias toward a particular “good” and “bad”bias toward a particular “good” and “bad”
student.student.

16. Test Administration ReliabilityTest Administration Reliability
 Test administration reliability deals with theTest administration reliability deals with the
conditions in which the test is administered.conditions in which the test is administered.
 Street noise outside the buildingStreet noise outside the building
 bad equipmentbad equipment
 room temperatureroom temperature
 the conditions of chairs and tables, photocopyingthe conditions of chairs and tables, photocopying
variationvariation

17. Test ReliabilityTest Reliability
 The test is too longThe test is too long
 Poorly written or ambiguous test itemsPoorly written or ambiguous test items

18. ValidityValidity
 A test is valid if it actually assess the objectivesA test is valid if it actually assess the objectives
and what has been taught.and what has been taught.
 Content validityContent validity
 Criterion validity (tests objectives)Criterion validity (tests objectives)
 Construct validityConstruct validity
 Consequential validityConsequential validity
 Face validityFace validity

19. Content ValidityContent Validity
 A test is valid if the teacher can clearly define theA test is valid if the teacher can clearly define the
achievement that he or she is measuringachievement that he or she is measuring
 A test of tennis competency that asks someoneA test of tennis competency that asks someone
to run a 100-yard dash lacks content validityto run a 100-yard dash lacks content validity
 If a teacher uses the communicative approach toIf a teacher uses the communicative approach to
teach speaking and then uses the audiolingualteach speaking and then uses the audiolingual
method to design test items, it is going to lackmethod to design test items, it is going to lack
content validitycontent validity

20. Criterion-related ValidityCriterion-related Validity
 The extent to which the objectives of the test have beenThe extent to which the objectives of the test have been
measured or assessed. For instance, if you are assessingmeasured or assessed. For instance, if you are assessing
reading skills such as scanning and skimmingreading skills such as scanning and skimming
information, how are the exercises designed to testinformation, how are the exercises designed to test
these objectives?these objectives?
 In other words, the test is valid if the objectives taughtIn other words, the test is valid if the objectives taught
are the objectives tested and the items are actuallyare the objectives tested and the items are actually
testing this objectives.testing this objectives.

21. Construct ValidityConstruct Validity
 A construct is an explanation or theory thatA construct is an explanation or theory that
attempts to explain observed phenomenaattempts to explain observed phenomena
 If you are testing vocabulary and the lexicalIf you are testing vocabulary and the lexical
objective is to use the lexical items forobjective is to use the lexical items for
communication, writing the definitions of thecommunication, writing the definitions of the
test will not match with the construct oftest will not match with the construct of
communicative language usecommunicative language use

22. Consequential ValidityConsequential Validity
 Accuracy in measuring intended criteriaAccuracy in measuring intended criteria
 Its impact on the preparation of test-takersIts impact on the preparation of test-takers
 Its effect on the learnerIts effect on the learner
 Social consequences of a test interpretation (exitSocial consequences of a test interpretation (exit
exam for pre-basic students at El Colegio, theexam for pre-basic students at El Colegio, the
College Board)College Board)

23. Face ValidityFace Validity
 Face validity refers to the degree to which a test looksFace validity refers to the degree to which a test looks
right, and appears to measure the knowledge or ability itright, and appears to measure the knowledge or ability it
claims to measureclaims to measure
 A well-constructed, expected format with familiar tasksA well-constructed, expected format with familiar tasks
 A test that is clearly doable within the allotted time limitA test that is clearly doable within the allotted time limit
 Directions are crystal clearDirections are crystal clear
 Tasks that relate to the course (content validity)Tasks that relate to the course (content validity)
 A difficulty level that presents a reasonable challengeA difficulty level that presents a reasonable challenge

24. AuthenticityAuthenticity
 The language in the test is as natural as possibleThe language in the test is as natural as possible
 Items are contextualized rather than isolatedItems are contextualized rather than isolated
 Topics are relevant and meaningful for learnersTopics are relevant and meaningful for learners
 Some thematic organization to items is providedSome thematic organization to items is provided
 Tasks represent, or closely approximate, real-Tasks represent, or closely approximate, real-
world tasksworld tasks

25. WashbackWashback
 Washback refers to the effects the tests have onWashback refers to the effects the tests have on
instruction in terms of how students prepare forinstruction in terms of how students prepare for
the test “Cram” courses and “teaching to thethe test “Cram” courses and “teaching to the
test” are examples of such washbacktest” are examples of such washback
 In some cases the student may learn whenIn some cases the student may learn when
working on a test or assessmentworking on a test or assessment
 Washback can be positive or negativeWashback can be positive or negative

26. Alternative Assessment OptionsAlternative Assessment Options
 Self and peer-assessmentsSelf and peer-assessments
 Oral production-student self-checklist, peer checklist,Oral production-student self-checklist, peer checklist,
offering and receiving holistic rating of an oral presentationoffering and receiving holistic rating of an oral presentation
 Listening comprehension- listening to TV or radio broadcastsListening comprehension- listening to TV or radio broadcasts
and checking comprehension with a partnerand checking comprehension with a partner
 Writing-revising work on your own, peer-editingWriting-revising work on your own, peer-editing
 Reading- reading textbook passages followed by self-checkReading- reading textbook passages followed by self-check
comprehension questions, self-assessment of reading habitscomprehension questions, self-assessment of reading habits
(page 416, Brown, 2001)(page 416, Brown, 2001)

27. Authentic AssessmentAuthentic Assessment
 Performance assessment- any form ofPerformance assessment- any form of
assessment in which the student constructs aassessment in which the student constructs a
response orally or in writing. It requires theresponse orally or in writing. It requires the
learner to accomplish a complex and significantlearner to accomplish a complex and significant
task, while bringing to bear prior knowledge,task, while bringing to bear prior knowledge,
recent learning, and relevant skills to solverecent learning, and relevant skills to solve
realistic or authentic problems (O’Malley &realistic or authentic problems (O’Malley &
Valdez, 1996; Herman, et. al., 1992).Valdez, 1996; Herman, et. al., 1992).

28. Examples of Authentic AssessmentExamples of Authentic Assessment
 Portfolio assessmentPortfolio assessment
 Student self-assessmentStudent self-assessment
 Peer assessmentPeer assessment
 Student-teacher conferencesStudent-teacher conferences
 Oral interviewsOral interviews
 Writing samplesWriting samples
 Projects or exhibitionsProjects or exhibitions
 Experiments or demonstrationsExperiments or demonstrations

29. Characteristics of performanceCharacteristics of performance
assessmentassessment
 Constructed responseConstructed response
 Higher-order thinkingHigher-order thinking
 AuthenticityAuthenticity
 IntegrativeIntegrative
 Process and productProcess and product
 Depth versus breadthDepth versus breadth

30. JournalsJournals
 Specify to students the purpose of the journalSpecify to students the purpose of the journal
 Give clear directions to students on how to get startedGive clear directions to students on how to get started
(prompts for instance “I was very happy when…)(prompts for instance “I was very happy when…)
 Give guidelines on length of each entryGive guidelines on length of each entry
 Be clear yourself on the principal purpose of the journalBe clear yourself on the principal purpose of the journal
 Help students to process your feedback, and showHelp students to process your feedback, and show
them how to respond to your responsesthem how to respond to your responses

31. ConferencesConferences
 Commonly used when teaching writingCommonly used when teaching writing
 One-on-one interaction between teacher andOne-on-one interaction between teacher and
studentstudent
 Conferences are formative assessment asConferences are formative assessment as
opposed to offering a final grade or a summativeopposed to offering a final grade or a summative
assessment. In other words, they are meant toassessment. In other words, they are meant to
provide guidance and feedback.provide guidance and feedback.

32. PortfoliosPortfolios
 Commonly used with the communicative languageCommonly used with the communicative language
teaching approach (CLT)teaching approach (CLT)
 It is a collection of students’ work that demonstrates toIt is a collection of students’ work that demonstrates to
students and others the efforts, progress andstudents and others the efforts, progress and
achievements in a given area. You can have a readingachievements in a given area. You can have a reading
portfolio or a writing portfolio, for instanceportfolio or a writing portfolio, for instance
 You can also have a reflective or assessment portfolioYou can also have a reflective or assessment portfolio
as opposed to collecting every piece of evidence foras opposed to collecting every piece of evidence for
each objective achieved in the courseeach objective achieved in the course

33. Portfolio GuidelinesPortfolio Guidelines
 Specify the purpose of the portfolioSpecify the purpose of the portfolio
 Give clear directions to students on how to get startedGive clear directions to students on how to get started
 Give guidelines of acceptable materials or artifactsGive guidelines of acceptable materials or artifacts
 Collect portfolios on a pre-announced dates and returnCollect portfolios on a pre-announced dates and return
promptlypromptly
 Help students to process your feedbackHelp students to process your feedback
 Establish a rubric to evaluate the portfolio and discussEstablish a rubric to evaluate the portfolio and discuss
it with your studentsit with your students

34. Cooperative Test ConstructionCooperative Test Construction
 Cooperative test construction involves theCooperative test construction involves the
students contribution to the design of test items.students contribution to the design of test items.
It is based on the concept of collaborative andIt is based on the concept of collaborative and
cooperative learning in which students arecooperative learning in which students are
involved in the processinvolved in the process
 (Brown, 2001, p. 420)(Brown, 2001, p. 420)