SlideShare une entreprise Scribd logo
1  sur  33
ILR 101 January 23, 2009 Language Testing and Assessment Unit, FBI Maria M. Brau and Rachel L. Brooks
Topics Covered ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Interagency Language Roundtable (ILR) ,[object Object],[object Object],[object Object],[object Object],[object Object]
ILR Skill Level Descriptions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
ILR Scale ,[object Object],[object Object],[object Object]
Typical Interval Scales Poor Excellent Good Fair Satisfactory 100 0 50 A B C D F Surveys: Academic:
0 1 2 3 4 5 Level 3 Range ILR Skill Level Descriptions
Cooking Scale ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Hiking Scale ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
ILR Scale 1 2 3 4 5 Moving upward is increasingly difficult.
5 4 0 1 2 3 Level 5 Level 4 Level 3 Level 2 Level 1 Level 0 Level 3 Range 3 ILR Skill Levels Each level represents a range.
e, t e.t.
e, t, a e.t. ate eat tea at ta
e, t, a, m e.t. ate eat tea at ta mate meat meta tame team mat met tam am me ma
e, t, a,  m, s e.t. ate eat tea at ta mate meat meta tame team mat met tam am me ma mates meats steam tames teams east eats mast mats mesa same sate seam seat tams stem teas sat sea as
ILR Skill Level Descriptions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Proficiency  (Single Skills) Performance  (Combined Skills)
Speaking 3  (General Professional Proficiency)   ,[object Object],Able to  speak the language with sufficient structural accuracy and vocabulary to participate effectively in most formal and informal conversations on practical, social, and professional topics. The individual  can  effectively combine structure and vocabulary to convey his/her meaning accurately.
ILR Levels 2, 2+, 3 Level Speaking Listening Reading Writing 2 Able to satisfy routine social demands and limited work requirements. Sufficient comprehension to understand conversations on routine social demands and limited job requirements. Sufficient comprehension to read simple, authentic written material in a form equivalent to usual printing or typescript on subjects within a familiar context. Able to write routine social correspondence and prepare documentary materials required for most limited work requirements. 2+ Able to satisfy most work requirements with language usage that is often, but not always, acceptable and effective. Sufficient comprehension to understand most routine social demands and most conversations on work requirements as well as some discussions on concrete topics related to particular interests and special fields of competence. Sufficient comprehension to understand most factual material in non-technical prose as well as some discussions on concrete topics related to special professional interests. Shows ability to write with some precision and in some detail about most common topics. 3 Able to speak the language with sufficient structural accuracy and vocabulary to participate effectively in most formal and informal conversations on practical, social, and professional topics. Able to understand the essentials of all speech in a standard dialect including technical discussions within a special field. Able to read within a normal range of speed and with almost complete comprehension of a variety of authentic prose material on unfamiliar subjects. Able to use the language effectively in most formal and informal written exchanges on practical social and professional topics.
ILR Level 2 Speaking Able to satisfy routine social demands and limited work requirements. Listening Sufficient comprehension to understand conversations on routine social demands and limited job requirements. Reading Sufficient comprehension to read simple, authentic written material in a form equivalent to usual printing or typescript on subjects within a familiar context. Writing Able to write routine social correspondence and prepare documentary materials required for most limited work requirements.
ILR Level 2+ Speaking Able to satisfy most work requirements with language usage that is often, but not always, acceptable and effective. Listening Sufficient comprehension to understand most routine social demands and most conversations on work requirements as well as some discussions on concrete topics related to particular interests and special fields of competence. Reading Sufficient comprehension to understand most factual material in non-technical prose as well as some discussions on concrete topics related to special professional interests. Writing Shows ability to write with some precision and in some detail about most common topics.
ILR Level 3 Speaking Able to speak the language with sufficient structural accuracy and vocabulary to participate effectively in most formal and informal conversations on practical, social, and professional topics. Listening Able to understand the essentials of all speech in a standard dialect including technical discussions within a special field. Reading Able to read within a normal range of speed and with almost complete comprehension of a variety of authentic prose material on unfamiliar subjects. Writing Able to use the language effectively in most formal and informal written exchanges on practical social and professional topics.
“ Knowing” a language ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Four Primary Skills Translation Interpretation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Types of Listening Interactive Listening (Participatory) Monitoring (Recorded Speech) Monitoring (Live Speech) Static Listening (Non-participatory) No opportunity to clarify Can request clarification
Translation Listening: FL Writing: Eng Listening: Eng Writing: FL Monitoring (Audio Translation) Prerequisite Skills Reading: FL Writing: Eng Reading: Eng Writing: FL Document Translation
Interpretation Listening: FL Speaking: Eng Listening: Eng Speaking: FL Consecutive & Simultaneous Interpretation Prerequisite Skills Reading: FL Speaking: Eng Reading: Eng Speaking: FL Sight Translation
Transcription (Monolingual) Listening: FL Writing: FL Listening: Eng Writing: Eng Transcription Prerequisite Skills
Capacity to Translate 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 NSL LSL Ideal Normal Reading Comprehension of the Source Language Writing Ability in the Target Language Maximum Translation Ability
From theory to practice ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Testing one prerequisite skill is not sufficient to predict translation ability.
Post 9/11 Arabic Applicants 1 10 12 (0.8%) Failed Reading/Passed TT 221 634 781 (54.3%) Passed Reading/Failed TT 369 875 1149 (79.9%) Failed Translation Test 113 202 289 (20.1%) Passed Translation Test 151 243 371 (25.8%) Failed Reading (DLPT) 331 834 1067 (74.2%) Passed Reading (DLPT) 253 563 683 (47.5%) Failed Listening (DLPT) 229 514 755 (52.5%) Passed Listening (DLPT) 482 1077 1438 Total Applicants L1: English L1: Arabic Total  
Percent Concordance: Final Derived VTE and EWT Score   Italian Overall (%) Vietnamese Overall (%) Turkish Overall (%) Average Overall (%) Exact Match 3.75 19.82 2.50 8.69 Within Level Match 28.75 50.16 26.30 35.07 One Level Difference (VTE higher or lower) 36.25 32.39 31.30 33.28 Greater than One Level Difference (VTE higher) 0.00 0.00 0.00 0.00 Greater than One Level Difference (VTE lower) 35.00 17.45 42.50 31.65
Practice Translation:  English to English Paraphrase the following sentences ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],A certain pre-adolescent human moved in an  awkward, unbalanced manner about that particular room purposed for recreation. The person with the intention to dispose of her  bread-enveloped meal is me.
Maria M. Brau [email_address] Rachel L. Brooks [email_address]

Contenu connexe

Similaire à ILR 101

Similaire à ILR 101 (20)

Abbott Student Learning Outcomes
Abbott Student Learning OutcomesAbbott Student Learning Outcomes
Abbott Student Learning Outcomes
 
Reading
ReadingReading
Reading
 
Speaking skills
Speaking skills Speaking skills
Speaking skills
 
Prep-B2-Workshop
Prep-B2-WorkshopPrep-B2-Workshop
Prep-B2-Workshop
 
SLPI-ASL Rating - Brandon Dockery
SLPI-ASL Rating - Brandon DockerySLPI-ASL Rating - Brandon Dockery
SLPI-ASL Rating - Brandon Dockery
 
International Evaluation System
International Evaluation SystemInternational Evaluation System
International Evaluation System
 
Module-2.pptx
Module-2.pptxModule-2.pptx
Module-2.pptx
 
Speaking
SpeakingSpeaking
Speaking
 
Interpretation
Interpretation Interpretation
Interpretation
 
English for life
English for lifeEnglish for life
English for life
 
ACTFL_SpanishProficiency_Certificate
ACTFL_SpanishProficiency_CertificateACTFL_SpanishProficiency_Certificate
ACTFL_SpanishProficiency_Certificate
 
Business Levels
Business LevelsBusiness Levels
Business Levels
 
Fluency vs accuracy
Fluency vs accuracyFluency vs accuracy
Fluency vs accuracy
 
EProScoreReportStrebe
EProScoreReportStrebeEProScoreReportStrebe
EProScoreReportStrebe
 
Communicative approach
Communicative approachCommunicative approach
Communicative approach
 
Plain language for professional prudence
Plain language for professional prudencePlain language for professional prudence
Plain language for professional prudence
 
Common European Framework. Level B2
Common European Framework. Level B2Common European Framework. Level B2
Common European Framework. Level B2
 
Brown.Elizabeth OPI Certificate
Brown.Elizabeth OPI CertificateBrown.Elizabeth OPI Certificate
Brown.Elizabeth OPI Certificate
 
Orsi_ACTFLResults
Orsi_ACTFLResultsOrsi_ACTFLResults
Orsi_ACTFLResults
 
Teaching the language and the skills RECEPTIVE.pptx
Teaching the language and the skills RECEPTIVE.pptxTeaching the language and the skills RECEPTIVE.pptx
Teaching the language and the skills RECEPTIVE.pptx
 

ILR 101

  • 1. ILR 101 January 23, 2009 Language Testing and Assessment Unit, FBI Maria M. Brau and Rachel L. Brooks
  • 2.
  • 3.
  • 4.
  • 5.
  • 6. Typical Interval Scales Poor Excellent Good Fair Satisfactory 100 0 50 A B C D F Surveys: Academic:
  • 7. 0 1 2 3 4 5 Level 3 Range ILR Skill Level Descriptions
  • 8.
  • 9.
  • 10. ILR Scale 1 2 3 4 5 Moving upward is increasingly difficult.
  • 11. 5 4 0 1 2 3 Level 5 Level 4 Level 3 Level 2 Level 1 Level 0 Level 3 Range 3 ILR Skill Levels Each level represents a range.
  • 13. e, t, a e.t. ate eat tea at ta
  • 14. e, t, a, m e.t. ate eat tea at ta mate meat meta tame team mat met tam am me ma
  • 15. e, t, a, m, s e.t. ate eat tea at ta mate meat meta tame team mat met tam am me ma mates meats steam tames teams east eats mast mats mesa same sate seam seat tams stem teas sat sea as
  • 16.
  • 17.
  • 18. ILR Levels 2, 2+, 3 Level Speaking Listening Reading Writing 2 Able to satisfy routine social demands and limited work requirements. Sufficient comprehension to understand conversations on routine social demands and limited job requirements. Sufficient comprehension to read simple, authentic written material in a form equivalent to usual printing or typescript on subjects within a familiar context. Able to write routine social correspondence and prepare documentary materials required for most limited work requirements. 2+ Able to satisfy most work requirements with language usage that is often, but not always, acceptable and effective. Sufficient comprehension to understand most routine social demands and most conversations on work requirements as well as some discussions on concrete topics related to particular interests and special fields of competence. Sufficient comprehension to understand most factual material in non-technical prose as well as some discussions on concrete topics related to special professional interests. Shows ability to write with some precision and in some detail about most common topics. 3 Able to speak the language with sufficient structural accuracy and vocabulary to participate effectively in most formal and informal conversations on practical, social, and professional topics. Able to understand the essentials of all speech in a standard dialect including technical discussions within a special field. Able to read within a normal range of speed and with almost complete comprehension of a variety of authentic prose material on unfamiliar subjects. Able to use the language effectively in most formal and informal written exchanges on practical social and professional topics.
  • 19. ILR Level 2 Speaking Able to satisfy routine social demands and limited work requirements. Listening Sufficient comprehension to understand conversations on routine social demands and limited job requirements. Reading Sufficient comprehension to read simple, authentic written material in a form equivalent to usual printing or typescript on subjects within a familiar context. Writing Able to write routine social correspondence and prepare documentary materials required for most limited work requirements.
  • 20. ILR Level 2+ Speaking Able to satisfy most work requirements with language usage that is often, but not always, acceptable and effective. Listening Sufficient comprehension to understand most routine social demands and most conversations on work requirements as well as some discussions on concrete topics related to particular interests and special fields of competence. Reading Sufficient comprehension to understand most factual material in non-technical prose as well as some discussions on concrete topics related to special professional interests. Writing Shows ability to write with some precision and in some detail about most common topics.
  • 21. ILR Level 3 Speaking Able to speak the language with sufficient structural accuracy and vocabulary to participate effectively in most formal and informal conversations on practical, social, and professional topics. Listening Able to understand the essentials of all speech in a standard dialect including technical discussions within a special field. Reading Able to read within a normal range of speed and with almost complete comprehension of a variety of authentic prose material on unfamiliar subjects. Writing Able to use the language effectively in most formal and informal written exchanges on practical social and professional topics.
  • 22.
  • 23.
  • 24. Types of Listening Interactive Listening (Participatory) Monitoring (Recorded Speech) Monitoring (Live Speech) Static Listening (Non-participatory) No opportunity to clarify Can request clarification
  • 25. Translation Listening: FL Writing: Eng Listening: Eng Writing: FL Monitoring (Audio Translation) Prerequisite Skills Reading: FL Writing: Eng Reading: Eng Writing: FL Document Translation
  • 26. Interpretation Listening: FL Speaking: Eng Listening: Eng Speaking: FL Consecutive & Simultaneous Interpretation Prerequisite Skills Reading: FL Speaking: Eng Reading: Eng Speaking: FL Sight Translation
  • 27. Transcription (Monolingual) Listening: FL Writing: FL Listening: Eng Writing: Eng Transcription Prerequisite Skills
  • 28. Capacity to Translate 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 NSL LSL Ideal Normal Reading Comprehension of the Source Language Writing Ability in the Target Language Maximum Translation Ability
  • 29.
  • 30. Post 9/11 Arabic Applicants 1 10 12 (0.8%) Failed Reading/Passed TT 221 634 781 (54.3%) Passed Reading/Failed TT 369 875 1149 (79.9%) Failed Translation Test 113 202 289 (20.1%) Passed Translation Test 151 243 371 (25.8%) Failed Reading (DLPT) 331 834 1067 (74.2%) Passed Reading (DLPT) 253 563 683 (47.5%) Failed Listening (DLPT) 229 514 755 (52.5%) Passed Listening (DLPT) 482 1077 1438 Total Applicants L1: English L1: Arabic Total  
  • 31. Percent Concordance: Final Derived VTE and EWT Score   Italian Overall (%) Vietnamese Overall (%) Turkish Overall (%) Average Overall (%) Exact Match 3.75 19.82 2.50 8.69 Within Level Match 28.75 50.16 26.30 35.07 One Level Difference (VTE higher or lower) 36.25 32.39 31.30 33.28 Greater than One Level Difference (VTE higher) 0.00 0.00 0.00 0.00 Greater than One Level Difference (VTE lower) 35.00 17.45 42.50 31.65
  • 32.
  • 33. Maria M. Brau [email_address] Rachel L. Brooks [email_address]

Notes de l'éditeur

  1. 01/13/10
  2. 01/13/10 Topics marked are those covered in the ILR 101 basic slide set, presented at the January 2009 Plenary. Other topics are added, depending on participant needs.
  3. 01/13/10
  4. 01/13/10
  5. 01/13/10 Purpose: introduce participants to the ILR scale by noting it is different from the interval scales with which they are familiar.
  6. 01/13/10 Purpose: to provide examples of familiar interval scales. The first is a Likert scale, used in consumer surveys. The second reflects the grading scale normally used in American educational institutions. Here, a score reflects one particular point on the scale. Only scores in the upper range are considered “good” or “acceptable.”
  7. 01/13/10 Purpose: to provide examples of familiar interval scales. The first is a Likert scale, used in consumer surveys. The second reflects the grading scale normally used in American educational institutions. Here, a score reflects one particular point on the scale. Only scores in the upper range are considered “good” or “acceptable.”
  8. 01/13/10 Purpose: to illustrate the principles behind the ILR scale by applying it to a familiar activity, such as cooking. Asking the participants to engage in this practical exercise helps them understand how acquisition of additional skills expands an individual’s abilities. The exercise also brings out the traditional distinctions between levels: a Level 1 can survive, a Level 2 has enough skills to satisfy daily needs, (a 2+ is the level at which individuals will normally resolve daily challenges successfully), a Level 3 is capable of producing professional results, a Level 4 is a chef using complex techniques that produce consistent results, and a Level 5 is recognized as a master chef.
  9. 01/13/10 Purpose: to illustrate the principles behind the ILR scale by applying it to a familiar activity, such as hiking. Again, below level 3, the hiker operates in a mostly familiar setting, without too many complications. The hiker above Level 3 requires specialized skills and equipment, and is able to lead.
  10. 01/13/10 Purpose: to illustrate graphically that achieving a higher score is progressively more difficult, since it involves not only increased vocabulary and structural control, but also linguistic competence to handle tasks at the upper levels.
  11. 01/13/10 Purpose: to show the inverted pyramid concept, traditionally used by ILR tester-trainers to illustrate ranges within levels.
  12. 01/13/10 Purpose: to show how additions multiply capacity. With 2 letters, 1 acronym is possible but 0 words.
  13. 01/13/10 Add 1 letter, and 5 words are possible.
  14. 01/13/10 Another letter jumps the number of words to 16.
  15. 01/13/10 One more letter allows for the formation of 36 words.
  16. 01/13/10
  17. 01/13/10 Purpose: to show the structure of an ILR skill level description. A bolded statement characterizes general ability at that level, followed by several statements providing details. There is also an examples section that is not shown in this slide.
  18. 01/13/10 Purpose: to show how similar concepts are conveyed across the proficiency skills.
  19. 01/13/10 Purpose: to show that work should not be assigned to a person based on a test score in a single skill.
  20. 01/13/10 Purpose: to show that (1) each of the proficiency skills has different attributes, and (2) each of the combined skills has different prerequisites. Therefore, it does not necessarily follow that a good translator can be a good interpreter, and vice versa. Translators use delayed skills, are able to craft a translation over time, with revisions and edits. Interpretation requires immediate language processing, involving the use of memory.
  21. 01/13/10 Purpose: to show that there are differences in a single skill, e.g., listening. In interactive listening, the person is a participant in the conversation and can request repetitions and clarifications. A non-participant can replay speech only if the source is a recording but can never request clarification.
  22. 01/13/10 Purpose: to show that the prerequisites for translation differ depending on the task.
  23. 01/13/10 Purpose: to show that the prerequisites for interpretation differ depending on the task. Note that those for audio translation (listening and writing) and those for sight translation (reading and speaking) are exact opposites.
  24. 01/13/10 Purpose: to show that since the prerequisite skills for transcription are monolingual, the task does not need to be assigned to a translator and much less to an interpreter, whose prerequisite skills do not include writing.
  25. 01/13/10 Purpose: to show that a person’s capacity to translate is limited by the lower prerequisite skill. A person who is a native of the source language (NSL) may be highly proficient in reading the source document, but a weakness in writing the target language will limit translation capacity. A person who is a learner of the source language (LSL) may have a lower reading comprehension and a higher writing ability in the target language. In both cases, the person will not perform higher than the lower prerequisite skill. It must be noted that prerequisites limit maximum capacity, but lack of congruity judgment (the ability to choose proper equivalents) may further bring down the quality of the product. The effect of congruity judgment is not shown in the chart.
  26. 01/13/10 Purpose: to show that FBI research tends to confirm that a reading test, a writing test, or a combination of the two, should not be used to replace a translation test. These are testing instruments that focus on assessing congruity judgment.
  27. 01/13/10 The study investigated the relationship between reading ability and translation ability. Examinees took the Defense Language Proficiency Test (DLPT) for both Reading and Listening in Arabic and the Arabic Translation Test. Passing rates for all tests were set at 2+. Although almost 75% of applicants passed the DLPT for Reading, only 20% of applicants passed the Translation Test, meaning that more than 50% of those who passed the Reading DLPT, failed the Translation Test.
  28. 01/13/10 The study investigated the relationship between writing ability and translation ability. Participants were natives of the foreign language, and could read the source text material. Both English Writing Tests (EWTs) and Verbatim Translation Exams (VTEs), from the foreign language into English, were administered to participants from three different language groups. On average, only 9% of the participants’ scores matched on the two tests, meaning 9% met their maximum capacity to translate. Approximately 35% had scores in the same level/plus level range. About 33% of participants had one level difference between the two tests, and each time the VTE score was lower than the EWT score. In almost 32% of the cases, the VTE score was two main ILR levels lower than the EWT score.
  29. 01/13/10 Purpose: to provide an opportunity for those who are not translators to experience what translation is like. The task was to paraphrase the sentence given with another sentence in English, making sure not to use any of the words in the original sentences. Two possible responses are given for the first two sentences.