3. 0:00 9:00 3:00 6:00 12:00 21:00 15:00 18:00 What time is it in Leuven ? What time is it in Hawaii ? “ Here and Now” condition “ There and… ” condition Now
4.
5. measurement, The essential roles of assessment, and evaluation in task-based language teaching
6. TBLT 2005: What’s at stake? ??? Language Education ??? Does it work? for whom, when, why? What’s it worth? to whom, when, why? How does it work? for whom, when, why? Can it be improved? by whom, when, why?
7. TBLT 2005: What’s at stake? … another L2 teaching method bandwagon… The critics say… … cognitive-interactionist SLA hegemony… … won’t work with _____ learners… … not possible in _____ program settings… … too much task, not enough language… … not happening in practice… T B L T
8. Setting the record straight: TBLT at work TARGET LANGUAGES English Dutch French Czech Chinese Arabic Ukranian Japanese Russian Spanish Korean German
9. Setting the record straight: TBLT at work PROGRAM TYPES Second Language Foreign Language Vocational Education General Education Local Innovation Small Classes National Ed. Policy Special Purposes Adult Child University Requirement Large Classes Beginning Learners Advanced Learners Face-to- Face Online
10. Setting the record straight: TBLT foundations “ All genuine education terminates in discipline, but it proceeds by engaging the mind in activities worthwhile for their own sake.” Dewey (1933), pp. 86-87
11. Setting the record straight: TBLT foundations “ This discussion rejects the doctrine that students should first learn passively, and then, having learned, should apply knowledge. […] In fact, the applications are part of the knowledge. For the very meaning of things known is wrapped up in their relationships beyond themselves. Thus, unapplied knowledge is knowledge shorn of its meaning.” Whitehead, 1947, pp. 218-219
12. Setting the record straight: TBLT foundations “ For starters, this means having students do tasks, or at least meaningful simulations, that experts do in the various disciplines. Second, it means teaching them to think in ways that experts do when they perform these tasks.” Sternberg (2003), p. 5
13. Setting the record straight: Other task-based domains Medical school Nursing Curriculum by design Successful intelligence Environmental studies Social work Task-Based Practice Music education
14. Setting the record straight: So what is TBLT? Task-Based Language Teaching Rationales and Principles Education Philosophy Psychology SLA Language teaching Sociopolitical values L2 Education Programs Learner needs Assessment Curriculum Instruction Teacher development Evaluation Materials
15. The potential of task-based practice: One example William Reid Reid. W. J. (1992). Task strategies: An empirical approach to social work practice . New York: Columbia University Press. Social work problems : truancy, aging, delinquency, substance abuse, family health
16. Task-Centered Practice Empirical Justification Empirical Implementation Theory Learning, behavior, cognition Research Psychosocial change, Problem solving Evaluation Case work practice, Intervention outcomes Needs Identify, agree, prioritize Action Plan, rehearse, attempt, feedback Assessment Task accomplishment, Problem change Development Practitioner-friendly methods Improvement Field-tested findings The potential of task-based practice: One example
17. The potential of task-based practice: One example “ Assessment, process, and outcome data are systematically collected in each case by means of structured recording guidelines. Continuing research is used to improve the model.” (Reid, 1992, p. 132). Development Improvement Field-testing
18. “ [Reid] was the single most significant figure in social work research” Shaw (2004) p. 113 The potential of task-based practice: One example
20. Task-Based Language Teaching: A(nother) method for language education practice. The potential of task-based practice: TBLT
21. Task-Based Language Teaching: An empirical approach to language education practice. The potential of task-based practice: TBLT Assessment Measurement Evaluation
22. The Role of Measurement in TBLT Interpreting research constructs… … to understand task-based learning
23. Measurement in TBLT: Responding to research purposes What is measurement? … trustworthy definition, elicitation, observation, analysis, and interpretation of research constructs…
24.
25. Measurement in TBLT: Responding to research purposes “ Perhaps the best recommendation would be that, with research in this area still developing techniques and understandings, the role of A/L work will be to offer suggestions for relevant variables and the standardization of operationalizations of variables…” Tavakoli & Skehan, 2005, p. 273
26.
27.
28. Task influence on L2 performance (adapted from: Skehan, 2001) Measurement in TBLT: Problems of practice 0 + 0 Transformations (planning) 0 + 0 Outcomes + 0 + Structure - + + Dialogic v. Monologic + 0 + Familiarity Fluency Complexity Accuracy Task characteristic
29. Task influence on L2 performance (adapted from: Skehan, 2001) Measurement in TBLT: Problems of practice 0 + 0 Transformations (planning) 0 + 0 Outcomes + 0 + Structure - + + Dialogic v. Monologic + 0 + Familiarity Fluency Complexity Accuracy Task characteristic
30. “… giving learners the opportunity to plan before a task is done consistently produces greater complexity of language…” (Skehan, 2003, pp. 395-96) Measurement in TBLT: Problems of practice “ To sum up, both modality and group condition affected complexity at the level of syntax…” (Ellis & Yuan, 2005, p. 184)
31. Measurement in TBLT: Problems of practice Task condition Planning L2 performance outcome Complexification Modality? writing/speaking Complexity? Subordination Complexity? Other measures Scope of measure? duration/timing Planning? Focus, structure, pressure Learner proficiency? “ L-H”, “N-I-A”, “450-600” Task features? Type, difficulty, form, mono-dialogic, interlocutors ?
32. Measurement in TBLT: Problems of practice Task condition + strategic planning L2 performance outcome Complexification Complexity? Clauses per C-Unit Learner proficiency? TOEFL: 427-670 Elder & Iwashita (2005) X X X
33. Measurement in TBLT: Problems of practice Task condition + strategic planning L2 performance outcome Complexification Complexity? Words/T-Unit Clauses per T-Unit Total Subordination Learner proficiency? TOEFL: 420-480 TOEFL: 510-580 TOEFL: 560-610 Kawauichi (2005)
34. Measurement in TBLT: An empirical approach to practice In what ways are what kinds of planning associated with what kinds of L2 performance for what kinds of L2 users under what conditions…and why?
35. Measurement in TBLT: An empirical approach to practice “ ... descriptive longitudinal studies which establish norms of performance for particular processes and phenomena in L2 acquisition…” Norris & Ortega, 2003, p. 738 “… attempting to ‘measure’ acquisition without a sound descriptive basis for meaningful differences in particular acquisition-related behaviors would be akin to timing a runner’s performance over a mile without knowing how many times around the track a mile happens to be…”
36. Measurement in TBLT: An empirical approach to practice Georgetown University German Department: “ Syntactic complexity development in task-based writing” Longitudinal (23) + Cross-sectional (329) Extended and extemporaneous writing tasks Multiple measures, including Multiple measures of SC End-of-curricular levels Across 4 years
37. Syntactic complexity in curriculum-embedded writing assessment: LONGITUDINAL Global compexification Phrasal elaboration Subordination
38. Syntactic complexity in curriculum-embedded writing assessment: CROSS-SECTIONAL Global compexification Subordination Phrasal elaboration
39.
40.
41. The Role of Assessment in TBLT Informing decisions and actions… … to enable task-based learning
42. Assessment in TBLT: Responding to educative purposes What is assessment? … the use of tests and other procedures for gathering trustworthy information about learners and making decisions and taking actions on the basis of that information, in support of teaching and learning…
43.
44. Assessment in TBLT: Responding to educative purposes “ In other words, by focusing on task-based assessment—here writing—faculty faced their own assumptions about long-term development in instructed L2 learning and appropriate learning outcomes at particular stages in that sequence.” Byrnes (2002), p. 428
45. Assessment in TBLT: Responding to educative purposes “… the quality of task completion gave teachers a good understanding of what students did and did not know—often, a better understanding than most tests could provide.” Leaver & Kaplan (2004), p. 52
46. Assessment in TBLT: Responding to educative purposes “ Focusing on what one can do instead of the competencies one has yields a more direct link with language performance that can be observed in the real world.” Gysen & van Avermaet (2005), p. 51
47.
48.
49. Assessment in TBLT: Problems of practice “… the inferences we want to make are about underlying ‘language ability’ or ‘capacity for language use’ or ‘ability for use’…” Bachman (2002), p. 454 “… assessments can have many different functions. What is appropriate for one assessment purpose may be inappropriate for another…” Council of Europe (2001), p. 180
50.
51. Assessment in TBLT: Problems of practice “… is not for the test designer to decide or prescribe. Rather, it should be the test users and stakeholders who inform exactly what needs to be done in the name of language assessment, and it is up to us to enable them to do so.” Norris (2002), p. 344
52. “ 1. The intended use(s) for task-based assessment within the language programme must be specified…” Long & Norris (2000), p. 600 Assessment in TBLT: An empirical approach to practice
55. Assessment in TBLT: Examples in practice, Specification products Intended assessment use specification Assessment policies Use specs for all tests Priorities for Development Assessment Constraints Quizzes Exams Performance Internal External Time Money Ethics Training Unknowns (Urgent!) Placement Learning outcomes External Accountability Developing Implementing Articulating Communicating Evaluating
56.
57. Task Content L conventions L focus Assignment 1 Assignment 2 Process writing 4-6 tasks per semester Assignment 3 Assignment 4 Prototypical Performance Writing Task: Level I Level II Level III Level IV Curricular level/genre learning expectations Feedback loop Explicit criteria Assessment in TBLT: Examples in practice, Instruments
58.
59.
60. The Role of Evaluation in TBLT Articulating empirical practice… … to improve task-based education … with practical value
61. Evaluation in TBLT: Responding to program purposes What is evaluation? … gathering trustworthy information about program elements in order to understand, demonstrate, and improve program value…
62.
63. Evaluation in TBLT: Responding to program purposes “ Given the checkered history of prescriptions for language teaching, the likelihood that all 10 TBLT MPs will turn out to have been well founded is minimal. Thus, designers … who draw upon these or other sources should build careful evaluation components into their delivery systems.” Doughty & Long (2003), p. 68 “ In contrast to the extensive research into tasks…there have been no more than a handful of evaluations of task-based teaching” Ellis (2003), p. 323
64. Evaluation in TBLT: Problems of practice Nunan (1989) Crookes & Gass (1993) Willis (1996) Skehan (1998) Lee (2000) Bygate, Skehan, & Swain (2001) Ellis (2003) Leaver & Willis (2004) Nunan (2005) Overview books on TBLT… … with chapters on program evaluation… Ellis (2003)
65.
66. Evaluation in TBLT: An empirical approach to practice Patton (1996) Utilization-focused Evaluation Intended uses Intended users Questions + Methods Understand Improve Educate Demonstrate worth Hold accountable Test theory Empower Teachers Administrators Curriculum writers Learners Parents/public Researchers Funders Values clarification Implementation Process-product Experimentation Synthesis
67. Evaluation in TBLT: Examples in practice Norris (2004) Evaluation of a US college German FL assessment program Van den Branden (to appear) Evaluation of task-based teacher development in Flanders
68. Assessment, measurement, and Evaluation in TBLT: Essential roles TBLT in situ Measurement Theory, Research Assessment Imple-mentation Evaluation Society, Value EMPIRICAL PRACTICE
72. REFERENCES Please see the following web site for a complete pdf file of references and resources from this presentation: http://www2.hawaii.edu/~jnorris/TBLT%20references.pdf
