Evaluating CALL resources and tools

1. Evaluating CALL
1 23 13

2. Housekeeping
• Start weekly blog (please complete before
Monday classes. Also comment on other blogs.
Blogs linked in Google site).
• Homework this weekend.
• Try to finish your wiki the week after your topic.
• Basic web design training: Friday 2/1 @ 12:30
(bring things for your website…pictures, CV, etc.)
• Presentations today: Karen, Jena, & Dan

3. Questions
• Types of CALL tools
• How do students/teachers use these
resources and in what contexts?
• Defining “evaluation”
• Four approaches to evaluation
• Challenge for class

4. Types of CALL tools
• Early in the history of CALL most software
focused on learner-centered, self-contained
software (for learning on your own).
• Software usually in media format, designed
around teaching/reinforcing a language skill or
skills.
• 90s and beyond: tools for teachers to develop
teaching materials.
• More recently: Using 2.0 apps/tools for
collaborative learning.

5. Types of CALL tools
• Dedicated CALL software (aka tutorial CALL
programs or dedicated CALL, usually paid)
• Dedicated CALL websites (usually free)
• Tools dedicated to teaching materials
• Tools that can be adapted for teaching (social
networking, CMC, multimedia).

6. CALL Dedicated Courseware
• Tell Me More
• Longman English Interactive
• WordSmart (vocabulary)
• Vocaboly (vocabulary, paid and free versions)

7. CALL Dedicated websites
• Randall’s Cyber Learning Lab
• ESL Independent Study Lab (from Lewis & Clark)
• Compleat Lexical Tutor
• Chemnitz Internet Grammar
• Duolingo (free, crowdsourced)
• Livemocha (free & paid, courses with social
networking)
• Busuu (freemium)

8. Tools for teaching
• Online activities (Hot potatoes)
• Flash cards (Anki, Quizlet, Flashcards+)
• Webquests (Webquests.org, Zunal)
• Testing tools (Quia, Yacapaca)

9. Tools/applications to adapt to teaching
• Collaborative tasks (Google docs, wikis)
• Communities (CMC, Second Life)
• Multi-media tools (Prezi, slideshare, podcasting,
screen capture, animators, audio/video
recording, flash games)
• Language input/exposure (Youtube, corpora,
archive.org, news sites, enthusiast press).

10. How do students use these tools &
resources?
• Interact with other learners, native speakers,
• Comprehension: with aided/scaffolded input
or not
• Production: organization/accuracy
• Production: publish
• Explore language

11. How do teachers use these resources?
• Elicit participation and communication.
• Guide learners to engage in meaningful tasks
and collaboration.
• Language practice with integrated skills.
• Assessment (grading, record keeping,
feedback, remediation)
• Present and reinforce content.
• Help achieve course goals?

12. Learning contexts for CALL
• Computer lab: CALL as supplemental,
supervised practice for classroom.
• Self-learning: CALL as sole/primary tutor for
independent learner
• In-class technologies (presentation, CMC,
multi-media)
• Hybrid learning: CALL used in conjunction with
in-class learning.

13. What does evaluation in CALL mean?
Hubbard (2011, p. 408):
1. Investigating a piece of CALL
courseware/website to judge its
appropriateness for a given language learning
setting.
2. Identifying ways it may be effectively
implemented in that setting.
3. Assessing its degree of success and determining
whether to continue use or to make
adjustments in implementation for future use.

14. Approaches to evaluation
1. Checklists
2. Methodological frameworks
3. SLA-based approaches
4. Action research

15. Checklists
• Good starting place for identifying important
factors. Needs tailoring to specific
contexts/purposes
• Large project checklists: NFLRC, ICT4LT
• Son’s (2005) model for evaluating websites…
– Seven sets of criteria
– Checklist w/ 15 criteria

16. Example: Son’s criteria
1. Purpose 9. Reliability
2. Accuracy 10. Authenticity
3. Currency 11. Interactivity
4. Authority 12. Feedback
5. Loading speed 13. Multimedia
6. Usefulness 14. Communication
7. Organization 15. Integration
8. Navigation

17. Methodological frameworks
• Description (not judgment) of CALL materials
and how they relate to larger class
goals/considerations (Hubbard, 1996):
• Approach – underlying theories assumed
• Design – consistent with approach, syllabus,
objectives of the method
• Procedure – techniques/activities to realize
the design.

18. Example: CALICO journal software
review
1. Technological features: installation, reliability,
user interface, etc.
2. Activities (Procedure): instruction,
collaborative, linguistic focus, linguistic skills.
3. Teacher fit (Approach): appropriate to
theories of language learning, SLA
4. Learner fit (Design): linguistic level,
feedback/correction, learning styles &
strategies

19. SLA-based approaches
• Evaluate CALL applications (or tasks) based on
SLA models of language learning.
• Six criteria from Chapelle (2001), Chapelle &
Jamieson (2010), & Jamieson, Chapelle, & Preiss
(2005):
1. Language learning potential: The degree of
opportunity present for beneficial focus on
form.
2. Learner fit: The amount of opportunity for
engagement with language under appropriate
conditions given learner characteristics.

20. SLA criteria for CALL evaluation
3. Meaning focus: The extent to which learners’
attention is directed toward the meaning of the
language.
4. Authenticity: The degree of correspondence
between the learning activity and target
language activities of interest to learners out of
the classroom
5. Positive Impact: The positive effects of the CALL
activity on those who participate in it.
6. Practicality: The adequacy of resources to
support the use of the CALL activity.

21. Action Research
• Classroom-based research
• Can measure learners’ attitudes/experience
with using a particular technology
• Learning outcomes (compare technologies or
traditional methods)
• Simple designs, short time periods, can give
more informed answers to evaluation criteria.

22. Challenge (2 steps)
• While we have evaluation tools for language
learning resources (courseware, websites,
applications), we do not have as well of an
established system for evaluating non-
language-specific tools and resources.
• What general criteria would you include for
evaluating these types of resources?

23. Step 1 (groups)
• Form a group of 3-4.
• Choose an area of language learning: speaking,
listening, reading, writing, integrated skills,
content-based instruction, TBLT.
• Develop a list of important criteria for evaluating
non-LL dedicated resources for this area. Use
handouts and the chapter to inform your ideas.
• You can focus on a specific type of application.
• Create a brief checklist/framework/rubric

24. Step 2
• Individually, choose an application/resource to
evaluate using your criteria.
• You can choose one of the relevant resources
mentioned in class today, from our textbook,
or from the Diigo group.
• Further elaborate your criteria as necessary,
evaluate the resource, and publish your
evaluation on your blog by the weekend.