Presentation at CALICO 2015

1. Adaptive
language
learning
Challenges & opportunities
Piet DESMET & Mieke VANDEWAETERE
CALICO 2015

2. ADAPTIVE
INSTRUCTION
CHALLENGES &
OPPORTUNITIES
WHAT
HOW
WHY
USE CASES

3. ADAPTIVE INSTRUCTIONI

4. WHAT IS ADAPTIVE INSTRUCTION?
“the method by which learners
are offered tailored instruction and support,
personalized to the individual
cognitive, affective and behavioral profile
of the learner.”
1

5. WHY ADAPTIVE INSTRUCTION?
Rationale
Learners differ
Cognitive
prior knowledge, metacognition, beliefs, goals, etc.
Affective
motivation, fear, anxiety, etc.
Behavioral
need for help and feedback, gaming the system, etc.
2

6. WHY ADAPTIVE INSTRUCTION?
Goal
To design supportive learning environments that account for individual
differences between learners (Shute & Zapata-Rivera, 2008)
To enhance performance & learning (Shute & Towle, 2003)
 individualized instruction is superior to the one-size-fits-all approach
(Cohen, Kulik, & Kulik, 1982; Kadiyala & Crynes, 1998; Kulik, Kulik, & Bangert-Drowns, 1990)
2

7. HOW TO PROVIDE ADAPTIVE INSTRUCTION?3

8. HOW TO PROVIDE ADAPTIVE INSTRUCTION?3
Vandewaetere, Desmet & Clarebout 2011 / Vandewaetere & Clarebout, 2012

9. HOW TO PROVIDE ADAPTIVE INSTRUCTION?3
Learner
cognition
Learner
behavior
Learner
affect
Vandewaetere & Clarebout, 2012

10. ADAPT TO WHAT?
Learner characteristics
Cognition
 field (in)dependency, prior knowledge, learning style, information
skills, working memory capacity, etc
Affect
 motivation, self-efficacy, frustration; relief, etc
Behavior
 need for help, number of attempts, duration, etc.
3a

11. HOW TO PROVIDE ADAPTIVE INSTRUCTION?3
static
dual pathway
dynamic
Vandewaetere & Clarebout, 2012

12. Static: prior to/after interaction
Eg. Student’s starting level is defined by teacher or pretest
Eg. Update of student’s level after a series of completed
tasks/exercises – after logging out.
Dynamic: during interaction
Eg. Update of student’s level after every completed task/exercise
Eg. Update of student’s parameters after every interaction with system (hint use ->
update in learner model)
Dual pathway
Eg. Combination: starting level is defined by teacher (pre-defined student model) –
during interaction student model is updated
WHEN TO ADAPT?3b

13. HOW TO PROVIDE ADAPTIVE INSTRUCTION?3
sequence
form
content
Vandewaetere & Clarebout, 2012

14. Adaptive curriculum sequencing
- From computerized adaptive testing
- Based on IRT:
- a measurement theory where the probability of a correct answer depends
on person characteristics and characteristics of the items.
 the difficulty of the items is adapted to the demonstrated level of knowledge
 This presumes:
• Difficulty level of exercises is known
• Skills/knowledge level of student can be tracked and sketched reliably
• An item selection algorithm that offers the most suitable exercise (wrt to difficulty)
to the learner at a certain time in the learning process, taking account of the
learners’ knowledge level.
ADAPT WHAT?3c

15. Adaptive form & content representation
- Adaptive content presentation of learning objects
 example: varying degrees of support
- with or without embedded support (eg. hints)
- with several degrees of feedback (eg. from 1|0 to faultspecific)
- with or without annotation of co-learners
- Adaptive form presentation of learning objects
 example: multimodality adjusted according to context
- only text when slow connection
- no audio in noisy environment
- video enhanced with annotations of co-learners when fast connection and
much time
ADAPT WHAT?3c

16. HOW TO PROVIDE ADAPTIVE INSTRUCTION?3
learner-controlled
shared control
program-controlled
Vandewaetere & Clarebout, 2012

17. Program or system-controlled
System reasoner decides what content is offered.
Evaluation:
 Lack of choice lowers motivation and fosters dependence (Hannafin & Rieber,
1989)
 More risk to become dependent of pre-structuralized instruction (Elen, 2000)
 Sparse interaction between learner & environment
 High investment and development costs (eg. ITS)
ADAPT HOW?3d

18. Learner-controlled
Learner selects what content is offered.
Evaluation:
 Not all learners can deal with choice: “the art of choosing”
 Not for novice, low-motivated learners
 Increases learners’ involvement, responsibility and self-regulation
strategies
ADAPT HOW?3d

19. Shared control
System preselects, learner chooses from preselection.
Best of both worlds
ADAPT HOW?3d

20. HOW TO PROVIDE ADAPTIVE INSTRUCTION?3
Vandewaetere & Clarebout, 2012

21.  Device: Mobile vs desktop adaptive learning
 Short duration – long duration
 Short term – long term
 Environment characteristics (eg. noise on train)
 Certification or not
 Quality of internet connection
 Etc.
ADAPT WHEN? CONTEXT3e

22. ADAPTIVE
INSTRUCTION
CHALLENGES &
OPPORTUNITIES
WHAT
HOW
WHY
USE CASES

23. Adaptive item selection based on combination of judgment and
data (What?) [Wauters, Desmet, & Van Den Noortgate, 2012]
- IRT: estimation of item difficulty taking into account a learner’s ability.
 Computationally intensive – a lot of data required
- Proportion correct
- ELO-rating system (Brinkhuis & Maris, 2010)
- Learners’ judgment “How difficult was the presented item to you?”
- One-to-many comparison by learners
- Expert ratings “How difficult do you think will this item be for your students”
This six techniques all provide reasonably accurate estimates of the difficulty of an
item, even with small sample sizes
Wauters K., Desmet P., Van Den Noortgate W. 2012. Item Difficulty Estimation: an Auspicious Collaboration Between Data and
Judgment. Computers and Education. Pergamon Press nr.58 , pp. 1183-1193 , ISSN 0360-1315
USE CASE 1 – ADAPTIVE ITEM SEQUENCINGII

24. Illusion of adaptivity might be as effective as adaptivity
(to what?) [Vandewaetere, Clarebout, Desmet, 2011]
 Adaptive instruction is motivating
 Illusion of adaptive instruction is also motivating
 Learners’ perceptions are important in the relation adaptive
instruction – motivation.
USE CASE 2 – ADAPTIVITY & MOTIVATIONII
adaptive
instruction
↗ learning
outcomes
perception
beliefs
motivation
Vandewaetere, M., Desmet, P., Clarebout, G. (2011). The contribution of learner characteristics in the development of
computer-based adaptive learning environments. Computers in Human Behavior, 27, 118-130.

25. Learner control as a means to provide adaptive instruction
(how?) [Vandewaetere & Clarebout, 2011]
 LC has to be perceived by learners:
 additional instruction of LC strengthens the perception
of control
 higher perception of control is related to higher learning
outcomes and motivation
 Study in language learning – N=165, age 18-20
 English tenses – 3 conditions: NC, LC, LC with additional
instruction of control
USE CASE 3: ADAPTIVITY & LEARNER CONTROLII
Vandewaetere, M., Clarebout, G. (2011). Can instruction as such affect learning? The case of learner control.
Computers and Education, 57(4), 2322-2332.

26. Learner control as a means to provide adaptive instruction
[Vandewaetere, Clarebout, Desmet, 2011]
USE CASE 3II

27. Learner control as a means to provide adaptive instruction
USE CASE 3II

28. Instruction of Learner control as a means to provide adaptive
instruction
 Direct effect of instruction of LC on perceptions, which in
turn were related to motivation
 Additional instruction of control: higher satisfaction with
control, higher interest/enjoyment, higher perceived
competence and higher interest in learning.
USE CASE 3II

29. ADAPTIVE
INSTRUCTION
CHALLENGES &
OPPORTUNITIES
WHAT
HOW
WHY
USE CASES

30. Adaptive feedback/support (adaptive representation) &LC
(What? & How?))
• Shed light on the effect of giving learners control on feedback levels. Different behavior
and use of feedback for learners having low and high prior knowledge, and for learners
with low and high self-regulated learning skills and motivation. Also, we expect the
selection of feedback to be different when learners ask feedback on difficult versus easy
items.
• Hypothesis 1.1: There is a main effect of learners’ prior knowledge on the type of requested feedback. Learners with
low prior knowledge will ask more frequently for detailed feedback as compared to learners with higher prior
knowledge.
• Hypothesis 1.2: There is a main effect of item difficulty level on the type of requested feedback. After completing a
difficult item, learners will ask more frequently for detailed feedback as compared to completing an easy item.
• Hypothesis 1.3: There is an interaction effect between item difficulty and prior knowledge. Learners with higher prior
knowledge will request less times detailed feedback after completing a difficult item than learners having lower prior
knowledge.
• four types of feedback:
• Faultspecific (wrong/correct + if wrong: correct answer + specific error feedback)
• General (wrong/correct + if wrong: correct answer + general attention remark)
• Correct answer (wrong/correct + if wrong: correct answer)
• Binary (wrong/correct)
OPPORTUNITIES: SOME EXAMPLESIII

31. Adaptive media enhancement (adaptive representation)
(What?)
OPPORTUNITIESIII

32.  Different types of enhanced audio-visual input may serve different learning goals (e.g.,
improve comprehension, stimulate learning of formulaic language). Yet, the visualisation can also
be adapted to learners’ profile.
• H 1: Different types of enhancement are chosen in function of the learner profile (proficiency level,
motivation, etc.).
• H 2: Different types of enhancement can be situated on an implicational scale going from less complex to
more complex viewing experiences.
 If the aforementioned hypotheses are confirmed, adaptive viewing paths (such as the one
suggested above) can be established in function of different learner profiles and objectives.
L1
subtitles
L2
subtitles
with L1
gloss
L2
subtitles
L2
keywords

33. CHALLENGESIII
Big data, big opportunities
From manually entering data to online massive storage
From self-reporting data to behavioral data
From single measurements to longitudinal measurements
From inaccessible to everywhere
Research domains:
-Learning analytics
-Educational data mining

34. Hype cycle for education (Gartner, 2013)

35. Contact
Piet Desmet
Piet.Desmet@kuleuven.be
http://wwwling.arts.kuleuven.ac.be/franling_e/pdesmet
be.linkedin.com/in/pietdesmet
@PietDesmet
ITEC
www.kuleuven-kulak.be/itec