AIED 2013 (Poster)

1. Intelligent Systems ProgramUniversity of Pittsburgh
Problem Definition
o Concept-level adaptive sequencing is a relatively mature
and well-known approach; however, there are still no
instances of its use in the context of exam preparation.
o Instead of gradual coverage of concepts, exam-time
sequencing should focus on bridging knowledge gaps
while trying to maximize the number of concepts that are
assessed and mastered by completing each suggested
problem.
Knowledge Maximizer
o Knowledge Maximizer (KM) is a concept-based problem
sequencing tool for Java programming exam preparation.
o Goal : providing a sequence of questions to help learner
address gaps in Java knowledge as quickly as possible.
o Learning content: 103 parameterized self-assessment
questions (activity) indexed by ontology concepts with
prerequisite and outcomes concepts.
o KM uses an overlay student model in conjunction with a
concept-level model of Java knowledge represented in
the form of Java ontology.
∑
∑
′
′
=
r
r
M
i
iw
M
i
iwik
K
∑
∑
′
′−
= o
o
M
i
i
M
i
ii
w
wk
I
)1(
1
1
+
−=
t
s
S
Evaluation
o A classroom study conducted for
Java-based undergraduate course.
o KZ was used for final exam
preparation, in addition to QuizGuide
(QG), and Progressor+ (P+), which
were available from the beginning of
the semester for accessing java
questions.
o 14 students had attempts from KM
while 17 students had attempts from
QG and P+.
Despite a remarkable increase
in complex questions in KM,
the success rates across all
systems were comparable
Method
How prepared is the student to do the activity?
More knowledge of prerequisite concepts (K) for an activity
makes it a better candidate for selection by the optimizer.
Mr : prerequisite concepts for the activity
wi’: log-smoothed weight for the concept
ki: level of the learner’s knowledge in i-th concept
What is the impact of the activity?
Impact of a certain activity (I) measures how well it addresses
the current lack of knowledge. An activity with a higher impact is
a better candidate for selection by the optimizer.
Mo: outcome concepts for the activity
Has the user already completed the activity?
Inverse of success rate measures the extent to which the
user needs to repeat the activity.
s : number of the success in activity;
t: total number of attempts in activity
Activity Rank :
)(S
SIKR ++=
-12.80%
-13.70%
11.53%
-15.00%
-10.00%
-5.00%
0.00%
5.00%
10.00%
15.00%
Class QG/P+ KM
Average Relative Progress Percentage
o KM pushes students
toward appropriate
complex questions
which targets more
concepts at once.
o This helps students
fill the gaps in their
knowledge in a more
efficient way.6.20%
43.50%
50.20%
34.60%
45.30%
20.10%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Easy Moderate Complex
Attempts per Question Complexity
KM QG,P+
58%
64%
0%
20%
40%
60%
80%
100%
KM QG,P+
Success rate