Ectel sem_info_rec_learning_resources_v6.0_20120921_ma

© author(s) of these slides including research results from the KOM research network and TU Darmstadt; otherwise it is specified at the respective slide
21-Sep-12
Prof. Dr.-Ing. Ralf Steinmetz
KOM - Multimedia Communications Lab
ECTEL__Sem_Info_rec_learning_resources_v6.0_20120921_MA.pptx
Exploiting Semantic Information for Graph-based
Recommendations of Learning Resources
Mojisola Anjorin
Thomas Rodenhausen
Renato Domínguez García
Christoph Rensing
EC-TEL 2012, Saarbrücken
Research
Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on
Basics
Activities
Find Related
Work
Friends
Friends
Friends
Blue Group

KOM – Multimedia Communications Lab 2
Resource-Based Learning

Application Scenario: CROKODIL
CROKODIL is a platform offering support for resource-based learning
§ Semantic Tag Types
§ Activities
§ Learner Groups
& Friendships
§ Recommendations
[Anjorin et al, 2011]
http://demo.crokodil.de

§ Motivation: Resource-based Learning
§ Application Scenario: CROKODIL
§ CROKODIL’s Extended Folksonomy Model
§ Ascore and AInheritScore
§ Evaluation Methodology, Metrics and Results
§ Conclusion & Future Work
Overview

A folksonomy is a quadruple
F:= (U, T, R, Y), where
U – Users
T – Tags
R – Resources
Y ⊆ U × T × R - tag assignment
Folksonomy Model
Research
Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
[Hotho et al. 2006]

CROKODIL Extends the Folksonomy Model …
Research
Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers

… with Semantic Tag Types
[Böhnstedt et al. 2009]
Research
Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Genre
Event
Person
Location
Other
Topic

… with Activities
Research
Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on
Basics
Activities
Find Related
Work

… with Learner Groups and Friendships
Research
Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on
Basics
Activities
Find Related
Work
Friends
Friends
Friends
Blue Group

CROKODIL‘s Extended Folksonomy
FC:= (U, TTyped, R, YT, (A, <), YA, YU, G, friends)
where
U – users
TTyped – typed tags
R – learning resources
YT ⊆ U × TTyped × R – tag assignment
(A, <) – activities with sub-activities
YA ⊆ U × A × R – activity assignment
YU ⊆ U × A – activity membership
assignment
G ⊆ P(U) – groups of learners
friends ⊆ U × U – friendship relation
Research
Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on
Basics
Activities
Find Related
Work
Friends
Friends
Friends
Blue Group

Resource Recommendations for CROKODIL
http://demo.crokodil.de

Graph-based recommender techniques can be classified as
neighbourhood-based collaborative filtering approaches
Graph-based Resource Recommendations
Graph-based
Ranking
Algorithm
Resource Score
r1 0.9
r2 0.7
r3 0.5
r4 0.2
1 1
2 1
P1
P2
P4
P3
3
4
2
1
2
Folksonomy Graph e.g. FolkRank based on
“Random Walk”
of PageRank
Recommendation List
(ranked resources)
[Desrosiers et al. 2011]

Overview

1.  Add activity nodes Vc = VF ∪ A
2.  Add edges:
§ activity assignments (u, r, a)
§ assignments of a user to an
activity (u, a)
§ activity hierarchies (asub , asuper)
4.  Assign weights to edges:
§ w(r,a) = w(r,u) = w(u,a)
= max(|Ut,r|)
§ w(u, a) = max(|Ru,t|)
§ w(asub,asuper) = max(|Ut,r|, |Ru,t|)
5.  Run graph-based ranking
algorithm e.g. FolkRank
AScore
[Abel et al, 2011]Inspired by GFolkRank
Extend the Folksonomy Graph F = (V, E) with Activities
Research
Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on
Basics
Activities
Find Related
Work

§ Depending on the tags of a user,
scores are “inherited” over the
activity hierarchy
§ Resources and users assigned
to activities influence the scores
as well
§ Scores are attenuated
depending on activity distance
§  Activity distance between two
activities: the number of hops
from one activity to the other
AInheritScore
[Abel et al, 2011]Inspired by GRank
Leveraging Activity Hierarchies to Calculate Scores
Research
Talk Ranking
Algorithms
Research
Talk Prepare Talk
Read-Up on
Basics
Find Related
Work
...
... ...

Overview

GroupMe! dataset
Evaluation Corpus and Evaluation Metrics
[Abel et al, GroupMe!]
Elements Count
Users 649
Tags 2580
Resources 1789
Groups of
Resources
1143
Posts 1865
Tag assignments 4366
The mean of the Average Precision over
several queries Q
Mean Normalized Precision:
The mean of the Precision@k over several
queries Q
MAP(Q) =
1
|Q|
|Q|

j=1
1
mj
mj

k=1
Precision(Rjk)
Mean Average Precision:
MNP(Q, k) =
1
|Q|
|Q|

j=1
Precisionj(k)
Precisionmax,j(k)
[Manning et al 2008]

Tango
Buenos
Aires
Dancing
Festival
Tango
Buenos
Aires
Dancing
Festival
A post is a Pu,r= {(u,r,t)|(u,r,t) ∈ Y}
For LeavePostOut, the recommendation task
with user as input is harder as with tag as input
Evaluation Methodology: LeavePostOut
[Jäschke et al. 2007]

RTr,t= {(u,r,t)|(u,r,t) ∈ Y}
For LeaveRTOut, the recommendation task
with tag as input is harder as with user as input
Evaluation Methodology: LeaveRTOut
Tango
Buenos
Aires
Dancing
Festival
Tango
Buenos
Aires
Dancing
Festival

A violin plot is a combination of a box plot and a density trace
Visualization of Results with Violin Plots
[Hintze et al. 1998]

A violin plot is a combination of a box plot and a density trace
Visualization of Results with Violin Plots
Median
3rd Quartile
1st Quartile
[Hintze et al. 1998]

Evaluation results with user as input
Evaluation Results for LeavePostOut

Approaches MAP
GFolkRank 0.70
AScore 0.70
AInheritscore 0.47
GRank 0.38
FolkRank 0.19
Popularity 0.00

Evaluation Results for LeaveRTOut

Evaluation Results for LeaveRTOut
Approaches MAP
AScore 0.20
GFolkRank 0.20
FolkRank 0.18
GRank 0.14
AInheritscore 0.11
Popularity 0.02

Exploiting hierarchical activity structures as found in CROKODIL can
improve the ranking of resources for the purpose of recommending
learning resources
§ AScore
§ AInheritscore
Future Work
§ Evaluation using a data set from CROKODIL
§ User Study
§ Hybrid approaches
Conclusion and Future Work
www.crokodil.de

Questions Contact

Statistical Significance Tests – LeavePostOut
More
effective
than à
Popularity Folk
Rank
GFolk
Rank
AScore GRank AInheritScore
Poularity
FolkRank X
GFolkRank X X X X X
AScore X X X X
GRank X X
AInheritScore X X X
Significance matrix of pair-wise comparisons of LeavePostOut results
Based on Average Precision with a significance level of p = 0.05

Statistical Significance Tests – LeaveRTOut
More
effective
than à
Popularity Folk
Rank
GFolk
Rank
AScore GRank AInheritScore
Poularity
FolkRank X X X
GFolkRank X X X X
AScore X X X X X
GRank X X
AInheritScore X
Significance matrix of pair-wise comparisons of LeaveRTOut results
Based on Average Precision with a significance level of p = 0.05

Adapted PageRank
!
!
!

# #
$%'()*+, Tango
0
Buenos
Aires
0
Buenos
Aires
0
Dancing
Festival
0
1
-.
#-.
#-.
-.
PageRank‘s intelligent surfer model
The ranking of a node is determined by how
often the surfer visits the node
Adjoining edges are followed with a certain
probability – determined by the edge weights
The query node acts as the starting point and
focus i.e. the surfer returns to this node with
a certain probability – determined by the
node weights
[Hotho et al. 2006]

Ectel sem_info_rec_learning_resources_v6.0_20120921_ma

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Similar to Ectel sem_info_rec_learning_resources_v6.0_20120921_ma

Similar to Ectel sem_info_rec_learning_resources_v6.0_20120921_ma (20)

Recently uploaded

Recently uploaded (20)

Ectel sem_info_rec_learning_resources_v6.0_20120921_ma