2024: Domino Containers - The Next Step. News from the Domino Container commu...
PolyCAFe and Social Learning Support for CSCL in LTfLL
1. PolyCAFe and Social Learning
Support for CSCL in LTfLL
Stefan Trausan-Matu
Traian Rebedea
Mihai Dascalu
Vlad Posea
“Politehnica" University of Bucharest
Computer Science Department
9th December 2010
3. The LTfLL - EU FP7 STREP Project
(2008-2011)
• Open Universiteit Nederland (coordinator)
• The University of Manchester
• Open University UK
• Universiteit Utrecht
• Eberhard Karls Universität Tübingen
• Wirtschaftsuniversität Wien
• Université Pierre-Mendès France, Grenoble
• Politehnica University of Bucharest (PUB)
• Institute for parallel processing of the Bulgarian Academy
• Aurus Kennis- en Trainingssystemen BV
• BIT MEDIA E-learning solution GMBH and CO KG
3 9 December 2010
4. Past International Research Projects
Related to LTfLL
The cognitive paradigm (e.g. Web semantic)
Portable AI Lab (PAIL) – IDSIA Lugano (1994-1995)
PeKADS - EU Copernicus (1994-1996)
LARFLAST – EU Copernicus (1998-2001)
IKF – EU EUREKA (2002-2003)
Towntology – EU COST Action (2005-2008)
The socio-cultural paradigm (e.g. Web2.0)
VMT –USA NSF (2005-2007)
EU-NCIT – EU FP6 (2005-2008)
Cooper – EU FP6 (2005-2007)
4 9 December 2010
5. The Outcomes of the LTfLL Project
Prototypes of next-generation services built on
advanced research on the application of
language technologies in education.
http://www.ltfll-project.org
5 9 December 2010
9. PUB Implementation Team
Prof.dr.ing. Stefan Trausan-Matu
Prof.dr.ing. Valentin Cristea
As.drd.ing. Traian Rebedea
As.drd.ing. Vlad Posea
As.drd.ing. Mihai Dascalu
As.drd.ing. Costin Chiru
As.drd.ing. Dan Mihaila
Ing. Alexandru Gartner
Ing. Erol Chioasca
Students which helped at the implementation:
Dan Banica
Mihai Nicolae
Iulia Pasov
Ionela Voinescu
Iulia Moscalenco
Oana Mihai
Alexandru Georgescu
9 9 December 2010
10. PUB Publications on LTfLL
29 papers, 12 in Proceedings ISI, 23 in
international databases,
2 NLPSL Workshops with Proceedings (Eds. S.
Trausan-Matu, P. Dessus)
4 book chapters (including published at Springer
and Hershey)
10 9 December 2010
11. The Problems Solved by
PolyCAFe
(a Polyphony-Based System for Collaboration
Analysis and Feedback Generation)
11 9 December 2010
12. Chat Conversations with Multiple
Participants
Multiple participants (≥3), conferencing style
Particular features – multiple, parallel discussion
chains !!!
There is a need for
Determining important utterances
Contributions of the participants
Degree of collaboration - inter-animation analysis
12 9 December 2010
13. Example: CSCL assignment
Students had to debate in chat sessions in
groups ranging from 3 to 8
In the first part of the conversation, each student
had to defend a technology by presenting its
features and advantages and criticize the others
by invoking their flaws and drawbacks
In the final part of the chat, they had to discuss on
how they could integrate all these technologies in
a single online collaboration platform
13 9 December 2010
14. CSCL assignment: Problems
How to assist teachers in evaluating students’
work in chats?
Offer assistance to students
Abstraction tools
Automatic feedback
14 9 December 2010
15. Experiments with Chat-based
CSCL
K-12 students solving mathematics problems both
individually and collaboratively in the VMT project at
Drexel University, Philadelphia, USA
Computer Science students at Bucharest “Politehnica”
University, Romania at
Human-Computer Interaction course in Romanian and French –
role playing and debate
Natural Language Processing - role playing and debate
Algorithm Design – problem solving
15 9 December 2010
19. Paradigms about Knowledge
Cognitive Socio-cultural
Newell, Simon Vygotsky, Bakhtin
“Knowledge is in the head” “Knowledge is in the community”
Artificial Intelligence, Theory of Activity,
Natural Language Processing Collaborative systems
Ontologies Folksonomies
Semantic Web Social Web (Web2.0)
Intelligent Tutoring Systems Computer-Supported Collaborative Learning
19 9 December 2010
20. Computer Supported Collaborative
Learning
A new paradigm in learning with computers
(Koshmann, 1999):
Knowledge is constructed socially (Vygotsky)
Induced by the spread of forums, chats, blogs,
wikis and folksonomies learning in (on-line)
virtual teams and/or communities
20 9 December 2010
21. Dialogism – Mikhail Bakhtin
• Basis for the CSCL paradigm (Koschman,
1999)
• “… Any true understanding is dialogic in
nature” (Voloshinov-Bakhtin, 1973)
• Opposed to de Saussure ideas, which are the
basis for Natural Language Processing
Polyphony
Inter-animation of voices
21 9 December 2010
22. The Polyphonic Model of CSCL
(Trausan-Matu, Stahl and Zemel, 2005,
http://mathforum.org/wikis/uploads/Stefan_Interanimation.doc)
A polyphony of voices characterizes any linguistic phenomenon
(Bakhtin) including CSCL chats
Inter-animation (Bakhtin, Wegerif) may be detected in
interactions and it may be used for analyzing collaboration and
assessing learners
Integrating NLP techniques with polyphony identification and
Social Network Analysis may provide a way for analyzing the
contributions of each participant and their collaboration.
Inter-animation and polyphony appears also in non-verbal
interactions
Consider threads (voices which last) rather than analyzing pairs
of utterances
22 9 December 2010
25. Words, voices and threads
Different positions assigned to participants – different
voices
Additional voices – frequent concepts – repeated
words become voices, stronger or weaker
Voices continue and influence each other through
explicit or implicit links.
Voices correspond to chains or threads of utterances:
repeated words
lexical chains
co-references
reasoning or argumentation
rhetorical schemas
25 9 December 2010
26. Analysis Units in the Polyphonic
Model
Words
Utterances
Pairs of utterances (links)
Threads
Voices
Participants
26 9 December 2010
27. Units of Interaction
Echoes of voices
Polyphonic-contrapuntal weaving
Inter-animation
Links between utterances and between words
Links may be:
implicit
explicit
27 9 December 2010
36. Social Network Analysis
• Degree
• Centrality
– Closeness
– Graph
– Eigen Value
• User Ranking
– Google Page Ranking
36 9 December 2010
37. Utterance evaluation
Social • Degree
• Semantic similarity
Qualitative • Predefined topics
• Overall discourse
• NLP Pipe
Quantitative • No of occurrences
mark(u) = ∑ words ( stem ) × (1 + log( no _ occurences )) × emphasis (u ) × social (u )
length
remaining
emphasis (u ) = Sim (u , whole _ document ) × Sim (u , predefined _ keywords )
social (u ) = ∏ (1 + log( f (u ))
all social factors f
( quantitative and qualitative )
37 9 December 2010
38. Collaboration (1)
• Utterance graph
– Explicit links – Attenuation
– Implicit links – Trust
• Social cohesion
• Quantitative collaboration
∑ all links l
with different speakers
attenuatio n(l) * trust(l)
quantitati ve collaborat ion =
total number of links (implicit/ explicit)
38 9 December 2010
39. Collaboration (2)
Qualitative - Gain based collaboration =
ECHO
Personal – individual knowledge building
Links to previous utterances with same speaker
Collaborative – collaborative knowledge
building
Links to previous utterances with different speaker
personal gain(u) = ∑ ((mark(v) + gain(v) ) * similarity(u, v) * attenuation(l) * trust(l))
link l exists between u and v,
v is an earlier utterance and
u and v have same speaker
collaborative gain(u) = ∑ ((mark(v) + gain(v) ) * similarity(u, v) * attenuatio n(l) * trust(l))
link l exists between u and v,
v is an earlier utterance and
u and v have different speakers
39 9 December 2010
41. Services & Widgets
PolyCAFe is an online platform:
Web services (Java and PHP-based)
Web widgets using W3C Widgets1.0 standard
The widgets can be integrated into any web
platform that has a W3C widget container
(e.g. Wookie)
There are services for maintenance tasks and
analysis tasks (process discussion, search,
etc.)
The widgets have been deployed in Elgg
(PLE)
41 9 December 2010
49. Validation Experiment 1.0
• Validation of PolyCAFe 1.0
– 9x students
– 5x tutors, 1x teacher
• Two chats with 4-5 students in a team
• All the students used PolyCAFe for the first time
• Only two tutors have previously used the system
• Students used the automatic feedback to get
insight about their discussion
• Tutors used PolyCAFe to offer manual feedback
49 9 December 2010
50. Validation Results – Tutors
The tutors validated all the instruments with
suggestions to change part of them
Quiz with 35 questions – all of them passed with an
average score between 3.50-5.00 / 5.00
The system is relevant and useful for their
activity
The time for providing final feedback to the
students is definitively reduced (between 30-
50%)
The quality of the feedback is improved
50 9 December 2010
52. Validation Results – Students
The students have validated most of the
instruments
Quiz with 32 questions – 5 where not validated; all
others have scores over 3.66/5.00
In the focus group, they reported that several
misleads have been found using this widgets
These errors or misleads were reported to be
only minor without influencing the overall
feedback
It has been suggested to try and fix them in
order to gain the full trust of the users
52 9 December 2010
53. Validation Results – Students (2)
For example, the students did not validate:
“Overall, I believe that the support for my
learning PolyCAFe (Chat Analysis and
Feedback Service) provides is close enough
to the current support provided by humans. ”
Average = 3.11
Perc. agree = 33%
However, we do not want to provide a
substitute for human evaluation!
53 9 December 2010
55. Validation Results - Widgets
Learners Tutors
Validation Statement
Agreement Agreement
PolyCAFe feedback is useful 100% 100%
PolyCAFe feedback is relevant 63% 80%
Conversation feedback is useful 78% 80%
Conversation visualisation is
89% 100%
useful
Utterance feedback is useful 83% 100%
Participant feedback is useful 78% 100%
Search conversation is useful 61% 100%
55 9 December 2010
56. Validation Experiment 2.0
Validation of PolyCAFe 1.5
25x students in experimental group
10x students in control group
6x tutors in experimental group
2x 7 chats with 5 students in a team
Most of the students used PolyCAFe for the first time
All the tutors have previously used the system
Experiment is still under-way
Preliminary results are very encouraging: very high
correlation between ranking of participants by the system
and by the participants themselves
56 9 December 2010
57. Verification
Accuracy of system ranking of participants
Accuracy of grading utterances
Accuracy of determining implicit links,
collaboration areas, discussion threads
Accuracy for determining speech acts
Accuracy for determining Model of Inquiry classes
57 9 December 2010
59. Conclusions and Future Work
(PolyCAFe)
The system is working well
The system passed the validation with students
and tutors
The visualization widget proved to be the most
useful
The interface might be improved
59 9 December 2010
60. Transferability Issues
• Domain
– The topic of the conversation should be easily
solved using discussions, no graphics or formulas
• Language
– Need for the components of the NLP pipe
– Corpus for training the LSA
– Maybe, a domain ontology
• Activity
– Collaborative activity
– Teams of 4-15 students (in the current design)
60 9 December 2010
62. Learning Scenario
Learner just uses social networking web sites
He connects to friends and possible tutors
Our application indexes the user’s friends and
resources and their peers friends and resources
Based on the data acquired we offer search and
recommendation services
62 9 December 2010
63. Tag and Social Network-based Search
Learner Community Knowledge
= = =
Web 2.0 user Social networking application Relevant content
63 9 December 2010
64. Find Relevant Resources
Understand the learner’s request and lead him to a
resource that is both relevant and trusted as it is
recommended by a member of the community
64 9 December 2010
65. Find Relevant Peers
We search for the right person to offer feedback
and guidance
65 9 December 2010
66. Semantic Indexing of Learning Objects
• Crawled data
converted in semantic
formats
•Vocabularies used:
SIOC, Neuman’s
tagging ontology,
SCOT, MOAT,
SKOS
•*using Amazon EC2
for crawling
66 9 December 2010
67. Search and Recommendations
Search
• Search algorithm based on PageRank R (v )
R(u ) = c ∑
• Folkrank - a user is important if he annotates important (v)
v∈B ( u ) N
resources with relevant (well ranked) tags.
• Fokrank works on a folksonomy - hypergraph - F=(V,E)
where V=U ∪R ∪T and E⊂UxRxT
Recommendation
• Uses clustering of resources, users and tags to
suggest relevant resources
67 9 December 2010
69. Using the Learning Services in a PLE
Search for Search for
resources users
username
Social networking platform
Recommended
Check the readings
learner’s
Relevant tags
profile and their
importance 69 9 December 2010
70. Conclusions and Future Work
(Search and recommendations service)
We use social data to provide feedback for
learners
We offer relevant resources to the user from his
social network
We help learners to find peers or tutors
The work was embedded in Moodle/WebCT and
validated in Bucharest and Utrecht
We’re centralizing the results
70 9 December 2010
