Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Comparing Automatically Detected  Reflective Texts with Human           Judgements   Thomas Daniel Ullmann, Fridolin Wild,...
Traditional methods• Questionnaires                • Time consuming  – Groningen Reflection    Ability Scale (GRAS)       ...
Related approachesLearning analytics•Associative connection between cuewords and acts of cognition•Machine learning=> rela...
Theory: Elements                               of ReflectionDescription of an experience      Personal                    ...
The ArchitectureUllmann, T.D 2011: An architecture for the automated detection of textual indicators of reflection.http://...
Benefits• Allows the mapping from low level  annotations to high level  constructs• Knowledge driven• Explanation of infer...
Example ruleFOR ALL sentences of the document:IF sentence contains a nominal subjectAND IF it is a self-referential pronou...
The experiment• Overarching goal:  – Evaluating the boundaries of automated    detection of reflection• Focus of the paper...
Parameterisation=> Weights for the automated detection of reflection
Text corpus• Text corpus: “The Blog Authorship  Corpus”• Experiment based on subset: 5176 blog  posts• 4.842.295 annotatio...
Questionnaire
Results
Results
Conclusions• Face values indicate the  anticipated difference between  reflective texts and not reflective  texts• Paramet...
Outlook• Further confirmatory testing• Fine-tuning of weights• Re-evaluation of rules and  annotations with larger corpus
Thomas Ullmannt.ullmann@open.ac.ukhttp://twitter.com/ThomasUllmann
Prochain SlideShare
Chargement dans…5
×

Comparing Automatically Detected Reflective Texts with Human Judgements

2 371 vues

Publié le

Slides from my presentation at the Awareness and Reflection in Technology-Enhanced Learning Workshop at the EC-TEL 2012 Conference. For more information about the workshop and the presentation please visit http://teleurope.eu/artel12.

Publié dans : Formation, Technologie, Business
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Comparing Automatically Detected Reflective Texts with Human Judgements

  1. 1. Comparing Automatically Detected Reflective Texts with Human Judgements Thomas Daniel Ullmann, Fridolin Wild, Peter Scott KMi - The Open University 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning 18 September 2012
  2. 2. Traditional methods• Questionnaires • Time consuming – Groningen Reflection Ability Scale (GRAS) • Delayed feedback – Reflective Dialogue Rating Scale • Personal nature of• Manual content reflection analysis – Overview see: Dyment, J. E., & O’Connell, T. S. (2011).=> Automated detection of reflection
  3. 3. Related approachesLearning analytics•Associative connection between cuewords and acts of cognition•Machine learning=> related but not on reflection
  4. 4. Theory: Elements of ReflectionDescription of an experience Personal Critical analysisReflection Frame-of-reference Outcome
  5. 5. The ArchitectureUllmann, T.D 2011: An architecture for the automated detection of textual indicators of reflection.http://ceur-ws.org/Vol-790/
  6. 6. Benefits• Allows the mapping from low level annotations to high level constructs• Knowledge driven• Explanation of inferences
  7. 7. Example ruleFOR ALL sentences of the document:IF sentence contains a nominal subjectAND IF it is a self-referential pronounAND IF the governor of this sentence iscontained in thevocabulary reflective verbsTHEN add fact "Sentence is of type personaluse of reflective vocabulary"
  8. 8. The experiment• Overarching goal: – Evaluating the boundaries of automated detection of reflection• Focus of the paper: – How does automated detection of reflection relate with human judgments of reflection? – What are reasonable weights to parameterise the reflection detector?
  9. 9. Parameterisation=> Weights for the automated detection of reflection
  10. 10. Text corpus• Text corpus: “The Blog Authorship Corpus”• Experiment based on subset: 5176 blog posts• 4.842.295 annotations• 178.504 inferences• Detection: 95 reflective; 54 not reflective ones
  11. 11. Questionnaire
  12. 12. Results
  13. 13. Results
  14. 14. Conclusions• Face values indicate the anticipated difference between reflective texts and not reflective texts• Parameterisation useful
  15. 15. Outlook• Further confirmatory testing• Fine-tuning of weights• Re-evaluation of rules and annotations with larger corpus
  16. 16. Thomas Ullmannt.ullmann@open.ac.ukhttp://twitter.com/ThomasUllmann

×