Aucune remarque pour cette diapositive
…d’où un questionnement normal sur « Comment évaluer les apprentissages de tous ces participants » ?
AES: (voir BALFOUR + discuter de la validité)
Donc….(voir diapo suivante)
Existe depuis 1973 (Shermis et al., 2010)
content analysis based on vocabulary measures
proportion of grammar errors
proportion of usage errors
proportion of mechanics errors
proportion of style comments
organization and development scores
features rewarding idiomatic phraseology
Accuracy, consistency, and reliability greater than human expert scoring
Web-based tools that are accessible anytime, anywhere
Scoring of both short-answer and extended response questions
Holistic and analytic scoring and feedback
Scoring capability in more than 20 different languages
Detection of non-legitimate essays, such as those that are:
Lack proper development
Are written in a language other than what was expected
Contain bad syntax
Copy the question
Contain messages of harm
- See more at: http://www.vantagelearning.com/products/intellimetric/#sthash.CNp1vPjp.dpuf
Q: What is the Intelligent Essay AssessorTM?
A: The Intelligent Essay Assessor (IEA) is a powerful internet-based service that has been proven to score written essays as accurately as trained human scorers. IEA scores essays based on content as well as on more mechanical aspects of writing, such as grammar and spelling.
human-centered interaction (HCI)
In all, 2,673 students submitted assignments in the first iteration (Spring 2012), and 3,203 in the second (Fall 2012)
1,000 samples of five assessments per submission avec méthode de Bootstrap
4 à 10 membres du personnel ont évalué au hasard les EPP. Pour les 2 itérations, le personnel ont évalué 99 travaux standardisés. Chaque étudiant à évalué au moins 1 travail standard (un travail standard correspond à 160 évaluations pour établir une note moyenne)
To establish a ground-truth comparison of self and staff grades, each assignment in- cluded 4 to 10 staff-graded submissions in the peer assessment pool (these were ran- domly selected). Across both iterations, staff graded 99 ground-truth submissions. Each student graded at least one ground-truth submission per assignment; a ground-truth assignment had a median of 160 assessments. (Some
Si la matière est mal expliquée, communiquée, l’erreur d’interprétation peut se traduire dans les évaluations.
Self-evaluation in Advanced Power Searching and Mapping with Google MOOCs
« The course authors define complex as problems that require multiple steps, have more than one correct answer, or have multiple ways to achieve the answer »
Students submitted a total of 3,948 assignments.
Out of the 3,853 assignments where students graded themselves, 2,708 (70.3%) awarded themselves full credit. 267 (9.9%) of the full credit submissions were blank or nonsense (e.g. ffwevrew)
After the course closed, course administrators provided researchers with an anonymized sample of assignment submissions. 13 of the course staff (including instructors, teaching assistants, content experts and instructional designers) graded 17% of the scored, accessible assignments. To ensure consistent interrater correlation before grading the sample set, graders trained together, independently evaluating assignments until they reached a point of being able to replicate the grading score across all of the graders. (It took five sample practice assignment-grading sessions to train to this level of consistency.)
16% passent à côté + 13% qui évaluent mal.