Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Using Transaction-Level Data to Diagnose
Knowledge Gaps and Misconceptions
Randy Davies, Rob Nyland, John Chapman, Gove Al...
Introduction
• Does assessment data give us an
accurate picture of student knowledge?
• Do assessments leave room for poss...
Types of Data
System Level Data
• Macro-level data: Data for groups of students,
universities. Typically the realm of acad...
Research Questions
1. How can we identify misconceptions from
student log data?
2. Does student log data tell a different ...
Progress Diagram
Individualize
Remediation
Process
Automate
Process
Actionable
Information
Manual Analysis
Phase 1 Phase 2...
Data Collection
• Data collected from online Intro to Excel
Class
– www.myeducator.com
• Assessments are situated and task...
Example Student Log
Knowledge Components
• Syntax
• Cell Referencing
• Calculation
• Absolute
References ($)
Types of Errors
Optimal
Solution
TEXT TEXT TEXT
Used $ when
not needed
=$C11*$C$8
=C11*$C$8
Major Issue
Failed to use
$ wh...
Transaction Level Data Example
D11
Error
rating D20
Error
ratin
g
Optimal Solution =C11*C$8 =F19*C$13/12
Step 1 =D11*C8 .5...
Knowledge Gap Analysis Results
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Solution Process Final Answer
First Attempt
2nd Attem...
Knowledge Gap Analysis Results
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Persisted Resolved Emerged No Error
Solution Process...
Knowledge Gap Analysis Results
0%
10%
20%
30%
40%
50%
60%
70%
80%
Persisted Resolved Emerged No Error
Solution Process
Fin...
Future Work
• Automating the process of discovering
patterns in student answers
• Give feedback to the student based on
th...
Questions?
Randy.Davies@byu.edu
robnyland@byu.edu
John.Chapman@byu.net
Prochain SlideShare
Chargement dans…5
×

Learning Analytics Conference 2015 Presentation

1 334 vues

Publié le

In this presentation, we discuss our initial approaches to learning analytics at the transaction level.

Publié dans : Formation
  • You can hardly find a student who enjoys writing a college papers. Among all the other tasks they get assigned in college, writing essays is one of the most difficult assignments. Fortunately for students, there are many offers nowadays which help to make this process easier. The best service which can help you is ⇒ www.HelpWriting.net ⇐
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Learning Analytics Conference 2015 Presentation

  1. 1. Using Transaction-Level Data to Diagnose Knowledge Gaps and Misconceptions Randy Davies, Rob Nyland, John Chapman, Gove Allen Brigham Young University LAK15, Marist College, Poughkeepsie, NY @robnyland @chapmjs
  2. 2. Introduction • Does assessment data give us an accurate picture of student knowledge? • Do assessments leave room for possible student misconceptions? • What could be the possible problems with these student misconceptions?
  3. 3. Types of Data System Level Data • Macro-level data: Data for groups of students, universities. Typically the realm of academic analytics. Individual Level Data • Assessment Data: Data about student performance in class activities Transaction Level Data • The individual transactions that create an assessment. Step level data.
  4. 4. Research Questions 1. How can we identify misconceptions from student log data? 2. Does student log data tell a different story than final answer data?
  5. 5. Progress Diagram Individualize Remediation Process Automate Process Actionable Information Manual Analysis Phase 1 Phase 2 Phase 3
  6. 6. Data Collection • Data collected from online Intro to Excel Class – www.myeducator.com • Assessments are situated and task-based • Step-level data for each task is captured
  7. 7. Example Student Log
  8. 8. Knowledge Components • Syntax • Cell Referencing • Calculation • Absolute References ($)
  9. 9. Types of Errors Optimal Solution TEXT TEXT TEXT Used $ when not needed =$C11*$C$8 =C11*$C$8 Major Issue Failed to use $ when it was needed =C11*C8 =$C8*$C11 Used $ incorrectly =C$11*C8 =C8*C$11 Type in value to avoid $ use =C11*0.0675 0.50.05 Error Weighting Optimal Solution: =C11*C$8 0.5 0.6 Minor Issue
  10. 10. Transaction Level Data Example D11 Error rating D20 Error ratin g Optimal Solution =C11*C$8 =F19*C$13/12 Step 1 =D11*C8 .5 =(F19*C13)/12 .5 Step 2 =C11*C8 .5 =($F$19*$C$13)/1 2 .55 Step 3 =C11*$C$8 .05 =(F19*C15)/12 .5 Step 4 =(F19*C13)/12 .5 Step 5 =F19*($C$13/12) .05 Final Solution =C11*$C$8 .05 =F19*($C$13/12) .05
  11. 11. Knowledge Gap Analysis Results 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Solution Process Final Answer First Attempt 2nd Attempt Amount of Error Detected Major Errors Only
  12. 12. Knowledge Gap Analysis Results 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Persisted Resolved Emerged No Error Solution Process Final Answer Only Major Errors
  13. 13. Knowledge Gap Analysis Results 0% 10% 20% 30% 40% 50% 60% 70% 80% Persisted Resolved Emerged No Error Solution Process Final Answer Including Minor Errors
  14. 14. Future Work • Automating the process of discovering patterns in student answers • Give feedback to the student based on their responses
  15. 15. Questions? Randy.Davies@byu.edu robnyland@byu.edu John.Chapman@byu.net

×