Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
COMPUTER SCIENCE
PASTEL: Evidence-based Learning
Engineering Method to create Intelligent
Online Textbook at Scale
Noboru ...
COMPUTER SCIENCE
Current MOOC Challenge
• Lack of individualization
– Ineffective learning (no learning!)
– Disengagement ...
COMPUTER SCIENCE
Current ITS Challenge
• Scalability / Generality
– Too expensive to build
– Mostly good for procedural sk...
COMPUTER SCIENCE
Summary of Challenges
• To overcome the issues of MOOC and ITS, there
is a critical need to innovate a te...
COMPUTER SCIENCE
Our Solution
• Evidence-based learning engineering methods
– PASTEL (Pragmatic methods to develop Adaptiv...
COMPUTER SCIENCE
CyberBook
• Online courseware that provide macro-
adaptive scaffolding
– Problem balancing
– Mastery prac...
COMPUTER SCIENCE
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 7
CyberBook: Example
COMPUTER SCIENCE
CyberBook: Adaptive Scaffolding
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 8
COMPUTER SCIENCE
Skill Name Association
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 9
COMPUTER SCIENCE
CyberBook: Cognitive
Tutor Integration
Matsuda & Shimmei
CyberBook @ Intelligent Textbook WS 2019
10
COMPUTER SCIENCE
Technological Challenges
• Automatic validation of courseware content
• Rapid creation of high-quality sk...
COMPUTER SCIENCE
Technology Innovations
• PASTEL: Evidence-based, iterative learning
engineering methods
Matsuda & Shimmei...
COMPUTER SCIENCE
Evaluation Study
• Does macro-adaptive scaffolding promote learning?
• Middle School Science—Newton’s Law...
COMPUTER SCIENCE
Evaluation Study (Cont.)
• Participants
– 175 middle school science students
– 168 high school geometry s...
COMPUTER SCIENCE
Results: Test Scores
15Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019
COMPUTER SCIENCE
Results – Num. Assessments
• Adaptive >> Non-adaptive for Science, but
not for Math
• No correlation betw...
COMPUTER SCIENCE
Results – Dynamic Link
• Surprisingly low use of dynamic link
– 0.4±1.8 in average
• Students figured out...
COMPUTER SCIENCE
Results – Video Use
• Adaptive == Non-adaptive
• Doer/Non-doer effect was not observed
Matsuda & Shimmei ...
COMPUTER SCIENCE
Results – Hint Use
• Hint on Failure Ratio
• Adaptive >> Non-adaptive for Science, but not for
Math. In p...
COMPUTER SCIENCE
Conclusions
• CyberBook with the macro-adaptive scaffolding facilitated
learning
– Controlling the amount...
COMPUTER SCIENCE
Future Works
• Redo the study with the balanced online
courseware
– Add cognitive tutors to Science
– Add...
Prochain SlideShare
Chargement dans…5
×

PASTEL: Evidence-based Learning Engineering Method to Create Intelligent Online Textbook at Scale

56 vues

Publié le

Presented at the First Workshop on Intelligent Textbooks (Chicago, IL, US; June 25, 2019)

Publié dans : Formation
  • Identifiez-vous pour voir les commentaires

  • Soyez le premier à aimer ceci

PASTEL: Evidence-based Learning Engineering Method to Create Intelligent Online Textbook at Scale

  1. 1. COMPUTER SCIENCE PASTEL: Evidence-based Learning Engineering Method to create Intelligent Online Textbook at Scale Noboru Matsuda & Machi Shimmei Center for Educational Informatics Department of Computer Science North Carolina State University
  2. 2. COMPUTER SCIENCE Current MOOC Challenge • Lack of individualization – Ineffective learning (no learning!) – Disengagement / drop-out • Lack of systematic content creation & validation – Where should we start from? – How can we iteratively make it better? Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 2
  3. 3. COMPUTER SCIENCE Current ITS Challenge • Scalability / Generality – Too expensive to build – Mostly good for procedural skill acquisition • What about conceptual learning? • Robustness of Learning – Luck of learning to solve with justifications Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 3
  4. 4. COMPUTER SCIENCE Summary of Challenges • To overcome the issues of MOOC and ITS, there is a critical need to innovate a technology that – provides adaptive instruction while promoting synergetic learning • An evidence-based curriculum development is essential – to build a large scale online course Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 4
  5. 5. COMPUTER SCIENCE Our Solution • Evidence-based learning engineering methods – PASTEL (Pragmatic methods to develop Adaptive and Scalable Technologies for next generation E-Learning) • Adaptive Online Courseware – CyberBook = MOOC + Cognitive Tutors + Adaptive Control Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 5
  6. 6. COMPUTER SCIENCE CyberBook • Online courseware that provide macro- adaptive scaffolding – Problem balancing – Mastery practice (cognitive tutoring) – Proactive detection of unproductive failure – Scaffolding on failure • Incorrect quiz attempt and/or wheel-spinning Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 6
  7. 7. COMPUTER SCIENCE Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 7 CyberBook: Example
  8. 8. COMPUTER SCIENCE CyberBook: Adaptive Scaffolding Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 8
  9. 9. COMPUTER SCIENCE Skill Name Association Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 9
  10. 10. COMPUTER SCIENCE CyberBook: Cognitive Tutor Integration Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 10
  11. 11. COMPUTER SCIENCE Technological Challenges • Automatic validation of courseware content • Rapid creation of high-quality skill-models • Affordable creation of cognitive tutors • Automatic creation of formative assessments • Reliable prediction of unproductive failure Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 11
  12. 12. COMPUTER SCIENCE Technology Innovations • PASTEL: Evidence-based, iterative learning engineering methods Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 12 SMART WATSON QUADL RADARS RAFINE Skill model discovery Cog tutor authoring Assessment generation Unproductive failure prediction Content validation
  13. 13. COMPUTER SCIENCE Evaluation Study • Does macro-adaptive scaffolding promote learning? • Middle School Science—Newton’s Law – 11 units / 17 videos / 83 quizzes / 0 cog tutor • High School Geometry—Coordinate Geometry – 23 units / 0 videos / 179 quizzes / 14 cog tutors • In-service teacher as a curriculum developer 13Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019
  14. 14. COMPUTER SCIENCE Evaluation Study (Cont.) • Participants – 175 middle school science students – 168 high school geometry students • Pre-test, 3 intervention days, post-test • Stratified RCT on pre-test 14Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019
  15. 15. COMPUTER SCIENCE Results: Test Scores 15Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019
  16. 16. COMPUTER SCIENCE Results – Num. Assessments • Adaptive >> Non-adaptive for Science, but not for Math • No correlation between # assessments and post-test when pre-test was controlled Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 16
  17. 17. COMPUTER SCIENCE Results – Dynamic Link • Surprisingly low use of dynamic link – 0.4±1.8 in average • Students figured out that corresponding resources were almost always on the same page Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 17
  18. 18. COMPUTER SCIENCE Results – Video Use • Adaptive == Non-adaptive • Doer/Non-doer effect was not observed Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 18 (Koedinger et al., 2015)
  19. 19. COMPUTER SCIENCE Results – Hint Use • Hint on Failure Ratio • Adaptive >> Non-adaptive for Science, but not for Math. In particular low-prior students Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 19
  20. 20. COMPUTER SCIENCE Conclusions • CyberBook with the macro-adaptive scaffolding facilitated learning – Controlling the amount of formative assessment and Dynamic link – Only for the middle school Science, but not for high school Math • Hint use on failure among low-prior students was associated with learning Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 20
  21. 21. COMPUTER SCIENCE Future Works • Redo the study with the balanced online courseware – Add cognitive tutors to Science – Add videos to Math • Redesign Dynamic Link Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 21

×