Games, Learning, andStealth Assessment               Valerie Shute              Florida State University     Games for Lea...
Games, Learning, Assessment   Claim 1              Claim 2                   Claim 3        +                      +      ...
Game Plan Define 21st century  competencies & good games Overview evidence-centered  design Describe stealth assessment...
It’s Time for a ChangeThe world has changed a lot in the past 100 years. Education has not.         Classroom photo, 1910 ...
21st       Century Skills Today’s kids need new  skills to be successful  and productive  (e.g., creativity, collabor  at...
CollaborationCritical Thinking                   Systems ThinkingCuriosity                                  Creativity  Gr...
Challenges One problem with  embracing new skills is  the lack of valid &  reliable assessments for  them. Old ways of t...
Games
Good Game Elements1. Interactive problem   solving2. Specific goals/rules3. Adaptive challenges4. Control5. Ongoing feedba...
Gestalt of Games                 Interactivity   Stimuli                            GoalsUncertainty                      ...
Evidence-Centered Design     (e.g., Mislevy, Steinberg, & Almond, 2003)
Assessment Design            Competency Model  What do you want to say about the              person?             Evidence...
Design & Diagnosis
StealthAssessment
Stealth Assessment Features        Seamless & Ubiquitous             Accurate & Rich            Assessment                ...
Stealth Assessment Advances in measurement let us administer  evidence-based assessments to:    Extract ongoing informat...
Elder Scrolls IVOblivion
Elder Scrolls IV: Oblivion First person 3D RPG set in a medieval world Can be one of many characters  (e.g., knight, mag...
Competency Model                                       Success in Oblivion                Cognitive                       ...
Example ECD Models                                                              Competency Model                      Crea...
Action Model w/Indicators Problem: Cross river filled with dangerous fish to get to the cave on the other side.           ...
Indicators Per ActionNovelty: 1 – frequencyEfficiency: Inverse fn (resources, time)Action:         Find a bridge over the ...
Bayes Model—Case 1                     CreativeProblemSolving                     Low          0.60                     Hi...
Wrapping it Up Bayes nets can be used in various ways to  improve learning and performance.     Continuously gather evid...
Summary To address educational challenges and harness potential of  immersive games, I presented an ECD-inspired idea re:...
Thank you!       Questions?      Email: vshute@fsu.eduWebsite: http://www.myweb.fsu.edu/vshute
Really Old Days In the Stone Age, kids  learned how to start a fire  with sticks. Hunting skills  were important too. Th...
Prochain SlideShare
Chargement dans…5
×

What is Stealth Assessment?

6 193 vues

Publié le

G4LI Games for Learning Day at G4C 2011

Publié dans : Formation, Technologie
0 commentaire
5 j’aime
Statistiques
Remarques
  • Soyez le premier à commenter

Aucun téléchargement
Vues
Nombre de vues
6 193
Sur SlideShare
0
Issues des intégrations
0
Intégrations
505
Actions
Partages
0
Téléchargements
91
Commentaires
0
J’aime
5
Intégrations 0
Aucune incorporation

Aucune remarque pour cette diapositive

What is Stealth Assessment?

  1. 1. Games, Learning, andStealth Assessment Valerie Shute Florida State University Games for Learning @ G4C (June 22, 2011)
  2. 2. Games, Learning, Assessment Claim 1 Claim 2 Claim 3 + + +Good games can act Learning is best Stealth assessmentas transformative when it is can collect dynamicenvironments to active, interesting, g evidence of learningsupport skill oal-oriented, and in real-time, atdevelopment and contextualized various grain sizesdeep, meaningful (i.e., features of good (and use info tolearning. games). support learning).
  3. 3. Game Plan Define 21st century competencies & good games Overview evidence-centered design Describe stealth assessment Illustrate approach in a worked example
  4. 4. It’s Time for a ChangeThe world has changed a lot in the past 100 years. Education has not. Classroom photo, 1910 Classroom photo, 2010
  5. 5. 21st Century Skills Today’s kids need new skills to be successful and productive (e.g., creativity, collabor ation, communication). The world in which we live is complex and interconnected (especially compared to the really old days).
  6. 6. CollaborationCritical Thinking Systems ThinkingCuriosity Creativity Grit Empathy
  7. 7. Challenges One problem with embracing new skills is the lack of valid & reliable assessments for them. Old ways of testing won’t work. Learning and succeeding in our complex world can’t be optimally be measured by MC tests. Re-think assessment!
  8. 8. Games
  9. 9. Good Game Elements1. Interactive problem solving2. Specific goals/rules3. Adaptive challenges4. Control5. Ongoing feedback6. Uncertainty7. Sensory Stimuli
  10. 10. Gestalt of Games Interactivity Stimuli GoalsUncertainty Challenges Feedback Control
  11. 11. Evidence-Centered Design (e.g., Mislevy, Steinberg, & Almond, 2003)
  12. 12. Assessment Design Competency Model What do you want to say about the person? Evidence Model What observations would provide best evidence for what you want to say? Task/Action Model Model What kinds of tasks let you make the necessary observations?
  13. 13. Design & Diagnosis
  14. 14. StealthAssessment
  15. 15. Stealth Assessment Features Seamless & Ubiquitous Accurate & Rich Assessment Learner ModelsAligned along 3 dimensions: vertically, horizontally, and temporally
  16. 16. Stealth Assessment Advances in measurement let us administer evidence-based assessments to:  Extract ongoing information from a learner  Make accurate inferences of competencies  React in immediate and helpful ways. Accomplished via automated scoring and machine-based reasoning techniques. When assessment is so seamlessly woven into the fabric of the learning environment that it’s invisible, this is stealth assessment.
  17. 17. Elder Scrolls IVOblivion
  18. 18. Elder Scrolls IV: Oblivion First person 3D RPG set in a medieval world Can be one of many characters (e.g., knight, mage, elf), each who has (or can obtain) various weapons, spells, and tools Primary goal—gain rank & complete quests Quests may include locating a person to obtain info, figuring out a clue for future quests, etc. Multiple mini quests along the way, and a major quest that results in winning the game (100s of hr of game play) Players have the freedom to complete quests in any order
  19. 19. Competency Model Success in Oblivion Cognitive Dispositions Creative Problem Solving Problem Persistence Creativity Solving ReadingAttention Efficiency Novelty Comp Working Background Reasoning Exploratory Reflection Memory Knowledge Skills Behavior
  20. 20. Example ECD Models Competency Model Creative Problem SolvingUnobservables Problem Solving Creativity Efficiency Novelty Evidence Model Scene 1 The Glue Scene 2 Action Model Scene 1 Observables Scene 2 Action Indicators
  21. 21. Action Model w/Indicators Problem: Cross river filled with dangerous fish to get to the cave on the other side. Relevant* Action Novelty Efficiency Swim across the river n = 0.12 e = 0.22 Levitate over the river n = 0.33 e = 0.70 Freeze water with a spell and slide across n = 0.76 e = 0.80 Find a bridge over the river n = 0.66 e = 0.24 Dig a tunnel under the river n = 0.78 e = 0.20 * Relevant refers to any action included in a successful solution.
  22. 22. Indicators Per ActionNovelty: 1 – frequencyEfficiency: Inverse fn (resources, time)Action: Find a bridge over the riverIndicators: Novelty = 1 - 0.34 = 0.66 Efficiency = 1 / [(3 0.4) + (5 0.6)] = 0.24• Resources Used = Weapon (1, fight monster w/sword) + Health (1, damage from monster) + Object (1, magic potion) = 3 resources (weight = 0.4)• Time expended = 5 minutes (weight = 0.6)
  23. 23. Bayes Model—Case 1 CreativeProblemSolving Low 0.60 High 0.40 ProblemSolving CreativityLow 0.64 Low 0.11High 0.36 High 0.89 Efficiency Novelty Low 0.86 Low 0.02 High 0.14 High 0.98 ObservedEfficiency ObservedNovelty 0 to 0.25 1 0 to 0.25 0 0.25 to 0.5 0 0.25 to 0.5 0 0.5 to 0.75 0 0.5 to 0.75 0 0.75 to 1 0 0.75 to 1 1 0.20 0.07 0.78 0.07 Dig a tunnel under the river: e = 0.20; n = 0.78
  24. 24. Wrapping it Up Bayes nets can be used in various ways to improve learning and performance.  Continuously gather evidence for accurate, real-time estimates of comp’s.  Info on competencies used by (a) teachers (to adjust instruction & give good feedback), (b) system (to select new gaming experiences), and/or (c) students (to reflect on how they’re doing). Current estimates of competency levels can be integrated into the game and displayed as progress indicators. This elevates valued competencies to the same level as health and weapons!
  25. 25. Summary To address educational challenges and harness potential of immersive games, I presented an ECD-inspired idea re: • Specifying competencies to be acquired from the game • Defining EMs that link game behaviors to competencies • Updating the learner model regularly Using ECD, stealth assessment, and automated data collection and analysis tools is meant to collect valid evidence of students’ emerging competencies, and reduce teachers’ workload allowing them to focus on fostering student learning. Next steps: adapt content (feedback, difficulty levels, scaffolding, etc.) in game to fit current needs of player, test plug ’n play of models, evaluate learning effects, etc.
  26. 26. Thank you! Questions? Email: vshute@fsu.eduWebsite: http://www.myweb.fsu.edu/vshute
  27. 27. Really Old Days In the Stone Age, kids learned how to start a fire with sticks. Hunting skills were important too. The world in which they lived was fairly simple and flat. Return

×