1. Processes, outcomes and metrics
for assessing synchronous and
asynchronous collaboration in
virtual worlds.
Supported by UK Prime Minister’s Initiative - PMI2,
1
and JAIST Japan.

2. • Technology adoption in Higher
Education:
• replicate = zero order change
• explore = first order change
• innovate = second order change
Cuban (2001) Oversold and Underused: Computers in the Classroom. Harvard University Press
2

3. Virtual Worlds + HEI
• Enthusiastic adoption of virtual worlds (e.g. Second Life)
– but most represent a ‘mirror world’ - mimics real world
(avatars sitting down, traditional lecterns, etc)
– such replication = zero order change.
• For first (explore) and second (innovate) order change,
educators require guidance on task construction,
implementation and assessment.
– how?
3

4. Reviewing research on teaching robotics
1) an effective tool for teaching science, engineering, and
technology
2) students who have engineered and programmed robots
are exposed to other disciplines that are important for
robotics, science and engineering
3) exposure to real world conditions with multiple possible
solutions
4) effective teamwork is a significant outcome
5) female students respond positively to working with robots.
Barker and Ansorge (2007) Robotics as a means to increase achievement
scores in an informal learning environment. Journal of Research on
Technology in Education,
39(3), 229-243.
4

5. Methodology
• Use NXT Mindstorms as a means to develop metrics for effective tasks
in a virtual world (e.g. Second Life ... later QWAQ).
• Group A in Japan: Group B in UK.
• Design Experiment (Brown, 1992)
– Iteratively collect and analyze data to develop an hypothesis for
good tasks in virtual spaces.
• Capture all actions (screen capture as video) and link to transcribed
communication.
– measure degree of success of task completion (by comparing
robots): TE + IE
– increase task difficulty
– follow up interviews + Flow questionnaire(Csikszentmihalyi)
5

6. Procedure
• Remotely located science students
• Robotics programming
• Communicating and collaborating in a virtual world.
• Metrics for assessing success of tasks in virtual worlds.
• Different collaborative processes (synchronous and
asynchronous)
• Develop framework for successful implementation of
collaborative tasks in virtual worlds
6

7. 7

8. 8

9. 9

10. 10

11. 11

12. 12

13. 13

14. 14

15. • Preliminary observations:
– Student enjoyment / challenge (affective
factors)
– Metacognition + role-shifting (second-order
ecology)
– Task Effectiveness + Interaction Effort
(anticipated metrics for assessing task
effectiveness)
– Learner engagement and environmental
instability (cognitive load theory)
– Communication (measured by discourse
15

16. Research status
• Aug+Sept 2009:
– tasks between JPN + UK being undertaken
in SL
– repeat in QWAQ
– transcription + tagging + iterative analysis
• Next
– financial - seek further funding
– technical - can an interface between a/any
virtual world and a robot be developed?
– Can you help? Email: michael@fun.ac.jp
16