1. Methods
Realism is emphasized through a number of design choices:
• Egocentric point of view (Figure 1)
• Physical crosswalk switch
• Still images from a real crosswalk (Figure 4)
Calming elements include:
• Muted colour palette
• Simple menu layout and graphics (Figure 3)
Technical details:
• Programmed in C# using Kinect for Windows SDK drivers
• Menu images created in Microsoft PowerPoint
• Still images of street collected using a digital SLR camera
• Images incorporated using Microsoft Visual Studio
• Crosswalk button is array of white ultrabright LEDs with green
plastic covering
Programming logic is illustrated in Figure 2.
Introduction
It is difficult to teach children with high-functioning autism how to
behave and communicate in social situations due to:
• Communication deficits
e.g. eye contact avoidance, social anxiety
• Inability to generalize
e.g. cartoon or dramatization vs. real world1
Educational tools for computers and mobile devices are useful be-
cause they do not rely on face-to-face communication2
. However
these tools do not allow for full-body interaction with a realistic
simulation (i.e. interactive dramatization). Therefore it is difficult
for the child to generalize what they learn to the real world.
Educational tools for the Microsoft Kinect gaming system (Kinect)
have shown promising results for teaching social skills3
. Kinect uses
motion capture to change visual and auditory output in response
to user motion. This allows for full-body participation in a simula-
tion, and can also provide feedback on performance. This makes it
a useful platform for teaching children how to cross the street.
How did the chicken cross the road?
Danielle Charrona
, Jirapat Likitlersuangb
, Rebecca Sinclair, Deanna Sochac
, Kasra Tajdarand
Institute of Biomaterials and Biomedical Engineering, University of Toronto
a
Ontario Cancer Institute, University Health Network, b
Toronto Rehabilitation Institute, University Health Network,
c
Toronto General Hospital, University Health Network, d
The Hospital for Sick Children Research Institute
Results
A tutorial for using a zebra crosswalk was prototyped. The tool dis-
plays a series of still images based on user hand and shoulder mo-
tion. The Kinect can successfully sense and respond to user motion.
Written instructions are displayed throughout the tutorial to guide
the user. We were unable to implement auditory guidance as this
was too technically demanding.
A menu was designed with working, selectable components. It in-
cludes calming elements and is also easily adapted for additional
users and levels.
Future steps include:
• Smoothing animations
• Prototyping additional levels
• Incorporating achievement points
• Beta testing with adult users to assess realism
• Demonstrations for educators and therapists to assess interest
• Case study with autistic children to asses effectiveness
Danielle Charron
d.charron@mail.utoronto.ca
Jirapat Likitlersuang
jirapat.likitlersuang@mail.utoronto.ca
Rebecca Sinclair
rebecca.sinclair@mail.utoronto.ca
Deanna Socha
deanna.socha@mail.utoronto.ca
Kasra Tajdaran
kasra.tajdaran@mail.utoronto.ca
Figure 1. Three-dimensional rendition of egocentric setup including crosswalk button and op-
tional zebra crossing floor mat.
Figure 3. The user menu includes reminders for the student, their achievement points, and
indicates the levels they have completed.
Figure 4. Prompts are provided during tutorial mode, including where to cross, when to start
crossing, and when to look for traffic.
Play introduction sequence
Display start screen
Prompt user to push the crosswalk button
Is button
pushed?
^
Yes
Conclusions
We have demonstrated a prototype educational tool for
teaching safe street crossing. We expect that by integrating
Kinect motion capture with still images our design is more
realistic than available tools.
Furtherdevelopmentandtestingisrequiredbeforetheproto-
typeis readyforcasestudytestingbychildrenwith high-func-
tioning autism and their educators and therapists. However,
we expect that this tool would be a useful intermediate step
in a child’s customized safety program.
Objectives
Our objective is to design a Kinect-based educational tool that
teaches how to safely cross the street. We hypothesize that by us-
ing Kinect our road safety tool will be more realistic than available
electronic tools.
Our design strategy focuses on the needs of children with autism.
Our primary design objective is to include realistic components to
help children generalize.
As part of a larger safety program the tool is required to:
• Follow established safe road crossing teaching strategies
• Progress in level of complexity and address a variety of road
crossing situations
• Provide feedback and warnings
• Make this feedback available to therapists and educators to
monitor progress
• Be self explanatory
^
^
^
Prompt user to look both ways
No >
<
Is user
looking?
Yes
^
^
Prompt user to begin crossing
No >
<
Is user
walking?
Yes
^
^
No >
<
Did user
cross?
Yes^
No>
Display success message
Display tips for success
Figure 2. Program logic during tutorial mode for response to user motion.
Acknowledgements
We wish to thank Dr. Biddiss for the kind use of the Kinect.
References
1. Brim, D., Townsend, D. B., DeQuinzio, J. A. & Poulson, C. L. Anal-
ysis of social referencing skills among children with autism. Res.
Autism Spectr. Disord. 3, 942–958 (2009).
2. Kagohara, D. M. et al. Using iPods(®) and iPads(®) in teaching
programs for individuals with developmental disabilities: a sys-
tematic review. Res. Dev. Disabil. 34, 147–56 (2013).
3. Bartoli, L., Corradi, C., Milano, P. & Valoriani, M. Exploring mo-
tion-based touchless games for autistic children’s learning. in In-
teract. Des. Child. (2013).