Seeds of empowerment mobile learning for blinds and visually impaired children

All Rights Reserved by Seeds of Empowerment

Spatial Intelligence and Executive
Spatial Intelligence and Executive
Function of Blinds
in Mobile Learning Environment
Paul Kim
Donggil Song
Arafeh Karimi

1


• The first study was conducted on November
The first study was conducted on November
11, 2010 in SMPK Setapak (Asarama) school
located at Jalan
located at Jalan Genting Kelang 53300 Kuala
Kelang,53300 Kuala
Lumpur, Malaysia
• Participants were 25 total blind children age
Participants were 25 total blind children age
12‐17 years old.

2


Entrance‐ November
11 2010‐ 3 PM
We met the principle and
teachers. They kindly gathered 30
total blind children in the library
for our study.
Our total blind participants
welcomed us warmly and then
welcomed us warmly and then
the teacher asked them to go to
the classroom beside the library.
Non of them used stick to find the
way. Some of them were holding
y g
each others hand for assistance.
After greetings, we explained the
purpose of the study.
We told them: we heard you are
We told them we heard you are
very smart and doing well in
school. We are working on
designing mobile games for you
and this is our device.
and this is our device

3


First Touch
•Then we handed in the
•Then we handed in the
devices to them. As we
didn’t have enough TMs
They shared the devices.
•They started to explore
TMs right away and were
discussing about it with
each other.
each other.
• 2 participants were
holding TMs with 90
degree rotation saying
they feel more
h f l
comfortable in that
position.
p
•We asked them simple
questions like how many
buttons does the device
have ? They started to talk
to each other and then
to each other and then
90% answered correctly .
•They were excited! 4


Exploration

•4 boys figured out
the up, down, right
the up, down, right
left concept of the
buttons very quickly. I
asked them if they
had any experience
h d i
working with
computers and the
answer was positive.
p
They said “they like
computers and they
want to play
computer games !
computer games”!
Later I checked with
their teacher and she
said these guys are
competent in using
computers.
5


Exploration
Different ways of
holding and using
TM is demonstrated
in pictures. We
suggest further
study on usability
and interface
and interface
design.. .

6


Exploration Hand/Mouth/Nose
Exploration‐Hand/Mouth/Nose

We observed 4
cases using their
mouth/nose for
exploration. They
were smelling the
lli th
device.
Any idea?

7


Connection

After the introductory
session, we have divided
session, we have divided
them to groups of 6 to be
able to conduct the study
with our 6 TM and be able
to manage it.
t it
We gave them the device
and asked them: Can you
help us turning on the
help us turning on the
devices?
They first were playing
individually then started to
discuss and collaborate
with each other to solve the
problem.

8


Connection‐ Found it!
Aina, 13yrs old was the first
one figuring out how to turn
on the device after 3 minutes.
She doesn’t have any
Sh d ’t h
experience of playing
computer games .

9


Connection
The audio test was set as the default
program in TM so it started playing
the audible test after Aina turned it
on and everyone were excited
,working harder on their device to
figure out how to turn it on.
fi th t t it
Next one was the boy with gray T‐
shirt (The Wong) with 5min record.
After they solved the problem they
were guiding each other and sharing
the experience.
We talked about the process. They
said “I knew the power is in corner
but I didn’t know I should hold it”

10


Interaction

After everybody learnt how to turn  on TMs it
was interaction time.
First, all of them listened to the first question
and then played with navigation buttons. They
had fun when they were pushing the buttons
and hearing North, South, West, East in their
g , , ,
own language (Malay).  Later they found out
they have to push blue button to confirm their
answer and get feedback. Exciting!
The program didn t include aural instruction
The program didn’t include aural instruction
on how to work with the device,  therefore  it
took them a while to figure out how to do it.
However we gave them time to explore by
themselves. For Next groups we guided them
themselves. For Next groups we guided them
on how to use the device because of the time
issue.
They were fast in finding what each button
does, unfortunately current version of audio
does unfortunately current version of audio
test doesn’t have any reaction for two upper
buttons (green, yellow). They were expecting
to hear something after touching each button.   11


Interaction Level1
Interaction‐Level1

Level 1 questions was described as “easy” by
q y y
the participants.
Almost 85% of them answered to the level 1
q
questions correctly. They were able to follow
y y
the instruction and answer the questions.
No Interaction and discussion happened
among the participants while were working
g p p g
on level 1 questions.
They seemed like they are enjoying it.
All of them knew the cardinal directions.
All of them knew the cardinal directions

12


Interaction Level 2
Interaction‐Level 2
• The moment they heard 180 degree rotation in questions, we observed
expressions like “wow, phww, mmm,oooh…” Mostly from boys.
expressions like “wow phww mmm oooh ” Mostly from boys
• 3 participants (2boy, 1girl) asked us to repeat level 2‐q2,3
• They started talking to each other and discussing the answers. More
collaboration.
• At Level 2 question, 2 girls started demonstrating the scenario. One of
them stood up and literally followed all the instructions such as 180
rotation. Facing North, turning to right, left. She answered all the
questions correctly.
questions correctly
• The other girl was following the instruction by demonstrating lines on
palm of her hand with her index finger .
• One boy (13yrs) started to show signs of frustrations, holding his head
with his hands, moving his leg, he only answered one question correctly.
i h hi h d i hi l h l d i l
• Whenever they received “correct” feedback from the device they were
excited sharing their success stories with their friends.
• Longer response time was reported compare to level 1
Longer response time was reported compare to level 1

13


Interaction Level 3
Interaction‐Level 3

More requests for repetition of
the questions.
the questions.
Demonstrating girls answered
all the questions correctly.
y g
More difficulty following the
scenario specially when
question had two levels of
rotation.
More interaction and
M i i d
discussion on answers. Sharing
success stories.
More incorrect answers
More incorrect answers
More frustration signs but
none of them stopped playing.
g p
Longer response time

14


Reflection
• We asked them who answered to all the questions correctly? 5 of them stood up,
we announced their name and everybody else applaud them and congrats them
• We interviewed them after finishing the data collection. What do you think about
the game, what was the problem,…
• “I liked it”, “I like to play science game, I don’t like geography”, “I want to answer
“I lik d it” “I lik t l i I d ’t lik h ” “I tt
English questions” “Level 3 was hard” “ I want repetition feature” “do you have
more games?” “ what does this button (yellow, green) do”
• Because of lack of time we weren t able to conduct TOL study with all of them so
Because of lack of time we weren’t able to conduct TOL study with all of them so
we decided to play the instruction on computer and receive feedback.
• They listened to the scenario carefully then gave us following comments:
– “too hard to follow”
– “why do you want to put fruits on sticks”
– "hard to imagine" , "why are we moving the fruits", "why we can only move one fruit at a time". One
girl (13yrs old asked me if I have the sticks and fruits somewhere she can touch it" “do you have any
other game?”

15


• Blind children collaborate with each other to solve more complicated problems
• Blinds need an authentic scenario and enjoy problem solving questions.
• Blind children can perform audible tests using TM however the Instruction and
scenario design, narration language, length, and authenticity should be further
studied.
t di d
• Usability and interface design study would add more richness to the experience.
• More study on Blinds’ cognition and comprehension process is required (spatial
intelligence , why did they started demonstrating the scenario physically, how do
intelligence why did they started demonstrating the scenario physically how do
they imagine the aural instruction? )
• The more cognition required the more response time needed.
• Participants requested for more aural feedback when they were interacting with
Participants requested for more aural feedback when they were interacting with
the device.

16


2nd Study: Tower Of London
Game

17


• Study was conducted on 16 November 2010 in
Study was conducted on 16 November 2010 in
SMPK Setapak (Asarama) school located at
Jalan Genting Kelang 53300 Kuala Lumpur
Kelang,53300 Kuala Lumpur,
Malaysia
• Participants were 12 total blind and 1 visually
Participants were 12 total blind and 1 visually
impaired children age 14‐17 years old.

18


Plan
• Due to School holiday number of participants
Due to School holiday, number of participants
was decrease to 12. However quantitative and
qualitative data was collected.
qualitative data was collected
• Quantitative data was collected through
TeacherMate while participants were playing
while participants were playing
with Tower of London game.
• Q li i d
Qualitative data was collected through semi‐
ll d h h i
structured interview and focus group sessions.

19


Entrance
• Upon our arrival we were welcomed by
Upon our arrival we were welcomed by
participants from previous study.
we said Hello guys they replied Hi, where is
we said “Hello guys”, they replied “Hi where is
your TM?”
Then they asked us to wait for them till they call
Th h k d i f h ill h ll
up their friends.

20


Introduction
We didn’t have much to do anymore. Our
little friends managed everything. We
just sat there and observed.
Old group gathered their friends and
passionately explained who we are and
what we do. Then they asked us to give
them some TeacherMates so they can
explain more accurately.
l i l
Next question “Do you have a new game
for us today?”
They invited us to their class and showed
us around.

21


Touch listen touch
Touch listen touch

22


New ideas
New ideas
• While we were waiting for teachers to prepare
While we were waiting for teachers to prepare
the new study setting, we started to discuss
with our friends about their last experience
with our friends about their last experience
with TM and asked them how do they want to
use it? What kind of game do they suggest?
use it? What kind of game do they suggest?
How are they planning to use TM for their
study?

23


New ideas
New ideas
• They started to
They started to
discuss with each
other and sharing
other and sharing
ideas.

24


New ideas
New ideas
After a fun
After a fun
discussion
with lots of
with lots of
jokes and
laugh they
laugh they
concluded:

25


“I like it when I listen to TM and I can control the game I like
I like it when I listen to TM and I can control the game. I like
playing with navigation buttons.”
26


“I like hearing different
I like hearing different
voices, I feel I’m getting to
p p
know new people.”
We had two English
narration with Korean and
Malaysian accent and one
sentence in Spanish, they
were excited about new
accents and were imitating
the accents and laughing
th t d l hi
among themselves .

27


“I want to have all my
school books in TM so I
don’t have to sit and read
all the braills. I can play
with TeacherMate and
listen to my lessons.”
Two other participants
instantly suggested they
will record the lessons for
him!

28


“We like audio
games. They
games ” They
patiently explained
to us how they
interact with the
environment and
emphasized on
p
auditory channel as
one of the most
important methods
important methods
of communication.

“When we listen to any instruction we imagine it in our mind You know
When we listen to any instruction we imagine it in our mind. You know
blind people have a better memory than sighted kids!!!!”
29


Other suggestions about using TM in class
continued as:
continued as:
“We can use TM to do our assignments.” Azli,
14years old asked us how we created the
14years old asked us how we created the
games with audible test and if it is easy for
their teacher to use the audio maker. Then
their teacher to use the audio maker. Then
problem solved.
“we will ask our teacher to record the
we will ask our teacher to record the
assignment in audio test maker and we will
submit our audio files to her”.
Currently they are using braille typewriters to do
their assignment.
30


Tie was excited “ wow, we don’t have to carry
typing machine everywhere anymore!”

TeacherMate vs braille typewriter
vs. braille

31


Next idea:
“Can we take all of our exam with TM?
It would be more fun
It would be more fun”
They discussed about pros and cons of taking
exam with TM but at the end agreed that it s a
exam with TM but at the end agreed that it’s a
better option for them if audio maker
supports repeating the question and they are
supports repeating the question and they are
able to revise the answer.

32


New Ideas
New Ideas
Next lesson for us was learning how they replied to
g y p
the “cardinal direction” game.
“I imagined myself walking while following the
instruction”
“I imagined I’m driving a car and am following the
direction to get to the destination.
direction to get to the destination.”
“I imagined leading my friend, holding his hand and
guiding him”
We continued our discussion after playing TOL
game.

33


Tower of London
Tower of London
• To design the audible version of TOL game we followed participatory design
approach and changed the original game based on feedback received from our
blind designers.
• V1‐Original TOL using color to differentiate the donuts‐ Our little fiends
commented that color is not authentic to them and it s rather a label and name
commented that color is not authentic to them and it’s rather a label and name
than a comprehensive concept.
• V2‐Using fruit and skewer concept. You have to move the fruits and rearrange the
order‐ We had to justify to them “why do you want to move them” why did you
j y y y y y
put the fruit on sticks”
• V3‐Using a barbecue party and role playing concept, problem solving‐ Players are
expert chefs who are taking different orders to prepare various arrangements of
chicken, fish and lamb.
hi k fi h d l b
• V4‐ Local version of barbecue party with fish, and shrimp ball and chicken
suggested by total blind participants. (Malay translation)

34


Sample Scenario
Sample Scenario
• Order 1. Start position: The left skewer holds up
O de . Sta t pos t o : e e t s e e o ds up
two foods, chicken and shrimp. The chicken is at
the bottom, and the shrimp is on the chicken. The
middle skewer holds up a fish ball at the bottom.
ddl k h ld f h b ll h b
The right skewer holds up nothing.
• G l
Goal position: The left skewer holds up nothing.
iti Th l ft k h ld thi
The middle skewer holds up two foods, fish ball
and chicken. The fish ball is at the bottom, and
and chicken. The fish ball is at the bottom, and
the chicken is on top of the fish ball. The right
skewer holds up a shrimp.

35


Game Design
Game Design
Original TOL has 5 levels with 52 questions.
Original TOL has 5 levels with 52 questions
For this design we planned to test only 3 levels
with 21 questions.
with 21 questions

36


TOL Study
TOL Study
12 Students
12 Students
gathered to
participate in
participate in
our study

37


TOL Study
TOL Study

We divided the participants to two groups to be able to use
our devices and facilitate observation process.
38


TOL Study
TOL Study
Participants had
a t c pa ts ad
enough time to
explore the TM.
We explained the
l d h
concept and what
they have been
they have been
asked to do and
they eagerly
y g y
started to play the
game.

39


TOL Level 1
TOL‐Level 1
Beginning level of the TOL game. Participants were focusing on
g g g p g
the instruction to figure out how this game works.

40


TOL Level 1
TOL‐Level 1
The level of participation and collaboration at level 1,beginning
of the game, was low and they separately were concentrating
of the game was low and they separately were concentrating
on their tasks.

41


TOL Level 2
TOL‐Level 2
After they passed level 1 and understood the game concept the
p
participation and collaboration started and they looked more
p y
relaxed and seemed they are having fun with the game

42


TOL Level 3
TOL‐Level 3
• Level 3 questions are a bit harder and more
Level 3 questions are a bit harder and more
complicated compare to previous questions
but participants had the highest level of
collaboration and they were talking together
about their performance.
• The TOL game provides feedback to players
and informs them if they replied correctly or
not. Participants showed high level of concern
P i i h d hi h l l f
about correct/incorrect feedback.

43


TOL Level 3
TOL‐Level 3

44


TOL Level 3
TOL‐Level 3

45


Su continued playing till level 5 although
everybody else has left ….
b d l h l f

46


More ideas
More ideas
• After data collection we had a chance of discussing more
g
about their feedback and learning from their suggestion.
• They labeled our game as Geography game (Cardinal
direction) and Mind game (TOL game).
di ti ) d Mi d (TOL )
• 10/12 participants found the mind game more challenging
and interesting rather than geography game
g g g p yg
• All participants asked us about our next game
• 2 participants called up their visually impaired friends to play
with TM.
• 3participants asked for science game (2boy 1 girl) and 8 asked
for more mind game and 1 asked for literacy game.
for more mind game and 1 asked for literacy game

47


More ideas
More ideas
• Natasha: “ I like mind game to improve my memory so I can
g p y y
remember more, it makes my brain smarter”
• All participants suggested to add repeat feature and more
control over the game flow.
t l th fl
• They asked for more group games.

They informed us
about their everyday
activities such as
activities such as
playing “goal ball”,
Chess, riding a bicycle,
listening to radio and
li t i t di d
TV.
48


African stories
African stories
We shared Seeds’
African stories with
f h
them (Jomopira and
Kakama), playing on
Kakama) playing on
iPad and Android
phone. They answered
phone They answered
to all the story
q
questions
correctly.

49


Zambia and India plan
Next we shared Seeds plan to go to India and
Next we shared Seeds’ plan to go to India and
Zambia and design games for blind children. We
explained children s educational situation to them
explained children’s educational situation to them
and our young friends started to propose solutions.

50


They asked many questions such as how old are they, if they don’t
y yq y, y
have typewriter how do they take exam?
What language they speak?
How could they learn more about them?
Why are they facing difficulties in their life?
What courses they like?
What courses they like?
Do you have more stories from them?

51


Zambia and India suggestions
Zambia and India suggestions
“Their teacher can record all of their courses on TM and they can
listen to them all the time, It’s OK if they don’t know braille“
“We can record all of our lessons for them and send it to them”
“They can use TM to practice their courses do assignments take
They can use TM to practice their courses, do assignments, take
exam”
“We can design audio games for them to play after school. Do you
have more mind games?”
h i d ?”
“We can record our stories for them and send on TM”
ey o e ed us t e e p a d suppo t a d p oposed e s g ou
They offered us their help and support and proposed revising our
future games to make it more suitable for blinds!
We were asked if we can give TMs to them to play in school,
unfortunately we didn t have enough devices to share.
unfortunately we didn’t have enough devices to share

52


Next?
• Revising TOL game and create multi‐lingual versions
• More study on Instructional design and narration instruction design for
blinds
• Pilot testing the audio game with children from varied socioeconomic
g g
background
• Increase the sample size
• Design game in different context: science, math, literacy,…
Design game in different context: science math literacy
• Study on collaborative and participatory learning among blinds
• User interface and tactile experience studies
• Study on blinds’ cognitive intelligence, ability and development
S d bli d ’ i i i lli bili dd l

53


Acknowledgement
Acknowledgement
Special thanks to Nurzyanti Mohammad Khan for
Special thanks to Nurzyanti Mohammad Khan for
her generous help in translating and narrating all
the questions and great help in conducting the
q g p g
studies. Thanks to Maryam Sarkhail for her
assistance in conducting the Cardinal direction
study. We couldn’t do any of this without you.
We acknowledge our appreciation toward principle
g pp p p
and teachers of SMPK SETAPAK school specially
MS Rosi.

54

Seeds of empowerment mobile learning for blinds and visually impaired children

Recommended

Recommended

More Related Content

Similar to Seeds of empowerment mobile learning for blinds and visually impaired children

Similar to Seeds of empowerment mobile learning for blinds and visually impaired children (20)

More from Seeds Empowerment

More from Seeds Empowerment (6)

Recently uploaded

Recently uploaded (20)

Seeds of empowerment mobile learning for blinds and visually impaired children