Short paper Vibro-Tactile Enrichment Improves Blind User Interaction with Mobile Touchscreens, presented at 'Interact 2013 - Designing for diversity' http://www.interact2013.org/

1. Maria Claudia Buzzi
Marina Buzzi
Barbara Leporini
Maria Teresa Paratore
Vibro-Tactile Enrichment Improves
Blind
User Interaction with Mobile
Touchscreens
INTERACT 2013 – Cape Town, 4-6 September 2013

2. The problem
Touchscreens are smooth
Detecting specific UI elements can be difficult for
blind users:
- No hardware keys
- Different way of interacting via touchscreen
INTERACT 2013 – Cape Town, 4-6 September 2013

3. The problem
Alternative ways to provide content on the touchscreen
are mainly based on speech and vocal channel
Noisy environments
Classes
Meetings
Speeches
Users may prefer something more silent
INTERACT 2013 – Cape Town, 4-6 September 2013

4. Usability Issues
Interacting with a Touch Mobile Device
• Lack of orientation on the UI
– E.g.: when reading an email, the user
has to read the message header
before catching its content
• Lack of logical navigation order
– When navigating content/elements
sequentially via swipe gestures
• Difficulty handling focus
– e.g.: when editing a text field with
several control UI elements
-> focus on the keyboard/edit fields
INTERACT 2013 – Cape Town, 4-6 September 2013

5. The proposal
• Preserve the original UI layout, providing extra
information/feedback for easier identification of the
UI elements or parts
• Applying the “Logical partitioning” of UI areas
• Using Reference Cues (haptic mechanism) to help
the user recognize those parts
• Prototype:
Samsung Galaxy Nexus running Android 4.2
+ Android’s support for accessibility
INTERACT 2013 – Cape Town, 4-6 September 2013

6. The proposal
• Any Android application should be 'natively'
MVC-compliant (logic and presentation are
strongly separated)
• Thus UIs are described by means of XML files
We developed a simple add-on with
customizable behavior in terms of accessibility
that can be used to enrich UIs
INTERACT 2013 – Cape Town, 4-6 September 2013

7. Customizable Accessibility Cue
• Graphical object with XML attributes to customize it
through spoken messages, sounds and vibrations
– Sounds/vibrations chosen for suggesting the function
• When inserted into a layout XML file, it is rendered
by Android as a ‘reactive’ colored ball
Points: misleading (too narrow area of the screen)
=> cue ‘stretched’ to become a horizontal strip
INTERACT 2013 – Cape Town, 4-6 September 2013

8. Preliminary User Test
• 2 blind users
• 2 applications - 5 tasks in laboratory
on our prototype (TalkBack enabled)
• Think Aloud protocol
• No indication on cues/markers
– where placed
– how implemented
INTERACT 2013 – Cape Town, 4-6 September 2013

9. Dial Pad
Main interaction areas
1.
2.
3.
4.
5.
status bar (battery indicator, network, time...)
numeric display area
numeric keyboard (to dial the number)
call and delete buttons
navigation bar (home and back buttons)
(4 Customizable Accessibility Cues to separate these areas)
Tasks: (1) dialing a call
(2) checking the phone number on the num. display area

10. Dial Pad: results
Blind users were able to perform the tasks
and reach the desired areas easily and rapidly
Enhancements to improve usability:
1. having each number announced both when
hovering over its button and when selected
2. a “spoken message event” added when
hovering over the number display

11. Email client
3 Main UIs
(1) border the email list
(2) mark the email text
(3) before the soft-keyboard
(to separate it from the editing area)
Tasks: (1) browsing the list of incoming emails
(2) reading an email
(3) composing an email

12. Email client: results
Blind users were able to perform the tasks more easily
than usual
Enhancements to improve the usability:
1. a sound was associated with the event: text field getting
the focus (thus becoming suitable for filling-in)
2. the text fields were announced by a 100-msec-vibration normally -- or a 300-msec-vibration -- if the text field held
the focus (to favor their detection)

13. Conclusion and Future work
• 2 blind users who provided very positive
feedback about the usefulness of the solution
• Future Work:
– Methodology: expanding the number of UIs;
different short sounds and vibrations (e.g.: for
highlighting when focus shifts from one area to
another, etc.)
– User test with a group of blind people (min. 10) to
evaluate their performance interacting with both
the simple and enriched user interfaces
INTERACT 2013 – Cape Town, 4-6 September 2013

14. Thank you 
claudia.buzzi@iit.cnr.it
INTERACT 2013 – Cape Town, 4-6 September 2013