The document discusses how mobile devices can leverage context awareness and sensors to improve the user experience. It describes how sensors like accelerometers, gyroscopes, microphones, and location sensors can provide information about the user's situation, environment and activity. With this context, devices can make smarter inferences and behave differently based on factors like location, time of day, activity, and the user's schedule and relationships. This will lead to devices that are less disruptive and more helpful. It also discusses challenges for user experience teams in designing for this new paradigm where inputs are no longer just from the user but also the environment and context.

1. Improving user experience
by leveraging the context
The rise of the mobile sensors
Florent Stroppa
Product Director

2. About me
• studied electronics, optics, computer science, network
architecture and MEMS
• joined the mobile scene 10 years ago and still
passionate about it
• not a graphic designer but a UX fanatic
• work with UX designers, software engineers, product
managers on projects ranging from Voice, SMS services
to iPhone and Web apps

3. Which elements define the
device user experience?
An engineer perspective

4. What form it is?
Mobile phones come in very
different form factors.
While manufacturers are
continually coming up with new
designs, there are several
common categories which can
be identified.
http://www.phonescoop.com/glossary/term.php?gid=4

5. How does it feel?
• Is it heavy or light?
• Is it cold or warm?
• Does it have curves or straight lines?
• Does it look expensive or cheap?
• How does it sound?

6. Which input?
• Does it have a keypad?
• Does it have a keyboard?
• Does it have a touch screen?
• Does it have a trackball?

7. What is the UI philosophy?

8. How fast it is?
• How long does it take to start an app?
• How long does it take to make a call, take a picture?
• How does it respond to user’s input?
• How fast does it change screens?
• How fast does it connect to the network?
• What about network latency?

9. How smart it is?
• Does it predict my input?
• Does it predict my intent?
• Does it know where I am and act upon it?
• Does it do what I want to do?

10. While there have been strong
improvements in device design,
operating systems, user interfaces,
application development and networks
it seems that...

11. Phones are still stupid

12. They are always with us,
in the palm of our hands,
in our pockets or purses
but...

13. while I am speaking

14. while she’s driving

15. while he’s dancing

16. while he’s sleeping

17. They don’t know
about it

18. This is improving...

19. Phones begin
to behave differently
based on...

20. orientation

21. location and directions

22. acceleration

23. This is not enough

24. Phones need to be aware of
the context
date, time, location, ambiance, our agenda
and our relationship with others.

25. What is the context?
Context
Situation Personal Data Patterns
Date and time Contacts
Location Calendar Short-term Long-term
Ambiance User Profile New contact Call logs
Proximity Photos Search Messaging
New locations Previous app uses
Crowd pattern Crowd pattern
Physical Social context
Position Social networks
Activity Local news
State Global news

26. date and time
• Date : day of week, week-end, holidays, seasons
• Time : day or night, alarm clock
• Mash-up with Calendar data

27. it’s not okay to be called

28. it’s okay to be called

29. Location
• Current user location
• Usual user location : home, office, home country
• Contact location : time zone, weather, price of calls
• Roaming: time zone, calling scheme

30. work or holidays?

31. doesn’t locate like work

32. Situation : ambiance
• Sounds : noise level, noise pattern
• Lights : light level, main colours
• Motion : still, slow motion, running

33. sounds like a meeting

34. doesn’t sound like a meeting

35. contacts
• Gender for contextual messages and contact picture
• Mother tongue for predictive text and spell checking
• Group: family, friends, colleagues, customers

36. events
• Day events : meeting, business trip, conference call,
reminders
• Evening : home, outside
• Inconsistencies between current location and
expected location
• Holidays

37. The enablers behind
context-aware applications
The rise of the mobile sensors

38. The enablers
• Processing power
• Mobile sensors
• Apps running in background
• Personal data synchronisation
• Communication patterns
• Artificial Intelligence
• The cloud

39. Mobile sensors
• Location sensors
• Accelerometers
• Gyroscopes
• Microphones
• Camera
• Compass
• Pressure sensors

40. Future is bright for sensors
• In2010, about 1 in 3 smart-phones will have
accelerometers.
• Pressure sensors and gyroscope will follow.
• The market for MEMS will top $1 Billion.

41. Background process
• To understand environment and context, apps
need to run in the background
• No smart context-aware apps without
background process
• Android, the best platform for background
process management. iPhone does not provide
support for developers.

42. The cloud
“The mobile platforms, Android and the
others, are so powerful now that you can
build client apps that do magical things
that are connected with the cloud”
Eric Schmidt, Google
http://www.techcrunch.com/2009/10/28/googles-eric-schmidt-on-magical-potential-of-mobile-cloud/

43. Leveraging them
Context AI
+
Situation
Cloud
Personal Data
Behavior
Learning
Expected
Perception reaction
Pattern
Patterns recognition
Recommendation
Data analysis
Short-term
Long-term Crowd

44. Some samples of
innovative concepts

45. Eye tracking concept
tat - 3D Eye Tracking
Reference : http://www.youtube.com/watch?
v=7SImOIMcMlk&feature=player_embedded

46. Schizophrenic device
MIT Media Lab
Handheld devices showing
different personalities
Reference: http://www.newscientist.com/
article/dn16617-gadget-reads-users-minds-
from-their-grip.html

47. Some Apple patents...
Motion sensitive UI
The filing, entitled Variable Device
Graphical User Interface, describes a
self-adjusting user interface
"For example, if the user wishes to
view the contact information for
'John Adams,' the user touches the
display over the area of the row for
the contact 'John Adams,'" Apple
says. "While the device is moving,
the motion of the device can be
detected. The device can change the
size of the rows of the contacts in
the contact list application to give
the user a larger target area for each
contact. For example, the height of a
row can be increased. This gives the
user a larger touch area with which
to select a contact. In some
implementations, the height of the
toolbar can be increased as well."

48. What does this
mean for user experience
team?

49. UX work will be harder
• We are going from digital to analog, from binary
logic to fuzzy logic
• The input is not only the user. There are also the
environment, the context and the user’s data
• New tools will need to be found to formalise those
new aspects

50. Agile methodology
• Agiledevelopment
• Cross-functional team (ux, ui, tech, marketing)
• Prototyping
• Rapid and iterative process

51. Our experience with
future mobile phones will
be radically different

52. Phones will be
Beautiful & Intelligent

53. Won’t interrupt me
while I am speaking thanks to
Location
Calendar
Clock
Simple AI

54. Will use the speaker
while she’s driving thanks to
Location
Accelerometer
Gyroscope

55. Will vibrate in his pocket
while he’s dancing thanks to
Microphone
Camera
Gyroscope

56. Won’t ring
while he’s sleeping thanks to
Location
Accelerometer
Clock
Microphone :)

57. Thank you