Presentation was made for my talk at World Usability Day Russia 2008

1. Use of Context Information for User Experience Design Mikhail Matveev,National Technology Officer,Microsoft Russia Daniel Kornev,Program Manager,Microsoft Russia Alexander Popov,MSTLab,Moscow State University

2. Agenda Existing examples of context information Context definition Sensors and their applications Applying context information Sensor and Location Platform Market opportunities Summary

3. // Existing examples of context information

4. Apple iPhone Natural User Interface: Gestures Zoom in/out Physical inertia applied to digital objects Context-Aware Interfaces: UI Orientation (changes when physical orientation changes) Light-aware UI Screen uses presence sensor to automatically turn off while on call Automatic switching between tasks with applying priorities (media player pauses while on call and after call resumes playing)

5. Contextual tabs in Office 2007 Natural User Experience: Ribbons for editing special objects like images, charts, tables etc appear only when these objects are selected by user

6. GPS and location-based applications Context-Aware User Experiences: Detection of user’s location Contextual information – POIs near user Directions Traffic and auto-correction of trip Show user’s friends location on map in real mode

7. Sound profiles in HTC Touch Pro Sound Profiles: Alarm can wake you even if you turned off system sound off Sound profiles – normal/vibration/without sound Special profile “Automatic” – switches app behavior to “Vibration” while on meetings (data is populated from Calendar)

8. Existing examples Ambient Light Sensors in cars, GPS and location-aware applications, IM Presence (Skype/Messenger/ICQ) Changing screen layout depending on device’s physical orientation (iPhone) Contextual Tabs inMicrosoft Office 2007 Sound Profiles in HTC Touch Pro …

9. // Context definition

10. Context is… Situation, in which the product is used and situation changes Let’s take a look on definitions of Context given by different companies and organizations, Intel and ISO

11. What is Context? (Intel) © Intel 2008, “Context Aware User Interface”, Intel Developer Forum Context-aware system can be aware of: Environmental Contexts Physical environment Devices Services User context Identity Physical state Physiological state Activities Contacts Preferences Platform Context Location Motion Network Batter Status Platform Features Running Apps Memory usage

12. What is Context? (ISO) Context combines users, their goals and tasks, infrastructure (hardware, software, materials), physical and social environment in which the product is used.* *as defined by ISO 9421-11

13. What are context-aware systems? Context-aware systems are those changing their behavior when the situation changes

14. // Sensors and their applications

15. Sensors Sensoris a device that measures a physical quantity and converts it into a signal which can be read by an observer Sensors make possible for computer devices to detect elements of contexts that are hard to impossible to discover by using traditional methods.

16. Sensors and context-aware solutionsare on the market already + Universities Research Institutes + Partners Software Developers

17. // Applying context information

18. Ideas to think about The computer volume automatically adjusts based on the noise level of the environment. For example, when I’m in a noisy location, my media player sets the volume level up. The display adjusts brightness and contrast based on current lighting conditions. Text, links, and buttons get bigger on my touch-enabled mobile computer when I’m on-the-go, so they are easier to use. For example, when I’m holding my device in hands, the text and images on the screen automatically get larger. The computer knows when it has moved from work to home and changes the desktop background. It can also hide work-related environment and show game and entertainment applications shortcuts. When the display is rotated, layout changes from landscape to portrait and vice versa. The computer can monitor the health status of me or a person I care about. For example, it can monitor their vital signs, including blood pressure, heart rate, and blood sugar levels.

19. // Sensor and Location Platform

20. Limitations of sensors today Location devices are exposed as virtual COM ports Exclusive application access Not secure Proprietary data formats (NMEA, others) GPS doesn’t work indoors Hard to support multiple technologies at once Sensors are integrated as vertical solutions Applications need to know sensorhardware specifics Limited adoption and scope

21. Sensor and Location Platform New platform for Windows 7 Provides unified driver model for all types of sensor devices Physical sensors (e.g., GPS devices, Light Sensors) Logical sensor (e.g., Wi-Fi triangulation resolver) Provides standard APIs for accessing sensors Sensor API: C++/COM Raw access to any sensor Location API: C++/COM, IDispatch (scriptable) Abstracted API for location data Puts user in control of information disclosure

22. Sensor and Location Platform Gadget or Script Application Application Location API Location IDispatch Interface Sensor API User Location and Other Sensors Control Panel System Sensor Class Extension Sensor Class Extension UMDF Sensor Driver UMDF Sensor Driver Sensor Device Logical Location Sensor (Triangulation)

23. Location Platform benefits Single API call to answer “Where am I?” Independent of provider used todetermine location Synchronous and Asynchronous models Script/automation compatible Automatic transition between providers Most accurate providers have priority Concurrent access for multiple applications Default Location Provided by user as fallback when no other sources are available

24. Types of location data Geographic Data (ILatLongReport) Latitude, longitude, altitude, associated error required Most common format Best format for precise location Can reverse geo-code later Civic Address (ICivicAddressReport) Zip, Country required Most human readable Best for ‘rough’ location estimates,street directions

25. Demo

26. Privacy and Access Control Location data is considered PII User consent is required to share data All sensors are disabled by default Admin rights requiredto enable a sensor Sensors can be configured on a per-user basis “Enable Dialog”invoked by applications

28. http://www.microsoft.com/whdc/sensors/

29. Dan Polivy(Lead PM, Sensor & Location Platform)presenting atMicrosoft PDC 2008:

30. http://channel9.msdn.com/pdc2008/PC25/

31. http://www.aeroxp.org/2008/11/windows-7-sensors-framework-interview-from-pdc/

32. MSDN Forum for developers:

33. MSDN Forum

34. Reverse Code (LatLong↔CivicAddress)with Virtual Earth:

36. http://blogs.msdn.com/gavingear/

37. Jim Travis (SDK Lead):

38. http://blogs.msdn.com/jimtravis/

39. Daniel Kornev:

40. http://blogs.msdn.com/semantics/

41. Jensen Harris (Office):

43. // Market opportunities

44. Maslow's hierarchy of needs

45. Microsoft’s position

46. Business Model: from Innovation to Platform ROI measurement Ecosystem satisfaction Customers satisfaction Investors satisfaction

47. Trends Sensor and actuator markets are experiencing a rapid growth Industry shifts towards active use of sensors and context information in user interface and consumer product development

48. // Summary

49. Summary Context-aware systemsPlatform ContextSensors

50. Call to action Utilize the Windows Sensor and Location platform to build location-aware applications, gadgets and services Implement light-aware UI in your applications to support betterreadability and UX under various environmental conditions Build environmentally-aware applications using additional sensors with the sensor and location platform

51. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. The End