MuMe Slide M. Wolpers 18 Nov

Context in mobile applications

How to achieve context sensitivity in mobile applications.

Martin Wolpers

© Fraunhofer-Institut für Angewandte Informationstechnik FIT

Agenda

 Introduction of context
 Sensors in mobile devices
 Conclusions based directly on sensor data
 Aggregating sensor data to derive conclusions
 Advanced sensor data processing to create higher order conclusions


Introduction of context

Any ideas?


Context awareness:
the essence of adaptability
Context awareness
 Resource awareness
 Adapt to available resources (connectivity, nearby devices
 Situation awareness
 Adapt to the situation (mode, location, time, event)
 Intention awareness (?)
 Adapt to what the user wants to do
Context awareness is found in humans
 We always adapt our behavior and actions according to the context (i.e. situation)
 Pervasive computing devices that ubiquitously accompany humans (such as
smartphones) must adapt accordingly
 Or risk being disruptive and annoying

 Taken from lecture slides CSE494/598


Defining Context

• Dictionary definition
• “the interrelated conditions in which something exists or occurs”
• Definition for pervasive computing
applications. In First International Workshop on Mobile computing Systems
Schilit, B., Adams, N. And Want, T.R. (1994), Context-aware computing

• “any parameters that the application needs to perform a task without being
explicitly given by the user”

One definition [Schilit et-al. 1994]: Another definition [Abowd & Mynatt]:
 Computing context: connectivity,  Social context: user identity and
communication cost, bandwidth, human partner identities
nearby resources (printers,  Functional context: what is being
displays, PCs)… done, what needs to be done
 User context: user profile, location,  Location context: where it is
nearby people, social situation, happening
activity, mood …
and Applications, pp. 85-90

 Temporal context: when it is
 Physical context: temperature, happening
lighting, noise, traffic conditions …
 Motivation context: why it is
 Temporal context (time of day, happening (purpose)
week, month, year…)
GREGORY D. ABOWD and ELIZABETH D. MYNATT (2000). Charting
 Context history can also be useful Past, Present, and Future Research in Ubiquitous Computing. ACM
Transactions on Computer-Human Interaction, Vol. 7, No. 1, March
2000, Pages 29–58.


An operational context definition

Definition:

Context is any information that can be used to characterise the situation of an entity
(Dey, 2001).

Elements used for the description of context information fall into five categories:
individuality, activity, location, time, relations

The activity predominantly determines the relevancy of other context information in
specific situations.

Location and time primarily drive the establishing of relations to other entities
enabling the exchange of context information among entities.

Based on Zimmermann et.al. 2007, Proceedings of Context 2007


Elements of context


Context Information: Individuality

 captures contextual information strongly related to the entity
 several types of entities possible:
 active and passive
 real and virtual
 mobile, movable, stationary
 human, natural, artificial, group entities


Context Information: Time

 covers temporal information related to the entity
 current time
 alternative representations
 overlay models
 time intervals
 recurring events
 process-oriented view
 historical context information
 access past contextual information
 analyse past contextual information


Context Information: Location

 covers spatial information related to the entity
 physical or virtual
 absolute or relative
 quantitative (geometric) and qualitative (symbolic) representations
 overlay models
 one entity possesses
 one physical quantitative location
 several different qualitative locations
 several different virtual locations


Context Information: Activity

 covers information about activities the entity is involved in
 described by goals, tasks and actions
 tasks are goal-oriented activities and small, executable units
 task models structure task into subtask hierarchies
 goals potentially change very frequently
 low-level and high-level goals
 determines the relevancy of other contextual information


Context Information: Relations

 covers information about relations the entity has established to other entities
 expresses semantic dependencies between two entities
 spatio-temporal coordinates of two entities are key-driver
 several relations can be established to the same entity
 each entity plays a specific role in a relation
 static and dynamic relations
 several types of relations:
 social relations
 functional relations
 compositional relations


Context (cont’d)
Sensor
Other classifications of context: data

 Low-level vs High-level Low-level
context
context relations

 Active vs Passive context individual activity

Putting it all together location

 Gather low-level context time

 Process and generate high- Context
high-level
context
level context processing
 Separate active from passive active passive
context context context

 Adjust
Context-aware
application


Context-Aware Application Design
How to take advantage of this context information?
Schilit’s classification of CA applications:
1. Proximate selection:
1. closely related objects & actions are emphasized/made easier to choose
2. Automatic contextual reconfiguration: adding/removing components or
changing relationships between components based on context
1. Switch to a different operation mode
2. Enable or disable functionality
3. Context-triggered actions: rules to specify how the system should adapt
3. Contextual information and commands: produce different results
according to the context in which they are issued
1. Narrow-down the output to the user using the context
2. Broaden the output to the user using the context


Problems with processing sensor data

From Junehwa Song. Mobile and Sensor OS. MobiSys 2008/TMC 2010/PerCom 2010


The usual approach

Requires costly operations for
 Continuous data updates from sensors
 Continuous context processing
 Complex feature extraction and context recognition
 Continuous change detection
 Repeated examination of numerous monitoring requests



Introducing feedback loops

 Early detection of context changes
 Remove processing cost for continuous context recognition
 Utilize the locality of feature data in change detection
 Reduce processing cost by evaluating queries in an incremental manner
 Turn off unnecessary sensors for monitoring results
 Reduce energy consumption for wireless data transmission


Sensors in mobile devices

 Touch screen  Navigation
 Several accelerometers  Browser history
 Gyroscope  Social networks
 GPS  Calendar
 Wifi  Contacts
 Microphone  Address resolver
 Camera  Music player
 Bluetooth
 Light
 Telephone (Call, SMS)


Conclusions based directly on sensor data

Sensor data generate first level observation data.
Examples
Accelerometer  indication that someone might be moving
Localization + Accelerometer  track of movement activity
Localization + Time  indication that someone might be moving
Localization + Feedback button  someone confirms an activity (e.g. app asks
the student to state that he attended a course after attending the course)
Time + Lightsensor  indication that someone might be outside
Real world examples
Location + Accelerometer + Time  Wake up timer
Location + Time + Calendar  Silence mobile phone, e.g. Tasker


Aggregating sensor data to derive conclusions

Combine sensor data to derive second level observation data.
Examples:
 Location + Contacts + Bluetooth log  Buddies near you; Buddy phone status
 Location + Calendar + Time + Sound  Identify if in a conversation
 Location + Accelerometers  Identify if someone is moving indoors and outdoors
 Time + Location + SMS activity  Identify if someone is waiting for someone else

Real world examples:
 ContextPhone
 VibN
 CenceMe
 Physical Activity measurement
 Time tracking: How do figure out if a task is completed.


The ContextPhone framework
http://www.cs.helsinki.fi/group/context/

(from 2004/2005: runs on Symbian OS 6 and 7 – Really old -- now part of
Google Jaiku http://www.jaiku.com/ )
Already then, most of today’s ideas have been addressed,e.g. using bluetooth
connections to determine how busy an environment is.
Or
Access to status of friends mobile phone:
Friends Phone
My Phone


VibN
http://sensorlab.cs.dartmouth.edu/vibn/
http://www.youtube.com/watch?v=U37G6uzTu5k Sound with iOS
Sound with Android
Using the microphone to collect environment information Sound with HTML5
(carefull, some problems)
Tagging of places with audio and statistics of people present
(To ensure privacy, voices are removed from the recording.)

 Points of Interest identified by sound
recording and time of stay

Uses microphone, localization and
accelerometers

 Note that accelerometer shut down on
iOS if app is in background (not so on
Android)

Good paper showing implementation at
http://sensorlab.cs.dartmouth.edu/pubs/sc
i906e-miluzzo.pdf


CenceMe – sensing and sharing presence
http://metrosense.cs.dartmouth.edu/projects.html#cenceme
http://cenceme.org/
http://www.youtube.com/watch?v=8rDFbTF47PA

Sensing presence captures a user’s status in terms of his activity (e.g., sitting,
walking, meeting friends), disposition (e.g., happy, sad, doing OK), habits
(e.g., at the gym, coffee shop today, at work) and surroundings (e.g., noisy,
hot, bright, high ozone).
iPhone access to calendar
Android access to calendar
Use of sensors:
 Accelerometers identify activity of user (sit, run, walk, etc.).
 Microphone identifies conversation, quite place, loud location, etc.
 Localization delivers web-based additional info like weather, etc.
 Access to contacts and calendar provides indications of with whom you are in
a conversation.


Example problem:
Physical Activity Measurement
using the iPhone
Task: identify the physical activity in terms of standing, sitting, walking,
jogging,moving upstairs and downstairs
Sensor: Accelerometer in mobile device at different places
Problem: Place where mobile device is on the body is unclear
Solution: Best place is the waist. If not possible, use transiton tables from
research, e.g.
 Jennifer R. Kwapisz, Gary M. Weiss, Samuel A. Moore. Activity Recognition
using Cell Phone Accelerometers. SensorKDD ’10, July 25, 2010,
Washington, DC, USA.
 Yuichi Fujiki. iPhone as a Physical Activity Measurement Platform. CHI 2010,
April 10–15, 2010, Atlanta, Georgia, USA.
Accelerometer on the iPhone
Accelerometer on Android


Time tracking – How to...

Solution 1: Ask the worker.
Solution 2: (Semi-) Automatic detection (one possible solution)
 Identify starting and ending events/activities of tasks or assignments
 Ask user to press button when starting a task
 Ask user to define task in terms of sensor input (change of location, result
sent, stop button pressed, participating partners, collaboration events, etc.)
 Integration with Calendar to ensure pausing at unrelated events
 Integrate with Telephone and Mic and Calendar to identify F2F collaboration
is ongoing
 Integrate with SMS to detect asynchronous collaboration
 ...
 Use facebook timeline upload/store data and to visualize activities


Advanced sensor data processing to create higher order
conclusions

A E D C
 Emoticon analysis
 Learning resource context C F E C G
 Basic learning analytics


Emoticon Analysis – Goals and Idea

Detecting positive sentiments from computer mediated communication (CMC)
between chat partners to qualify the degree of positivity in a relationship

 Positive emoticons in CMC do convey positivity and respective emotions
 Take emoticons as a substitute for non-verbal communication. Disregard all
verbal information -> ease and speed of processing

Question:
Does positivity as calculated by emoticon extraction correlate with sympathy?


Emoticon Analysis: Experimental Setup & Indicators

Extract chats from Skype for test users. Anonymize contacts and user
information and store emoticon parameters on central DB
Calculated Positivity value:

 = Positive Emoticon Quotient
 = Global Emoticon Quotient
 = Emoticon Mimicry Quotient

 PEQ relates to positive emoticons per chat session to all chat sessions.
 GEQ relates to emoticon usages per chat session to all chat sessions.
 Mimicry rate grabs the amount of mimiced emoticons between chat partners.
 Scalar weight vector (G) open for modification.

Emoticon Analysis: Evaluation & Results

Questionnaires for participants (N=6)
 Top ten ranking of skype contacts with pseudonyms to guarantee
anonymousity
 Build pairs of partners to detect differences in relationship interpretation

Results
 Calculated top ten ranking of algorithm includes 50% of the most sympathetic
Skype contacts
 Pairing leads to very interesting results showing emoticon use and mimicry
can differ widely in chat communication. Hinting towards personal tendencies
and inequalities in relationships


Paradigmatic Relations
 Background (corpus linguistic)
 Words that occur in similar contexts are commonly semantically related
 Example: beer and wine

 Research question
 Do (learning) objects with similar usage contexts have similar content?

 Approach
 Each object holds a usage context profile comprising all its usage contexts
 A usage context (UC) consists of a pre- and a post-contexts

pre-contexts post-contexts
A E D C UC 1 A D C
E
C F E C G UC 2 C F C G


Paradigmatic Relations

First results using CAM collected in the MACE project:
 Medium correlation between metadata similarity and object
context similarity (0.32), significant due to large sample size (>
65.000.000 object pairs)
 Manual comparison: 92% of the 100 object pairs with the highest
object context similarity are strongly related.

 The found context similarity was in many cases not entailed
in the metadata.


PPP – Data Collection

Engineering program at Universidad Carlos III de Madrid
 C programming course from Sep 6 - Dec 16, 2010 (244 students) and Sep 5 –
Oct 19, 2011 (342 students)
virtual machine with all tools needed, configured by teaching staff
learning management system (.LRN then Moodle) for forums, course material,
etc.
reminder about data collection at every start of the VM (should be used for
course-related work only)
existence of a concrete folder functions as a switch (students can move it easily)
people in charge can be contacted and request for insight and deletion is
possible


PPP – Analysis and Results

Extracting key actions to identify user patterns and tendencies throughout the
whole course
 keywords semantically represent the text they are taken from
 key actions represent the session they are taken from

Year 1: ~120,000 events and 19 event types
 visualization of key actions showed key action sequences clearly pointing to
corrective actions to be deployed
 analysis of errors also showed problems to discuss in class

Year 2: ~125,000 events and 34 event types
 teachers think key actions to be a very useful form of data distillation  use
results for course evaluation
 teachers liked getting better information from the key actions than from the
logs themselves.


0
50
100
150
200
250
300
350
400
12/9/2010
14/9/2010
16/9/2010
18/9/2010
20/9/2010
22/9/2010
24/9/2010
26/9/2010
28/9/2010
30/9/2010
2/10/2010
4/10/2010
6/10/2010

8/10/2010
10/10/2010
12/10/2010
14/10/2010
16/10/2010

number of times the error occurred
18/10/2010
20/10/2010
22/10/2010
24/10/2010
26/10/2010
28/10/2010
30/10/2010
1/11/2010
3/11/2010
5/11/2010
PPP – Example Visualizations Year 1

7/11/2010
9/11/2010
11/11/2010
13/11/2010
15/11/2010
17/11/2010
19/11/2010
21/11/2010
number of students getting the error

23/11/2010
25/11/2010
27/11/2010
29/11/2010

PPP – Example Visualizations Year 2


A final word...

About social media apps
 Used to communicate context
 Used to consume context
 Respect privacy and ensure security of data

Don’t be too overly ambitious:
 Semi-automatic rule-based volume control is an app that sells for 6 US-$.
Don’t try to duplicate it – use it (if possible).
 Joint To-Do lists including calendar access are already existing, e.g. Family
Organizer
 Follow the principles of architecture design:
Copy and improve rather then re-invent.


Questions??


MuMe Slide M. Wolpers 18 Nov

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à MuMe Slide M. Wolpers 18 Nov

Similaire à MuMe Slide M. Wolpers 18 Nov (20)

Dernier

Dernier (20)

MuMe Slide M. Wolpers 18 Nov