Urban sensing, participatory sensing, and user activity recognition can provide rich contextual information for mobile applications such as social networking and location-based services. However, continuously capturing this contextual information on mobile devices consumes huge amount of energy. In this paper, we present a novel design framework for an Energy Efficient Mobile Sensing System (EEMSS). EEMSS uses hierarchical sensor management strategy to recognize user states as well as to detect state transitions. By powering only a minimum set of sensors and using appropriate sensor duty cycles EEMSS significantly improves device battery life. We present the design, implementation, and evaluation of EEMSS that automatically recognizes a set of users' daily activities in real time using sensors on an off-the-shelf high-end smart phone. Evaluation of EEMSS with 10 users over one week shows that our approach increases the device battery life by more than 75% while maintaining both high accuracy and low latency in identifying transitions between end-user activities.

1. A Framework for Energy Efficient Mobile Sensing
for Automatic User State Recognition
Yi Wang, Jialiu Lin, Murali Annavaram, Quinn A. Jacobson,
Jason Hong, Bhaskar Krishnamachari and Norman Sadeh

2. OUTLINE
 Motivation
 EEMSS introduction
 System modules
 Case implementation
 Performance evaluation
 Conclusion

3. PEOPLE-CENTRIC MOBILE SENSING
 Health monitoring
 Keeping track of patients/elders
 Social networking
 Automated “Twittering”
 Automated profile updates
 Ringtone adjustment
 Location based services
 Mobile advertising

4. CHALLENGE: LIMITED BATTERY
 Battery capacity of mobile device is low
 Sensors are main source of energy usage
 Blind sensing drains battery soon
 Need intelligent sensor management
 Energy Efficient Mobile Sensing System
(EEMSS) is important
 Possibly trade some detection accuracy for
much longer device lifetime

5. AN EXAMPLE
Office Library

6. SENSOR MANAGEMENT METHODOLOGY
 Only utilize a minimum set of sensors
 Recognize state & detect state transition
 E.g.: No need for GPS when indoor
 Hierarchical management
 Sensors are activated when necessary
 E.g.: Accelerometer -> WiFi scan -> GPS
 If multiple sensors achieve the same task:
Use energy efficient ones
 How to determine sensor energy efficiency?

7. DETERMINE ENERGY EFFICIENCY
 Energy = Power drain × Operating duration
 Sensor power consumptions on N95 devices:
 Keep an eye on sensor operating duration

8. EEMSS SYSTEM ARCHITECTURE

9. USER STATE DESCRIPTION
 Task 1: Identify the states to be detected
 Working/meeting/walking/driving outdoor …
 Task 2: For each state define entry criteria
 Sensing thresholds from one or more sensors
 E.g.: “Outdoor” + “High travel speed” = “Vehicle”
 A state is entered when the criteria are
satisfied
 Task 3: For each state also specify the
necessary sensors to be monitored
 Detect state transition

10. XML BASED STATE DESCRIPTOR

11. DESIGN BENIFITS
 Scalable
 Add or remove a state upon user’s interest
 Different criteria for different individuals
 Real-time update
 User’s habits may change
 Sensing criteria can to be refined in real-time

12. CLASSIFICATION MODULE
 Classification algorithms are the key to high
state recognition accuracy
 Mobile phone is the only sensing resource
 Capability limitations
 Computing issues
 Real-time
 Computing power

13. REAL-TIME AUDIO RECOGNITION
 Based on microphone
sensing
 Focus on the detection of
speech
 Audio features
 Energy
 Silence ratio
 SSCH peak1
 3 outputs:
 Speech
 Loud/Noisy
 Silent
1 B. Gajic and K.K. Paliwal. “Robust speech
recognition in noisy environments based on
subband spectral centroid histograms”

14. SPEECH DETECTION PERFORMANCE
 Algorithm is tested on 1085 speech clips
 91.14 % are classified as speech
 Complexity of algorithm is O(N2)
 N: Number of frequency samples
 Overall processing time of a 4 seconds
sound clip on N95
 ~10 seconds

15. EEMSS CASE IMPLEMENTATION
 Implemented and tested on Nokia N95
 Sensors operated include:
 Accelerometer, Microphone, WiFi detector, and
GPS
 User states are featured by:
 Location, background sound, and user motion
information
 E.g.: “office + quiet + still” => User working in office

16. STATES INTERESTED
State Name State Features Sensors Monitored
Location Motion Background Sound
Working Office Still Quiet Accelerometer, Microphone
Meeting Office Still Speech Accelerometer, Microphone
Office_loud Office Still Loud Accelerometer, Microphone
Resting Home Still Quiet Accelerometer, Microphone
Home_talking Home Still Speech Accelerometer, Microphone
Home_entertaining Home Still Loud Accelerometer, Microphone
Place_quiet Some Place Still Quiet Accelerometer, Microphone
Place_speech Some Place Still Speech Accelerometer, Microphone
Place_loud Some Place Still Loud Accelerometer, Microphone
Walking Keep on changing Moving Slowly N/A GPS
Vehicle Keep on changing Moving Fast N/A GPS

17. SENSOR DUTY CYCLES
 When activated, sensors are turned on and
off periodically
 Trade-offs in sampling:
 Frequent sampling: Wastes energy and
provides redundant information
 Infrequent sampling: Saves energy but low state
detection accuracy
 Current implementation provides small
event detection delay

18. EEMSS PERFORMANCE EVALUATION
 We evaluate EEMSS in terms of:
 Ability to record one’s state in real time
 State recognition accuracy
 Energy efficiency
 User study at CMU & USC with 10 users
 Each user carried Nokia N95 phone as daily
used cell phone with EEMSS running
 Ground truth was manually recorded by
each user
 Fine-grained entries with time and state
records

19. STATE RECOGNITION RECORD
 State records of two sample users
A CMU user A USC user

20. STATE RECOGNITION ACCURACY
 Individual user state recognition accuracy
for all 10 participants:

21. STATE RECOGNITION ACCURACY
 Confusion matrix of recognizing “Walking”,
“Vehicle”, and all other states*
All other
states
Walking Vehicle
All other states 99.17% 0.78% 0.05%
Walking 12.64% 84.29% 3.07%
Vehicle 10.59% 15.29% 74.12%
* “All other states” include “Working”, “Meeting”, “Office_loud”,
“Resting”, “Home_talking”, “Home_entertaining”, “Place_quiet”,
“Place_speech”, and “Place_loud”.

22. DEVICE LIFETIME
 Average device lifetime comparison
 More than 75% gain compared to existing
systems

23. EEMSS ENERGY USAGE AT A GLANCE
 User worked in office, then walked to library
and stayed (20 min empirical interval)

24. CONCLUSION
 User state recognition based on mobile
sensing is popular
 Energy efficiency is required due to low
device battery capacity
 Our sensor management methodology:
 Utilizing minimum number of sensors to
accomplish sensing tasks
 Manage sensors hierarchically
 EEMSS achieves good state recognition
accuracy and energy efficiency

25. THANK YOU!