Presentation made for my Ph.D. final exam (dissertation) at Politecnico di Torino, 20 March 2014

1. Interacting with
Smart Environments
USERS, INTERFACES, AND DEVICES
Luigi De Russis
XXVI Cycle

2. Scenario
• Smart Environment
– small world where all kinds of smart devices are
continuously working to make inhabitants’ lives
more comfortable
• Important to investigate the user’s perspective
in interacting with her surrounding
– no predefined interaction situation or context
– interaction can happen casually or accidentally
• Large and sparse field
– the home is the selected target environment
3/24/2014 Interacting with Smart Environments 2

3. Main Problems
• Effective interaction between users and their
Smart Environment is still a challenging aspect
– few solutions exist, in the literature
• People must (and want to) remain in control
– they want to do “the job”
• Most of the existing solutions
– are deeply integrated inside a specific system
– do not present a set of requirements for
reproducibility and further validations
3/24/2014 Interacting with Smart Environments 3

4. Contributions and Goal
3/24/2014 Interacting with Smart Environments 4
This thesis aims
at improving
the interaction
between users and
Smart Environments
– by exploring challenging and
different approaches in key areas
– by providing a set of tools and applications, loosely
coupled with the underlying intelligent system
– based on solid and explicit requirements
– by allowing replicability of the found solutions

5. DOGeye
• Challenge: effective
eye-based interaction
patterns
• Multimodal eye-based
application
– enable people with motor disabilities to control and manage
their homes
• Requirements from the COGAIN European Network of
Excellence
• Good results from user testing with both experienced
and not experienced eye-tracker users
3/24/2014 Interacting with Smart Environments 5
Publication
• D. Bonino, E. Castellina, F. Corno, L. De Russis, “DOGeye: Controlling your
Home with Eye Interaction”, Interacting with Computers, 2011

6. WristHome
• Challenge: effective interaction
pattern with ubiquitous devices
(tabs size)
• Wearable Home Access Point
– low-cost and off-the-shelf
– handle messages coming from the environment
– quick access to commands
• Requirements from literature
• Preliminary user testing and focus group shows interest
in real-world usage
3/24/2014 Interacting with Smart Environments 6
Publications
• D. Bonino, F. Corno, L. De Russis, “dWatch: a Personal Wrist Watch for
Smart Environments”, 3rd International Conference on Ambient Systems,
Networks and Technologies, 2012
• L. De Russis, D. Bonino, F. Corno, “The Smart Home on Your Wrist”,
HomeSys: a Ubicomp workshop, 2013

7. RulesBook
• Challenge: effective visual
environment programming
for end-users
• Rule-based task delegation
– empower end-users to define desired autonomy level
• Requirements from unstructured interviews
– with people living in and managing smart homes
• Preliminary user testing confirms
– selected features
– viability of the approach
3/24/2014 Interacting with Smart Environments 7
Publication
• D. Bonino, F. Corno, L. De Russis, “A User-Friendly Interface for Rules
Composition in Intelligent Environments”, International Symposium on
Ambient Intelligence, 2011

8. WattsUp: User Survey
• Challenge: design of an innovative user interface
– to incentivize responsible and “green” energy
consumption behaviors in domestic environments
• Development of a user survey, published on the
Internet
– presented the designed interface working in 2 different
modalities
– collected information about user understanding, and
opinions or ideas on the proposed interaction paradigms
• 992 replies were received
– suggestions were given to improve the presented
interface
3/24/2014 Interacting with Smart Environments 8
Publication
• D. Bonino, F. Corno, L. De Russis, “Home Energy Consumption Feedback: A
User Survey”, Energy and Buildings, 2012

9. WattsUp
• Challenge: use AI to
enable fine-grained UI
feedback in a
cost-effective way
• An ontology has been designed
– to enable the intelligent system to tackle the survey’s
requirements
• Mobile application
– built upon the design and results of the user survey
• Small user testing of the overall system
– 6 people
– not in a real home (in lab)
– AI results indistinguishable from sensor-based values
3/24/2014 Interacting with Smart Environments 9

10. Summary of Contributions
Name Requirements Area Validation Publication
DOGeye EU project
interaction,
eye-tracking
User testing
Journal
WristHome Literature
interaction,
wearable
User testing
Conference
(2 papers)
RulesBook
Unstructured
interviews
visual
programming,
mobile
User testing
Conference
WattsUp Web Survey
Semantic
Web, mobile
User testing
Journal
3/24/2014 Interacting with Smart Environments 10
Other activities - a book chapter and two journal papers

11. Future Works
• Interactions using sensing and actuating devices
already present in the environment
– e.g., lighting system could provide a viable and
unobtrusive output mean in several conditions
• Wearable Computing for sensing and
communication
– in “medical” settings, like a nursing house (for people with
disabilities)
– for gaming and education
• On-Body Interaction
– relatively new field
– move input/output on the body, e.g., projected user
interfaces or bio-feedbacks
3/24/2014 Interacting with Smart Environments 11

12. Thank you!
QUESTIONS?
Luigi De Russis
luigi.derussis@polito.it
XXVI Ph.D. Cycle

13. License
• This work is licensed under the Creative Commons “Attribution-
NonCommercial-ShareAlike Unported (CC BY-NC-SA 3,0)” License.
• You are free:
– to Share - to copy, distribute and transmit the work
– to Remix - to adapt the work
• Under the following conditions:
– Attribution - You must attribute the work in the manner specified by the
author or licensor (but not in any way that suggests that they endorse you
or your use of the work).
– Noncommercial - You may not use this work for commercial purposes.
– Share Alike - If you alter, transform, or build upon this work, you may
distribute the resulting work only under the same or similar license to this
one.
• To view a copy of this license, visit
http://creativecommons.org/license/by-nc-sa/3.0/
3/24/2014 Interacting with Smart Environments 13