We present an approach that combines semantic metadata and reasoning with a visual modeling tool to enable the goal-driven configuration of smart environments for end users. In contrast to process-driven systems where service mashups are statically defined, this approach makes use of embedded semantic API descriptions to dynamically create mashups that fulfill the user's goal. The main advantage of the presented system is its high degree of flexibility, as service mashups can adapt to dynamic environments and are fault-tolerant with respect to individual services becoming unavailable. To support end users in expressing their goals, we integrated a visual programming tool with our system. This tool enables users to model the desired state of their smart environment graphically and thus hides the technicalities of the underlying semantics and the reasoning. Possible applications of the presented system include the configuration of smart homes to increase individual well-being, and reconfigurations of smart environments, for instance in the industrial automation or healthcare domains.

1. | 1
Simon Mayer
http://people.inf.ethz.ch/mayersi
Simon Mayer, Nadine Inhelder, Ruben Verborgh,
Rik Van de Walle, and Friedemann Mattern
Internet of Things 2014 Conference | Cambridge, MA | Oct. 7 2014
Configuration of Smart Environments Made Simple
Combining Visual Modeling with Semantic Metadata and Reasoning

2. | 2
Simon Mayer
http://people.inf.ethz.ch/mayersi
 Embedded sensors and actuators measure and actuate
values in the real, physical world and allow for faster
and smarter decisions
 Smart devices allow well-informed decisions
 Umbrella lights up if forecast shows rain
 Smart environments can perform complex tasks
 Assembly line adjusts to current business needs
Smart Devices in a Smart World

3. 3
|
Simon Mayerhttp://people.inf.ethz.ch/mayersi
Enablehuman userstobettermonitorandcontrolentiresmart environments

4. 4
|
Simon Mayerhttp://people.inf.ethz.ch/mayersi
Internet ofThings forbasicnetwork-levelconnectivitybetweensmart devices
Challenge: Isolated“Intranets of Things”on theapplicationlevel
Web ofThings: Application-level convergence
FullyintegratesensorsandactuatorswiththeWorld Wide Web
The IoTandtheWeb ofThings

5. | 5
Simon Mayer
http://people.inf.ethz.ch/mayersi
Integrate Functionality across WoT Devices
Smart
Thermostat
Environmental
Sensor
Smart Umbrella
www
Web
Services

6. | 6
Simon Mayer
http://people.inf.ethz.ch/mayersi
Integrate Functionality across WoT Devices
www
I know that!
Ask me!
Is it going to rain
today?
What’s the
temperature?
76% chance!

7. Embed semantic functional service descriptionsin smart things representations
How?
Describe service API
What?
Describe what a service does

8. | 8
Simon Mayer
http://people.inf.ethz.ch/mayersi
What?
Describe what a
service does
Given a temperature value, I
can set this room to that
temperature!
Temperature Value Modify Room State
How?
Describe service API
푃푟푒푐표푛푑푖푡푖표푛 → 푃표푠푡푐표푛푑푖푡푖표푛

9. | 9
Simon Mayer
http://people.inf.ethz.ch/mayersi
How?
Describe service API
Given a temperature value, I
can set this room to that
temperature!
Temperature Value Modify Room State PUT to thermostat.com
푃푟푒푐표푛푑푖푡푖표푛 → 푃표푠푡푐표푛푑푖푡푖표푛 ∧ 퐻푇푇푃 푅푒푞푢푒푠푡

10. | 10
Simon Mayer
http://people.inf.ethz.ch/mayersi
Given a temperature value, I
can set this room to that
temperature!
Notation3
{
?newTemp a dbpedia:Temperature;
ex:hasValue ?celsiusValue;
ex:hasUnit "Celsius".
}
=>
{
?state :Office44 ex:hasTemperature ?newTemp.
_:request http:methodName "PUT";
http:requestURI
(<http://thermostat.com?temp=>?celsiusValue).
}.
Precondition
Postcondition
Request

11. | 11
Simon Mayer
http://people.inf.ethz.ch/mayersi
Given a temperature value, I
can set this room to that
temperature!
Notation3
{
?newTemp a dbpedia:Temperature;
ex:hasValue ?celsiusValue;
ex:hasUnit "Celsius".
}
=>
{
?state :Office44 ex:hasTemperature ?newTemp.
_:request http:methodName "PUT";
http:requestURI
(<http://thermostat.com?temp=>?celsiusValue).
}.
Postcondition
Request

12. | 12
Simon Mayer
http://people.inf.ethz.ch/mayersi
Given a temperature value, I
can set this room to that
temperature!
Notation3
{
?newTemp a dbpedia:Temperature;
ex:hasValue ?celsiusValue;
ex:hasUnit "Celsius".
}
=>
{
?state :SomePlace ex:hasTemperature ?newTemp.
_:request http:methodName "PUT";
http:requestURI
(<http://thermostat.com?temp=>?celsiusValue).
}.
Request
How can this service
be invoked?

13. | 13
Simon Mayer
http://people.inf.ethz.ch/mayersi
Given a temperature value, I
can set this room to that
temperature!
Notation3
{
?newTemp a dbpedia:Temperature;
ex:hasValue ?celsiusValue;
ex:hasUnit "Celsius".
}
=>
{
?state :SomePlace ex:hasTemperature ?newTemp.
_:request http:methodName "PUT";
http:requestURI
(<http://thermostat.com?temp=>?celsiusValue).
}.

14. | 14
Simon Mayer
http://people.inf.ethz.ch/mayersi
OK. So I know the device URL.
How can I find the functional metadata?

15. | 15
Simon Mayer
http://people.inf.ethz.ch/mayersi
HTTP OPTIONS is a good fit!
“This method allows the client to determine the options and/or requirements
associated with a resource, or the capabilities of a server, (…)” (RFC 2616)
Descriptions are returned using the Link Header (RFC 5988)
OPTIONS example.org/thermostat
Link: <therm.n3>; rel=“describedBy“

16. Semantic reasoning engines can process these descriptions and automatically combine services to achieve a user goal
SemanticMetadata
Aha!

17. This enables the goal-driven configurationof smart environments!
Do *it*!
I canfind out howtodo *it* myself!

18. | 18
Simon Mayer
http://people.inf.ethz.ch/mayersi
I want my environment to
have a temperature of 23°C
You should send a PUT
request to thermostat.com
that contains the number
“23” as payload!

19. | 19
Simon Mayer
http://people.inf.ethz.ch/mayersi
I want my environment to
have a temperature of 23°C
No way!
Only °F
inputs…
This will fail if no compatible
services are available…

20. :temp23 a dbpedia:Temperature;
ex:hasValue"23";
ex:hasUnit"Celsius".
?state a st:State;
log:includes{
:Office ex:hasTemperature:temp23.
}.
"PUT";

21. :temp23 a dbpedia:Temperature;
ex:hasValue"23";
ex:hasUnit"Celsius".
?state a st:State;
log:includes{
:Office44 ex:hasTemperature:temp23.
}.
"PUT";

22. "PUT";
The requestscanbeexecutedbythereasonerortheclient, orcanbeparsedtomobile code….

23. A reasoning engine enables the dynamic creation of service chainsfrom the functional semantic metadata of individual services
SemanticMetadata

25. | 25
Simon Mayer
http://people.inf.ethz.ch/mayersi
Advantages
 Dynamic service composition in smart environments
 Services becoming unavailable, new services emerging…

26. | 26
Simon Mayer
http://people.inf.ethz.ch/mayersi
Advantages
 Dynamic service composition in smart environments
 Rich vocabulary: Lots of ontologies out there

27. | 27
Simon Mayer
http://people.inf.ethz.ch/mayersi
Advantages
 Dynamic service composition in smart environments
 Rich vocabulary
 Extremely efficient
300ms to chain
2000 services…

28. | 28
Simon Mayer
http://people.inf.ethz.ch/mayersi
Advantages
 Dynamic service composition in smart environments
 Rich vocabulary
 Extremely efficient
Chaining 100 services with
up to 1000 options per step

29. | 29
Simon Mayer
http://people.inf.ethz.ch/mayersi
Challenges
 Not straightforward in stateful smart environments
 A room cannot have two different temperatures at the same time!
 Approach: Helper ontology for state management
Details in the paper!

30. | 30
Simon Mayer
http://people.inf.ethz.ch/mayersi
Challenges
 Not straightforward in stateful smart environments
 Users need to be supported during goal formulation
 Approach: Integration with visual programming language

31. | 31
Simon Mayer
http://people.inf.ethz.ch/mayersi
Challenges
 Not straightforward in stateful smart environments
 Users need to be supported during goal formulation
 Approach: Integration with visual programming language

32. | 32
Simon Mayer
http://people.inf.ethz.ch/mayersi
Challenges
 Not straightforward in stateful smart environments
 Users need to be supported during goal formulation
 Users require feedback on the interactions
 Approach: Verbalization and Visualization

33. | 33
Simon Mayer
http://people.inf.ethz.ch/mayersi
Monitoring Interactions: Verbalization
I want my environment to
have a temperature of 23°C
?A?h?a?!

34. | 34
Simon Mayer
http://people.inf.ethz.ch/mayersi
Monitoring Interactions: Visualization
What is my smart
electricity meter up to?
[Mayer, Hassan, and Sörös, 2014]

35. | 35
Simon Mayer
http://people.inf.ethz.ch/mayersi
Monitoring Interactions: Visualization
So I can actually see
HTTP messages!
[Mayer, Hassan, and Sörös, 2014]

36. | 36
Simon Mayer
http://people.inf.ethz.ch/mayersi
 Making functional semantics and reasoning
work in smart environments
 Facilitation of the goal formulation step and provisioning
of methods to supervise composite services
Summary: Automatic Service Composition

37. | 37
Simon Mayer
http://people.inf.ethz.ch/mayersi
 Scaling up to industrial automation scenarios
 Enabling users to control device interactions
Future Work
YOU shouldn’t be
interacting…

38. | 38
Simon Mayer
http://people.inf.ethz.ch/mayersi
Thank You!
R. Verborgh, V. Haerinck, T. Steiner, D. Van Deursen, S. Van Hoecke, J. De Roo, R. Van de Walle, and J.
Gabarró Vallés: Functional Composition of Sensor Web APIs. Semantic Sensor Networks 2012
S. Mayer and G. Basler: Semantic Metadata to Support Device Interaction in Smart Environments. Web of
Things 2013
S. Mayer, N. Inhelder, R. Verborgh, and R. Van de Walle: User-friendly Configuration of Smart
Environments. PerCom 2014
S. Mayer, Y. N. Hassan, and G. Sörös: A Magic Lens for Revealing Device Interactions in Smart
Environments. SIGGRAPH Asia MGIA 2014
S. Mayer, N. Inhelder, R. Verborgh, R. Van de Walle, and F. Mattern: Configuration of Smart Environments
Made Simple. Internet of Things 2014
Slides are on www.slideshare.net/smnmyr

39. | 39
Simon Mayer
http://people.inf.ethz.ch/mayersi
 http://andrewbleakley.com
 http://www.webmarchand.com
 http://www.patentspostgrant.com/
 http://www.orkin.com
 http://sweetclipart.com
 http://la-matrice.org
 http://learn.randolph.k12.ma.us
 http://theparisreview.org
 http://bestclipartblog.com
 http://freepik.com
 http://http://www.rugdots.com
 http://www.productdose.com
 http://westwoodblinds.co.uk
 http://www.ambientdevices.com
 http://www.pats.ua.ac.be
 http://www.vesternet.com
 http://www.rovingnetworks.com
 http://www.qualitymechanicalservices.com
 http://chantalteakettle.org
 http://blog.eeye.com
 http://cartographersguild.com
 http://haverford.edu
 http://clker.com
 http://stickers-addict.fr
 http://en.wikipedia.org
 http://academycomicalarts.blogspot.com
Image Sources