The document describes a system called PERSEO that allows for personalization of smart environments through mobile devices and smart objects. It uses a mobile-instrumented interaction method where a mobile device acts as a trigger and orchestrator. A specific use case of a "smart night table" is presented, which allows controlling devices like audio, photo frames, TV through a tablet. The architecture uses DLNA standards for content sharing and rendering between devices. An example workflow is also shown.

1. Grupo de Procesado de Datos y Simulación
ETSI de Telecomunicación
Universidad Politécnica de Madrid
PERSEO: a system to
Personalize the Environment Response
through Smart phonEs and Objects
SENAmI 2013
5th International Workshop on Smart Environments and Ambient Intelligence
(presented by Josué Iglesias)
Luca Bergesio, Íñigo Marquínez, Ana M. Bernardos, Juan A. Besada, José R. Casar
luca.bergesio@grpss.ssr.upm.es

2. contents
 introduction and motivation
 interaction method fundamentals
 ‘smart night table’ use case
 functionalities
 architecture & implementation
 example
 video-demonstration
 discussion and future works
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8

3. contents
 introduction and motivation
 interaction method fundamentals
 ‘smart night table’ use case
 functionalities
 architecture & implementation
 example
 video-demonstration
 discussion and future works
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8

4. introduction and motivation
smart
space
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8

5. introduction and motivation
smart
space
smart
objects
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8

6. introduction and motivation
smart space behaviour
smart personalization
space
smart
objects
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8

7. introduction and motivation
smart space behaviour
smart personalization
space
focus
general approach
 mobile-instrumented
smart interaction method
objects
particular implementation
 device-objects
coordination for content
representation &
management
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 1/8

8. contents

 introduction and motivation
 interaction method fundamentals
 ‘smart night table’ use case
 functionalities
 architecture & implementation
 example
 video-demonstration
 discussion and future works
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8

9. interaction method fundamentals
mobile-instrumented
concept
trigger
orchestrator
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8

10. interaction method fundamentals
mobile-instrumented
concept
trigger
orchestrator
physical-based interaction
maintaining traditional interaction
paradigm
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8

11. interaction method fundamentals
mobile-instrumented
concept
trigger
orchestrator
mobile device – smart object
coordinated response
logical state of mobile device
physical-based interaction
maintaining traditional interaction
paradigm
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8

12. interaction method fundamentals
mobile-instrumented
concept
trigger
orchestrator
device-hosted
content sharing
audio, video, pictures, etc.
mobile device – smart object
coordinated response
logical state of mobile device
physical-based interaction
maintaining traditional interaction
paradigm
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8

13. interaction method fundamentals
mobile-instrumented multiple & heterogeneous
concept smart objects interfaces
trigger TV, Hi-Fi, tablet, PC, photo frame, etc.
orchestrator
device-hosted
content sharing
audio, video, pictures, etc.
mobile device – smart object
coordinated response
logical state of mobile device
physical-based interaction
maintaining traditional interaction
paradigm
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8

14. interaction method fundamentals
rule-based reasoning
ECA model
Events – Conditions – Actions
mobile-instrumented multiple & heterogeneous
concept smart objects interfaces
trigger TV, Hi-Fi, tablet, PC, photo frame, etc.
orchestrator
device-hosted
content sharing
audio, video, pictures, etc.
mobile device – smart object
coordinated response
logical state of mobile device
physical-based interaction
maintaining traditional interaction
paradigm
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8

15. interaction method fundamentals
context
user context-responsive
environment context analysis before executing
etc. actions
rule-based reasoning
ECA model
Events – Conditions – Actions
mobile-instrumented multiple & heterogeneous
concept smart objects interfaces
trigger TV, Hi-Fi, tablet, PC, photo frame, etc.
orchestrator
device-hosted
content sharing
audio, video, pictures, etc.
mobile device – smart object
coordinated response
logical state of mobile device
physical-based interaction
maintaining traditional interaction
paradigm
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 2/8

16. contents

 introduction and motivation

 interaction method fundamentals
 ‘smart night table’ use case
 functionalities
 architecture & implementation
 example
 video-demonstration
 discussion and future works
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 3/8

17. ‘smart night table’ use case
functionalities
hi-fi audio
digital photo frame
television
‘smart
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 3/8

18. ‘smart night table’ use case
functionalities
hi-fi audio
Voice (call)
digital photo frame
Photos Music
Internet
Videos
television
‘smart
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 3/8

19. ‘smart night table’ use case
functionalities
Voice (call) Music
hi-fi audio
Voice (call)
Photos
digital photo frame
Photos Music
Internet
Videos
Videos
television
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 3/8

20. contents

 introduction and motivation

 interaction method fundamentals
 ‘smart night table’ use case

 functionalities
 architecture & implementation
 example
 video-demonstration
 discussion and future works
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8

21. ‘smart night table’ use case
architecture & implementation
Voice (call) Music
hi-fi audio
Voice (call)
Photos
digital photo frame
Photos Music
Internet
Videos
Videos
television
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8

22. ‘smart night table’ use case
architecture & implementation
Android
service
Voice (call) Music
hi-fi audio
Voice (call)
Photos
digital photo frame
Photos Music
NFC tag
Internet
Videos
Videos
television
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8

23. ‘smart night table’ use case
architecture & implementation
Android
service
Voice (call) Music
hi-fi audio
Voice (call)

Photos Airplay Mirroring (Apple)
 Intel Wireless Display (WiDi) digital photo frame
 Wireless HD
Music
Photos
 Miracast
NFC tag DLNA
Internet
Videos
(DMS) Digital Media Server
(DMR) Digital Media Renderer
Videos
(DMC) Digital Media Controller
television
 maturity
 interoperability
(S.O., comm. tech., brands, etc.)
 certified devices
 SDK available
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8

24. ‘smart night table’ use case
architecture & implementation
DLNA Renderer
Android (DMR)
service ID: Renderer003
Voice (call) Music
hi-fi audio
DLNA Renderer
(DMR)
ID: Renderer001 DLNA Controller DLNA Renderer
(DMC) (DMR)
DLNA Server ID: Renderer004
Voice (call)
M-DMS RESTful API
Photos
digital photo frame
Photos Music
NFC tag DLNA DLNA Renderer
Videos
Internet (DMR)
(DMS) Digital Media Server ID: Renderer002
(DMR) Digital Media Renderer
Videos
(DMC) Digital Media Controller
television
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8

25. ‘smart night table’ use case
architecture & implementation
Bluetooth
DLNA Renderer
Android (DMR)
service ID: Renderer003
Voice (call) Music
hi-fi audio
DLNA Renderer
(DMR)
ID: Renderer001 DLNA Controller DLNA Renderer
(DMC) (DMR)
DLNA Server ID: Renderer004
Voice (call)
M-DMS RESTful API
Photos
digital photo frame
Photos Music
NFC tag DLNA Renderer
Videos
Internet (DMR)
ID: Renderer002
Videos
television
sockets
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8

26. ‘smart night table’ use case
architecture & implementation
Bluetooth
DLNA Renderer
Android (DMR)
service ID: Renderer003
Voice (call) Music
hi-fi audio
DLNA Renderer
(DMR)
ID: Renderer001 DLNA Controller DLNA Renderer
(DMC) (DMR)
DLNA Server ID: Renderer004
Voice (call)
M-DMS RESTful API
Photos
digital photo frame
Photos Music
NFC tag DLNA Renderer
Videos
Internet (DMR)
ID: Renderer002
Videos
television
sockets
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 4/8

27. contents

 introduction and motivation

 interaction method fundamentals
 ‘smart night table’ use case

 functionalities

 architecture & implementation
 example
 video-demonstration
 discussion and future works
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8

28. ‘smart night table’ use case
example
Bluetooth
DLNA Renderer
Android (DMR)
service ID: Renderer003
Voice (call) Music
hi-fi audio
DLNA Renderer
(DMR)
ID: Renderer001 DLNA Controller DLNA Renderer
(DMC) (DMR)
DLNA Server ID: Renderer004
Voice (call)
M-DMS RESTful API
Photos
digital photo frame
Photos Music
1
 Videos
Internet
NFC tag DLNA Renderer
(DMR)
ID: Renderer002
Videos
television
sockets
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8

29. ‘smart night table’ use case
example
SELECT render Bluetooth
2 GET renderers
3 DLNA Renderer
(DMR)
Android
service ID: Renderer003
Voice (call) Music
hi-fi audio
DLNA Renderer
(DMR)
ID: Renderer001 DLNA Controller DLNA Renderer
(DMC) (DMR)
DLNA Server ID: Renderer004
Voice (call)
M-DMS RESTful API
Photos
digital photo frame
Photos Music
1
 Videos
Internet
NFC tag DLNA Renderer
(DMR)
ID: Renderer002
Videos
television
sockets
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8

30. ‘smart night table’ use case
example
Bluetooth
2
3 DLNA Renderer
(DMR)
Android
service ID: Renderer003
Voice (call) Music
hi-fi audio
DLNA Renderer 4 STOP 5
render
(DMR)
ID: Renderer001 DLNA Controller DLNA Renderer
(DMC) (DMR)
DLNA Server ID: Renderer004
Voice (call)
M-DMS RESTful API
Photos
digital photo frame
Photos Music
1
 Videos
Internet
NFC tag DLNA Renderer
(DMR)
ID: Renderer002
Videos
television
sockets
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8

31. ‘smart night table’ use case
example
Bluetooth
6 START playing
3 DLNA Renderer
(DMR)
Android
service ID: Renderer003
Voice (call) Music
hi-fi audio
DLNA Renderer 4 5
(DMR)
ID: Renderer001 DLNA Controller DLNA Renderer
(DMC) (DMR)
DLNA Server ID: Renderer004
Voice (call)
M-DMS RESTful API
Photos
digital photo frame
Photos Music
1 7 START playing
 Videos
Internet
NFC tag DLNA Renderer
(DMR)
ID: Renderer002
Videos
television
sockets
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8

32. ‘smart night table’ use case
example
Bluetooth
6
3 DLNA Renderer
(DMR)
Android
service ID: Renderer003
Voice (call) Music
hi-fi audio
DLNA Renderer 4 5
(DMR)
ID: Renderer001 DLNA Controller DLNA Renderer
(DMC) (DMR)
DLNA Server ID: Renderer004
Voice (call)
M-DMS RESTful API
Photos
digital photo frame
Photos Music
1 7
 Videos
Internet
NFC tag DLNA Renderer
(DMR)
ID: Renderer002

Videos
Videos 8 (continuous streaming)
television
sockets
‘smart
Internet
night table’ tablet
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 5/8

33. contents

 introduction and motivation

 interaction method fundamentals

 ‘smart night table’ use case

 functionalities

 architecture & implementation

 example
 video-demonstration
 discussion and future works
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 6/8

34. ‘smart night table’ use case
video-demonstration
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 6/8

35. contents

 introduction and motivation

 interaction method fundamentals

 ‘smart night table’ use case

 functionalities

 architecture & implementation

 example

 video-demonstration
 discussion and future works
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8

36. discussion and future works
smart space behaviour personalization
⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
objects through mobile devices. Proc. of the 9th ACM International Symposium on
Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8

37. discussion and future works
smart space behaviour personalization
⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
objects through mobile devices. Proc. of the 9th ACM International Symposium on
Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011
 smart objects orchestration architecture
mobile-centred
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8

38. discussion and future works
smart space behaviour personalization
⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
objects through mobile devices. Proc. of the 9th ACM International Symposium on
Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011
 smart objects orchestration architecture
mobile-centred
 mobile-instrumented interaction method
mobile media content/services ↔interface device transfer
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8

39. discussion and future works
smart space behaviour personalization
⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
objects through mobile devices. Proc. of the 9th ACM International Symposium on
Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011
 smart objects orchestration architecture
mobile-centred
 mobile-instrumented interaction method
mobile media content/services ↔interface device transfer
 specific use case
‘smart night table’
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8

40. discussion and future works
smart space behaviour personalization
⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
objects through mobile devices. Proc. of the 9th ACM International Symposium on
Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011
 smart objects orchestration architecture
mobile-centred
 mobile-instrumented interaction method
mobile media content/services ↔interface device transfer
 specific use case
‘smart night table’
future extensions
 extending the set of interaction actions to perform
(generic approach)
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8

41. discussion and future works
smart space behaviour personalization
⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
objects through mobile devices. Proc. of the 9th ACM International Symposium on
Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011
 smart objects orchestration architecture
mobile-centred
 mobile-instrumented interaction method
mobile media content/services ↔interface device transfer
 specific use case
‘smart night table’
future extensions
 extending the set of interaction actions to perform
(generic approach)
 interface device interaction
e.g., mobile-device movement detection, Kinect, etc.
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8

42. discussion and future works
smart space behaviour personalization
⁷ Bernardos, A.M, Casar, J.R., Cano, J., Bergesio, L. 2011. Enhancing interaction with smart
objects through mobile devices. Proc. of the 9th ACM International Symposium on
Mobility Management and Wireless Access, Miami (USA), 31 October - 4 November 2011
 smart objects orchestration architecture
mobile-centred
 mobile-instrumented interaction method
mobile media content/services ↔interface device transfer
 specific use case
‘smart night table’
future extensions
 extending the set of interaction actions to perform
(generic approach)
 interface device interaction
e.g., mobile-device movement detection, Kinect, etc.
 context-responsiveness enhancements
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 7/8

43. any question?
Smart Environments and Ambient Intelligence – SENAmI 2013 luca.bergesio@grpss.ssr.upm.es 8/8