Talk given at the NFC 2012 Workshop, March 13 2012, Helsinki Finland. The talk presents the research performed on NFC-based user interfaces at the University of Oulu, Department of Computer Science and Engineering, Oulu, Finland.

1. NFC-Based
User Interfaces
Jukka Riekki, Ivan Sanchez, Mikko Pyykkönen
Department of Computer Science and Engineering
University of Oulu
Oulu, Finland
http://www.oulu.fi/cse
NFC 2012
4th International Workshop on Near Field Communication
March 13th, 2012
Helsinki, Finland
These slides were modified slightly after the presentation. Three videos were replaced with text and
figures. Two of the videos are available at YouTube, links are provided on the corresponding slides.

2. Table of Contents
• Introduction
• Previous work
• Interaction model
• Advertising NFC tags
• Conclusions

3. Introduction
• NFC can have more profound effect on our
everyday lives than has been discussed in public
– Transit and Ticketing, Payment, and Advertising applications are
just the tip of the iceberg
• NFC-based user interfaces can be built to
any Internet or mobile phone application;
to any device connected to Internet
– Generally, NFC can be used to replace manual typing, menu
selections, and other user interface actions with acts of touching.
– For example: household appliances with an Internet connection;
see the next slide

4. Introduction
When will this application be published?

5. Introduction
• We see significant potential in interactive
spaces
– Environments equipped with rich user interfaces and
offering a large variety of services for users
– Focus on user control – instead of intelligent environments
observing the users and activating services autonomously
• The challenge is to provide easy-to-use user
interfaces
– The available services need to be communicated clearly to
users; how they can be controlled, and the devices
the services use
– The actions for controlling the services need to be easy to
perform.

6. Introduction
• NFC excels in tackling this challenge
– Graphical icons advertise services, commands & devices
– NFC tags are placed under icons
– Available services can be identified by visual scanning
– Commands performed by touching icons with NFC phones
• We study interactive spaces
– We have built many prototypes containing NFC-based user
interfaces
• This talk presents some of these prototypes and
our current work on:
– Interaction model for NFC-based user interfaces
– Graphical language for advertising NFC tags

7. Previous Work
• Advertising NFC tags
• Prototypes:
– Positioning Prototype 2006: NFC + WLAN positioning
– Information Board Prototype 2007: Information board map + tags
– Touch & Control Prototype 2007: Controlling a wall display
– Touch & Run Prototype 2008: Outdoor game
– Touch & Collect Prototype 2008: Pupils collect content from a
museum
– EPG Control Prototype 2009: UI for Electronic TV Program Guide
– PlaceMessaging Prototype 2009: Leaving messages on a place
– Touch & Learn Prototype 2009: Learning languages
– iPoster Prototype 2010: Poster presentation with NFC UIs
– Touch & Learn to Read Prototype 2010: Children learn to read
• Next, some of this work is presented in more detail

8. Previous Work: Icons
• Research on NFC icons is quite rare
• We have designed several sets of NFC icons

9. Previous Work:
Icons
EPG control board: over 30 icons & tags
iPoster: a cartoon contains the icons
and gives instructions

10. Touch & Control Prototype 2007
• NFC tag is placed next to a wall display
• Touching the tag starts a multimedia
player on the wall display
• The player can be controlled
– With phone UI (BOTTOM LEFT) ControlCube
– By touching icons on the Control Panel (BOTTOM MIDDLE)
– By touching icons on the Control Cube (TOP RIGHT)
• Minimalistic phone UI suffices for the panel and
the cube – a single image (BOTTOM RIGHT)
• Video: http://www.youtube.com/isgoulu

11. Previous Work: Lessons learnt
• Carefully designed graphical icons facilitate recognizing:
• The services available in the local environment
• How the services can be controlled
• But not much has been done to develop the visual
appearance of NFC user interfaces
• Touching is a natural way of interacting with
our everyday environment
• Users can focus on their activities (calm computing!)
• This interaction model results in users being present
in the environment
• And interacting directly with other users
• Instead of each user focusing on own mobile terminals'
display (or on PCs’ or tablet PCs’ display)

12. Previous work: Lessons learnt cont’d
• Most of our prototypes have been tested with real users
• Feedback has been very positive
• In fact, phones with very modest resources can be used
to build impressive user interfaces
• Large touch displays are not mandatory
• Rather, a small phone easy to hold is better!
• Lots of different user interfaces can be built with
the current technology
• Many applications need software only for phones (no server);
the tag is the database!
• So applications can be kept simple and robust
• And using the applications is cheap (no communication costs)

13. Interaction model: Motivation
• Interactive space UIs differ from traditional GUIs
– UI is spread in the environment
– Tangible UI, i.e. physical objects are handled
• An interaction model can be used to
– Identify and define the UI components (objects, devices,…)
– Structure and model the interaction
– Study alternatives
• In addition
– The model might be used to present
recommended interaction patterns,
for example

14. Interaction model: Elements
• User: a person in an interactive space
• Token: a physical object in the space
– Users can control and observe the states of tokens
• Service: a digital object consisting of a set of
functionalities and policies controlling their usage
– Users achieve their goals with these functionalities
• Resource: a device in the physical space
• Services use resources to realize
their functionalities
– To recognize the states of tokens
– To change the states of tokens and
other environment
– To present information to users
– To get input from a user (e.g. touch
display)

15. Interaction model: Tokens & Resources
• The mapping between tokens and resources
determines
– How services experience the user-token interaction
– How users experience the service-resource interaction
• An object (e.g. an NFC phone) can be
– From the user perspective a token (handled as physical object)
– From the service perspective a resource
• NFC tags are resources as well
• When NFC readers are
placed in the environment
– User handles tokens equipped
with NFC tags

16. Interaction model: Stages
• Discovery Stage: a user scans visually the space
to find the tokens to interact with
• Composition Stage: the user composes the
application he or she wants to use
– By selecting services and tokens
– E.g. by touching tokens with a phone; NFC tags in tokens
• Usage Stage: The tokens
are handled
– User communicates with the
application by interacting
with the tokens in the physical
space
– Services interact with resources
in the digital space

17. The model elements in the environment

18. Advertising NFC tags
• We advertise NFC tags with service advertisements
– Graphical advertisements communicating information about
interaction possibilities
– And offering the means for realizing the interaction

19. Advertising NFC tags
• A service advertisement specifies
– The exact position to touch
– All details a user needs to know when making
a decision on whether to touch a tag or not
• Attention element: draws a user's attention
to the advertisement
– This element is included into each advertisement
• Technology element: indicates the technology
utilized in the interaction
– Also other interaction technologies,
like 2D barcodes and Bluetooth,
can be used

20. Advertising NFC tags
• Interaction element: advertises the exact point
to touch
– NFC tags are placed under these elements
• Action element: indicates the action the system
performs when a user touches the interaction
element
– The related NFC tag stores the command to execute that action, or
the data the system maps to the command
• Context element: can be
used to present, for example,
the service to be commanded

21. Advertising NFC tags
• Instruction element: explains how the interaction
is to be performed; text, cartoon, etc
– Separate instruction elements can give general instructions about
services in a certain area
• A service advertisement does not have to contain
all these elements
• When users become familiar with NFC and some
details are known otherwise,
less details suffice in
service advertisements

22. Advertising NFC tags: Examples
• Service advertisement for a photo album service presenting
photographs on a wall display:

23. Advertising NFC tags: Examples
• Advertisement for a voting service:

24. Advertising NFC tags: Examples
• Multimedia player service for a tourist attraction:

25. Advertising NFC tags: Examples
• Service advertisement for connecting a phone to a photo printer. Left:
printing is free. Right: printing costs 2,5 Euros:

26. Advertising NFC tags: Examples
• Service ad for downloading and joining a game shown on a wall display:

27. Advertising NFC tags: Examples
• Maps and contact information by touching:

28. Conclusions
• NFC can revolutionize the way we use applications
in our everyday life – a single touch to give a command
• We have good experiences on:
• Applications related to a place and/or using local resources
• Using objects in environment as data containers
• Learning applications
• NFC is ready for large scale usage – for a rich set of
different applications
• Simple phones are sufficient
• We continue building prototypes and developing the
model and the language
• The model can be used when designing applications and to
communicate good practices
• The visual language can advance common understanding

29. Contact information and acknowledgements
• Contact information: jukka.riekki@oulu.fi
– http://www.oulu.fi/cse
– http://www.youtube.com/ISGOulu
• Acknowledgements
– This presentation was enabled by numerous researchers from
the University of Oulu and from our partners. The EPG
application was developed at the University of Tampere. The
work has been funded by Tekes, EU, regional funding agencies
and companies

30. SELECTED PUBLICATIONS
Riekki J, Sanchez I, Pyykkönen M (2012) NFC-Based User Interfaces. 4th International Research Workshop with focus on Near Field
Communication (NFC2012), March 13th, 2012, Helsinki, Finland. Invited paper. (THIS TALK)
Riekki J, Cortés M, Hytönen M, Sánchez I and Korkeamäki RL (2012) Touching Nametags with NFC Phones: A Playful Approach to
Learning to Read. LNCS Transactions on Edutainment. Springer Verlag. In press.
Sanchez I, Cortes M, Riekki J & Pyykkönen M (2011) NFC-Based Physical User Interfaces for Interactive Spaces. In: Near Field
Communications Handbook (Internet and Communications). Ed Syed a. Ahson and Mohammad Ilyas. Auerbach Publications. 175-230.
Pyykkönen M, Riekki J, Alakärppä I, Sánchez I, Cortés M & Saukkonen S (2012) Designing Tangible User Interfaces for NFC Phones. In
Kiyokawa, K. (Ed.) Advances in Human Computer Interaction. Hindawi Publishing Corporation. Volume 2012.
http://dx.doi.org/10.1155/2012/575463
Riekki J, Sanchez I & Pyykkönen M (2010) Remote control for pervasive services, Int. J. Autonomous and Adaptive Communications
Systems, Vol. 3, No. 1, pp.39–58. http://dx.doi.org/10.1504/IJAACS.2010.030311.
Riekki, J. (2007) RFID and smart spaces, Int. J. Internet Protocol Technology, Vol. 2, Nos. 3/4, pp.143–152.
http://inderscience.metapress.com/link.asp?id=d3t8lmph214k0tl2
Riekki J, Salminen T & Alakärppä I (2006) Requesting Pervasive Services by Touching RFID Tags. IEEE Pervasive Computing, Jan.-
Mar. 2006. 5(1)40-46. http://dx.doi.org/10.1109/MPRV.2006.12.
Riekki J, Cortés M, Hytönen M, Sánchez I & Korkeamäki R-L (2011) Learning to Read by Touching Nametags with NFC Phones. The
Fourth International eLearning Baltics Conference (eLBa 2011), 26-27 May 2011, Rostock, Germany.
Sanchez I, Davidyuk O & Riekki J (2009) Towards User-Oriented Application Composition, In Proc. the IEEE Workshop on Pervasive
Service Computing and Applications (PSCA'09), December 17-19, Shanghai, China.
Sanchez I, Riekki J, Rousu J & Pirttikangas S (2008) Touch & Share: RFID Based Ubiquitous File Containers. In Proceedings of 8th
International Conference on Mobile and Ubiquitous Multimedia (MUM08), December 3-5 2008, Umeå, Sweden. 57-63.
http://dx.doi.org/10.1145/1543137.1543148
Sanchez I, Riekki J, Pyykkönen M (2008) Touch & Control: Interacting with Services by Touching RFID Tags. In Proceedings of the 2nd
International Workshop on RFID Technology - Concepts, Applications, Challenges (IWRT 2008), In conjunction with ICEIS 2008.
Barcelona, Spain, June 12-13, 2008. pp 53-62.
Riekki J, Sanchez I, Pyykkönen M (2008) Universal Remote Control for the Smart World. F.E. Sandnes et al. (Eds.): UIC 2008, LNCS
5061, Springer-Verlag, pp. 563-577. http://dx.doi.org/ 10.1007/978-3-540-69293-5_44
Sánchez I, Cortés M & Riekki J (2007) Controlling Multimedia Players using NFC Enabled mobile phones. In Proceedings of 6th
International Conference on Mobile and Ubiquitous Multimedia (MUM07), Oulu, Finland, December 12-14 2007.
http://dx.doi.org/10.1145/1329469.1329485