The web in the world

1. The web in the world Timo Arnall elasticspace.com nearﬁeld.org I do design and research, based at the Oslo School of Architecture & Design, where I run a design research project that looks at emerging technologies. We work in interaction and industrial design mainly, so we are in a good position to be thinking about the current shift in technology, from screen-based interaction towards physical interaction with the world around us.

2. So what do I mean by The web in the world? When we think of the mobile web many of us think about small screen rendering of web pages. And this is clearly happening, the web is moving towards an enormous number of tiny screens.

3. And it’s leaking out into a lot of other devices. This is the result of having Twitter sending out SMSes for a few days. There are of course a lot of subtler ways that the web is ﬁltering out into the world.

4. The iPhone is also clearly making a platform for the mobile web, with it’s assumed always-on network connectivity. Applications are starting to take advantage of this always-on-ness and to make use of some of it’s sensing abilities.

5. But we are working towards a ‘ubiquitous web’ has greater implications than everyone constantly engaged in tiny-screen interaction. In fact I think we should set out to NOT create an overload of this interaction. The ‘internet of things’ may instead increase awareness of the physical and social environment, and to do more, with less screen-time.

6. Marc Smith of Microsoft Research said last week at the Social Computing Symposium that: “Most digital interaction takes place in another interaction order – in another space – not in the physical interaction order of co-presence.” “This is about to change...” So digital interaction takes place, mentally and physically, somewhere else. We go onto the internet... And what is enabling this to happen? Spimes (Bruce Sterling) Data shadows (Mike Kuniavsky) The Internet of things...

7. Things & links So I’ve put together a number of ‘patterns’ that show some issues and directions for the future web in the world. Then a few design directions that are needed to make these happen. What is the basic building block of an internet of things?

8. Well according to many, it is the humble link. If we go right back to basics, we can see that URLs have been inhabiting the physical world for a few years. From gra!ti artists to large companies, the url has become a standard part of public marketing and branding.

9. And an ever increasing number of objects are used to attempt to link people back into the web. http://ba.com

10. But the interaction experience for this is not yet well deﬁned. Shortcodes for instance are quite successful, short text input yields richer information sent back, usually via a URL.

11. So many projects and platforms are setting out to ﬁnd other ways of linking the physical to the digital. Yellow arrow used a kind of shortcode, or unique ID for places, and encouraged storytelling (didn’t really work on a wide scale) http://www.yellowarrow.net

12. Thinglink attempts to create unique identiﬁers for physical products. They have a lovely idea about creating greater visibility for the long tail. http://thinglink.org

13. So there are many incompatible barcode systems that have emerged, some like QR coding are designed for linking, particularly on the mobile phone.

14. But experiences of barcoding stuquot; beyond grocery goods haven’t gone very well. This is the Cuecat, the massive fail project that partnered with Wired.

15. This is one ‘unique identiﬁer’ project that I really like, made by Maya Lotan at Ivrea. It has unique and decorative patterns that can be converted through image recognition into a unique ID. She designed a range of products, from clothes, to accessories to paper material that contained the codes. I really like the integration with fashion. Urban Seeder. http://www.urbanseeder.com

16. This is another way of linking, a box that asks you to turn on bluetooth and lets you download content. The weirdness with this is that it’s di!cult to tell where the bluetooth begins and ends. It’s not ‘direct manipulation’.

17. Near Field Communication is perhaps the most advanced kind of object linking. It uses battery-less RFID chips that store and transfer data. The idea is that we’ll use phones for ticketing, payment, and for discovering ‘services’ in the physical world.

18. • index_us.html Raﬁ probably showed Mir:ror yesterday. It uses the same RFID technology http://www.violet.net

19. And Tikitag.com But what is required is a more nuanced vision of the physical link. I think the hyperlink is a ﬂawed model for physical interaction. Yes it allows for objects to communicate, for things to be tracked as part of logistics systems. But does it really make sense from the user perspective? What is the user-experience? What is the context? What is useful or playful in this interaction?

20. We have also created products that use physical objects to trigger interactions with media. These wooden objects have embedded RFID chips, that create reactions when they are placed inside the bowl. We really tried to concentrate on objects and interfaces that would ﬁt in the home environment (on the coquot;ee table!) and would allow for subtle and tangible interactions with media (particularly with kids) Which leads us to... www.nearﬁeld.org/2007/12/bowl-token-based-media-for-children

21. Tangible in What I think we need to be thinking about from a user-perspective is tangible interaction. This is the use of physical objects and gestures to control interactions. The aim is to use more senses and more of the body in our interactions.

22. Nintendo Wii controller.

23. Nokia 5500 and iphone have accelerometers.

24. Eyetoy or sixaxis controller. Sixaxis playstation

25. GPS in, in lots of things, cars, mobiles. I really hate GPS, as someone who lives most of their lives walking in dense cities, it doesn’t work well...

26. But lets not overlook the humble microphone... The games produced that include sound and breath input for the DS are astounding. Nintendo DS

27. And of course, tangible interaction with objects is perfect for games and toys, this is where it is starting to emerge. Brio Network http://brio.hosting.mrfriday.com/network/

28. Ambient out I use the word ambient loosely (there is a whole research equot;ort on ‘ambient computing’) but I think the concept is interesting. Instead of the focused, attention grabbing interaction with the web, what if we could use more of our senses to get information: Peripheral, background, sound, haptic

29. Perhaps the quintessential ambient displays are made by “Ambient Devices” This is their orb, that can be set up to display a few variables of data, through brightness and colour.

30. Their HomeJoule device also oquot;ers background information in speciﬁc places. To do this they have set up their own network in the US.

31. Their ‘dashboard’ oquot;ers lovely analogue pointers that display information. This is the kind of display that can be glanced, or is easily understood without direct attention.

32. One really lovely thing about this dashboard is the custom plates that allow customisation of the displays. Not only does this change the functionality, it changes the scale or the data that is presented.

33. There are a lot more products on the horizon that oquot;er some sense of ambient output. The chumby is equot;ectively ‘widgets for physical space’. Photo CC by dalager: http://ﬂickr.com/photos/dalager/2587380983/

34. Sound objects Sound is a critical aspect of interfaces that inhabit the world with us Don’t bleep!!

35. And ﬁnally, haptics. The use of vibration to indicate interaction output. This is a lovely use of haptic feedback, Chris Woebken at the RCA made a simple digital compass that vibrated whenever it pointed north. People who wore this apparatus gained a new sense of direction. Part of the animal superpowers project by Chris Woebken http://www.ﬂickr.com/photos/sutje/493456056/

36. In our Touch project one of my students created ‘Sniquot; the dog’ that sniquot;s RFID tags and gives sound and vibration feedback. This is a lovely explorative toy where the sound and vibration adds up to create emotional responses to the environment. http://www.nearﬁeld.org/2007/06/sniquot;-wins-prize-for-design-for-all

37. Context I use the word context rather than location. It’s more general, and has more interesting connotations. Context is hard. We also have a whole research domain called ‘context aware computing’ that has been for decades looking for ways of making devices respond smarter to their surroundings. Also, contextual design practices will have to be brought to the forefront.

38. But one definition of context that is useful right now is: Devices that are aware of other devices... Plazes turns the proximity to wifi routers into one way of determining simple context *The use of fields to determine context* Founder says that it was important for Plazes location lookups to be implicit in the use of mobile devices and laptops, not explicitly chosen by the user (but with user control over privacy)

39. Simply mapping devices creates a lot of rich data. This is the ‘Mapping Graz’ project by MIT http://web.mit.edu/newso!ce/2005/cellphones.html

40. And this data can be turned back into interface Here is ‘Citysense’ allowing people to visualise the number of people (cellphones) around them

41. The other thing that is clearly happening is visualising the geo-located data from the web, in place. I’m skeptical about Augmented Reality, particularly the goggles/glasses variety but this is quite a compelling demo Sekai phone http://www.tonchidot.com

42. Here is one piece of evidence, ‘The future of internet search’ by Mac Funamizu He has successfully visualised ideas about using data in the physical world An interface that adds to physical experience rather than making us withdraw to small screens. http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/

43. Visibility Visibility is one of the biggest issues for ‘ubiquitous computing’.

44. And Donald Norman is taking those ideas further. There is an equot;ort towards embedding computing in everything, and making it ‘disappear’. But the consequences of that from a user-perspective have not been fully thought through.

45. Every modern electronic device we carry has some sort of invisible interaciton with the world around it, through various ‘bubbles of radio’ of varying sizes.

46. RFID in particular has been a short-fuse of controversy around ubiquitous computing. This is both because it can be invisible (Embedded) And because it has features that are not visible (Radio)

47. These radio bubbles surround us most of the time. Wiﬁ Bluetooth GSM 3G DMB GPS RFID Zigbee

48. 3G coverage in Stavanger, Norway This stuquot; is starting to have an impact on our everyday lives. It’s experienced as dropped connections, as unreliability, as slowdowns. The mapping of this stuquot; is important.

49. Skyhook wireless is mapping out these ﬁelds, and turning them into material we can use in interactions... Their coverage can be visualised on their website http://www.skyhookwireless.com/howitworks/

50. Very practically: How will you explain to users that their location lookups depend on a number of bubbles of radio waves? Or how will they know that they are in a GPS shadow? This is my student’s work on visualising radio fields, she imagined fictional ‘species’ of radio waves that she turned into an encyclopaedia. http://www.nearfield.org/2007/12/fictional-radio-spaces

51. This is some work I did with Adam Greenﬁeld towards making a set of symbols for his book “Everyware” We attempted to explain various elements of ubiquitous experience through simple symbols. Work in progress.

52. Tracking & tracing A lot of these technologies lend themselves to tracking and tracing. So something has emerged in what Matt Jones and Tom Coates are calling “Personal Informatics”, “the ability to leverage data about yourself and your friends to your advantage”

53. The Nike+ is a fascinating example of new forms of entertainment It is designed, a physical object, using high technology. But the connection to the web is a critical part of the experience.

54. The sharing and reﬂection on data is a large part of the experience

55. Nokia Sports Tracker: http://sportstracker.nokia.com

56. It’s the same with the Wattson from DIYKyoto. Plugging this thing in for the ﬁrst time is something that gets Flickr’ed over and over. http://www.diykyoto.com/

57. And the resulting data visualisations are then excellent ‘social objects’ with which to compare and reﬂect.

58. http://ﬂickr.com/photos/antimega/2371642867/

59. Scarcity & distance If the mobile phone and internet is compressing time and space, any move back towards the interaction with the physical world re-introduces scarcity and distance.

60. This is what I got when I loaded Geode for the ﬁrst time. A total lack of content. Not having an active community near you means that many services will only work in certain geographical contexts. http://labs.mozilla.com/2008/10/introducing-geode/

61. If we have ‘smart’ posters oquot;ering content, is this the only place that it is available, can I take the content with me and access it again? These are questions that need to be answered.

62. Poken is a physical (rﬁd based) object that lets you connect your social networks in physical space, by touching them together you create a connection. It is open-API based. underestimating the di!culty of achieving critical mass or: overestimating their ability to aquot;ect culture There is an enormous equot;ort to reach the critical point at which cluster equot;ects can take place, and many users will feel the full brunt of the ‘ﬁrst fax machine’ problem. http://www.doyoupoken.com/

63. Adaptability Which also leads to adaptability. The mouse, keyboard and screen is a wonderfully adaptable interface, so much so that it has stuck with us for over 20 years. And the web is a fantastically adaptive environment, built out of layers of generalised code (material). But physical and tangible interfaces tend to be much more specialised. In particular the economics of adaptability.

64. Single use-interfaces This is a toy called Swinxs, it connects to the web and allows lots of games to be played, but the interactions are limited to touching this console. Suitable for toys and museums, and that is exactly where we are seeing the ﬁrst of these kinds of interfaces. http://www.swinxs.com

65. So there are modular approaches to hardware, which should provide some means of getting some of the adaptable features of the web onto into the physical world. http://www.buglabs.net

66. This is the Olinda radio from Schulze & Webb. It goes some of the way towards creating an adaptive, social piece of hardware, that mirrors the web. They even made a hardware API. Matt Webb says that “Some things are di!cult to do in social software: quot;But these things I get for free by having them in a physical objectquot; eg quot;Storing my identity in an object, and gifting it to someonequot;.” http://schulzeandwebb.com/2008/olinda/

67. Designerly directions So what does this mean for design practice?

68. The good news is that this stuquot; is already emerging from the web. Web practices are very applicable, dealing with networks, connections, social media, etc. Schulze & Webb for example are actively encouraging web-practices in physical product design... But this stuquot; is di!cult, it needs to be platformed out so that practices, methods and data can be shared. http://www.pachube.com oquot;ers a wide platform for starting some of these linkages.

69. This stuquot; is di!cult, even compared with the web. Arduino and other hardware prototyping platforms make it easier. This is something that Mike Kuniavsky talks about: Sketching in hardware, the ability to use relatively simple hardware tools to make and revise quick prototypes. We could also call it rapid prototyping.

70. Both this and the last image are the iterations towards a ﬁnished “Sniquot;”. The challenges come in making physical objects which are robust, which have the qualities of good products, and work interactively. This requires lots of skills, from interaction design to software and industrial design.

71. Some of this is made easier through visualisation. Visualisation skills become important in actually uncovering the materials that we are working with.

72. This is a series of sketches that work towards understanding how RFID works. We wanted to sketch out the aquot;ordances of RFID: quot;the perceived and actual properties of the thing, primarily those fundamental properties that determine just how the thing could...” “A quality of an object, or an environment, that allows an individual to perform an action.”

73. And we ended up with a diagram that explains the important aspects of RFID for designers.

74. We can also start to explain some of the invisible aspects of RFID, here we are looking at the ﬁelds that surround an RFID object.

75. Evidence: create the stuquot; that we will all talk about, and can either agree or disagree about. This is from Urban Seeder by Maya Lotan.

76. Fiction is really important, especially when the

77. Emerging themes Design approaches Things linked Sketching Tangible in and out Visualisation Ambient and haptics Fiction Context (In)visibility Adaptability Scarcity In summary

78. Thank you elasticspace.com nearfield.org aho.no