We are living in interesting times. Technology that at one time would have been considered the ravings of a mad men has now become reality: pocket computers, ubiquitous Internet access, cars that can drive themselves... All of this exist for us to use, now. And among these technology advances has come two more fanciful notions: interaction using gesture and voice. What was once science fiction is now here, now, and we are at the cusp of a breakthrough in how we interact with our electronic devices.
In this presentation, Joseph Dickerson, technologist and user experience architect for Fortune 500 company Fiserv, discusses these newest technology innovations and looks forward, to the "death" of the UI.
24. Bake three potatoes.
12:25 PM
Say a command
Three potatoes will take AUTO AUTO
12 minutes on high.
DEFROST REHEAT
Starting now.. 1 2 3
4 5 6
7 8 9
POWER +30
LEVEL 0 SEC
CLEAR PAUSE START
25.
26.
27. TV, record the new
Breaking Bad.
It's set to record on
Wednesday at 9. Do you
want to record all new
episodes?
Sure.
OK, all new episodes
will be recorded.
36. Voice Control
• Decide whether voice is your primary or
secondary way of interacting
• Be conversational but not too personal
37. Voice Control
• Decide whether voice is your primary or
secondary way of interacting
• Be conversational but not too personal
• Recover gracefully
38. Voice Control
• Decide whether voice is your primary or
secondary way of interacting
• Be conversational but not too personal
• Recover gracefully
• Be reliable
39. Voice Control
• Decide whether voice is your primary or
secondary way of interacting
• Be conversational but not too personal
• Recover gracefully
• Be reliable
• Get out of the way as quickly as possible
42. Gesture control
• Decide how and when gesture control should
be used... and choose wisely
• Keep gestures simple
43. Gesture control
• Decide how and when gesture control should
be used... and choose wisely
• Keep gestures simple
• Gestures has to make sense
44. Gesture control
• Decide how and when gesture control should
be used... and choose wisely
• Keep gestures simple
• Gestures has to make sense
• Use common gestural “affordances”
45. Gesture control
• Decide how and when gesture control should
be used... and choose wisely
• Keep gestures simple
• Gestures has to make sense
• Use common gestural “affordances”
• People are lazy
50. Augmented Reality
• Keep it simple
• Be easy to turn off
• Don’t block people’s view
• Integrate with speech and location
51. Augmented Reality
• Keep it simple
• Be easy to turn off
• Don’t block people’s view
• Integrate with speech and location
• Be useful, helpful but unobtrusive
52. Augmented Reality
• Keep it simple
• Be easy to turn off
• Don’t block people’s view
• Integrate with speech and location
• Be useful, helpful but unobtrusive
• It’s not “Pop-up Video”
I’m Joe Dickerson, UX Architect and Innovations Lead\n
\n
So what do I mean The Death of the UI? To answer a little context and background on new ways of interacting with technology that is changing how people do things.\n\n
A user interface is not just a screen you stare at on a computer or a mobile phone. Your home telephone has a user interface, Your oven, etc... I consider user interfaces predominantly fall into two groups. There’s interfaces where you review where you are at and what’s going on and interfaces where you do things. New technology is starting to impact both types of interfaces...\n
Voice interaction and control. First example that many of us saw was HAL 9000 in 2001. The movie that, if you followed the Apple and Samsung patent case, also showcased the first iPads...\n
SIRI from Apple, while still "Beta", is much more "intelligent"Majel, being worked on by Google, is also reported to have very good artificial intelligence“Google Now” is a sneak peak of Majel\n It's good and getting better...\n
When it comes to gesture, this first came into our popular culture through this movie...\n
Now we have Kinect and before that the Wii... but these are still first-gen products...\n
\n
The Leap is coming out this November... $99, works with any computer, has an open SDKINCREDIBLY sensitive down to the fingertip level - You can do air signatures, control and rotate objects... \n
Near Field Communication is another technology that we need to pay attention to... and even though Apple didn’t put an NFC chip in their new iphone, I’m sure that they are looking at it...\n
And there’s “augmented reality.”\nGoogle Glasses - the UI is very minimal and there will be ways you can turn it off and simply speak commands\n
Yes, even brain control is being looked at as a way to interact with technology - some simple applications of this have already been produced and sold to consumers.\n
The thing about these new technologies is, whether they get widespread adoption and use or not, they are not going away. People are going to get used to these systems and way of doing things and in many instances they begin to expect them. This is called the KANO model, used in product development...\n
\n
All these new interactions, Where will all this lead us? I think the answer is obvious.\n
Yes, in the future we will all look like we are in the Wiggles.\n
I’m going to get off on a tangent... I am a firm believer that the future is formed by geeks who read and watch SF. Newest Wired magazine 3D printer is called the replicator...\nSo if you want to come up with some good ideas, read some sci fi.\n
All these technologies, all these new methods of interacting with our devices... it will not lead to the death of the UI... it will lead to the death of the UI as we know it. \nTraditional UIs like an ATM screen or a mobile screen will probably never go away. What I think will die off are redundant or bad interfaces, and the different interaction methods will cause many UIs to change and become obsolete.\n\nAI - Memory\nSoftware should be smart, we use it every day. One of the real breakthroughs that is yet to come is artificial intelligence\nWe have more processing power in our free mobile phones that we had in the entire building at Kennedy Space center. The smart companies in the future will use it \nANother Star Trek Reference. When Captain Picard on Star Trek The NExt Generation called for Tea, Earl Gray, Hot... That command was needlessly complicated. He shoould have just said Tea, and the computer should have remembered that most of the time he liked Earl Grey and he liked it hot. That is the type of Smart Interfaces\n\nPeople will use and engage with technology in different ways... but even then, we should never lose sight of our primary interaction method\n
All these technologies, all these new methods of interacting with our devices... it will not lead to the death of the UI... it will lead to the death of the UI as we know it. \nTraditional UIs like an ATM screen or a mobile screen will probably never go away. What I think will die off are redundant or bad interfaces, and the different interaction methods will cause many UIs to change and become obsolete.\n\nAI - Memory\nSoftware should be smart, we use it every day. One of the real breakthroughs that is yet to come is artificial intelligence\nWe have more processing power in our free mobile phones that we had in the entire building at Kennedy Space center. The smart companies in the future will use it \nANother Star Trek Reference. When Captain Picard on Star Trek The NExt Generation called for Tea, Earl Gray, Hot... That command was needlessly complicated. He shoould have just said Tea, and the computer should have remembered that most of the time he liked Earl Grey and he liked it hot. That is the type of Smart Interfaces\n\nPeople will use and engage with technology in different ways... but even then, we should never lose sight of our primary interaction method\n
All these technologies, all these new methods of interacting with our devices... it will not lead to the death of the UI... it will lead to the death of the UI as we know it. \nTraditional UIs like an ATM screen or a mobile screen will probably never go away. What I think will die off are redundant or bad interfaces, and the different interaction methods will cause many UIs to change and become obsolete.\n\nAI - Memory\nSoftware should be smart, we use it every day. One of the real breakthroughs that is yet to come is artificial intelligence\nWe have more processing power in our free mobile phones that we had in the entire building at Kennedy Space center. The smart companies in the future will use it \nANother Star Trek Reference. When Captain Picard on Star Trek The NExt Generation called for Tea, Earl Gray, Hot... That command was needlessly complicated. He shoould have just said Tea, and the computer should have remembered that most of the time he liked Earl Grey and he liked it hot. That is the type of Smart Interfaces\n\nPeople will use and engage with technology in different ways... but even then, we should never lose sight of our primary interaction method\n
All these technologies, all these new methods of interacting with our devices... it will not lead to the death of the UI... it will lead to the death of the UI as we know it. \nTraditional UIs like an ATM screen or a mobile screen will probably never go away. What I think will die off are redundant or bad interfaces, and the different interaction methods will cause many UIs to change and become obsolete.\n\nAI - Memory\nSoftware should be smart, we use it every day. One of the real breakthroughs that is yet to come is artificial intelligence\nWe have more processing power in our free mobile phones that we had in the entire building at Kennedy Space center. The smart companies in the future will use it \nANother Star Trek Reference. When Captain Picard on Star Trek The NExt Generation called for Tea, Earl Gray, Hot... That command was needlessly complicated. He shoould have just said Tea, and the computer should have remembered that most of the time he liked Earl Grey and he liked it hot. That is the type of Smart Interfaces\n\nPeople will use and engage with technology in different ways... but even then, we should never lose sight of our primary interaction method\n
Here’s some simple case studies on how some of these interaction methods can be used\n
My Microwave has a lot of buttons and settings. It works, but it’s overly complicated.\nWhat if it looked like this...\n
Better!\n
Unlocking your house and disabling your security system - a custom gesture you have setup with your device and an NFC chip that is in your phone\n
Here’s my remote for my satellite TV. I just want to record a show! So, how do I do that... I go to the guide, I scroll through the listings... and then I click a button. Or I type in a search word with the keypad... Ugh.\n
My \n
Better!\n
A brain-controlled car? This is actually being developed by Google.\n
\n
Designer are already defining standards and best practices for these new interaction methods\n\n
Enablers that lets you do things quicker and easier\nAdvisors that recommend you do something other than what you are doing - GPS recommending alternate route\nAssistants that do things on your bequest or behalf - Send a text message back when you don’t answer a call\n
Enablers that lets you do things quicker and easier\nAdvisors that recommend you do something other than what you are doing - GPS recommending alternate route\nAssistants that do things on your bequest or behalf - Send a text message back when you don’t answer a call\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
In closing, looking at all these we should never lose sight of the first interaction method...\n
The hand. it’s still going to be the primary way we engage with and use our tools. Just because different methods come to being, doesn’t mean that we will stop grasping pulling and typing. And when we design we should never lose sight of the user, the people attached to those hands. \n
\n
\n
Shameless plug #1 - My pal Mark Frost has written a new book that has just come out. It’s called the Paladin Prophesy and it’s really good. It’s available on Amazon and wherever fine books are sold... which is mostly Amazon.\n