Presented at Talk To Me, Berlin June 14th, 2018

1. DESIGNING THE FUTURE,
WITH VOICE
Karen Kaushansky
@kjkausha
Talk To Me Berlin
June 14, 2018

2. For 20+ years I’ve been designing experiences with emerging technology -
speech recognition, conversational interfaces, biometrics, mixed reality, AI.
EXPERIENCE DESIGN
Tellme for Windows
Mobile 6.5 - 2009

3. We’ve already seen huge advancements in speech recognition.
SOME HISTORY
NLU MODELS CONNECTED DEVICES “PART OF THE OS”
Conversational
experiences
Integrated
hardware and
software
Using speech to
deep link

4. Ok Hound. What are
some Italian restaurants
near me with
reservations for 2 people
for tomorrow night.
Now today it’s possible to use wake words
to start a conversation, and string
together real human like questions, that
go well beyond the top layer to get results
we want.

5. What is the future?

6. CONVERSATIONAL INTERFACES
App Fatigue has settled in.
We are moving towards an app-less
world, and conversational
interfaces allow us to “reach people
in the channels they already
massively use most of the time,
instead of fighting a hopeless
battle to drive them to niche,
standalone apps they infrequently
visit.” - Meydad Marzan

7. VOICE IS THE KEY TO AN APP-LESS WORLD
Voice will let users work broadly across
apps and experiences.
Source: Mercedes-Benz

8. THE RISE OF CONVERSATIONAL INTERFACES
Chatbots provide some of the same
advantages of voice:
‣ Lets users instantly interact to get
what they want eliminating browsing,
searching and clicking.
‣ Offers the promise of frictionless
access to goods, services and
information.
KLM Facebook Messenger Bot

9. Source: Magic Leap
THE WORLD IS YOUR SCREEN

10. CAMERAS EVERYWHERE
Source: Amazon

11. THE RISE OF ASSISTANT(S)
“There are going to be multiple
successful intelligent agents, each with
access to different sets of data and with
different specialized skill areas…"
Jeff Bezos, FT Times

12. ASSISTANTS
Source: MSC

13. Step 1: Assistants connected to
other assistants (Cortana&Alexa)
THE RISE OF THE ASSISTANT
Step 2: Assistants talking to
other assistants
Step 3: Assistants acting on
our behalf (Google Duplex)

14. AGENTIVE TECH
Chris Noessel:
Designing Agentive
Technology

15. SENTIMENT ANALYSIS/EMOTION

16. • “Social robots bring high-tech to high- touch
for humanized engagement” - JIBO
• This is a new kind of relationship; they are
not trying to replace someone or something
that exists
SOCIAL ROBOTS
Kuri JIBO Pepper

17. “Voice assistants will
probably have an
emotive voice
indistinguishable from
a real human voice
and have a virtual
body to more directly
interact with the user.”
Source: Unity
MIXED-REALITY CHARACTERS

18. Industries and use cases

19. "Voice does something that
other UIs don’t do; it's a conduit
between channels, and removes
friction." - Tom McElreath
CONTENT HYPERLINKING/TAGGING

20. CONTENT HYPERLINKING/TAGGING
Source: Amazon

21. NBA example
CONTENT HYPERLINKING EXAMPLE
Example of interactive media - asking
for replays, standings, player stats.

22. STORYTELLING/BRANCHING NARRATIVES

23. INTERACTIVE NARRATIVES

24. HEALTHCARE
Sensely

25. HOSPITALITY
Source: Naomi
Hotel, TripAdvisor

26. IMMERSIVE APPS WITH VOICE
“Instead of having to teleport
between crew-member
stations or robotically issuing
instructions to AI teammates
from a menu, you can just
hold down a button then
shout out your
instructions….Just say
"Prepare for warp," and the
crew will line up a destination
and charge the warp coils.“ -
Cnet
Star Trek Bridge Crew

27. Breaking it down

28. Alexa, is it going
to rain today?
“Rain is likely in
San Francisco
today, there’s an
85% chance.”
INPUT OUTPUT

29. Alexa, show me
kids shows to
watch. Here are kids
shows…”
INPUT OUTPUT
Voice + Touch Audio + Screen
Alexa, play
number 3.

30. INPUT OUTPUT
Voice Screen
Alexa, show me
my front door.

31. OK Google, show me photos
of Margaret on my TV.
INPUT OUTPUT
Audio + Screen
User directed output
to a screen
OK Google, what movies
are playing near me?
Ok Google, send
that to my phone.
Voice

32. BALANCING AUDIO AND VISUALS
There are different ways to use
audio, with a screen:
• Replace what is on the screen
• Complement the screen
• Indication to look at the screen
Audio can and should change based
on context
Google Assistant/
Smart Displays

33. MULTI-INPUT
Methods of interaction:
• Voice
• Touch
• Text
• Gestures
• Eye gaze
• Brain
Contextual input:
• Position, body
• Camera
• Sensors
• Emotions
• Speed, GPS
• Heart rate, temperature
• The speaker (who it is)

34. MULTI-OUTPUT
Response/output:
• Audio
• Screens
• Haptics
• Lights, LEDs
• Actions
• Embodiment, gesture
• Smells

35. MULTI-USER
The user is an input, and affects
the output.
With Google/Amazon, we are able
to set up different profiles, using
voice identification, so when one
user asks for their calendar or
playlist, the experience is
personalized.

36. The experience you are building is not always the first interaction with
your customer; your voice experience could start mid-conversation.
CROSS DEVICE/MULTI-DEVICE
CONSISTENT EXPERIENCE CONTINUOUS EXPERIENCE COMPLEMENTARY EXPERIENCE

37. MULTI-CHANNEL
With BMW Connected, you can ask questions or give commands such as:
• What is my next trip?
• When should I leave for my next trip?
• What time is my next appointment?
• Send my next trip to the car.
• Activate the climate control.
• Lock the doors.
• Is my BMW locked?
• Are my windows up?
• What’s my fuel range?
Is my trunk locked? (New!)
BMW has an Alexa skill. The “conversation”
spans outside the car.

38. WHAT NOW?

39. Thank you
Karen Kaushansky
@kjkausha