TAT Webinar on Dynamic UIs

1. Dynamic UIs
James Haliburton, Concept Lead

2. CONTENTS
•Today & Tomorrow’s Mobile
•Future Visions
•Emerging Tech
•Dynamic UIs
• Legacy of Mobile
• What’s in a phone today
• Patterns for Achieving Dynamic UIs
• Transitions are the new state
• Design tools & processes
•Inspirational Seminars

3. What is a Dynamic UI?
A dynamic UI experience is about valuing the in-the-
moment use of the mobile phone, and not only the
product’s overall usefulness. A dynamic UI creates a
pleasurable experience.

4. What is a Dynamic UI?
4 Principles
•Provide apt feedback
•Invite playful, non-task based behavior
•Respond to “incorrect” interaction
•Design forms of interaction which support natural behavior

5. Philosophy about the future
At TAT we believe that it is necessary to examine and innovate the future of technology. We are examining strong use
cases which justify and direct the need for this development. We are designing for these scenarios now, as the
future is arriving quickly.

6. TODAY’S MOBILE

7. TODAY’S MOBILE
•Capacitive touchscreens •Accelerometers
•Multitouch •Compasses
•GPS •High res
•Bluetooth •More processing power than it
•Wifi took to put a man on the moon!

8. TOMORROW’S MOBILE
projected UI:s
3D Gestures
Gyros
Augmented reality

9. UI/UX in mobile moving forward
Changed Input Methods
We have recently moved from phones like the Sony Ericsson T610 with indirect
input (keys) and a color graphical output to the time of the iPhone. Changes in
the UI are partially enabled by the change from indirect input to direct input
(touch).
But do our designs actually take into account that we have constructed
new types of devices?

10. FUTURE VISIONS

11. FUTURE VISIONS

12. FUTURE VISIONS

13. EMERGING TECH

14. PROJECTOR PHONES

15. PROJECTOR PHONES: FUTURE

16. PROJECTOR PHONES
A Radical Shift
•Not merely a matter of scaling up existing UIs and mobile content
•Offers a paradigm shift in the way we interact
•Phones have a social presence in our lives more than other
technology
•They are not fixed to a location or specific task - they aim to be
flexible and align to our behavior and to enhance our experiences
•Our behavior and experiences are more often than not, situated
amongst other people
•Private device in a public setting causes a disparity and disconnect
•The key difference with procam enabled UIs will be that we can begin
to truly interact, work, and play together. We will not be forced to
remove ourselves from a conversation to check a mapping application
or record a phone number - these can be collaborative actions.
•The UI can become a conversational tool.

17. FLEXIBLE DISPLAYS

18. MULTIMODAL
A multimodal user interface is a UI that involves more than one modality, or human sense
A multimodal UI is thus a UI that provides feedback to several human senses, e.g. visual,
auditory and haptic (tactile) feedback
18

19. MULTIMODAL
Sounds can be added to augment the visual design, adding a new dimension to user
immersion
Compared to haptics, sounds can convey information efficiently, e.g. through sound
effects, music or even speech
Globe Sejf Trum
19

20. MULTIMODAL
Sound can help users to better understand the effects of their interactions, especially
when using touchscreens
Visual design themes can have corresponding sound themes, enhancing the metaphor of
the GUI
Water Fire Paper
20

21. MULTIMODAL
As proven by film and computer games, sounds can be very expressive
Sounds should be subtle and varied, to avoid annoying people around the person using
the device
Example: Asia cute UI with sound
21

22. MULTIMODAL: FUTURE
22

23. MULTIMODAL
A UI ECOSYSTEM
Mobile UIs are no longer just about button input and visual output
New input paradigms require new approaches to output, for example:
Touchscreen input -> usability need for haptic and/or sound feedback
Gesture input (via touchscreen or motion sensor) -> expectation of visual output that
follow gestures realistically, e.g. through physics modeling
3D aesthetics and realism -> increasing expectations that UIs should mimic the real
world, e.g. by including sound and haptics
In a well designed device, output is closely linked and matched to user input
Changing one part of a UI will affect other aspects
23

24. AUGMENTED REALITY

25. GESTURES
What is a gesture?
•Essentially any movement can be a
gesture
•Rich and subtle
•Occurs over space and time
•Until recently we’ve been limited mostly to
1-dimensional button pressing
•But touch brings in new possibilities

26. GESTURES
A model for understanding gestures Symbolic
•Symbolic •3x3 Grid
•Iconic •Must learn what the gesture means
•Metaphoric •“automagical”
•Gesture is symbolic of the action occuring

27. GESTURES
A model for understanding gestures Iconic
•Symbolic •Uses “real world” gestures matched to actions on
•Iconic the wcreen
•Metaphoric •Flipping like you do your CD collection
•Motion matches something we already know how
to do

28. GESTURES
A model for understanding gestures Metaphoric
•Symbolic •Uses a familiar metaphor
•Iconic •Doesn’t necessarily match real world interaction
•Metaphoric •Needs support of sound, and visuals

29. GESTURES
SYMBOLIC ICONIC METAPHORIC
•Pro-user •Instantly understood •easily remembered
•magic quality •slower to perform than •dependent on other UI
•good for supporting surprise arbitrary gestures elements
•can be seperate from the •context dependent •can be used to discover new
graphic UI •highly satisfying functionality
•‘But supported by feedback in •can find inspiration in industrial •gesture is enabler
other UI elements design affordances
•Can be difficult to remember •gesture is the meaning
•Gesture is its own layer
A model for understanding gestures

30. 3D & PHYSICS

31. 3D & PHYSICS

32. 3D & PHYSICS

33. 3D & PHYSICS
Focus Declutter
Flexible Information Visualization (FIV)
Describes the elements of 3D which allow a UI to
better display large amounts of data, like playlists, in
Zoom & Overview
FIV Realtionships
Optimize
interesting ways relevant to the user and to the
application being used.
Reduce Cognitive Load
Show Options
Naturalized Interaction (NI)
Describes the elements of a 3D UI which minimize how
much a user needs to think about how theyʼre using an
application, and allows them to simply use it.
NI
Flatten Learning Curve
Show Affordances
WOW - Effect
Visual Style and Feedback (VSF)
Describes how 3D can add a “wow effect” to a
product and better differentiate itself from
competitors.
Visual Immersion
VSF
Holistic & Seamless
Feedback

34. 3D & PHYSICS

35. QUALITIES FROM THIS NEW TECHNOLOGY
Integrating our devices with the real world
•We can use our devices as social tools
•They can be flexible to our needs
•Encourage fun and playfulness
•Designs can be rich and creative

36. TODAY’S MOBILE
•Capacitive touchscreens •Accelerometers
•Multitouch •Compasses
•GPS •High res
•Bluetooth •More processing power than it
•Wifi took to put a man on the moon!

37. WHAT’S THE RESULT OF ALL THIS TECH?
A nicer box with a longer feature list.
Minimal UI innovation.

38. WHAT’S THE RESULT OF ALL THIS TECH?
What about the UI & UX?
UIs are still based around static screen states.
This is legacy of WIMP (windows, icons,
menus, pointing device), but we’ve been
talking about post-WIMP for 15 years.
This makes sense with simple task based
operations. Screen states support consistency
and simple flows.

39. WHAT’S THE RESULT OF ALL THIS TECH?
PC Mobile Evolution So Far
Windows Static screens
Icons Icons, Widgets
Menus Menus in soft keys
Pointing Device (mouse, D-Pad, Trackball, Touch
trackpad) pointing

40. WHAT’S THE RESULT OF ALL THIS TECH?
Input innovation has been big (qwerty, touch,
accelerometer, predictive text)
•But output and presentation have been mostly
"cosmetics" (if that)
•iPhone did a big jump in creating a more natural
interface (Realtime, direct manipulation)
•There are more steps in making devices less like
machines - interface with real world feel (dynamic)

42. DYNAMIC UIs - Reacts to real world behaviour

43. DYNAMIC UIs - Persistency of elements

44. DYNAMIC UIs - Soft Values, Generative visuals

45. DYNAMIC UIs - Movable light sources

46. DYNAMIC UIs - Adaptive Elements

47. DYNAMIC UIs - Situated Change
Sony Ericsson S500i
Theme changes appearance
depending on time of day,
date and special events.
Appearance changes based
on the time of day and special
events.

48. DYNAMIC UIs - Personalized Space

49. DYNAMIC UIs - Personalized Behavior

50. DYNAMIC UIs - Multimodal Design
Dynamic sound can help user to better understand the effects of
their interactions, especially when using touchscreens.
Water Fire Paper
50

51. DYNAMIC UIs - Analogue Manipulation, Physics, Transition Free

52. DYNAMIC UIs
TRANSITIONS ARE THE NEW STATE
Dynamic Paradigm
UI
UIs support real behaviour.Type
Transition User Reaction
Static Screen No Transition Where am I?
This makes sense when youscreen transition
Static Screen Full use Something happened, I’m
technology that connects your device to somewhere else
the real world.
Static Screen Smart full screen transition I understand where I’ve arrived
Dynamic UI Link flow transition I can do some new things now
Dynamic UI Transformation of UI elements I can repurpose these UI
elements for what I want
Dynamic UI User controlled space I’m casually browsing, but have
control to do what I like

53. DYNAMIC UIs
TRANSITIONS ARE THE NEW STATE
Dynamic UIs support real behaviour.
This makes sense when you use
technology that connects your device to
the real world.

54. DYNAMIC UIs
Dynamic UI Qualities Static State Based UI Qualities
Adaptive Fixed
User expectations are different PC based expectation
Metaphoric, natural behavior Abstract behavior
Empathetic Unforgiving
Creates dialogue with user Directing
Explorative Functional
Playful Task based
Holistic Fragmented
Can communicate soft values Hard values

55. DYNAMIC UIs
Dynamic UI Patterns
Persistency of UI Elements
Generative Visuals
Apt Reactions to Physical Input
Movable Light Sources
Contextual Change
Personalization of Space
Adaptive UI Elements
Synchronized Multimodal Design
Analogue Manipulation Patterns
Free Moving Elements With Physics Models

56. DYNAMIC UIs
Design tools & processes
Clear mental / behavioural models
•The device has a persona
•Range of behaviour and personality is the consistency which pulls a
dynamic UI together - aligns to situations
•Can derive both design patterns & tech requirements
Agile Work Process
•All stakeholders involved
•Horizontal, not vertical process
•Rapid iterations
•Important to sketch on device!

57. Concept Sketching
•wireframes Typical design production and documentation
v2 v2 •functional specifications
•paper prototypes
•flash movies
has difficulty communicating the qualities of a
v1 •UI flows dynamic user experience.
•etc.
The tools we use for sketching also leave us
short as designers in actually testing the
experience of what the UI will be like.
We should be able to experience and communicate the rich behavior
and interactions of our UIs.

58. Concept Sketching
Spatial metaphors can be used to reduce
cognitive load since you can work with
familiar principles.
Using a metaphor can make it easier to
achieve consistency throughout the UI.
Communicating camera angles in 3D UIs
It is the first step in creating a consistent
behavioral model.
Communicating 3D or behvioral UIs within
teams can be troublesome, to have a
spatial metaphor can make discussions
more direct.
A shape alone is not a metaphor.
Sketching of 3D UIs using spatial metaphors
However, don’t go overboard! It’s still a UI and you can break the rules!

59. Concept Sketching

60. Concept Sketching
Shape vs. Metaphor
•Metaphors have behaviour
•You can put your shape in an
environment
•Simple is good
•Spatial metaphors are all about being
consistent
•Like personas for interaction behaviour
•You are testing how the UI behaves and
feels

61. Traditional work process
Planned

62. Traditional work process
Actual

63. Iterative work process with Cascades

64. DYNAMIC UIs
Patterns & Tools
3D, Touch, Gestures
Physics
Generative movement & animation
Multimodal design
Clear mental models, Agile work processes
A dynamic UI experience is about valuing the in-the-moment use of the
mobile phone, and not only the product’s overall usefulness. A dynamic
UI creates a pleasurable experience.

65. DYNAMIC UIs
Patterns & Tools
3D, Touch, Gestures
Physics
Generative movement & animation
Multimodal design
Clear mental models, Agile work processes
There is no best practice.
Our UI design patterns will come from practices of physical
computing, industrial design, and good field research - not the
desktop PC.

66. Inspirational Seminars
•3D
•Android
•Multimodal
•Advanced Displays
•Physics
•Sketching & Prototyping

67. Creating Real Customer Value With 3D
This seminar explores the use of 3D
graphics in UIs on mobile devices
• Inspirational examples
• Framework for analyzing and working with 3D
• What’s the real customer value of 3D?
• Provide a 3D experience from low-end to high-end
devices
• A 3D roadmap to the future
• TAT designs & concepts for you
Typically the palette of "3D techniques" has been comprised of lighting, shadows, focus,
depth, camera angles and similar visual elements. At TAT, we believe that 3D is more
than 3D graphics.
We understand "3D" in user interfaces as a paradigm where the appearance of the
interface displays three-dimensional or real-world qualities, while the technology
behind the visualization is not necessarily built up as a real 3 dimensional simulation. To
benefit from the full potential of 3D in mobile user interfaces we also need to consider
the total user experience.

68. Android Seminar & Workshop
TAT presents an inspiring look at the options
with Android today and tomorrow.
• Inspirational examples for you
• Differentiating your UI on Android
• Configurations for 3D, hardware acceleration, silo apps, more
• Suggested roll-out and strategy for this new platform
• How does the Cascades UI Framework enhance Android?
• How to get better performance and enhance hardware efficiency
• Hands-on demonstration of Android development
The new kid on the block is making waves, but how do we harness all that Android has
to offer? What is Android? What can it do for you today? What can it do for you
tomorrow?
By providing the interaction design and the visual look & feel of the G1, TAT has been
part of Android right from the beginning. Already we have deep experience and insight
into both the technical and UX opportunities available and can help you push the limits
of 3D, unique transitions, work processes, prototyping, and more.
Mixing pragmatism and inspiration TAT will help you address key issues for your
organization with Android.

69. Multimodal UIs - User Experience and WOW
An exploration of a cohesive use of
sound, haptics, & animation
• Inspirational examples
• How can sound and haptics enhance UX?
• A taxonomy for multimodal UIs
• Usability factors & aesthetic concerns
• Synchronizing animations with sound and haptics
• TAT designs & concepts for you
Like films and computer games, the mobile UI can become more engaging and
immersive by including sounds and haptics.
Sound and haptics can improve usability on touchscreen devices where visual
feedback is not always sufficient.
To achieve a true WOW factor, sound, haptics, and visuals must all be synchronized
together.

70. ProCams and the Future of Mobile Displays
Novel displays and the potentials of
ProCams (projectors combined with
cameras)
• Inspirational examples
• Overview of emerging display technologies
• Opportunities and challenges of new displays
• Different aspects of interaction with projected UIs
• New interaction paradigms of flexible and transparent
displays
• TAT designs & concepts for you
Projected graphics will enable mobiles to become far more social than today.
Collaboration can be enabled by using the projection as input surface
Since pressing buttons will interfere with projections, other ways of interaction, such as
camera captured gestures or speech input, become necessary
Flexible and transparent displays open up new interaction possibilities, such as “bend to
zoom”

71. Realism and Physics
New levels of realism in UIs enabled by
physics modeling
• Inspirational examples
• How can physics and realism enhance UX?
• To what parts of a UI can physics be applied?
• How to design realistic output that corresponds to input
gestures
• Benefits and drawbacks of realism
• Simulating physics on low end and high end devices
• TAT designs & concepts for you
Mobiles with touchscreen or motion sensor input can become more intuitive and
impressive if graphics behave realistcally, modeling physics.
UIs that model physics are responsive to subtle gestures, which makes them inviting
and playful.
When used right, realistic UIs can both enhance usability and add a considerable
WOW-factor.

72. Thinking Beyond Flash Prototyping
How to capture creativity and design strong
products through efficient prototyping
• Inspirational workflow examples
• What is a sketch? What is a prototype?
• An overview of prototyping tools and methods
• Communicating ideas through a multitude of sketches
• Useful skills for efficient prototyping
• TAT process and tools for prototyping
Introducing new prototyping methods can open up new opportunities by
increasing product differentiation
At TAT we believe that many important aspects of new products and services can be
designed and evaluated before entering a resource-heavy prototyping phase.
By introducing practices for improved sketching at your company, a more creative and
flexible prototyping culture will emerge.
TAT believes that artefacts radically improve communication and that all aspects of
products, services, processes and business models can be visualized.

73. THANK YOU!