Slides from the Introduction and Theoretical Foundations of New Media course of the Interactive Media and Knowledge Environments master program (Tallinn University).

1. Introduction and Theoretical Foundations of New Media Interaction Styles

2. Contents Etymology The relation between the evolution of computing and the main interaction styles The technological hype cycle and adoption timings Related knowledge domains Beyond interacting with digital media David Lamas, TLU, 2011 2

3. Personal computing 1978 It is generally thought that a computer must cost under USD 1000.00 to have mass-market appeal. A machine at that price today is a minimal computer system. It has as little as 8KB of user memory, uses audio cassettes for mass storage, and has a CRT display for output. Today’s computer is programmed in BASC. Small amounts of application software are available on cassettes. David Lamas, TLU, 2011 3

4. Data storage 1978 A new approach to storing data in computers, using a tunable dye laser, is described in US Patent 4,101,976 awarded to scientists at IBM’s San Jose Research Laboratory. Based on a photochemical process called ‘hole burning’, the new system provides a unique method for increasing the amount of information that can be packed into a given space. David Lamas, TLU, 2011 4

5. Mobile computing 1994 Recent advances in technology have provided portable computers with wireless interfaces that allow networked communication even while a user is mobile. Whereas today’s first-generation notebook computers and personal digital assistants are self-contained, networked mobile computers are part of a greater infrastructure. Mobile computing will very likely revolutionize the way we use computers. David Lamas, TLU, 2011 5

6. Interaction styles Inter Among, between Action the fact or process of doing something, typically to achieve an aim Interaction Reciprocal action or influence Interaction is a kind of action that occurs as two or more objects have an effect upon one another The idea of a two-way effect is essential in the concept of interaction, as opposed to a one-way causal effect Style A manner of doing something A way of painting, a way of writing… A way of interacting David Lamas, TLU, 2011

7. Interaction styles In our case… Ways of interacting with and through interactive media Ways of communicating with and by means of computerized environments David Lamas, TLU, 2011 7

8. The evolution of computing David Lamas, TLU, 2011 8 Waldner, J-B. 2007. Nano-informatiqueet intelligence ambiante: inventerl'ordinateurdu XXIe siècle. Hermes Science Publications, 2007

9. The evolution of computing David Lamas, TLU, 2011 9 Waldner, J-B. 2007. Nano-informatiqueet intelligence ambiante: inventerl'ordinateurdu XXIe siècle. Hermes Science Publications, 2007

10. Main interaction styles Command line interfaces Graphical user interfaces Natural user interfaces David Lamas, TLU, 2011 10

11. The evolution of computing David Lamas, TLU, 2011 11 Waldner, J-B. 2007. Nano-informatiqueet intelligence ambiante: inventerl'ordinateurdu XXIe siècle. Hermes Science Publications, 2007

12. Physical programming David Lamas, TLU, 2011 12 In the beginning it was all about interacting with the computer

13. Card punching and reading David Lamas, TLU, 2011 13 Batch processing

14. A teletypewriter David Lamas, TLU, 2011 14 The birth of the Command Line Interface

15. Early graphic workstation David Lamas, TLU, 2011 15 An initial Graphic User Interface

16. A mouse prototype David Lamas, TLU, 2011 16 Invented by Douglas Engelbart

17. A video-display unit David Lamas, TLU, 2011 17 The oN-Line System featuring a display, a keyboard and mouse

18. The oN-Line System …or the Augmentation of Human Intellect A system envisioned by Douglas Engelbart, to help Increasing the capability of a man to approach a complex problem situation, to gain comprehension to suit his particular needs, and to derive solutions to problems Increased capability in this respect is taken to mean a mixture of the following: more-rapid comprehension, better comprehension, the possibility of gaining a useful degree of comprehension in a situation that previously was too complex, speedier solutions, better solutions, and the possibility of finding solutions to problems that before seemed insolvable Complex situations we include the professional problems of diplomats, executives, social scientists, life scientists, physical scientists, attorneys, designers--whether the problem situation exists for twenty minutes or twenty years… David Lamas, TLU, 2011 18 http://www.dougengelbart.org/pubs/augment-3906.html

19. The oN-Line System The system was called oN-Line System, because it was also networked between multiple computers Computers were no longer isolated The display system was based on vector graphics technology and could display both text and solid lines on the same screen Because of limited memory space in the mainframe computer, it could only display upper-case characters, although true upper-case was displayed by the use of a short horizontal line directly above any capitalized letters David Lamas, TLU, 2011 19

20. The evolution of computing David Lamas, TLU, 2011 20 Waldner, J-B. 2007. Nano-informatiqueet intelligence ambiante: inventerl'ordinateurdu XXIe siècle. Hermes Science Publications, 2007

21. The Xerox Alto David Lamas, TLU, 2011 21 The Alto was not a microcomputer as such, although its components did fit under a desk

22. The Xerox Star David Lamas, TLU, 2011 22 The Star had some differences from the Alto, most significantly the ability to overlap windows was removed as it was thought too confusing for the general public…

23. The Apple Lisa David Lamas, TLU, 2011 23 The Lisa user interface invented some of the Graphical User Interface concepts we still use today. Icons could represent all files in the system and the drag and drop was used for file

24. The Apple Macintosh David Lamas, TLU, 2011 24

25. The Apple Macintosh David Lamas, TLU, 2011 25

26. Other early graphic user interfaces David Lamas, TLU, 2011 26

27. The evolution of computing David Lamas, TLU, 2011 27 Waldner, J-B. 2007. Nano-informatiqueet intelligence ambiante: inventerl'ordinateurdu XXIe siècle. Hermes Science Publications, 2007

28. A graphic user interface timeline Examples of graphic user interface styles are… Menu selection Forms fill-in Direct manipulation Metaphors (ie. The desktop) Web navigation David Lamas, TLU, 2011 28

29. The evolution of computing David Lamas, TLU, 2011 29 Waldner, J-B. 2007. Nano-informatiqueet intelligence ambiante: inventerl'ordinateurdu XXIe siècle. Hermes Science Publications, 2007

30. The evolution of computing David Lamas, TLU, 2011 30 Waldner, J-B. 2007. Nano-informatiqueet intelligence ambiante: inventerl'ordinateurdu XXIe siècle. Hermes Science Publications, 2007

31. Natural user interfaces …is the common designation used by designers and developers of computer interfaces to refer to a user interface that is effectively invisible, or becomes invisible with successive learned interactions, to its users The word natural is used because most computer interfaces use artificial control devices whose operation has to be learned Such an interface relies on a user being able to carry out relatively natural motions, movements or gestures that they quickly discover control the computer application or manipulate the on-screen content David Lamas, TLU, 2011 31

32. Natural user interfaces The most distinct identifier of a natural user interface is the lack of a physical keyboard and or mouse Hence, the most common examples are… (multi-)touch interfaces; and voice-operated interfaces The natural user interface removes the metaphors, and many of the artificially learned devices, to allow users to more directly manipulate content using more natural movements, motions and gestures Enthusiast defend that these interfaces are fast to learn and, as such, freely apply the adjective 'intuitive’ to describe how users interact with them David Lamas, TLU, 2011 32

33. Perceptive pixel David Lamas, TLU, 2011 33

34. Microsoft Surface David Lamas, TLU, 2011 34

35. Xbox Kinect David Lamas, TLU, 2011 35

36. Natural user interfaces are… not natural According to Don Norman Fundamental principles of knowledge of results, feedback, and a good conceptual model still rule The strength of the graphical user interface has little to do with its use of graphics It has to do with the ease of remembering actions, both in what actions are possible and how to invoke them Visible icons and visible menus are the mechanisms, and despite the well-known problems of scaling up to the demands of modern complex systems, they still allow one to explore and learn The important design rule of a GUI is visibility: through the menus, all possible actions can be made visible and, therefore, easily discoverable. The system can often be learned through exploration Systems that avoid these well-known methods suffer. Are natural user interfaces natural? No, he says, but they will be useful. David Lamas, TLU, 2011 36

37. Other user interfaces David Lamas, TLU, 2011 37

38. Other user interfaces David Lamas, TLU, 2011 38

39. Other user interfaces David Lamas, TLU, 2011 39

40. Technological hype cycle David Lamas, TLU, 2011 40 Linden, A. and Fenn, J. 2003. Understanding Gartner's Hype Cycles. Strategic Analysis Report R-20-1971. 30 May 2003. Gartner Research.

41. Technological hype cycle David Lamas, TLU, 2011 41

42. Technological hype cycle and adoption timings David Lamas, TLU, 2011 42 Linden, A. and Fenn, J. 2003. Understanding Gartner's Hype Cycles. Strategic Analysis Report R-20-1971. 30 May 2003. Gartner Research.

43. Related knowledge domains Human-computer interaction The study of how people interact with computers and to what extent computers are or are not developed for successful interaction with human beings Recent advances in mobile, ubiquitous, social, and tangible computing technologies have moved human-computer interaction into practically all areas of human activity This has led to a shift away from the usual stress on usability to a much richer scope of user experience, where user's feelings, motivations, and values are given as much, if not more, attention than efficiency, effectiveness and basic subjective satisfaction David Lamas, TLU, 2011 43

44. Related knowledge domains Interaction design A design discipline dedicated to defining the behavior of artifacts, environments and systems User experience design The field of user experience was established to cover the holistic perspective to how a person feels about using a system The focus is on pleasure and value rather than on performance David Lamas, TLU, 2011 44

45. Human-computer interaction David Lamas, TLU, 2011 45

46. Human-computer interaction David Lamas, TLU, 2011 46

47. Interaction design David Lamas, TLU, 2011 47

48. User experience design David Lamas, TLU, 2011 48

49. User experience design David Lamas, TLU, 2011 49

50. But… Interactivityis not limited to technological systems People have been interacting with each other as long as humans have been a species From this broader viewpoint, reasoning about interaction styles should also address the interaction between human beings by means of a interactive media David Lamas, TLU, 2011 50

51. Interaction styles recap Etymology The relation between the evolution of computing and the main interaction styles The technological hype cycle and adoption timings Related knowledge domains Beyond interacting with digital media David Lamas, TLU, 2011 51

52. Two final questions How do you see the evolution of interacting with and through interactive media? Are natural user interfaces the future or part of the future? Are the previous interaction styles dead or condemned? Should mobile user interfaces be regarded as a completely new interaction style? If so, what would their distinctive characteristics be? David Lamas, TLU, 2011 52