Contenu connexe



  1. chapter 8 (modified) implementation support
  2. Introduction how does HCI affect the programmer? levels of abstraction hardware specific → interaction-technique specific layers of development tools – windowing systems – interaction toolkits – architecture styles and frameworks
  3. windowing systems device independence resource sharing application management
  4. windowing systems device independence different devices: mouse, trackpad, joystick keyboard layouts screen resolution
  5. device independence layers of abstraction Application: what the user really wants to do Toolkit: (e.g. Java AWT) more abstraction! Windowing System: abstract pointer (mouse) abstract display: pixels, Postscript, vector graphics Operating System: hardware interface, low-level device drivers
  6. windowing systems resource sharing (usually) many one screen one pair of eyes applications one keyboard one set of fingers one mouse
  7. resource sharing window system virtual terminals abstract abstract abstract abstract each app its own PC! terminal 1 term. 2 term. 3 term. 4 OS: resource manager shares disk, CPU, etc. device drivers
  8. types of window layout • tiled (space shared) – web sidebars, expanding widgets • single app (time shared) – phone, early Windows • overlapping (time and space) – most common for PCs
  9. resource issues not just the screen! users care about power management network utilisation ... especially when it costs!
  10. windowing systems application management many applications ... so – consistent look and feel – inter-application sharing cut & paste, drag & drop scripting and automation – UI to manage it all e.g. Mac Finder, Windows Explorer app swopping, preferences +extras: dashboard
  11. Architectures of windowing systems three possible software architectures – all assume device driver is separate – differ where management is implemented early Mac & MS Windows 1. each application manages everything assumed ‘cooperative’ apps – everyone worries about synchronization – poor portability and robustness may be used for – can be efficient for low-end devices dedicated devices 2. window mgt. tightly coupled with operating system – applications tied to operating system Mac OS and MS Windows 3. management role as separate application – maximum portability X Windows
  12. X Windows architecture
  13. X Windows architecture (ctd) pixel imaging model with some pointing mechanism X protocol defines server-client communication separate window manager client enforces policies for input/output: – how to change input focus – tiled vs. overlapping windows – inter-client data transfer
  14. not just the PC phone – similar issues – sharing screen, keyboard, etc. – ‘full screen’ apps, not overlapping windows but Vodafone 360 had semi-open apps web – browser takes role of window manager web micro-apps (e.g. Facebook apps) – access shared data, screen space dedicated devices (e.g. microwave control panel) – mostly code direct to hardware but appliance-oriented Java, Windows, ...
  15. programming the application toolkits paint models event models
  16. toolkits what they provide interaction objects (widgets) e.g. menus, buttons, simple text areas, file selection N.B. input and output intrinsically linked layout support for resizable windows abstractions for windowing & OS services e.g. copy/paste, sound
  17. toolkits higher level of abstraction than raw WM promotes consistency – look and feel, widget interactions easier to code – amenable to object-oriented programming easier to port – new versions of same platform – cross-platform • e.g. Java for desktop apps, jQuery for web
  18. interfaces in Java Java toolkit – AWT (abstract windowing toolkit) Java classes for buttons, menus, etc. Notification based – AWT 1.0 – need to subclass basic widgets – AWT 1.1 and beyond – callback objects Swing toolkit – built on top of AWT – higher level features – uses MVC architecture (see later)
  19. paint models what appears on the screen and when it happens
  20. screen painting layers screen your application your code toolkit layer(s) void main() { > cat fred.txt Frame f = new Frame(); This is frd.txt AWT > ls } fred.txt tom.txt Swing harry.c other application(s) > cat fred.txt This is frd.txt > ls fred.txt tom.txt harry.c
  21. paint models what do you draw direct to the screen direct to screen via viewport – clipping, coordinate transformation e.g. raw Java AWT separate buffer – double buffer to reduce flicker, retained bitmap option in Java Swing display list / model – list of operations line, image etc. toolkit worries about showing these the screen e.g. OpenGL, ? MacOS display PDF, JS-DOM
  22. buffering in toolkit and/or window manager e.g. Mac OS X three levels: • nonretained – no buffering • retained – wm buffers hidden parts • buffered – app/toolkit writes to buffer
  23. paint models when do you need to draw internal events – data has changed – application ‘knows’ external events – user / network does things – window manager ‘knows’ … and then tells toolkit
  24. paint models when do you actually draw internal control – code does things when it wants – but what about external events? works OK with display list or retained bitmap external control – called by the toolkit / window manager – but what about internal events? … purpose of repaint() in Java draw once per frame (game engines) – variant of external control … but code rather like internal combined with polling model for input (check button state)
  25. event models when things happen
  26. event models – when things happen read-evaluation loop repeat read-event(myevent) case myevent.type type_1: do type_1 processing type_2: do type_2 processing ... type_n: do type_n processing end case end repeat
  27. event models notification-based void main(String[] args) { Menu menu = new Menu(); menu.setOption(“Save”); menu.setOption(“Quit”); menu.setAction(“Save”,mySave) menu.setAction(“Quit”,myQuit) ... } int mySave(Event e) { // save the current file } int myQuit(Event e) { // close down }
  28. going with the grain notification style affects the interface – modal dialogue box • easy with event-loop (just have extra read-event loop) • hard with notification (need lots of mode flags) – non-modal dialogue box • hard with event-loop (very complicated main loop) • easy with notification (just add extra handler) beware! if you don’t explicitly design, it will just happen implementation should not drive design
  29. screen your code java api AWT invokes Listener repaint sets myListener(…) { user ‘dirty’ flag // update state clicks repaint(); } notices component dirty flag paint(Graphics g) { asked to g.fillRect(…) paint itself // draw things } waits for more input …
  30. asynchrony ... things happen in any order hard to test code – usually one test user at a time – low network load – fast networks and fast machines race conditions – unexpected event orders – sometimes may seem unlikely, but ...
  31. Birthday surprise problem people connections 1 0 2 1 3 3 4 6 5 10 6 15 … … n n(n-1)/2
  32. architecture and frameworks separation and presentation abstraction: Seeheim component models: MVC and PAC
  33. origins of separation User Interface Management Systems (UIMS) a level above toolkits (e.g. for non-programmers) separation: application semantics and presentation – colours & wording different from deep functionality improves: – portability – runs on different systems – reusability – components reused cutting costs – multiple interfaces – accessing same functionality – customizability – by designer and user virtually extinct (!) – but the concepts live on – the Seeheim workshop ...
  34. Seeheim model lexical syntactic semantic Functionality Dialogue USER USER Presentation (application APPLICATION Control interface) switch
  35. conceptual vs. implementation Seeheim – arose out of implementation experience – but principal contribution is conceptual – concepts part of ‘normal’ UI language … because of Seeheim … … we think differently! e.g. the lower box, the switch • needed for implementation presentation dialogue application • but not conceptual
  36. semantic feedback different kinds of feedback: – lexical – movement of mouse – syntactic – menu highlights – semantic – sum of numbers changes semantic feedback often slower – use rapid lexical/syntactic feedback but may need rapid semantic feedback – freehand drawing – highlight trash can or folder when file dragged
  37. what’s this? lexical syntactic semantic Functionality Dialogue USER USER Presentation (application APPLICATION Control interface) switch
  38. the bypass/switch lexical syntactic semantic Functionality Dialogue USER USER Presentation (application APPLICATION Control interface) switch direct communication rapid semantic between application and presentation feedback but regulated by dialogue control
  39. monolithic vs. components Seeheim has big components often easier to use smaller ones – esp. if using object-oriented toolkits Smalltalk used MVC – model–view–controller – model – internal logical state of component – view – how it is rendered on screen – controller – processes user input
  40. MVC model - view - controller view model controller
  41. MVC issues • MVC is largely pipeline model: input → control → model → view → output • but in graphical interface – input only has meaning in relation to output e.g. mouse click – need to know what was clicked – controller has to decide what to do with click – but view knows what is shown where! • in practice controller ‘talks’ to view – separation not complete
  42. PAC model • PAC model closer to Seeheim – abstraction – logical state of component – presentation – manages input and output – control – mediates between them • manages hierarchy and multiple views – control part of PAC objects communicate • PAC cleaner in many ways … but MVC used more in practice (e.g. Java Swing)
  43. PAC presentation - abstraction - control A P A P C C abstraction presentation control A P C A P C
  44. ... and on the web
  45. on the web salami sliced Seeheim between browser and server NOT the obvious split presentation dialogue semantics browser web server
  46. on the web salami sliced Seeheim between browser and server data updated on page (esp. AJAX) presentation dialogue semantics browser links and css and javascript browser styling updating server backend databases web server scripts templates, XLST
  47. on the web salami sliced Seeheim between pages/scripts – consistency and control? presentation dialogue semantics script 1 script 2 script 3 script 4 CSS and dialogue flow business logic templates maintaining state
  48. Summary Levels of programming support tools • Windowing systems – device independence, resource sharing – managing multiple user tasks and applications • Toolkits – programming interaction objects • Programming the application – event paradigms: read-evaluation vs. notification – paint paradigms • Architecture – conceptual architectures for separation – need to think about dialogue and state

Notes de l'éditeur