Chapter3

Chapter 3
Evaluation Techniques
Prepared by Mdm PYTan

LEARNING OUTCOME
• Identify the concept of user interface
design to elevate technical complexity
for usable product that accepted by
user. (C3, PLO1)

What are They doing?

What is evaluation?
• Evaluation tests the usability,
functionality and acceptability of an
interactive system
• Tests the system to ensure that it
actually behave as we expect and meet
user requirements

Questions…
• When we should do evaluation process?
• What is the advantage and
disadvantage of doing evaluation during
design process?

Goals of Evaluation
• assess extent of system functionality
• assess effect of interface on user
• identify specific problems

Evaluate the Design by Expert
Analysis
Cognitive Walkthrough
Heuristic Evaluation
Review-based evaluation
Model-based evaluation

Cognitive walkthrough
• Proposed by Polson et al.
• Origin - The code walkthrough familiar in software
engineering
• Walkthrough require a detailed review of a sequence
of actions
• The main focus is to establish how easy a system
is to learn or learning through exploration
– evaluates design on how well it supports user in
learning task
– usually performed by expert in cognitive
psychology
– expert ‘walks though’ design to identify potential
problems using psychological principles
– forms used to guide analysis

• You need FOUR (4) things:
1) A specification or prototype of the system
2) A description of the task the user is to
perform on the system
3) A complete, written list of the actions
needed to complete the task with the
proposed system
4) An indication of who the users are and
what kind of experience and knowledge
the evaluators can assume about them

• Details in step 3:
1) Is the effect of the action the same as the
user’s goal at that point?
2) Will users see that the action is available?
3) Once users have found the correct action,
will they know it is the one they need?
4) After the action is taken, will user
understand the feedback they get?

Cognitive walkthrough : example

Screen shots
Before
After

• Proposed by Nielsen and Molich.
• Is a guideline or general principle or rule
thumb that can guide a design decision or
be used to critique a decision that has
already been made
• Can be performed on a design
specification so it is useful for evaluating
early design
• Is a method for structuring the critique
of a system using a set of relatively
simple and general heuristics

Heuristic evaluation
• It is useful for evaluating early design
• It also can be use on prototypes,
storyboards and fully functioning
systems
• It is flexible and cheap approach
• Heuristic evaluation `debugs' design.

• Nielsen’s ten heuristic:
1. Visibility of system status : Always keep users
informed about what is going on, through
appropriate feedback within reasonable time
2. Match between system and the real world :
the system should speak the user’s language,
with words, phrases and concepts familiar to the
user
3. User control and freedom : users often choose
system functions by mistake and need a clearly
marked ‘emergency exit’ to leave the unwanted
state without having to go through an extended
dialog

Nielsen’s ten heuristic:
4. Consistency and standards : users
should not have to wonder whether words,
situations or actions mean the same thing
in different contexts
5. Error prevention : make it difficult to
make errors
6. Recognition rather than recall : the
users should not have to remember
information from one part of the dialog to
another
7. Flexibility and efficiency of the use :
allow users to tailor frequent actions

Nielsen’s ten heuristic:
8. Aesthetic and minimalist design :
dialogs should not contain information
that is irrelevant or rarely needed
9. Help users recognize, diagnose and
recover from
errors : error messages should be
expressed in plain language
10. Help and documentation : system can
be used with no instructions so it may be
necessary to provide help and
documentation

Neilson’s Heuristics (Improved)
• H2-1: Visibility of system status
• H2-2: Match between system & real world
• H2-3: User control & freedom
• H2-4: Consistency & standards
• H2-5: Error prevention
• H2-6: Recognition rather than recall
• H2-7: Flexibility and efficiency of use
• H2-8: Aesthetic and minimalist design
• H2-9: Help users recognize, diagnose, and
recover from errors
• H2-10: Help and documentation

H2-1: Visibility of system status
• keep users informed about what is
going on
– example: pay attention to response time
• 0.1 sec: no special indicators needed, why?
• 1.0 sec: user tends to lose track of data
• 10 sec: max. duration if user to stay focused
on action
• for longer delays, use percent-done progress
bars
searching database for matches

H2-1: Visibility of system status
• keep users informed about what is
going on
– Appropriate visible feedback
What did I
select?
What mode
am I in now?
How is the
system
interpreting
my actions?

H2-2: Match between system & real world
• speak the users’ language
• follow real world conventions
• (bad) example: Mac desktop
– Dragging disk to trash
• should delete it, not eject it!

H2-2: Match between system & real world
• speak the users’ language
• follow real world conventions
• e.g. withdrawing money from a bank
machine

H2-3: User control & freedom
 “exits” for mistaken choices, undo, redo
 don’t force down fixed paths
How do
I get
out of
this?

H2-3: User control & freedom
 “exits” for mistaken choices, undo, redo
 don’t force down fixed paths
 Strategies:
– Cancel button (for dialogs waiting for user input)
– Universal Undo (can get back to previous state)
– Interrupt (especially for lengthy operations)
– Quit (for leaving the program at any time)
– Defaults (for restoring a property sheet)

H2-4: Consistency & standards
• Consistency of effects
– same words, commands, actions will always have the same effect in
equivalent situations
• predictability
• Consistency of language and graphics
– same info/controls in same location on all screens/dialog boxes
– forms follow boiler plate
– same visual appearance across the system (e.g. widgets)
• e.g. different scroll bars in a single window system!
• Consistency of input
– consistent syntax across complete system

H2-4: Consistency & standards
These are labels with a
raised appearance.
Is it any surprise that
people try and click on
them?

H2-5: Error prevention
• try to make errors impossible
• modern widgets: only “legal commands”
selected, or “legal data” entered
• Provide reasonableness checks on input data
– on entering order for office supplies
• 5000 pencils is an unusually large order. Do you really
want to order that many?

H2-6: Recognition rather than recall
• Computers good at remembering things, people aren’t!
• Promote recognition over recall
– menus, icons, choice dialog boxes vs command lines, field
formats
– relies on visibility of objects to the user (but less is more!)

H2-7: Flexibility and efficiency of use
• Experienced users should be able to perform
frequently used operations quickly
• Strategies:
– keyboard and mouse accelerators
• abbreviations
• command completion
• menu shortcuts
• function keys
• double clicking vs menu selection
– type-ahead (entering input before the system is
ready for it)
– navigation jumps
• e.g., going to window/location directly, and avoiding
intermediate nodes
– history systems
• WWW: ~60% of pages are revisits

H2-7: Flexibility and efficiency of use
Keyboard
accelerators for
menus
Customizable
toolbars and
palettes for
frequent actions
Split menu, with
recently used
fonts on top
Scrolling controls for page-
sized increments
Double-click
raises object-
specific menu
Double-click
raises toolbar
dialog box

H2-8: Aesthetic and minimalist design
• No irrelevant information in dialogues
• Bad example:

H2-9: Help users recognize, diagnose,
and recover from errors
• error messages in plain language
• precisely indicate the problem
• constructively suggest a solution

H2-10: Help and documentation
• Help is not a replacement for bad design!
• Simple systems:
– walk up and use; minimal instructions
• Most other systems:
– feature rich
– some users will want to become “experts” rather than “casual” users
– intermediate users need reminding, plus a learning path
• Many users do not read manuals
– prefer to spend their time pursuing their task
• Usually used when users are in some kind of panic, need immediate help
– indicates need for online documentation, good search/lookup tools
– online help can be specific to current context
– paper manuals unavailable in many businesses!
• e.g. single copy locked away in system administrator’s office
• Sometimes used for quick reference
– syntax of actions, possibilities...
– list of shortcuts ...

Types of help (cont.)
• Reminders
– short reference cards
• expert user who just wants to check facts
• novice who wants to get overview of system’s
capabilities
– keyboard templates
• shortcuts/syntactic meanings of keys; recognition
vs. recall; capabilities
– tooltips
• text over graphical items indicates their meaning
or purpose

3 stages for doing heuristic
evaluation
• Briefing session to tell experts what to
do.
• Evaluation period of 1-2 hours in which:
– Each expert works separately;
– Take one pass to get a feel for the product;
– Take a second pass to focus on specific
features.
• Debriefing session in which experts
work together to prioritize problems.

Before
After

• Results from the literature used to
support or refute parts of design.
• Care needed to ensure results are
transferable to new design.

• Certain cognitive and design models
provide a means of combining design
specification and evaluating into the
same framework
• The GOMS (goals, operators, methods
and selection) model predicts user
performance with a particular
interface and can be used to filter
particular design options

• GOMS:
– Goals : specify what the user wants and
intends to achieve.
– Operators : are the building blocks for
describing human-computer interaction at the
concrete level.
– Methods : are programs built with operators
that are designed to accomplish goals.
– Selection : rules predict which method will be
used. For example, “If the mouse is working,
select point to an item on screen, if not, select
choose OPEN option in file menu”.

Using GOMS
Repeat
until
Satisfied
Choose Top Level
Goals
Start
Draft Operators and
Methods for Each Goal
Select Methods
Check for Goal Attainment
Further
Refinement
Needed?
Stop
No
Yes
Figure 3.1 A flow chart for building GOMS

• Design methodologies, such as design
rationale also have role to play in
evaluation at the design stage
• Dialog models can also be used to
evaluate dialog sequences for problems
• Eg. Unreachable states, circular dialogs
and complexity.

Group Activity (3 persons)
• Choose ONE application or system or
website or product.
• Use them.
• Discuss your evaluation through expert
analysis based on Cognitive Walkthrough
criteria.
• Identify any problem with the system /
application / website / product based on
the task that you had specified. (if any)
• Suggest how to solve the problem(s).

Group Activity
Discuss the Advantages and Disadvantages
of Each type of Evaluating Designs below:
Cognitive Walkthrough

THANK YOU

Positives :
• Real insights gained into potential issues with
new users;
• Extremely practical and easy to implement;
• Encourages team members to view with a
fresh eyes;
• Throws up surprises.
Negatives
• Can be subject to personal ‘quirks’;
• Not always easy to draw generalisable
conclusions;
• Danger of over-reaction.

Positives:
• Navigation and design flaws highlighted;
• External input is more objective;
• Severity rating allows easy setting of priorities.
• Provides reassurance about things that are going
well
Negatives:
• Need to strike a balance between carrying out
formal evaluation early enough to inform the
design process, and maximising cost-
effectiveness;
• Some of the points made referred to ‘known
issues’;

Advantages and problems
heuristic evaluation
• Few ethical & practical issues to
consider because users not involved.
• Can be difficult & expensive to find
experts.
• Best experts have knowledge of
application domain & users.
• Biggest problems:
– Important problems may get missed;
– Many trivial problems are often identified;
– Experts have biases.

Model-Based Evaluation
Advantages
• a particular interaction
can be calculated with
little effort, at little
cost, and in a short
amount of time
• each specific task has
been previously
measured
experimentally to a
high degree of
accuracy
Disadvantages
• generally less rigid
with basic cognitive
actions
• none of the GOMS
models allow for any
type of error

Review-Based Evaluation
Advantages
• Expert review of
relevant literature can
avoid to repeat
experiments
continually
Disadvantages
• Expensive to repeat
experiments
continually

Chapter3

Recommandé

Recommandé

Contenu connexe

Similaire à Chapter3

Similaire à Chapter3 (20)

Dernier

Dernier (20)

Chapter3