Usability testing (or user testing) involves measuring the ease with which users can complete common tasks on your website. The results of the analysis are a huge eye-opener and their implementation often leads to: Increased sales and task completion and a high rate of return site visitors A greatly improved understanding of your customers’ needs A significant reduction in call centre enquiries A much more user-focused in-house development team Source: http://www.wbcsoftwarelab.com/wbcblog/read-basics-of-usability-testing

1. Usability testing
Usability testing (or user testing) involves measuring the ease with which users can complete
common tasks on your website. The results of the analysis are a huge eye-opener and their
implementation often leads to:
 Increased sales and task completion and a high rate of return site visitors
 A greatly improved understanding of your customers' needs
 A significant reduction in call centre enquiries
 A much more user-focused in-house development team
 Usability testing is truly an eye-opening experience. You'll have the opportunity to witness
your site visitors struggle to complete what you perceive to be the easiest and most intuitive of
tasks.
Deliverables from the usability testing
Written report & presentation of usability testing findings
The usability testing report is very much recommendation-led and highlights the problem areas of
the site from the usability testing. Clear recommendations are provided for how to remedy each issue.
Recommendations are defined as high, medium and low priority, as appropriate.
The usability testing report is written in layman's terms with all relevant issues explained
thoroughly and appropriate screenshots included.
Webcredible will also make a short presentation of the main findings from the usability testing. We'll
shows video clips during the presentation to highlight some of the main points. We'll then run an
implementation workshop whereby we all brainstorm the best possible way to implement our
recommendations.
DVDs of usability testing
We can provide you with a DVD featuring all of the usability testing videos, as well as a DVD showing
some of the main highlights from the usability testing. The highlights DVD is a great way to quickly
show the usability testing to colleagues who were unable to watch the testing live.
Our usability testing facilities
The usability testing takes place in our purpose-built usability lab, complete with microphones,
cameras and eye tracking. Using state-of-the-art software we're able to capture what the user is
doing on-screen, together with audio commentary and a video of the user carrying out the test.

2. There's a viewing room next door with a large one-way mirror between the two rooms so you can
see the testing live, in real time.
Additionally, a 45 inch plasma TV is mounted in the corner of the viewing room. A screenshot of the
participant's computer screen along with a picture of their face is displayed in real-time on the TV:
We strongly advise you watch the usability testing live - it's truly an eye-opening experience. The
viewing room is an airy room with wireless Internet access, a large table and a number of comfortable
chairs. You also get access to our break out area, where you can relax on our comfy sofa between
sessions and during lunchtime. Refreshments are provided throughout your time here.
During the usability testing
During the usability testing participants are instructed to think aloud at all times. If they ever fall
silent for more than a few seconds they're encouraged to keep talking. Our usability testers are
expertly trained to ask questions and elicit thoughts in a non-biased and neutral manner.
Participants are assigned specific tasks to carry out and their ability to complete these tasks is
monitored. Once the testing is complete we'll conduct a brief interview with each person. We'll
typically conduct the usability testing with 6-12 participants per user group.
Before the usability testing
Before the usability testing is carried out we work closely with you to identify your precise goals for
both the website and the usability testing. Together, we identify the different user groups using
your site and source a number of people representative of your target audiences.
We then design and prepare a list of tasks for participants to complete so they're going through
realistic journeys on your site. As standard, we run a pilot testing session before launching into the
full usability testing.
User-centered design (UCD) - 6 methods

3. User-centered design (UCD) is a project approach that puts the intended users of a site at the centre
of its design and development. It does this by talking directly to the user at key points in the
project to make sure the site will deliver upon their requirements.
The stages are carried out in an iterative fashion, with the cycle being repeated until the project's
usability objectives have been attained. This makes it critical that the participants in these methods
accurately reflect the profile of your actual users.
ISO 13407 outlines four essential activities in a user-centered design project:
 Requirements gathering - Understanding and specifying the context of use
 Requirements specification - Specifying the user and organisational requirements
 Design - Producing designs and prototypes
 Evaluation - Carrying out user-based assessment of the site
The following is a typical top-level characterisation of the most popular user-centered design
methods:
Method Cost Output Sample size When to use
Focus groups Low Non-statistical Low Requirements gathering
Usability testing High Statistical & non-
statistical
Low Design & evaluation
Card Sorting High Statistical High Design
Participatory
design
Low Non-statistical Low Design
Questionnaires Low Statistical High Requirements gathering &
evaluation
Interviews High Non-statistical Low Requirements gathering &
evaluation
Focus groups
What are they?
A focus group involves encouraging an invited group of intended/actual users of a site (i.e.
participants) to share their thoughts, feelings, attitudes and ideas on a certain subject.

4. Organising focus groups within an organisation can also be very useful in getting buy-in to a project
from within that company.
When to use
Focus groups are most often used as an input to design. They generally produce non-statistical data
and are a good means of getting information about a domain (e.g. what peoples' tasks involve).
Issues
It's necessary to have an experienced moderator and analyst for a focus group to be effective.
Usability testing
What is it?
Usability testing sessions evaluate a site by collecting data from people as they use it. A person is
invited to attend a session in which they'll be asked to perform a series of tasks while a moderator
takes note of any difficulties they encounter.
Users can be asked to follow the think-aloud protocol which asks them to verbalise what they're
doing and why they're doing it.
You can also time users to see how long it takes them to complete tasks, which is a good measure of
efficiency (although you should bear in mind that using the 'think aloud' protocol will slow users down
considerably).
Two specialists' time is normally required per session - one to moderate, one to note problems.
When to use
Usability testing can be used as an input to design or at the end of a project. It represents an
excellent way finding out what the most likely usability problems with a site are likely to be.
Usability testing can be used generate non-statistical or statistical data.
Issues
Usability testing requires some form of design to be available to test - even if it's only on paper.
Testing works best if it focuses either on gathering non-statistical feedback on a design through 'talk
aloud' or statistical measures.
Card sorting

5. What is it?
Card sorting is a method for suggesting intuitive structures/categories. A participant is presented
with an unsorted pack of index cards. Each card has a statement written on it that relates to a page of
the site.
The participant is asked to sort these cards into groups and then to name these groups. The results of
multiple individual sorts are then combined and analysed statistically.
When to use
Card sorting is usually used as an input to design. It's an excellent way of suggesting good
categories for a site's content and deriving its information architecture.
Card sorting can be used generate statistical data.
Issues
Providing participants with a trial run on some easy cards (e.g. sports, animals, etc.) can reassure
about what they are expected to do and result in a more productive session.
Participatory design
What is it?
Participatory design does not just ask users opinions on design issues, but actively involves them in
the design and decision-making processes.
When to use
Participatory design is usually used within a mini-project to generate prototypes that feed into an
overall project's design process.
An example would be a participatory design workshop in which developers, designers and users work
together to design an initial prototype. This initial prototype would then feed into a more traditional
design process.
Projects which only utilise participatory design are very rare.
Issues
Participatory design sessions can be very fluid and require an experienced moderator with thorough
knowledge of the domain to guide them.
Questionnaires

6. What are they?
Questionnaires are a means of asking users for their responses to a pre-defined set of questions and
are a good way of generating statistical data.
When to use
Questionnaires are usually employed when a design team:
 Can only gain remote access to users of a site
 Is seeking a larger sample size than can be realistically achieved through direct contact
It is for this reason that questionnaires are usually administered through post or electronic means.
Issues
Questionnaires allow statistical analysis of results, which can increase a study's credibility through its
scientific appearance. This makes it all the more important that the questionnaire is well-designed and
asks non-biased questions.
Interviews
What are they?
An interview usually involves one interviewer speaking to one participant at a time.
The advantages of an interview are that a participant's unique point of view can be explored in detail.
It is also the case that any misunderstandings between the interviewer and the participant are likely
to be quickly identified and addressed.
The output of an interview is almost exclusively non-statistical - it's critical that reports of interviews
are carefully analysed by experienced practitioners.
When to use
Interviews are usually employed early in the design process in order to gain a more detailed
understanding of a domain/area of activity or specific requirements.
Issues
Interviewing places a high premium on the experience and skill of the interviewer and analyst.
Conclusion

7. This has been an introduction to the major user-centered design methods. It's vital to remember that
although each can be extremely valuable, using them in the right way, for the right reasons and at the
right time is critical.
Exactly which method to use, and when and how to use it will differ from project to project.
Split A/B testing
Following on from any website usability study a number of usability problems are usually found. There
can often be debate within any organisation as to the best solution for each problem, with no one
really knowing the optimal solution. Rather than letting the person that shouts the loudest get his or
her own way, a better solution can be to test 2 solutions in a live environment. Whichever
performs the best is clearly the superior solution. Welcome to split A/B testing!
Buridan's ass and A/B testing
Did you hear the story of the donkey stood in the middle of 2 equally appealing stacks of hay? He
spent so long trying to decide which one to start eating first that the poor animal starved to death
without having moved a single inch. This paradox, first discussed by Aristotle is known as 'Buridan's
ass' and is an often discussed psychological phenomenon.
If you're managing a website you might face similar situations when you need to decide which of 2
different designs to opt for. Nowadays, you need to continuously improve and evolve your site by
making small frequent adjustments. But how do you know which change will have the highest impact
on the customer experience?
Split A/B testing is a way of finding out which changes help your users' performance. It
provides a controlled method of measuring the effectiveness (or not) of alterations to your site. It's
often used for small tweaks (e.g. 'Is this style heading clearer than the original?') but can also be used
to test bigger wholesale changes (e.g. 'Is this new 1-click checkout process better?'). In essence, it
involves running 2 different versions side by side to see which is more effective.
A typical A/B scenario
You feel your users might not be finding the 'Proceed to checkout' button hidden below the fold of the
page and think this might be causing people to leave your site before making a purchase. You've
created a new design which you feel is more effective but you'd like to know for sure.
To A/B test your new page, you serve the regular page to say 90% of your visitors as usual, but a
randomly selected 10% would be shown your new design. Then you sit back, pour yourself an ice
cold gin and tonic, wait and watch your web statistics.

8. If your new page has a positive effect as you suspect, then you should see an improvement in the
10% group, as measured by conversion rates. Proof that you should publish the more successful page
to all visitors.
Advantages of split A/B testing
There are a number of benefits to A/B testing:
 Low risk approach
 Cheaper than other methods such as focus groups
 Provides proof
 Invisible to most of your users
 Great way to do 'test run' new designs before full roll-out (to avoid negative surprises on the
launch day)
 Can solve internal disputes
Disadvantages of split A/B testing
A/B testing isn't always suitable. Some of its disadvantages include:
 New designs might have to 'wear in' before you can measure their real performance (visitors'
initial response might be negative because they're used to the old solution)
 You can only compare 2 versions with a single factor that differentiates both designs (for more
factors / variations you need to deploy multivariate testing which is more difficult to analyse)
 It takes technical know-how to set up and analyse the results
 You're testing in a live environment so external factors might have an impact on the outcome
 If your site is really in bad shape, you'll still need to do a proper overhaul
In a nutshell
A/B testing is powerful stuff and a useful method for a quick comparison for 2 different designs.
However, bear in mind that it's no substitute for getting proper feedback from your users. Only this
will give you the whole picture straight from the ass's mouth.
Expert usability review vs. usability
testing
One question we're often asked is which method is best: usability testing or expert usability
reviews? Well, if they were sports cars, expert usability reviews might be a Porsche (pretty decent

9. car and better than no car at all), but usability testing would be in a different league, namely Formula
1.
So, what's the difference?
 An expert usability review is when a usability specialist inspects a website to identify
potential usability problems
 Usability testing involves getting people from the target audience to evaluate your site
whilst performing tasks
They're quite similar in many ways in that both:
 Find and prioritise usability problems
 Evaluate designs in the context of tasks
Do they find the same problems?
The answer depends on how precisely you measure this, on balance no, not reliably. Generally
speaking, expert usability reviews tend to find high level breaches of design rules and
consistency. Usability testing is better at finding issues related to special domain knowledge and task
flows.
If you were to carry out an expert usability review before conducting usability testing, and compare
the two sets of findings, you can see how the issues you found compared to their real experiences. Did
you find all the problems? Which ones did you miss? Which ones that you thought might be issues
turned out not to be? This will give you a good feel for how expert usability reviews hold up against
usability testing.
Often, expert reviews will:
 Miss usability issues that arise during usability testing
 Find some issues that usability testing didn't
 Report false alarms (i.e. not real issues)
And that's the problem. No matter how good an expert you are if you just rely on an expert usability
review you'll find different sorts of issues and overlook some potentially serious ones.
Consider the case of a health website aimed specifically at new mums. The expert usability review
found plenty of great issues, but might have missed the main killer one that caused participants to
pass comments like “That's just so insensitive!” and “I wouldn't use this site anymore”. And all
because of a seemingly benign bit of banner advertising for a local gym placed next to an article about
breast feeding. A reviewer would have difficulty predicting the indignant reaction of new mothers who
felt the site was rubbing their nose in it. After all there would be little chance of them going anywhere
near a gym with a newborn.

10. How much overlap is there between expert usability
reviews & usability testing?
Naturally, there is some overlap in the problems found in both expert usability reviews and usability
testing. Research conducted by Fu, Salvendy and Turley in 2002 estimated this to be around 41%.
Not as high as you might expect. Interestingly, expert usability reviews tend to find more problems
compared to usability testing. But it's quality over quantity.
Expert usability reviews are good...
Expert usability reviews are more common than usability testing simply because they're cheaper and
quicker to do. Typically, it'll only take a few days to inspect a site and write a report and because
you don't need to hire labs nor recruit users, costs are significantly cheaper. Because of this, they're
often preferred to usability testing, especially when time and budget are squeezed.
There are however some pitfalls. For a start, a lot depends on the reviewer's level of experience. No
two usability experts will find exactly the same issues so the method is also open to practitioner
variation. The biggest problem however is simply that they sometimes miss the real problems that
cause users to fail tasks. This can be especially true when the target audience has a particular skill
set. For example, an accountant may be better placed to diagnose certain types of issues related to
online accountancy tasks than a non-accountant reviewer.
...But usability testing is da Daddy!
Usability testing takes more time to plan and organise and is more expensive too - recruiting and
incentivising people from a target audience can be costly. But it's definitely worth the effort! The
results give a truer picture of the real problems people encounter because they're derived from real
users in the first place.
It's inconceivable how sites go live without checking to see that they're actually usable. It's like
building a car but not test driving it. "Yes, we've built this great website. Umm, well no, we've not
checked to see if people can use it yet. But don't worry because all the links work and the pages
download quickly."
Another advantage of usability testing is that there's less conjecture and feedback comes straight from
the horse's mouth. Videos of user sessions in particular, can provide a powerful persuasion tool for
reluctant stakeholders. It's hard to argue with recordings of people in tears of frustration.
Use real users and accept no substitute!
Both expert usability reviews and usability testing have their time and place. In practice, people often
use expert usability reviews early on to straighten up their design in preparation for usability testing.

11. (Indeed this is critical for accessibility testing, since it's a waste of time to recruit and test users who
can't even get into your site in the first place.)
Whilst an expert usability review is better than nothing, it's still vital to put your website in front of
users as early as possible too, rather than leaving it too late. Key elements such as site structure and
navigation should be assessed with users at the earliest opportunity. Otherwise you risk having your
website completely overhauled at a late stage.
Ultimately, the key is to appreciate that expert usability reviews and usability testing are different
beasts. The most effective approach is to try and integrate both techniques. Remember though, you
must test your site with real users - accept no substitute!
Have us conduct usability testing for you!
Find out whether site visitors can actually use your website by having us run usability testing for
you.
Visit: http://www.wbcsoftwarelab.com/