2. Contents:
1) Executive Summary
2) Introduction
3) Product description
4) A Detailed Heuristic List
5) Cooperative Evaluation of Prototype (Task List)
6) Setting up the Task Environment & Equipment
7) Performing the Evaluation
8) Results from evaluation
9) Design recommendations
10) Conclusions
3. Executive Summary
This is a heuristics evaluation report, the aim of this report is to identify a system, and explore its
interactive qualities. All evaluations are based on heuristics, which are a set of guidelines set out by
experts to evaluate an Interactive System’s Usability. This report was undertaken to summarize and
evaluate User Interface ability of the program Skype and better help the design of usable Interactive
Systems.
“The goal of heuristic evaluation is to find the usability problems in the design so that they can be
attended to as part of an iterative design process. Heuristic evaluation involves having a small set of
evaluators examine the interface and judge its compliance with recognized usability principles”
(http://www.useit.com/papers/heuristic/)
The heuristics used in this report, we’re that of Usability expert, Jakob Nielsen, and his 10 heuristics.
1. Visibility of system status.
2. Match between system and the real world.
3. User control and freedom.
4. Consistency and standards.
5. Error prevention.
6. Recognition rather than recall.
7. Flexibility and efficiency of use.
8. Aesthetic and minimalist design.
9. Help users recognize, diagnose, and recover from errors.
10. Help and documentation.
These 10 recommended heuristics by Jakob Nielsen will be used to explore the Interactive usability of
the Skype interface and allow us to better understand its use for online Calling, Video Calling, and
Instant Messaging amongst other user’s.
4. Introduction
This report discuss’s the heuristic evaluation process for Skype, a free internet based video calling
service. It begins by describing the product and discussing the target population for the program. It
then gives a look at the heuristics used to review the program and what major issuses, if any, were
found in this review of the program. We plan to outline these problems and suggest possible measures
which could be taken to fix or avoid them in the future therefore creating a greater overall experience
for skype users everywhere..
Product Description
Skype, which is now part of Microsoft, is software which allows a community of people from all over
the world interact for free via Phone, Live stream Video Calling, or even through Instant Messenging.
It allows its users to free call anyone, anywhere, at any time of the day once downloaded.
You can even obtain Skype for your business through paying by Pay as You Go, or committing to a
small fee Bill Pay package, allowing its customers to do more. Paying users have more access to
colleagues, customers, whoever, with their low cost Skype fee’s to Landlines and Mobile phones,
which may not have the Skype tool. The extra fee benefits can mean better corporate calls, by being
able to interact by Live Stream Video conference calls to up to four Skype accounts at a go.
Skype can be used anywhere, at any time now thanks to the use of Smartphones. Anyone who has
access to a PC, MAC, Laptop, or even smartphone can now download Skype for free and avail of its
free interaction services. Although its recognised for its Live stream Video Calling, users without a
web cam can still engage in online communications through Instant Messaging, or by connecting a
Skype phone up to their computer.
5. A Detailed Heuristic List
1. Visibility of system status
To sign into Skype it is clear to the user where to enter the username and password.
While the program is “signing in”, a loading circle appears indicating it is in the process of
doing so.
On start up of the Skype program, there is an audio cue to let the user know it is ready for use.
A pop up interface appears when a user is receiving an incoming call or video call. The
options to answer, answer with video or decline are given in a clear manner.
When an instant message is received a small orange dot appears beside the other user’s name
in the contacts. Also an audio cue can be heard at the same time. The conversation also
appears in the recent tab which is highlighted orange. The Skype icon flashes on the task bar.
2. Match between system and the real world
The Skype program avoids using technical jargon, instead using familiar language where any
user can comprehend. It presents information in such a way that the user understands easily.
Information is presented in appropriate order.
There is a slight repetitiveness with regard to the Skype options. Many of the sub categories
from the drop down menus lead to the same screen. For example, if a user selects privacy
from the Skype menu, or audio settings from the call menu, or options from the tool menu,
they are led to the same window.
3. User control and freedom
If the user wishes to go back to the log in page while the loading circle is on screen, there is no
option to do so. The user must wait for the program to finish loading.
The grouping together of the end call button, adding participants to the conversation, the
microphone and video buttons is of poor design. The user could easily make the mistake of
ending the call rather than choosing the other options.
In the options window, the user is given the choice of either saving the changes made or
returning by clicking cancel.
There is a clearly visible “close” button in the conversations window to exit.
4. Consistency and standards
Skype uses typical language accepted by numerous video calling and instant messaging
services.
Regular and novice users can easily understand what action leads to what result as no jargon is
used in the program.
Skype adheres to these standards and obeys them. Making it an easy program to learn for
anyone, novice or expert.
6. 5. Error prevention
Skype is a well built program which rarely crashes
The program can crash however due to poor Internet connection, typically during a video call.
Skype automatically tries to reconnect the call without any actions being taken by the user.
If connection cannot be re-established then Skype shows a pop up message explaining the
issue in plain English to the user.
After every call Skype asks the user to rate the quality of the call to help further development
of the platform.
6. Recognition rather than recall
Typically Skype’s layout is quite simple and easy to use.
During video calls however a menu bar appears along the bottom of the received video. This
menu bar contains images referencing certain actions that can be made during the call.
However, it is not clearly defined what these buttons do and can only really be discovered by
trial and error. After a few use’s it becomes second nature but the user should never have to
do this. The actions the buttons take should be clear for the users.
7. Flexibility and efficiency of use
Brightly coloured, simple layout.
Few buttons leaving for simplified efficiency.
Live stream calling at the click of a button.
Visually aesthetic, simple on screen navigation and easily able to tell who’s online, currently
busy, or offline.
8. Aesthetic and minimalist design
6 basic toolbars, Skype, Contacts, Conversation, Call, View, Tools and Help, allowing the
user simple access.
Stored contacts are easily displayed, relieving the hassle trying to find friends contact details.
Miniature green Skype logo show’s online friends activity, a yellow one to state the user is
online, but currently busy, and a red logo for user’s who are not to be interrupted.
The dial function displays a visual of a mobile key pay, with the 0-9 digits, *, Hash, Save,
Call and SMS contact features for users who want to contact unsaved user’s.
9. Help users recognize, diagnose, and recover from errors
7. A pop up window is displayed to show error.
Plain simple text, no coding references apart from connection port statuses.
Direct links to Skype’s online support website.
10. Help and documentation
Help feature in toolbar re-directs the user to https://support.skype.com/en/ where the users can
brows a selection of Frequently Asked Questions, and various How-to Guides.
FAQ’s and How-to Guide are designed with the beginner Skype user in mind and are
presented with a mixture of basic short and sweet sentences mixed with screenshots to aid
their users both mentally and aesthetically.
The use of these visuals aid the users in a more direct manner, by showing what each action
taken should look like on their own screen.
8. Cooperative Evaluation of Prototype (Task List)
The Task List will be broken down into three stages;
1) Setting up the Equipment and Environment.
2) Performing the Task’s.
3) Results and Evaluation of the completed task.
1) The set up of our task will be a very basic, but effective one. All that is needed of our user is a
laptop, microphone, a Screen Capture tool (ViewletCam), a USB for backing up our project,
and the Skype software with an already set up account.
2) The only approach the user need’s to keep in mind is the “Think Aloud” method we have
decided to use. With this method, all the user is asked to do is give a running commentary on
any actions they are required to do, and how they go about pursuing it, therefore allowing us
to have an audio recording of their evaluation.
3) Once the user has finished up their experience using the Skype program, we will then be able
to take their audio and visual recordings away for review, and see how easily (or confusing)
the user found being able to navigate their way through the program.
Setting up the Task Environment & Equipment
In order to properly record and monitor our user during this evaluation report, we decided upon using
the “Think Aloud” process, which would aid our ability to read into what the users are thinking
throughout the process. With the use of the “Think aloud” process, and a microphone to record, we will
be able to achieve a crystal clear recording of our user’s thoughts on navigating through Skype.
As a pre-task trial, we firstly set up a laptop with a plug in microphone to record ourselves participating
in the task to make sure all goes well before our subject enter’s the room to clear up any possible flaws
he might encounter subjects speech throughout the process as well as recording our actions on the
monitor by using the “ViewletCam” screen capture software.
Eventually we could bring in our user ands aloud him work his way though the program. The screen
capture and audio recording files will then be backed up onto a USB for further evaluation. This set up
would leave us with a step-by-step guide though the mind of the user as his speech and on screen
mouse movements will all have been recorded and stored to allow any time reviewing of our user.
Performing the Evaluation
In order to properly monitor our chosen user we decided to video record him as he attempted to use the
Skype interface. We began by explaining the consent form to our user; we reassured him that he had
full access to all the data and how it would be used. We also informed him that he could leave any time
he liked. After agreeing to the consent form with our user we set up a laptop, turned on the Screen
Capture software, and began recording through the microphone. We also asked him to think aloud as
he worked his way through the program.
As we wished to get a full cooperative evaluation of Skype we decided to begin by getting our user to
open the program and try to call a friend. We watched as he clicked on the Skype symbol on the
desktop and the program opened. Our user was confronted with a log in screen where he entered the
username and password we had already set up in order to gain access to the Skype account. Once this
information was entered the program started up. Upon opening the program the user was presented
with tips from the program. For proper evaluation we asked him not to use these tips and so he closed
the window and continued with the evaluation.
We watched and listened as our user told us how he was scrolling down through the friends list looking
for the one we had asked him to call. Upon arriving on the chosen friend our user double clicked the
name opening the friends Skype page. From this window our user could see all the options available to
him, which he made clear by listing out what he could see on screen.
9. Our user then clicked on the green phone or ‘call’ button on the chosen friends page. Skype then took
over and began trying to connect the call. Our user told use he knew the call was in action as Skype
gave him a dial tone as he awaited an answer from the other end. The call was then connected and the
chosen friend appeared on screen.
However, our users video feed was not being sent to the friend and so we asked him to try to figure this
problem out, aloud. He scanned over the options available on the call screen until he came upon a
video camera symbol. Upon clicking it his video feed appeared on screen. He mentioned he could tell
the feed was being sent by the reaction of his friend on screen when the video appeared.
We then asked him if he could send an instant message to the friend whilst in the call. This posed
slightly more difficulty than expected as our user decided to scroll through his friends list again and
attempted to send a message this way. However he quickly discovered his mistake and returned to the
video call screen, scanning the options there for a message icon. He announced he found what he called
a ‘speech bubble’ icon. Upon clicking this a small chat window opened below the video feed and he
began writing a message, which was sent immediately once the ‘send’ button was clicked. Our user had
found the instant messaging option.
Now the time had come to end the call. We asked our user to say goodbye and hang up. Our user
quickly spotted the red phone symbol and clicked it ending the call. Video feed disappeared and he was
assured that the call was over via an on screen message. Once the call had ended Skype gave our user a
option to review the call for quality purposes. He went to ignore it but for the purposes of this
evaluation we asked him to do it. He was given about 5 questions about call clarity and video quality,
which he could rate from 1-5 depending on the standard he felt. Upon completing this he sent the
feedback to Skype and the window closed returning him to the same friends page he had been on prior
to starting the video call. Our user then logged out of his profile and closed the application, completing
his part in our evaluation. We thanked him for his time and patience and he left us to continue our
report.
Results from Evaluation
Once the tasks set out had been completed by our user we were left with a lot of information regarding
this use of the Skype program. We felt using “ViewletCam” to record the users mouse movements
along with recording the “out-loud thoughts” gave a greater understanding of how the user was
working his way through the program. Therefore, we could easily notice any mistakes made, or wrong
paths taken and fully understand why or how the user ended up in any particular situation. From this
evaluation we have found that the basic uses’ of Skype are quite simple to understand, however, certain
aspects of the program need re-designing in order to avoid potential problems such as sending an SMS
rather than instant messages and making some options more easily visible from the video call screen as
to avoid confusion for the novice user.
Design recommendations
There are many possible re-design recommendations that can be made to improve the Skype program.
User Interface design is an iterative process that never stops, so therefore the program can undergo
numerous re-designs.
Skype could introduce a form of parental control within the program. There are options to block people
from contacting the user, but there are no controls in place as to whom a child could contact. This
would increase the safety and security of the user greatly.
There are possible issues with the program when it is integrated with the toolbar of the operating
system. An option to turn off the flashing notifications during an active conversation could be
provided. As it is, it is only possible to turn them off when initial contact is made.
Another issue found with the toolbar and Skype, is that conversations are shown as different windows.
This could be integrated so that only one icon for Skype appears rather than the multiplying effect that
can be seen in the following image,
10. The location of the icon that changes text conversation into SMS in the conversation window should be
changed to a different location. This is because it is far too easy, when using emoticons, to scroll over
the SMS icon on the way back to the text box, and end up sending an SMS.
Skype could create an option whereby it is possible to see who is viewing your profile. This would help
with the user’s privacy and they may then be able to block contacts. Or perhaps create an option where
only people in a certain country or area of the world can add you to their contacts.
The group chat function is very popular amongst the Skype community. However, if one member of
the group is not online they are not updated with the rest of the conversation. This means the user may
miss an important part to the discussion. If the messages would still be sent to the offline user, and they
could receive them the next time they log in, nothing will be missed out on.
Conclusions
Product Analysis
Upon evaluating Skype. We found the software quite simple to use and user friendly. The aesthetics of
the user interface are very basic and direct, as we discovered throughout our evaluation, our users had
no major difficulties navigating themselves around the layout. However certain buttons and options
were not made easily visible for the user. This small problem could be easily fixed with the addition of
text underneath the symbols so as to avoid confusion.
Overall Summary
Overall, the main flaws we found whilst evaluating Skype were, the layout of buttons during user
interaction, and the representation of icons which confused our user as to which icon meant Instant
Message or send an SMS, as well as smaller issues such as video feed lagging during calls, and voice
clarity being lost due to poor connection.
However outside of the video call window the general layout prior to calling someone is very simplistic
and understandable for both the novice and advanced users.
11. “Think-Aloud” Evaluation Consent Form
I hereby acknowledge my free and willing participation in a “think-aloud” project designed to uncover
flaws on the User Interface of Skype .
I understand that _______________________________________ will take notes and may record my
speech as I “think aloud” while navigating through the program I have agreed to review, and that
she/he will use this recording only to evaluate and improve the material I review.
I also understand (1) that no one except __________________________________________ will listen
to my recording, and (2) that any use of the information contained in the recording will not reveal my
name or other information that could identify me.
Name of Participant: _______________________________________________
Participant’s Signature: ______________________________________________
Date: ________________
(https://sites.google.com/a/ceunh.unh.edu/writing-editing-for-the-living-web/Do-you-need-and-editor-
/Think-aloud-testing/Think-alouds/think-aloud-consent-form)
