2. INTRODUCTION
Bowling Green State University offers an online degree completion program that is
designed for working adults who wish to complete a Bachelor of Science degree in Technology.
The Advanced Technological Education (ATE) program consists of nine core courses that teach
students how to integrate technological concepts into their working environments. Students can
choose between two courses of study: Technology in Training and Development or Information
Studies. All of the courses are online offering convenience and flexibility for working students.
An applied associate degree is required for the program, as well as work experience.
Prospects wishing to pursue a degree online will tend to look for information online. It is
critical that the ATE website be enticing, informational and usable. Therefore, a usability test
was conducted on the ATE website to determine if prospects and existing students are able to
easily retrieve key information. In addition, the process in which prospects contact ATE
personnel to receive additional information was also analyzed, as it is an integral component of
the website and recruiting process.
In order to encompass all components of the website and the information process, this
usability analysis consists of the following sections:
• Pre-analysis
• Methodology
• Results
• Conclusions and Recommendations
3. PRE-ANALYSIS
In order to develop productive test questions, it was important to determine relevant facts
about the ATE program. For instance, what do prospects and students want and need to know?
Who is the target audience? What types of tasks do they want to perform from the website?
What are the objectives of the website and key points it should contain?
What is the purpose of the website?
Currently, the purpose of the site is to inform prospective students about the ATE program. A list
of questions usually asked by prospects was compiled. Answers to the following frequently asked
questions should be found easily and quickly:
• How long will it take me to finish?
• What is required to be eligible for the program?
• Do I need an associate degree?
• How much does it cost?
• What can I do with an ATE degree?
• What does ATE stand for?
• What will I learn in the ATE program?
• What courses will I be required to take?
• Can I take courses elsewhere and transfer them to the ATE program?
• Will my degree say “online?”
• When do classes begin?
• What if my employer pays my tuition?
The four questions below were found to be questions asked not of prospects, but of those that
have already applied. Therefore, it was discovered that the purpose of the website needs to expand to
providing information to those that have just applied as well.
• When will I know I’ve been accepted into the ATE program?
• I’ve applied – what happens next?
• I don’t think a course transferred correctly. What do I do?
• What is a transfer evaluation vs. an official audit?
Who is the audience?
The website can be accessed by both prospects and current students. However, for the most part,
the website is currently designed for future students. Is it not normally used as a tool for students already
4. in the program. However, for the purposes of the usability test, the future and current student audiences
are defined.
Future Students
Users will be most likely between the ages of 25 and 60. Computer skills will vary from
beginner to expert. Most will have some college experience. Connection speed will vary
from dial up to high speed.
Current Students
Users will most likely be between the ages of 25 and 60. Computer skills will vary from
beginner to expert. All will have some college experience. Connection speed will be
high speed.
What tasks need to be performed from the website?
Students and prospects viewing the ATE website will need to have access to the following links:
• Financial aid
• Course Descriptions
• All online courses available
• Course Schedules
• Online application
• Tuition deferral program
• Ability to request more information
• Access to a sample online classroom
• Video/Audio of ATE explanation
Information and features that should be present:
• Explanation of the enrollment process including time involved
• Application instructions
• Curriculum
• Course Descriptions
• Class Schedule
• Faculty Information
What are the measurable usability objectives?
In order to create test questions that would provide meaningful information, it was important to
define the following measurable objectives:
• Were all of the users’ questions answered?
• Was the user able to request more information easily?
5. • Was the user able to quickly identify the correct links to answer their immediate
questions?
• Was the user able to easily print the information they wanted to save?
• Did the user have an emotional reaction to the site? Was the user excited about
starting the ATE program or was he/she discouraged, confused or indifferent?
What are the key points?
The following key points are important components and should be included in the
website:
• Course Descriptions
• Costs
• Completion Time
• Current Promotions
• Accreditation
METHODOLGY
Participants
Five testers were selected for this usability test. Three out of the five were female and all
five were between the ages of 40 and 45 which is also the approximate average age of most ATE
prospects. All use the internet and email to some degree for work and/or pleasure. Hours of
usage vary from 10 hours to 40 plus hours per week.
Test Questions
The following information, directions and questions were provided to each participant.
Usability Test Instructions and Questions
I would like you to view a website for me and answer a few questions. The process will
probably take approximately 15 minutes.
• Please know that I am not testing you, I am testing the site. You can’t make any
mistakes during the test.
• I want you to be completely honest – you won’t hurt my feelings. The goal is to
make the site better and easier to use.
6. • As you think about each page and decide where to click, please do me a favor and
include any thoughts you have in your answers so I can understand your thought
processes during the test.
Before you look at the website, I have a few questions for you, OK?
1. What is your occupation?
2. How many hours a week do you spend using the internet, including email?
3. What types of tasks do you use the internet for? What kinds of sites do you like to visit?
4. What is your age? 18 to 25, 26 to 39, 40 to 59, 60 to 74
Now its time to open the website. Go to http://ideal.bgsu.edu/programs/ATE/index.php
5. Have you ever seen this website before?
a. If so, why and when?
6. Just from looking at this site, who do you think the site is designed for?
b. Why do you think that?
7. By glancing at the website but without clicking on anything, can you tell me 4 types of
information the site is intended to provide?
8. Click on a link you think you would click on first if you were accessing this website for
the first time at home. Why did you choose that link?
9. By just glancing at this page, what information is provided?
Scenario 1
You want to know the eligibility requirements for the ATE program. You’ve heard an
associate’s degree is required, but you don’t have one. You want to know if you can still apply.
Can you find the answer to your question?
Not completed Completed with difficulty or help Easily completed
Scenario 2
You would like to know how much it will cost to complete the program. Can you find the
answer to your question?
Not completed Completed with difficulty or help Easily completed
7. Scenario 3
You have an associate degree but want to know how many courses will transfer to the program.
Can you find the answer to your question?
Not completed Completed with difficulty or help Easily completed
Scenario 4
You want to know what courses you are required to take as a part of the ATE program. Can you
find the answer to your question?
Not completed Completed with difficulty or help Easily completed
Scenario 5
You’ve heard some employers don’t hold online degrees in as high esteem as other degrees.
However, BGSU is a reputable and highly regarded by employers. You want to know if your
degree will indicate that you attended online or if it is the same type of degree you would receive
if you attended on campus. You also would like to know ATE’s accreditation. Can you find the
answers to your questions?
Not completed Completed with difficulty or help Easily completed
Scenario 6
You want more information about the program and would like more information. Can you
request more information?
Not completed Completed with difficulty or help Easily completed
Complete the request for more information form on the website
10. Did you feel uncomfortable about providing any of the information asked of you on the
request for information form?
Open the email sent to you in response to your request for more information
11. Do you feel you’ll read the entire email?
12. What information are you inclined to look at first?
13. Is the information too much information, not enough information, or just enough?
14. Is the information organized and easy to read?
15. Are you confused by anything indicated in this information?
16. Would you open each and every document?
8. Open the documents you would open in a real situation
17. Can you now answer any of the scenario questions you couldn’t answer before?
18. What is your overall impression to the ATE website and request for information process?
19. What did you like best?
20. What did you like least?
21. What is the first thing you would do to improve the site?
22. Do you have any other comments and/or questions?
Rate the website and the request for information process:
Strongly Strongly
disagree agree
1. Overall I found the information I needed.
1 2 3 4 5
2. I found the information unnecessarily
complex
1 2 3 4 5
3. I thought the information was easy 1 2 3 4 5
to use
1 2 3 4 5
4. I found the various links in
the website were well integrated
1 2 3 4 5
5. I thought there was too much
inconsistency in this website
1 2 3 4 5
6. I would imagine that most people
would learn to navigate this website quickly 1 2 3 4 5
7. I found navigating the website very
1 2 3 4 5
cumbersome
8. I felt very confident using the 1 2 3 4 5
Website and the request for information
1 2 3 4 5
9. Test Procedures
The ATE program is online, so there is a presumption that individuals requesting
information about the program have access to the internet and email. Therefore, all informative
documents about the ATE program are emailed to prospects. Statistics show that a majority of
prospects find the ATE website and then complete the Request for Information Form. In order to
simulate this procedure as accurately as possible, testers were emailed the test instructions. They
were asked to complete the tasks and answer the questions without a test administrator present.
Part of the test was to utilize the “Request for Information” feature on the ATE website, and then
read an email with 5 attached documents sent to them in response. Participants were asked
questions about both the website and the emailed information. Testers recorded their answers
electronically and submitted their results via email.
11. Tester Information Tester 1 Tester 2 Tester 3 Tester 4 Tester 5
Secretary/Medical Recruiting Coordinator &
1 Occupation IT/Contractor Transcriber/Student Secretary Administrative Secretary Database Administrator
How many hours a
week do you spend
using the internet,
2 including email? 10 hours 10-15 hours A lot 20-40 hours a week 40+ hours
Email at work, internet for work procedures.
I rarely surf for pleasure but frequently visit
What types of tasks do my bank site and sometimes will view
you use the internet Banking, Investments, Google calendar, weather, self.com or diet and exercise sites. I have a
for? What kinds of sites Shopping, Email, Email, online classes, news, paycheck, bursar bill, business website that I check and sometimes Google, databases, email,
3 do you like to visit? News shopping book orders weather.com research
4 What is your age? 40 to 59 40 to 59 40-59 40-59 40-59
Website Information - http://ideal.bgsu.edu/programs/ATE/index.php
Have you ever seen this I thought I had been to this website but it
5 website before? no No No looks different than I remember. no
At the very first glance, I would think the site
is for any BGSU student. It references the
degree name and that it is a bachelor of
20-somethings who science. Of course, knowing about the
didn't go to a 4-year program, I know it is for adult students and it
Just from looking at this school. It never says To get info about a degree in is an online program, but you wouldn't know
site, who do you think Designed for adults anywhere it's for adult Tech Education. I think that that unless you read the first three bulleted Students wanting to earn a
the site is designed for? that want a four year learners or non- because it says "Bachelor of (sort of) lines under the description. I t think Bachelor of Science in Advanced
6 Why do you think that? degree traditional students Technological Education." that is because of the template Tech Education
By glancing at the
website but without Four things I should be able to get are
clicking on anything, information about the program, the specific Phone number, email address,
can you tell me 4 types Contact information if curriculum to finish the degree, info about qualifications: need an associate
of information the site they are interested in the faculty and contact information to sign degree & work experience, can
7 is intended to provide? learning more Not really Contact info up. take classes online
It is a degree from
BGSU info about the degree
You can find more
details by the links on
the left
It is an online degree
(although that isn't
terribly obvious)
Click on a link you think
you would click on first
if you were accessing About the Program - I clicked on About the I clicked on About the Program
this website for the first because there isn't a Program because I because I know nothing about About the Program - I would choose it About the program - want to
time at home. Why did lot of information on wanted to figure out the program and figured that because I would want to know what the learn more about it, plus first link
11 you choose that link? the home page. what it was about. was a good starting point. program is about and if it would interest me. in list
12. Results Analysis
• None of the participants had seen the ATE website before completing the test.
• At first glance, the participants were asked what the website was designed for. The
testers all indicated the website provided information about a bachelor degree. When
asked to identify four types of information provided on the website’s home page, one
of the five reported what they should be able to find – not what they did find. Another
tester said they could not identify four types of information. The remaining three all
agreed they could locate contact information and information about the degree. Two
participants said they identified the program was online, though one reported that it
was not necessarily obvious.
• All participants agreed they would be inclined to click on the “About the Program”
link first.
• One participant did not read the information provided on the “About the Program”
page. All remaining participants indicated that the “About the Program” page
provided more information, who the program is designed for, and how it works. One
participant appeared to be a bit confused as this was the first time “degree
completion” was mentioned, indicating that it should have been included on the home
page.
• When asked if the testers could find out if they could apply without an associate
degree, one of the four said they could easily complete that task. The remaining four
testers could not find this information. It is unclear how one tester found this
information as it is not provided on any of the ATE website pages.
• When asked if the participants could locate costs associated with the program, only
one reported finding the information but with difficulty or help. All others could not
locate cost information.
• When asked to locate information about transferring courses to BGSU, the answers
varied. Two completed the task with difficulty or help, one could not complete the
task, and one easily completed the task. One tester experienced an error on the
transfer credit page.
• All participants agreed they could easily locate the courses required to complete the
ATE program.
13. • When asked to find out what type of degree and accreditation the program held, one
participant reported they completed the task with difficulty or help, three easily
completed the task and one reported they found the information on the “Recent
Updates” page and it should be placed on the “About the Program” page as well.
• All five participants easily requested more information about the program. One tester
suggested being able to ask specific questions on the Request for Information form
and felt the contact information and the request for information pages should be
combined.
• All five testers felt comfortable providing information asked of them on the Request
for Information form.
• One participant felt they would only read the first few paragraphs of the email sent to
them in response to their Request for Information. The remaining testers said they
would read the entire email. One tester felt a lot of the information in the email
duplicated the website information.
• When asked which information in the email they were inclined to look at first, one
reported they would like to see course compatibility, though that information is not
currently included in the email. Another tester reported they would click on the link
to the web page if they hadn’t already viewed it. Another said they would first read
about what the program is and then view the curriculum. The remaining participant
said they would read the email first.
• What asked if the email provided too much information, not enough information or
just enough, two participants felt it was just the right amount of information. One
participant remarked that the highlighted and bulleted information was helpful. One
participant reported it was not enough information and more transfer information
needed to be provided. Two remarked that the letter repeats too much information
from the web page. The remaining tester reported there is never too much
information.
• All agreed the information was organized and easy to read. One participant remarked
they did not understand what some of the information meant, however.
14. • When asked if any of the information confused them, three participants reported they
were not confused. One reported they were confused, and the remaining tester
indicated the requirements should be better explained.
• When asked if they would open every document attached to the email, three
participants reported they would and two reported they would not.
• Only one out of the five participants said they found answers to earlier test questions
they could not answer after reading the email and opening an attachment.
• The overall impression of the website was positive. The available links, contact
information, the appearance and navigation capabilities were reported as the best
features of the website. The small font size, the look of the Recent Updates and the
Request for Information form, the number of attachments on the email, and the ability
to understand the information were the least liked features of the website and request
for information process.
• Suggested improvements were:
o Make the heading smaller and provide more information on the home page.
o Provide more links to transfer information
o Change the color of the Request for Information form and add a text box for
specific questions
o Increase the font size
Conclusions/Recommendations
Conclusions
The ATE website and request for information process scored in the middle range overall.
The average rating for positive features ranged between 3.25 and 4, with 1 being the lowest score
and 5 the highest.
However, the pre-analysis and testing processes revealed information that is missing and
areas of the website that are in need of improvement.
15. Website Information
The website appears to be informative in a limited capacity. The way in which the
information is presented could be improved. For instance, key information should be accessible
at the top of the page where users are most likely to focus their attention. This should include
that the program is online, is for adults, is a degree completion program, and is accredited. In
addition, the following information is missing from the website:
• Financial Aid (although this is included in the emailed information)
• Costs
• Available online courses
• Class Schedule
• Online Application
• Tuition Deferral Program
• Tutorial
• Enrollment process (although this is included in the emailed information)
• Completion Time
• Accreditation Agency
• Current Promotions
• Options for individuals without an associate degree
• Detailed transfer information
• Video/Audio files verbally describing the program
• FAQ’s
Overall, there was not the impression from the test results that the website generated an
emotional reaction. If anything, one user’s answers indicated frustration; though it appears that
could be from a previous bad experience.
The Request for Information page did not seem to intimidate the users so it’s possible to
conclude that the information requested from the prospect is not perceived as being invasive.
For those using smaller computer screens, the graphical heading at the top is too large. In
addition, some felt the font was too small and was bothersome.
16. The website does appear to navigate well. Users were inclined to select the “About the
Program” link from the home page which would be the logical choice. In addition, the courses
required for the program were easily obtained by all participants.
Emailed Information
It appears most testers read the entire email or at least skimmed the information. It was
not apparent which attached emailed document was opened first. The question pertaining to this
subject should have been worded differently in order to determine which document was of most
interest to the tester. The amount of emailed information did not appear to overwhelm a majority
of the testers, which was an initial concern. Most found the email easy to read and navigate,
though it may be necessary to provide clearer explanations.
Recommendations
Based on the pre-analysis information, the test results, and the conclusions the following
changes to the website and emailed information are recommended:
• The home page should provide links to: “Future Students”, “Current
Students”, and “I’ve just applied. What Next?" Each will direct the user to a
page which has information specifically relevant to their situation.
• Links that need to be added to the website in their appropriate category(s) are:
ATE Community
Class Schedule
MyBGSU
Online Classes Available
FAQ (different questions for future, current, or just applied)
Application Instructions
Tuition Deferral Application
ATE Checksheet
Video/Audio describing ATE
Online tutorial
Online Application
• The website needs to include a printer-friendly version for all pertinent
information and documents.
17. • The following information should be at the top of the home page where users
will quickly identify it:
The program is online
The program is for adults
ATE is a Bachelor Degree Completion Program
How to retrieve more information/contact information
FAQ
• One of the FAQ should be “What if I don’t have an Associate Degree?” A
page should be dedicated to information specific to those individuals.
• A text box should be added to the Request for Information form allowing
users to enter specific questions. A link to the Future Students FAQ would be
helpful from this page.
• The email attachments should be converted to websites and links should be
provided to the websites as opposed to attachments.
