This document describes a process for developing customer satisfaction surveys for assessing the quality of support provided by an IT Customer Service organization.

1. OBAN IT SUPPORT OPERATIONS
Preliminary Guide for Customer Survey Development
Overview...........................................................................................................................................................2
Purpose.............................................................................................................................................................2
Plan...................................................................................................................................................................2
Relative Measures............................................................................................................................................3
Background....................................................................................................................................................3
Examples.......................................................................................................................................................3
Performance Factors.......................................................................................................................................4
Proposed Questions..........................................................................................................................................5
Help Desk Support.........................................................................................................................................5
General Questions..........................................................................................................................................6
Open-ended Questions....................................................................................................................................7
Recommendations............................................................................................................................................7

2. Overview
The rapid expansion of the Oahu Base Area Network (OBAN) infrastructure and customer
base has created a necessity to gauge the overall effect of this tremendous growth. A
mechanism for measuring the satisfaction of the user community as it relates to
Information Technology support services and system performance is required. In order to
achieve the standards stated in the Service Level Agreement (SLA) put forth by the
Regional Information Technology Service Center (RITSC), a method of collecting the
feedback necessary for continuous improvement must be developed.
Purpose
The development and publication of a comprehensive customer survey is a commonly
accepted method of collecting the data used to establish a performance baseline. From
that baseline, management can periodically evaluate any deviation from established
performance metrics and proactively take corrective action. Depending on the nature of a
given survey question and the associated scale, a variety of measures may be derived
from the resultant data gathered.
Plan
The ongoing workstation upgrade project has placed a sense of urgency on the creation of
a comprehensive customer survey. While this is not the only factor driving the survey, it is
one of the primary reasons to have a survey in place quickly. With all of the changes
occurring to the OBAN network infrastructure, there is a great opportunity to establish a
working baseline over the next three to six months. The remainder of this document will
focus on completing this task.
The following sections will give you an opportunity to participate in creating the survey. A
variety of proposed survey questions are presented along with various methods for
collecting and measuring the data. In each section, you will have guidelines and an
explanation for using a given methodology.
The key to success in this exercise is objectivity – the survey questions should not
contain bias in any form, or lead the participant to a desired response. Please do not
interpret the inclusion of a distinct question as bias. Certain occasions require focusing on
evaluating specific support functions or service features.
I would like everyone that participates to respond no later than Wednesday, December 16,
1998. You may e-mail your responses to me, or circle your selections and write in your
comments (legibly, please!) on the printed document. I will be glad to discuss the process
in person, but only if lobbying is kept to a minimum. Time is ticking away…
Sincerely,
Jim Soltis
Senior Metrics Analyst
u651jds@cpf.navy.mil
471-8590
2

3. Relative Measures
Background
Surveys designed to collect performance data need a mechanism to identify what the
customer believes is important. This is an essential concept in that it helps management
to identify the areas that generate the most customer satisfaction.
Too often, we overlook an opportunity to improve customer support due to a poorly
designed survey. By limiting a response to a level of goodness or badness, the question
does not indicate whether the customer is concerned with a given issue. To correct this
drawback, a two staged response is required – one indicating importance, the other
stating opinion. The result is called a “relative measure,” and can be displayed as a
functional or performance metric.
Relative measures are highly valuable to support organizations since they allow team
members to focus their efforts where they will have the most impact. As data is collected
over time, the process should result in a streamlining of task assignments and team
knowledge. Each organization will have different focused outcomes, as each LAN
community has different needs. However, the streamlining process should still occur.
Examples
Importance Satisfaction
1 = Very Unimportant 1 = Very Dissatisfied
Question / Measure
5 = Very Important 5 = Very Satisfied
N = Not Applicable N = Not Applicable
1 2 3 4 5 N 1 2 3 4 5 N
1. In general, how satisfied are you with the responsiveness of the Help Desk?
1 2 3 4 5 N 1 2 3 4 5 N
2. In general, how satisfied are you with submitting trouble tickets via the web site?
Importance Agreement
1 = Very Unimportant 1 = Strongly Disagree
5 = Very Important 5 = Strongly Agree
N = Not Applicable N = Not Applicable
1 2 3 4 5 N 1 2 3 4 5 N
3. I am satisfied with my support from the Help Desk staff.
1 2 3 4 5 N 1 2 3 4 5 N
4. The Help Desk staff always answers my questions accurately.
Importance Frequency
1 = Very Unimportant 1 = Very Rarely
5 = Very Important 5 = Very Frequently
N = Not Applicable N = Not Applicable
1 2 3 4 5 N 1 2 3 4 5 N
5. I have reliable e-mail delivery through the OBAN LAN.
1 2 3 4 5 N 1 2 3 4 5 N
6. I have a reliable connection to the Internet through the OBAN LAN.
3

4. Performance Factors
Identifying specific performance factors completes the process of collecting survey
information as relative measures. Performance factors are normally called metrics, since
they categorize the collected data into measurable groups. Deciding on the performance
factors is a matter of preference based on tracking and collecting desired information. For
the OBAN IT support function, the following seven factors are suggested as a starting
point:
• Accessibility
• Technical Quality
• Speed of Solution
• Completeness
• Clarity
• Success
• Courtesy
Seven categories may seem like too much information, but the data is not difficult to
collect. That issue is more dependent on the set of survey questions selected. A
minimum of five factors should provide a reasonable set of measurable data.
Once the performance factors are determined, the survey questions are grouped together
in sets to collect the metric input data. Collecting the data with a form like the example on
page 3 allows the information to be displayed in the following format:
RELATIVE MEASURES
Accessibility
Technical Quality
Speed of Solution
FACTOR
Importance
Completeness
Performance
Clarity
Success
Courtesy
0 1 2 3 4 5
RATING
This chart is an example of a static display. As the survey data is collected over time,
another chart can be developed to plot the dynamic changes in a given factor. This is a
simple method of tracking both performance and evolving customer opinion.
The following sections will provide a list of potential survey questions. The questions will
be grouped according to the performance factors stated above. Of course, the questions
may be reorganized in the event that fewer than seven factors are used.
4

5. Proposed Questions
Help Desk Support
Listed below is a group of 14 potential survey questions related to measuring Help Desk
support issues. Each question has a suggested rating scale. The list is not absolute.
However, new questions will only be considered if this list is inadequate, and multiple
respondents suggest a similarly worded question for use. Again, turn-around time for the
survey development is crucial. As we learn more about customer opinion, the survey
content will surely evolve to address timely questions.
A suggested approach would be to consider 10 questions covering 5 performance factors,
with 2 questions per factor if possible. An added variable is to team complimentary or
opposing questions in each pairing.
Accessibility:
• What is your preferred method of contacting the Help Desk?
• In Person, Phone, E-mail, Web Site
• I am able to submit my trouble ticket without any difficulty.
• Always, Often, Neither, Rarely, Never, N/A
• In general, how satisfied are you with submitting trouble tickets via the web site?
• Very Satisfied, Satisfied, Neither, Dissatisfied, Very Dissatisfied, N/A
Technical Quality:
• Was the staff member professional?
• Excellent, Good, Fair, Poor, Unacceptable
• Did the staff member have a clear method of troubleshooting your problem?
• Excellent, Good, Fair, Poor, Unacceptable
• Rate the technical knowledge of the Help Desk staff member.
• Excellent, Good, Fair, Poor, Unacceptable
Speed of Solution:
• Was your problem resolved in a timely manner?
• Excellent, Good, Fair, Poor, Unacceptable
• In general, how satisfied are you with the responsiveness of the Help Desk Staff?
• Very Satisfied, Satisfied, Neither, Dissatisfied, Very Dissatisfied, N/A
Completeness:
• Did the staff member provide complete, accurate information?
• Excellent, Good, Fair, Poor, Unacceptable
Clarity:
• Did the staff member communicate clearly?
• Excellent, Good, Fair, Poor, Unacceptable
5

6. Success:
• Was the problem corrected to your satisfaction?
• Excellent, Good, Fair, Poor, Unacceptable
• I am satisfied with my support from the Help Desk.
• Strongly Agree, Agree, Disagree, Strongly Disagree, No Comment or N/A
Courtesy:
• Do you find the Help Desk staff courteous and helpful?
• Excellent, Good, Fair, Poor, Unacceptable
• My overall experience with the Help Desk staff was positive.
• Strongly Agree, Agree, Disagree, Strongly Disagree, No Comment or N/A
General Questions
The proposed questions in this section are system-level nature and do not contribute
directly to the Help Desk performance factors. They do not require a rating of importance
as they are specifically directed at one issue. A suggestion for this section is to select 5 to
7 questions that cover a broad array of issues.
• I am able to log into the LAN without any problems.
• Always, Often, Neither, Rarely, Never, N/A
• I consider the performance of the e-mail system acceptable.
• Always, Often, Neither, Rarely, Never, N/A
• I have reliable e-mail delivery through the LAN
• Always, Often, Neither, Rarely, Never, N/A
• I consider the performance of my Internet/Intranet connection acceptable.
• Always, Often, Neither, Rarely, Never, N/A
• I have a reliable connection the Internet/Intranet.
• Always, Often, Neither, Rarely, Never, N/A
• The software applications installed on my workstation are easy to use.
• Strongly Agree, Agree, Disagree, Strongly Disagree, No Comment or N/A
• The software applications installed on my workstation enhance my daily tasks.
• Strongly Agree, Agree, Disagree, Strongly Disagree, No Comment or N/A
• I require additional training to become more productive with my software applications.
• Strongly Agree, Agree, Disagree, Strongly Disagree, No Comment or N/A
• The software training offered makes me more productive.
• Strongly Agree, Agree, Disagree, Strongly Disagree, No Comment or N/A
• The software training offered is effective and easy to understand.
• Strongly Agree, Agree, Disagree, Strongly Disagree, No Comment or N/A
6

7. Open-ended Questions
The questions in this section are primarily for categorizing customers and accepting
general commentary. Knowing how long an individual has been a customer is important
in determining their needs, expectations and familiarity with the environment. Knowing
how frequently a customer has requested support can uncover hidden problem areas with
support, system reliability and customer knowledge. A suggested approach for this
section is to select 5 questions - the first 2 are preferred.
• How long have you been an OBAN LAN customer?
• 1-3 Months, 4-6 Months, 7-12 Months, 1 Year or more
• How many times have you contacted the Help Desk for support?
• 1-3, 4-6, 7-9, 10 or more
• What did we do well?
• Open-ended comment box
• What can we do better?
• Open-ended comment box
• What service-related improvements can you recommend?
• Open-ended comment box
• Please indicate the name(s) of any staff member you would like to commend:
• Open-ended comment box
• What do you consider your most important source of computer advice?
• Commercial, Internet, Print Media, Friends/Coworkers, Help Desk
Recommendations
Ideally, most people will certainly prefer to keep this survey as short as possible.
Realistically, a minimal survey will decrease the knowledge we can gain from the collected
data. The idea is to structure the survey to cover a comprehensive array of issues while
retaining a level of simplicity for the participant.
The recommended approach is as follows:
• Select questions that allow the use of relative measures and performance factors for
tracking the Help Desk services. A block of 10 questions should be sufficient.
• Select general questions that focus on essential issues. A block of 5-7 questions
should be sufficient.
• Select open-ended questions that facilitate valuable feedback from the customer. No
more than 5 questions are needed in this area.
• Make the survey fun to use in order to encourage participation.
• Make the survey web-based, using radio buttons and drop down lists to decrease the
time spent completing and submitting the survey.
7