My survey research I made when working in VNG, website department :)

1. How to make a survey?
Web Quality 2012
Presenter: TúTC
Date: 13/3/2012

2. Previous surveys:

3. Our weakness:
• The questions aren’t well written enough.
• The result doesn’t really reflect the user experience.
• The report doesn’t have much details about other
criteria: layout structure, text... on specific level.

4. Training objectives:
After the presentation, the listeners will:
Understand the meaning of survey and its types.
Know how to plan and conduct a survey.
Can apply the survey to get more detailed
information about user experience.

5. Table of contents
I.
What’s survey?
II.
Planning a survey
III.
Conducting a survey
IV.
Our experiences
V.
Surveys apply in WO
VI.
References

6. I. What’s survey?
1.
Definition
2.
Benefits of survey
3.
How to collect the survey?
4.
Surveys in WO

7. I. What’s survey?
1
2
3
4
Definition
Survey is a quantitative research method which to investigate
the customer in large population about:
Opinions
Favorite choice
Feedback and
satisfaction about the
product.

8. I. What’s survey?
1
2
3
4
Benefits of survey
◦ To know how most of the users feel about the product
◦ To have statistic data to make design decisions
◦ To evaluate the design or website based on a large number
of target users.
 To improve overall UX of the product!

9. I. What’s survey?
1
2
3
4
Benefits of survey
Ex: Homepage design survey of ZT2:

10. I. What’s survey?
1
2
3
4
How to collect the survey?
Online survey
Telephone
Street survey
In-home survey
Mail

11. I. What’s survey?
1
2
3
4
Surveys in WO
◦ In project:
Design
• PSD file
• HTML version
• Vportal version
• input content
Gather information
UI Testing and publish
• Usability Testing
• Publish
Coding - Deploying
Homepage design survey
1. Expectation survey
2. Post-session survey

12. I. What’s survey?
1
2
3
4
Surveys in WO
◦ In website operation:
Info collecting survey

13. II. Planning a survey
1.
Set goals and objectives
2.
Identify the target users
3.
Recruit the participants
4.
Calculate the sample size
5.
Set timeline and resource

14. II. Planning a survey
Step 1
2
3
4
5
Set goals and objectives
Goal
Tìm hiểu xem game thủ
yêu thích yếu tố nào
trong một teaser giới thiệu
game mới.
Objective
Thu thập 1000 mẫu phản hồi ý kiến
trong thời gian 5 ngày để xem game
thủ yêu thích yếu tố nào, từ đó quyết
định thiết kế teaser.

15. II. Planning a survey
1
Step 2
3
4
5
Identify the target users:
◦ Ask the PG for more details:
Gamers (casual + MMORPG)
Internet user (no game or SNS
game background)

16. II. Planning a survey
1
2
Step 3
4
Recruit the participants
◦ Web Quality recruit list from forum, website game.
◦ CS center or ask PG marketing support.
Forum post
CS center
Online survey tool
5

17. II. Planning a survey
1
2
3
Step 4
Calculate the sample size
◦
Taro Yamane formula
n: sample size (số mẫu cần thiết)
N: population size (tổng số game thủ)
e: level of precision (sai số)
5

18. II. Planning a survey
1
2
3
Step 4
5
Calculate the sample size
Example:
HBTH teaser survey on 5.000.000 MMORPG gamers
(N=5000000) with level of precision is 3% (e=0.03), we need
to have:
n = 5000000/[1+ 5000000*(0.03)²]= 1110 samples

19. II. Planning a survey
1
2
3
Step 4
Calculate the sample size
Example:
n = 5000000/ [1+ 5000000*(0.03)²]= 1110 samples
5

20. II. Planning a survey
1
2
3
Calculate the sample size
◦
Offline survey in WO:
Only 50-100 samples because of:
1. Shortage of human resource.
2. Limited time.
Step 4
5

21. II. Planning a survey
1
2
3
Set timeline and resource
1. How long the survey runs?
2. Who’s in charge?
3. Survey cost (gifts for participants)
4
Step 5

22. III. Conducting a survey
1.
Choose question types
2.
Design question sequence
3.
Check the questions
4.
Prepare to launch
5.
Make the report

23. III. Conducting a survey
Step 1
2
3
4
5
Choose question types
1) Opened-ended question:
Feelings
Likes/Dislikes
Opinions
Pros
Get a lot of user’s info and
details: feedback, suggestions….
Additional comments
Cons
• Time consuming.
• Difficult to analyze.
• People will skip easily.

24. III. Conducting a survey
Step 1
2
3
4
5
Choose question types
2) Multiple choice – one answer:
Pros
To know the best possible
choice of the user over the
others.
Cons
Don’t know how the user feel
about the rest of the other
choice.

25. III. Conducting a survey
Step 1
2
3
4
5
Choose question types
3) Multiple choice - multiple answers:
Pros
Cons
To know the additional choice of Don’t know the level of choice
the user in a list.
(which is better and the order of
choice)

26. III. Conducting a survey
Step 1
2
3
4
Choose question types
4) Ranking question
Pros
To know order of choice over
the others.
Cons
Confuse user
5

27. III. Conducting a survey
Step 1
2
3
4
5
Choose question types
5) Rating question (Likert scale)
1 - Không thích
Pros
To know level of all specific
choice.
5 - Thích
Cons
Take a lot of user’s time.
Take time to analyze the scores.

28. III. Conducting a survey
1
Step 2
3
4
Design survey sequence
1) Opening question
• First questions should be easy and interesting
• Motivate user going forward to the rest of survey
5

29. III. Conducting a survey
1
Step 2
3
4
5
Design survey sequence
2) Question flow
• Categorize questions into groups of related topic
• Let users skip or switch back and fort between questions.

30. III. Conducting a survey
1
Step 2
3
4
Design survey sequence
3) Location of sensitive questions:
•
•
Age
•
Income
•
Marriage
•
Sensitive
questions
Name
Habits
Should be placed at the end of the survey or
can be skipped.
5

31. III. Conducting a survey
1
Step 2
3
4
5
Design survey sequence
Example:
Bạn có phải là game thủ VNG hay không?
Có
Bạn yêu thích thể loại game nào?
Webgame
Nhập vai
Casual
Không
Bạn sử dụng sản phẩm nào dưới đây của VNG?
Zing News
Zing MP3
Zing Chat

32. III. Conducting a survey
1
2
Step 3
4
5
Check the questions
◦
Do the respondents understand the survey’s objective?

33. III. Conducting a survey
1
2
Step 3
4
Check the questions
◦
Do the respondents feel comfortable answering the
questions?
5

34. III. Conducting a survey
1
2
Step 3
Check the questions
◦
Is the wording of the survey clear?
4
5

35. III. Conducting a survey
1
2
Step 3
4
5
Check the questions
◦
Are the answer choices compatible with the respondents’
experience in the matter?

36. III. Conducting a survey
1
2
Step 3
4
5
Check the questions
◦
Does the survey require the respondent to think too long or
responding?
◦
Which items produce irritation, embarrassment, or
confusion?
◦
Do any of the questions generate response bias?
◦
Do the answers collected reflect the purpose of the survey?
◦
Is there enough diversity in the answers received?
◦
Is the survey too long?
….

37. III. Conducting a survey
1
2
Step 3
4
5
Check the questions
Just do a PILOT TEST with a friend, colleague, family
member… who is a target user.

38. III. Conducting a survey
1
2
3
Step 4
5
Prepare to launch
Consider about the response rate:
Number of complete surveys
=
Number of paticipants contacted
Response
rate
E.g.
• Completed: 40
• Contacted: 100
 Response rate = 40%

39. III. Conducting a survey
1
2
3
Step 4
Prepare to launch
Acceptable response rate :
50% adequate, 60-70% good to very good
80% good
40% average, 50-60% good to very good
30% average
80-85% good
5

40. III. Writing a survey
1
2
3
Step 4
Prepare to launch
Don’t forget bring them
the gift as rewards 
5

41. III. Conducting a survey
1
2
Make the report
3
4
Step 5

42. III. Conducting a survey
1
2
3
4
Step 5
Make the report
• Focus the objectives of
the survey and result.
• Analyze and make
recommendations.

43. IV. Experiences
1.
Good vs. bad questions
2.
Our experiences

44. IV. Experiences
1
2
Good vs. bad question
1. Be brief: Keep questions short and ask one question at a time.
E.g.:
Bad: “Bạn thấy giao diện trang này có dễ nhìn không, bố cục có
hợp lý không?”
Good: “Bạn thấy giao diện trang web này như thế nào?”

45. IV. Experiences
1
2
Good vs. bad question
2. Be objective: Pay attention to the neutrality of the words.
E.g.:
Bad: “Trang web chúng tôi vừa có một tính năng Tìm kiếm rất tiện
dụng. Bạn có thấy thế không?”
Good: “Bạn thấy tính năng Tìm kiếm mới của trang web này thế
nào?”

46. IV. Experiences
1
2
Good vs. bad question
3. Be simple: use language simple in both words and phrases.
E.g.:
Bad: “Bạn có thường download client game, wallpaper và
screenshot trên trang web này không?”
Good: “Bạn có thường tải game, hình nền và hình trong game trên
trang web này không?”

47. IV. Experiences
1
2
Good vs. bad question
4. Be specific: Avoid things that are too general, too complex or
undefined.
E.g.:
Avoid: “thông thường”, “nói chung”, “đại loại”, “hay”…

48. IV. Experiences
1
2
Good vs. bad question
5. Some other writing guidelines:
Use legible questions
Use relevant questions
Use painless (less amount of effort) questions

49. IV. Experiences
1
2
Our Experiences
• Choose the right place to survey
• Choose Monday to survey.
• Be open and natural with the user
• Need to put tracking code to “Take
the survey” and “Submit” button to
calculate the response rate.
• Use the same device on a person.

50. V. Surveys apply in WO
1.
In Usability Testing
2.
In choosing homepage design
3.
In website review

51. V. Surveys apply in WO
1
2
3
In Usability Testing
1) ASQ (After-Scenario Questionnaire)
Effectiveness
Efficency
Satisfaction

52. V. Surveys apply in WO
1
2
In Usability Testing
2) Expectation Measure
Before
After
3

53. V. Surveys apply in WO
1
2
3
In Usability Testing
Expectation Measure (cont.)
After
Before

54. V. Surveys apply in WO
1
2
3
In choosing homepage design
QUIS (Questionnaire for User Interface Satisfaction)

55. V. Surveys apply in WO
1
2
3
In choosing homepage design
QUIS (Questionnaire for User Interface Satisfaction)(cont)

56. V. Surveys apply in WO
1
2
3
In choosing homepage design
QUIS (Questionnaire for User Interface Satisfaction)(cont)

57. V. Surveys apply in WO
1
2
3
In website review
1) SUS (System Usability Scale)

58. V. Surveys apply in WO
1
2
3
In website review
SUS (System Usability Scale) (cont.)

59. V. Surveys apply in WO
1
2
3
In website review
2) USE (Usefulness, Satisfaction and Ease of use)

60. V. Surveys apply in WO
1
2
3
In website review
USE (Usefulness, Satisfaction and Ease of use) (cont.)

61. V. Surveys apply in WO
1
2
3
In website review
USE (Usefulness, Satisfaction and Ease of use) (cont.)

62. V. Surveys apply in WO
1
2
3
In website review
USE (Usefulness, Satisfaction and Ease of use) (cont.)

63. VI. References
1.
SurveyMonkey: Smart Survey Design (2008)
2.
Measuring the user experience (2008)
3.
A project guide to UX design (2009)

65. Thank you 
Survey will make user HAPPY!!!