SlideShare une entreprise Scribd logo
1  sur  106
Design Better Surveys and Boost Response Rates
When It Comes to Surveys, Much to Complain About

            Too time consuming


Ask too many personal questions


    Come at inconvenient times


                   Are too long


               Can't be trusted

                                  0%   5%   10%    15%      20%      25%
                                                  Source: Vovici, n = 100
                                                                            2
What Makes a Successful Survey?

                    Meets its original goal
                    Produces accurate data that can be used
                    to make decisions from
                    Improves the satisfaction level of its
                    respondents
                    Used to improve the sponsoring
                    organization




                                                              3
The Seven Steps of Highly Successful Surveyors

                   1. Focus on a Goal
                   2. Survey the Right Group of People
                   3. Craft Your Invitation Carefully
                   4. Order Questions Logically
                   5. Write Objective Questions
                   6. Shorten the Survey
                   7. Close the Feedback Loop


                                                         4
Agenda

 Response Rate Targets
 Drivers of Response Rates
 Jumping the Hurdles to Response
 Craft Your Invitation Carefully
 Reporting
 Charts
 Q&A
 Multi-Language Surveys



                                   5
Response Rate Targets
Oversimplified Look at Market Research “Rates”
                                          General Population




                                                   Invitees

                            Incidence Rate = Invitees ÷ General Population



                                           Invitation Readers




                                                 Participants

  Participation Rate = Participants ÷ Invitees           Click-Thru Rate = Participants ÷ Invitation Readers



                                                 Completes
   Response Rate 1 ~= Completes ÷ Invitees                    Completion Rate = Completes ÷ Participants


                                                                                                               7
Response Rate Goals




                      Compliance
                      Representative at Higher Confidence
                      Representative by Cell
                      Representative Overall




                                                            8
High Response Rates Not Needed

Higher response rates do not produce much more accurate results
Fear is that low response rates indicate a potential non-response bias
Pew Research Center found a 25% response rate did not reduce quality of
survey estimates compared to a 50% response rate
   Source: “Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey”

Holbrook, Krosnick and Pfent found:
   Little variance by response rate
   But aim for response rates of at least 20% (“Our findings might not generalize ... to
   declines in response rates below 20 percent.”)
   Source: “The Causes and Consequences of Response Rates in Surveys by the News Media and Government
   Contractor Survey Research Firms”
Drivers of Response Rates
Key Drivers to Response Rates




                                                                Target
                                                  Quantity of   Audience
                                                  Reminders
                                     History of
                                     Invitation
                                     Frequency
                       Empanelment



         Invitation
         Salience

         Recency
         Exclusivity
         Incentives

                                                                           11
Target Audience Matters
Response Rates Proportional to Strength of Relationship
100%
  80%
  60%
  40%
  20%                                              Low
                                                   High
   0%
                                                   Average




Source: Vovici
                                                             12
Use Reminders to Improve Response Rate

                    Cumulative Response Rate
35%

30%

25%

20%

15%                                        Optimize rather than
                                           maximize response rates
10%

 5%

 0%
       Invitation    1st Reminder   2nd Reminder     3rd Reminder



                                                                     13
History of Invitation Frequency

                   For customers who aren’t members of a
                   panel:
                     The Boy Who Cried “Survey!” – too
                     frequent requests for feedback lead to a
                     decline in response rates
                     Rare requests to participate in a survey
                     produce a higher response
                     Response rates suffer if Marketing sends
                     out a high volume of email (“spam”)


                                                                14
Empanelment

              Respondents who have opted in to a panel
              expect to participate
              Engaged panelists will complain if they are
              not surveyed each month




                                                            15
Invitation Salience: Effect of Topic & Text on Response


       • Participate and earn a free gift card from Home Depot

       • Participate and win one of 3 cash prizes
           • Participate and $5 will be credited to the credit card of your choice




        • We’d like to know your thoughts about gasoline prices •This survey is super-short! (we promise)
        • Shorter than the typical survey! Takes only 10 minutes… • We only need to ask you about 10 questions
         • Your answers will remain completely confidential • We only need to ask you about 25 questions
       • We’d like to ask you questions about home improvement
                                                         • We’d like to ask you questions about Space Tourism
        • Get a free copy of the results after you participate!
                                            • The results of the survey will be published in the New York Times!


        • Earn frequent flyer miles on the credit card of your choice


                                                   Source: CMOR, “Creating a Respondent Cooperation Playbook”




                                                                                                              16
Recency

          The longer the lag between the event and
          the survey the lower the response rate.
          The response rate for one transactional
          survey jumped from 15% to 63% by
          sending invite within 15 minutes of event
          instead of 24 hours later.




                                                      17
Exclusivity

              Emphasizing the exclusivity of the
              invitation will improve response rates.
              Example: "You are one of a small, select
              group of customers that we have invited to
              provide us feedback."
              Of course, don't say it if it isn't true!




                                                          18
Incentives & Response
                    “It has been repeatedly shown that providing financial
                    incentives as a persuasive motivator for reluctant
                    respondents is a viable and effective technique for
                    increasing response rates…” - “Demographics and Online
                    Survey Response Rates”, Kurt Knapton and Steve Myers
                    “....Goetz, Tyler and Cook…concluded that financial
                    incentives increased response rates and showed no
                    differences in the demographics of the incentive and
                    non-incentive groups.”
                    “....Mason, Lesser and Dillman (2003) found that non-
                    response error may actually be reduced when financial
                    incentives are used...”




                                                                             19
Meta Analysis

                “Factors affecting response rates of the
                web survey: A systematic review”
                  Computers in Human Behavior (March 2010)
                  Weimiao Fan and Zheng Yan

                Synthesizes results of 300 studies into a
                model of the contributing factors of
                response rates
                Response rate = completes / invites




                                                             20
Factors Affecting Response Rates to Web Surveys

                  Content of
  Target                                                   Invitation                 Technology
                     web             Survey delivery
 audience                                                   designs                     issues
                questionnaires

                   Nature of the
   Type of        sponsors of the         Sampling        Personalization
  population          survey              methods
                                                                                      Survey software

                 What the topic is                            Scarcity
                                       Contact delivery
 Demographics
                                           modes
                   How long the
                  survey takes to                          Access control
                    complete                                                              Internet
                                                                                       transmission
  Personality                          Contact quality                                  capabilities
                 Question writing                         Use of incentives




                                                                              Source: Fan and Yan, 2010

                                                                                                        21
The Science of Email Marketing – Applied to Surveys

  Businesses are consumers.
  Try sending survey invitations on the weekends.
  Send invites very early in the morning.
  Use lots of links in your survey invites.
  Include reference information in your survey invitation, providing recipients
  other ways to provide feedback or seek assistance after the survey is closed.
  Subject lines should use the word survey and should not emphasize incentives.
  Emphasize in the invite that this is an exclusive opportunity to provide
  feedback (for those studies for which it is true, of course).
                     Source: Dan Zarrella, “The Science of Email Marketing”, 2011



                                                                                  22
Jumping the Hurdles to Response
The Third Step: Craft Your Invitation Carefully

     On                     • For research to house lists, make sure
   Email List
                              that lists are double opt-in
                    Not on
                              •
                Unsubscribe List For general market research, do not
                                Not Flagged as
                                 buy lists
                                    Spam
                              • Strongly consider using external panel
                                                Message
                                                Opened
                                 from a reputable provider
                                                                Link
                                                               Clicked




                                                                       24
The Third Step: Craft Your Invitation Carefully

      On                          • Maintain a suppression list of
    Email List
                                    addresses that have opted-out
                   Not on
                 Unsubscribe     • Provide different categories of
                    List     Not Flagged as
                                 Spam
                                     opt-out (e.g., marketing email vs.
                                     newsletters vs. surveys)
                                               Message
                                                Opened
                                 • Remove suppressed addresses
                                                                Link
                                     prior to emailing         Clicked




                                                                          25
The Third Step: Craft Your Invitation Carefully

      On                                             CAN-SPAM Compliance
    Email List
                                                     • Truthful subject lines
                     Not on
                 Unsubscribe List                    • Valid “From” e-mail
                                    Not Flagged as
                                        Spam           address
                                                     • Message
                                                        Physical address
                                                      Opened
                                                     • Valid opt-out link
                                                                       Link
                                                                     Clicked


• Avoid SPAM filter trigger words
  (“Free, “$”, “act now”)


                                                                                26
The Third Step: Craft Your Invitation Carefully

       On                                             •   Personal email address
     Email List                                           rather than corporate
                      Not on                              mail box
                  Unsubscribe List
                                     Not Flagged as
                                         Spam
                                                      Message
                                                      Opened
                                                                        Link
 •   Compelling subject line
                                                                       Clicked
      o   “Help us improve our
          products”
      o   “Share your opinion”




                                                                                   27
Clearing the Final Hurdle: Craft Your Invitation Carefully

  Less than 8 seconds to make a first impression
       On
  Tell recipients
   Email List     what you want them to do as soon
                   Not on
  as possible in the invitation
               Unsubscribe List
                         Not Flagged as
  Know your audience and write the invitation with
                             Spam
  this knowledge in mind                  Message
                                              Opened
  For transactional surveys, keep it timely             Link
                                                       Clicked
  For random samples, mentioning the exclusivity
  of the invite improves response rates
  Let them know how long the survey will be
  Consider using HTML e-mails vs. text
                                                                 28
Hurdles in Race to Turn Recipients into Respondents

     On                                             A well-crafted invitation is
   Email List                                   essential to jumping response
                    Not on                                            hurdles.
                Unsubscribe List
                                   Not Flagged as
                                       Spam
                                                    Message
                                                    Opened
                                                                       Link
                                                                      Clicked




                                                                                29
Craft Your Invitation Carefully
Sample Invitation




                    31
Clearing the Final Hurdle: Six Points to Cover
Intro/Basic Appeal   Your                               You've been selected for a research
                     OpinionCounts!                     study!
Importance of        Your participation in this survey Your participation in this survey will
Participation        will help improve products and help make improvements to a
                     services.                         government program.
Survey Subject       We'd like to know your thoughts We'd like to know your thoughts
Matter               about gasoline prices.          about dairy products.
Flexibility/         We only need to ask you about This survey will only take a few
Time Burden          10 questions.                 moments of your time.
Incentive Offered    Participate and $5 will be         Get a free copy of the results after
for Participation    credited to the credit card of     you participate!




                                                                                                Source: CMOR
                     your choice
Privacy/Data         As member of the Better            Your privacy is important to us, your
Confidentiality      Business Bureau we take your       answers will be combined with
                     privacy seriously and will         others, and will never be linked with
                     respect your confidentiality.      you personally.



                                                                                                32
Common Elements of Email Invitations

Incentive amount

Expected length

      Contact info

 Opt-out/Decline

Subject included

   Privacy Policy

          Deadline

  Panel reminder

                      0%             20%          40%   60%   80%   100%


Source: Vovici review of 30 ESOMAR 26 documents
                                                                       33
Invitations: To Disclose or Not to Disclose
100%                                                 “The invitation text does not
                                      Exclude
 90%                                                 include…the topic of the
                                      Include
 80%                                                 survey, as this may affect the
 70%                                                 panelist’s willingness to
 60%                                                 participate and thus create
 50%                                                 bias.” - Norstat
 40%                                                 “Research shows that people
 30%                                                 may be more likely to
 20%                                                 respond when the invitation
 10%                                                 references a local geography,
  0%                                   Normalized.   a topic of interest, or an
                                                     appealing reward. However,
                                                     care must be taken not to
 Source: Vovici review of 30 ESOMAR 26 documents     bias the responses.” - SSI
Effect of Topic & Text of Invitation on Response Rate


       • Participate and earn a free gift card from Home Depot

       • Participate and win one of 3 cash prizes
           • Participate and $5 will be credited to the credit card of your choice




        • We’d like to know your thoughts about gasoline prices •This survey is super-short! (we promise)
        • Shorter than the typical survey! Takes only 10 minutes… • We only need to ask you about 10 questions
         • Your answers will remain completely confidential • We only need to ask you about 25 questions
       • We’d like to ask you questions about home improvement
                                                         • We’d like to ask you questions about Space Tourism
        • Get a free copy of the results after you participate!
                                            • The results of the survey will be published in the New York Times!


        • Earn frequent flyer miles on the credit card of your choice


                                                   Source: CMOR, “Creating a Respondent Cooperation Playbook”




                                                                                                              35
Different Constituencies Respond Differently
  American College of Cardiology Case Study
    Surveys sent between 10 am and 2 pm Tuesday through Thursday
    Discovered by analyzing response patterns to past surveys
    “Strike at the best time, even if it means waiting.”

  Timing of invitations
    What works well for consumers may work poorly for businesses
    “Is Sunday the new Tuesday?” for B2B surveys
    A staggered series of reminders makes optimizing delivery time less important

  A/B test
    Adapt the split tests used by marketing departments to pit invitations against one
    another
    Rinse and repeat to find what works best

                                                                                         36
Reporting
Good Reports Require Upfront Planning

   Sampling’s Impact on Reporting
  Creating questionnaires that facilitate results.


   Construct Reports
  Easily consumable reports that accurately detail trends and
  highlight conclusions.


   Present & Share Reports & Data
  Efficiently format and share your detailed reports and raw data




                                                                    38
Qualitative/Quantitative Research




• Random samples                                                   • Convenience samples
• Closed questions             Quantitative Research               • Polls with comments
• Limited response options                                         • Online Communities
• Can’t get the story behind                                       • Focus Groups
  the story                                                        • Provide the Story behind
• Provides hard data that                   Qualitative Research     the story
  can be extrapolated to a                                         • Can’t extrapolate to a
  larger audience                                                    larger audience




                                                                                          39
Ramifications for Reporting

  Sometimes lots of numbers are really qualitative research,
  because you can’t extrapolate from them
  Forget sophisticated reporting techniques if research is based on
  a convenience sample
  But… you don’t need a probability sample to learn that a single
  customer is unhappy




                                                                      40
Structure the Questionnaire for Good Data


Preload as much respondent demographic
  data as possible




Use conditional logic and piping to keep
 respondents from answering questions that
 aren’t applicable to them




Use validation to ensure data is provided in
 the correct format


                                               41
Structure Questions to Fit Analytic Models


Set up parallel questions if you want gap analysis
  between importance and satisfaction




Include questions to support the quadrant analyses
  and segmentations you will want to report




When designing recurring surveys, develop multi-
 question indices to report on



                                                     42
Sample Gap Prioritization Matrix




                                   43
Use Test Data Before Fielding

  Populate the survey with random test data
  Generate reports on the mock data to verify your questions will
  give you the information that you are seeking




                                                                    44
Understand Your Audience

                Who, What, When

         Who will be looking at your data?


         What are their expectations? Format, delivery,
         depth.


         How often?

                                                          45
Building your Report

      Use a top-down approach
         Start by understanding the major themes, then drill down.
         Use filtering and cross-tabulation to find hidden patterns and trends.
         Preloaded historical and demographic data will provide essential context.
         Do others reach the same conclusions when they work with the data?

      Let the results shape the report
         Question order should not determine reporting order
         Abandon preconceived notions/self-fulfilled ‘realizations’/dramatic over-
         generalizations
         However, understand, explain and develop conclusions that seem contradictory or
         illogical




                                                                                           46
Logically Consolidate & Summarize
•   Keep it simple and assume nothing.

        •    Be precise; remove platitudes; use examples and draw logical
             parallels

        •    Include: an executive summary, your methodology, any special
             notes. Always cite all sources.

        •    Call out significant findings. Use the “tell them, then tell them
             again” approach.

•   Avoid unintentional conclusions. Reconcile easily
    misunderstood or contradictory results.

•   Provide links to secondary research and articles that:

    •       Provide perspective

    •       Reinforce conclusions

                                                                                 47
Sharing Your Insight

       Build Reports that are Scalable & Repeatable
          Researcher’s quandary: People always want more.
          Provide a method for other business users to work with reports
               Real-time self service & Raw data (filtered or in its entirety)

       Choose The Right Format & Distribution Method
          Self-service vs. email vs. print
          Word, Excel, PowerPoint, PDF, etc.

       Consider the Future
          The most cost effective research is that which you have already done
          Integrate data into future research



                                                                                 48
Create Hierarchical Reports
  Hierarchical reports eliminate manual processes and offer personalized data per report
                                           user

Case Study:
Corporate Climate Survey, reviewed by 1000 individuals, with different views:
    Branch manager and assistant branch manager: branch results
    compared to district, region, nation

    District manager: each branch in district compared to
    district, region, nation

    Regional VP: each branch in district, each district in
    region, compared to nation

    Corporate executives: national results; drilldown to any level


                                                                                           49
Five-Part Essay   vs.   Inverted Pyramid

 Introduction              Conclusion
                             Essential
 Narration                 Information
                             Important
                            Information
 Affirmation
                            Supplemental
                             Information
 Negation
                                Data
                               Tables

 Conclusion


                                           50
Reports & Presentations in Opposition
Reading a Report                 Attending a Presentation

Not scheduled in advance         Scheduled

No agenda                        Follows agenda

Private                          Social

No interaction with author       Interaction with presenter

Competing demands on reader      No competing demands on attendee

Recommends first                 Recommendations last

                              Source: Lawrence Gibson, Eric Marder Associates

                                                                            51
Charts
“My focus
group likes
the green.”




              53
Bar Charts are Easier to Interpret than Pie Charts

     Question 1            Question 2           Question 3




 A     B   C   D   E   A   B   C   D    E   A   B    C   D   E
Bar Charts Often Inconsistent

                 % Completely                        % Completely Likely to
                   Satisfied                             Repurchase
                                                                     30%
Product B                                     Product B                35%
                                                                          43%
Product D                                     Product D                 37%
            0%        20%     40%     60%                 0%   20%    40%       60%


            % Completely Likely to                   % Completely Likely to
                Recommend                              Expand Purchases
   Product B                                  Product B
   Product D                                  Product D

                 0%     10%     20%     30%           10%      12%    14%       16%
Summary Statistics Lose Information
                            Mean = 3.0
5                      5                   5
4                      4                   4
3                      3                   3
2                      2                   2
1                      1                   1
0                      0                   0
                           Top Box = 20%
    5                  5                   5
    4                  4                   4
    3                  3                   3
    2                  2                   2
    1                  1                   1
    0                  0                   0
Common Mistakes with Trend Reports

                     Showing one year of data
                     Grouping data by short time periods
                     Using trend reports at all
                     Including recent responses in trend
                     reports




                                                           57
Random Samples Converging As Sample Size Increases

                 Series 1   Series 2   Series 3
10.0
 9.0
 8.0
 7.0
 6.0
 5.0
 4.0
 3.0
 2.0
 1.0
         1
        10




       109




       208
        19
        28
        37
        46
        55
        64
        73
        82
        91
       100


       127
       136
       145
       154
       163
       172
       181
       190
       199

       217
       226
       235
       244
       253
       262
       271
       280
       289
       298
       118




                                                     58
Rich Media Implications
Use of Faces in Surveys

  A face of an interviewer or researcher will increase
  the accuracy of reporting
  Accuracy is maximized when the gender of the
  respondent and the displayed face match
  Male panelists are more likely to report accurately
  when the introduction is formal, rather than
  friendly
  Accuracy differed depending on the interaction of
  gender of the respondent, presence or absence of a
  face, gender of the face and style of introduction
                                    Source: NPD Group


                                                         60
Continuum of Rich Media Offerings




        More like                   More like
      face-to-face                   paper
       interviews                   surveys


                                      Source: Gina K. Walejko

                                                                61
Continuum of Rich Media Offerings

                               Avatars with speech
                               Faces with recordings
                               Faces
                               Surveys routed to human
        More like              interviewers
         face-to-              Automating traditional F2F
           face                probes
       interviews
                            Reintroduces social desirability
                            bias

                                                               62
Rich Media Can Increase Completion Rates




                      Source: Deborah Sleep & Jon Puleston, Quirks – December
                                                                         2008

                                                                                63
Element54 Research

   Data Consistency   Respondent Usability




                                             64
Balanced 5 Cell Design: 2000 Total Interviews

 Radio Button

 Anchor @ „1‟

 Anchor @ „5‟

Anchor @ ‟10‟

 Experimental
                                                65
Anchoring of Slider Position Matters




                                       Source: Element54

                                                           66
Inference vs. Instruction

                            42% did not
                             read the
                             question




                                              33% did not read
                                               the "traditional"
                                                  question

                                          Source: Element54


                                                                   67
Equal Ratings for ‘Engagement’ Metrics


                       1.0               2.0
    Survey
  Enjoyment            8.20              8.25
  Ease of Use          8.86              8.85
   Desire to
    Repeat             7.02              6.56

                                         Source: Element54

                                                             68
Tips for Migrating to Rich Media

  Discretion is the better part of
  valor
  Split ballot test rich media
  questions against traditional
  questions to identify differences
  Visual choices introduce bias,
  especially for more abstract
  choices
  Use rich media in a considered
  manner

                                      69
Q&A
Multi-Language Surveys
Benefits of Multi-Language Surveys

     Increase response rates
     Improve data quality
     Language-specific surveys
     Use country as a basis for reports and conditional
     branching
     Highly efficient means of conducting a single
     survey among a global customer base
     Allows you to consistently communicate with and
     gain feedback from global customers

72                                                        72
Translation throughout the Survey Research Process




     Study       Survey
                               Fielding      Analysis
     Design      Design




73                                                      73
Study Design

     What languages do you need to
     translate to?
     What internal and external
     resources will you use for
     translation?
     Once the results come in, how
     will you translate open-ended
     comments?




74                                   74
Remember: Countries are Not Monoliths

  Most countries have multiple
  languages
  There is no Belgian language
  There is no Chinese language
  Even when two countries share a
  common language, you may need to
  budget for “localization”




                                        75
                                         7
Biggest Mistake in Multi-Language Studies: Unrealistic Schedules


                 • Prepare a conservative schedule
 Study Design




                 • Time-consuming translation phase after the master
 Survey Design     questionnaire is complete

                 • International holiday schedules can extend the time the
     Fielding      survey is in the field

                 • Translation of open-ended comments adds weeks to the
     Analysis      analysis phase


76                                                                           76
Translation throughout the Survey Research Process




     Study       Survey
                               Fielding      Analysis
     Design      Design




77                                                      77
Write Master Questionnaire with Translation in Mind

     Using your typical questionnaire would
       Increase translation costs
       Reduce representativeness of results

     Instead, structure for translation
       Avoid jargon, slang and technical terms
       Rewrite for readability
       Use as few open-ended questions as possible
       Use language-specific skip patterns




78                                                     78
Some Questions Need Alternates, Not Translations

     Think localization, not just translation
     Country-specific questions are often called-for
     No sense translating the choices of the question,
     “Which hotel did you choose?”
     Choice lists may need to be tailored for major
     brands in different markets
     Entire sections of the questionnaire may differ
     for structure of industry in that country
     Graphics and icons may be inappropriate in
     some countries or for some cultures



79                                                       79
Translation                    vs. Localization
                                                     English

                                     United States

                                                     Spanish
           French
                                                     English
English   Spanish       Master
                                       Canada
                     Questionnaire
                                                      French
          Japanese
                                        Mexico       Spanish



                                        Japan        Japanese




                                                                80
Areas of Special Interest

     Weights and measures
     Currency
     Time – 24 hour clock, GMT, Zulu, UTC,
     WET
     Use images without text to best use in
     many cultures
     May need to reduce questionnaire
     length to take into account overall
     budget for translations


81                                            81
Generational Considerations

     Use Age Groupings rather than Generational Groupings for
     reporting across cultures
       Exception: Generational Groupings may be used for the US and Canada

     What is Generation X?
       Group born between 1964 and 1975 (US)
       Younger Westernized generation (India)
       Generation Golf (Germany)
       Génération Bof (France)
       Generation Niks (Netherlands)




82                                                                           82
Scales Matter

     Some scales don’t translate well
       They become too wordy, too subjective

     Some scales exaggerate cultural issues
       Likert scales can be particularly difficult

     Safe choices
       ConstantSum
       MaxDiff




83                                                   83
Answers Patterns for Common Response Styles
                                                    Completely              Somewhat    Neither agree   Somewhat           Completely
                                                                 Disagree                                          Agree
                            Response Style           disagree                disagree   nor disagree      agree              agree

                            Optimal Responding

                            Extreme Response
                            Style (ERS)
Truncated Scales




                            Response Range (RR)

                            Mild Response Style

                            Midpoint Response
                            (MPR)
                            Acquiescence
Social/Anti-Social Styles




                            Response Style (ARS)
                            Disacquiescence
                            Response Style (DARS)
                            Socially Desirable
                            Responding (SDR)
                            Noncontingent
                            Responding (NCR)


                                                                                                                                    84
Response Styles by Country: Informed by Culture




                          Source: Johnson, Kulesa, Cho, Shavitt, 2003; Vovici

                                                                                85
Constant Sum Example

  “Please distribute 100
  points amongst the                      Item         Points
  following 5 items, based     Optical zoom of 3X or    30
                               greater
  on their importance to you   Resolution: 10           40
  when buying digital          Megapixels or greater
                               Available in              5
  cameras. An item that has    multiplecolors
  no importance to you         Battery Life             25

  should be given no points.   Weight                    0
                                                        100
  The total must equal 100
  points.”


                                                                86
MaxDiff Example




87                 87
Important Safety Tips for Multi-Language Surveys

     Translation is the last part of the survey
     cycle
     Surveys must be finalized before having
     them translated to avoid confusion and
     re-work
     Do not trust machine translations!




88                                                  88
A Survey Translation is Not Just a Translation of the Survey

     Invitation and reminder emails
     Introductions
     Button captions
     Validation messages
     End pages
     Thank you messages
     Alerts




89                                                          89
Choose Your Translator Well

     The best translators have excellent
     understanding of the originating language, the
     target culture, the industry and survey
     research
     Too often, translation is “insourced” to
     employees who do not have the requisite
     experience
     Survey translations are more demanding than
     traditional translations
     Use only native speaking professional
     translators (preferably certified translators)


90                                                    90
Best Practices for Working with Translators

     Brief the translators on the intent of the
     survey
     Create a glossary of English terms and the
     target language for consistency
     Use native speakers to review the translation
                                                     Kathleen Bostick
     and provide feedback                            VP Global Marketing
                                                     Kathleen.bostick@lionbridge.com
                                                     @Lion_MktgVEEP
     Pilot test of translation of survey may be
                                                     http://blog.lionbridge.com/
     required and will lengthen the process
                                                     www.lionbridge.com

     A backtranslation may be recommended
                                                     Source: Lionbridge
     Choose a tool that saves the question and       Technologies

     translations.


91                                                                                     91
Translation throughout the Survey Research Process




     Study       Survey
                               Fielding      Analysis
     Design      Design




92                                                      92
Deadlines and Timing

     Timing can affect survey response rates
     Be knowledgeable of the holidays and cultural
     norms of your survey populations
       Avoid survey deadlines on holidays or weekends (in some
       cultures Friday/Saturday constitute the weekend, not
       Saturday/Sunday)
       Be mindful of the International Date Line and individual times
       zones within countries
       Be careful of time indicators – many cultures use a 24-hour
       clock




93                                                                      93
Respondent Language Selection

     •   Translated email invitations with links toappropriate translation
     •   Or self-selection based on all available languages:




94                                                                           94
Respondent Language Selection

  Auto-detect based on primary
  language set in browser
  Conditional logic, alerts and triggers
  based on language-specific surveys
  Respondents’ browser settings may
  not match their native language




                                           95
                                            9
Translation throughout the Survey Research Process




     Study       Survey
                               Fielding      Analysis
     Design      Design




96                                                      96
Analysis Considerations

     Some countries skew towards “positive” responses (Latin
     America, Japan)
     Others skew negative (Germany)
     Take special care with Product Concept Testing and Brand
     Perception Surveys
     Always analyze data by country, not just by region
       Even if you plan to aggregate by region for reporting purposes
       Within some countries, important regional differences are also worth investigating




97                                                                                          97
Optimal Data Storage for Survey Results




    Base
   Survey


    English




                                          98
Optimal Data Storage for Survey Results


                     French
                                French




                     Italian
                                Italian




                    German
    Base                       German


   Survey
                    Spanish
                               Spanish




    English         Chinese
                               Chinese




                    Japanese
                               Japanese




                                          99
Optimal Data Storage for Survey Results


                     French
                                French




                     Italian
                                Italian




                    German
    Base                       German     Response
   Survey                                 Database*
                    Spanish
                               Spanish




    English         Chinese
                               Chinese




                    Japanese
                               Japanese
                                                       *Safe
                                                      Harbor
                                                      Certified
                                                                  100
Optimal Data Storage for Survey Results


                     French
                                French
                                                      Data
                                                     Export
                     Italian
                                Italian




                    German
    Base                       German     Response
   Survey                                 Database
                    Spanish
                               Spanish




    English         Chinese
                               Chinese




                    Japanese
                               Japanese




                                                              101
Optimal Data Storage for Survey Results


                     French
                                French
                                                      Data
                                                     Export
                     Italian
                                Italian




                    German
    Base                       German     Response
   Survey                                 Database
                    Spanish
                               Spanish


                                                      View
    English         Chinese                          Report
                               Chinese




                    Japanese
                               Japanese




                                                              102
Translation throughout the Survey Research Process




      Study      Survey
                               Fielding      Analysis
      Design     Design




103                                                   103
What is the Role of the Translator Seat?

  User role that provides access to translate specific surveys into
  specific languages
  Economically allows translators to provide value
  Provides security by limiting what the translator can see and do
  Option to export surveys as XLIFF file, an XML version of the
  survey which translation houses can rapidly use to translate a
  single survey into a variety of languages




                                                                      104
Bad Translations: More Than Just Bad Data

      Significant schedule slips
                                            Due to re-fielding,
      Significant budget overruns            new translation
      Non-blind surveys: Your brand loses
      credibility




105                                                               105
Thank You For Your Time

Feel free to email questions:
  Jeffrey Henning, PRC

  VP of Strategy & Founder

  Vovici

  jhenning@vovici.com




           http://blog.vovici.com

                                    106

Contenu connexe

En vedette

Cloudworks Presentation Mar 2010
Cloudworks Presentation Mar 2010Cloudworks Presentation Mar 2010
Cloudworks Presentation Mar 2010Ryo Ooishi
 
Presentacion de hipertexto
Presentacion de hipertextoPresentacion de hipertexto
Presentacion de hipertextoedipau2007
 
Global Health Informatics: Novel Approaches & Applications: Walter Curioso
Global Health Informatics: Novel Approaches & Applications: Walter CuriosoGlobal Health Informatics: Novel Approaches & Applications: Walter Curioso
Global Health Informatics: Novel Approaches & Applications: Walter CuriosoUWGlobalHealth
 
Film Posters with similar Themes of my film
Film Posters with similar Themes of my filmFilm Posters with similar Themes of my film
Film Posters with similar Themes of my filmguest2cc3ef
 
Survey Software--Increasing Survey Response Rates
Survey Software--Increasing Survey Response RatesSurvey Software--Increasing Survey Response Rates
Survey Software--Increasing Survey Response RatesQualtrics
 
Node Js, AngularJs and Express Js Tutorial
Node Js, AngularJs and Express Js TutorialNode Js, AngularJs and Express Js Tutorial
Node Js, AngularJs and Express Js TutorialPHP Support
 

En vedette (14)

PACIS Survey Workshop
PACIS Survey WorkshopPACIS Survey Workshop
PACIS Survey Workshop
 
Bulletin #263
Bulletin #263Bulletin #263
Bulletin #263
 
Cloudworks Presentation Mar 2010
Cloudworks Presentation Mar 2010Cloudworks Presentation Mar 2010
Cloudworks Presentation Mar 2010
 
Presentacion de hipertexto
Presentacion de hipertextoPresentacion de hipertexto
Presentacion de hipertexto
 
Global Health Informatics: Novel Approaches & Applications: Walter Curioso
Global Health Informatics: Novel Approaches & Applications: Walter CuriosoGlobal Health Informatics: Novel Approaches & Applications: Walter Curioso
Global Health Informatics: Novel Approaches & Applications: Walter Curioso
 
Public Health - Bellinda Schoof
Public Health - Bellinda SchoofPublic Health - Bellinda Schoof
Public Health - Bellinda Schoof
 
Film Posters with similar Themes of my film
Film Posters with similar Themes of my filmFilm Posters with similar Themes of my film
Film Posters with similar Themes of my film
 
module1
module1module1
module1
 
Government Affairs Update - Bob Hall
Government Affairs Update - Bob HallGovernment Affairs Update - Bob Hall
Government Affairs Update - Bob Hall
 
Survey Software--Increasing Survey Response Rates
Survey Software--Increasing Survey Response RatesSurvey Software--Increasing Survey Response Rates
Survey Software--Increasing Survey Response Rates
 
4.3. survey
4.3. survey4.3. survey
4.3. survey
 
Mobile app ux_principles
Mobile app ux_principlesMobile app ux_principles
Mobile app ux_principles
 
Developing a National Telemedicine Network
Developing a National Telemedicine NetworkDeveloping a National Telemedicine Network
Developing a National Telemedicine Network
 
Node Js, AngularJs and Express Js Tutorial
Node Js, AngularJs and Express Js TutorialNode Js, AngularJs and Express Js Tutorial
Node Js, AngularJs and Express Js Tutorial
 

Similaire à Vovici Vision 2011: New Design Better Surveys and Boost Response Rates b

2010 Healthcare Exhibitors Association Annual Meeting: Mix-it-up Metricsl
2010 Healthcare Exhibitors Association Annual Meeting: Mix-it-up Metricsl2010 Healthcare Exhibitors Association Annual Meeting: Mix-it-up Metricsl
2010 Healthcare Exhibitors Association Annual Meeting: Mix-it-up MetricslCindy McCormick
 
Tips to make you better at collecting customer feedback
Tips to make you better at collecting customer feedbackTips to make you better at collecting customer feedback
Tips to make you better at collecting customer feedbackBaptiste Debever
 
InfoQuest Intro PowerPoint
InfoQuest Intro PowerPointInfoQuest Intro PowerPoint
InfoQuest Intro PowerPointJohn Coldwell
 
The complete guide to increasing quality and quantity of survey responses
The complete guide to increasing quality and quantity of survey responsesThe complete guide to increasing quality and quantity of survey responses
The complete guide to increasing quality and quantity of survey responsesXoxoday
 
Crowdsourced Probability Estimates: A Field Guide (FAIR Institute)
Crowdsourced Probability Estimates: A Field Guide (FAIR Institute)Crowdsourced Probability Estimates: A Field Guide (FAIR Institute)
Crowdsourced Probability Estimates: A Field Guide (FAIR Institute)Tony Martin-Vegue
 
Are In-Context Respondents More "Honest"?
Are In-Context Respondents More "Honest"?Are In-Context Respondents More "Honest"?
Are In-Context Respondents More "Honest"?Instantly
 
Better Bidding for Facilities Management Contracts
Better Bidding for Facilities Management ContractsBetter Bidding for Facilities Management Contracts
Better Bidding for Facilities Management Contractschrisdpayne
 
Crowdsourced Probability Estimates: A Field Guide
Crowdsourced Probability Estimates: A Field GuideCrowdsourced Probability Estimates: A Field Guide
Crowdsourced Probability Estimates: A Field GuideTony Martin-Vegue
 
Survey Research
Survey Research Survey Research
Survey Research atrantham
 
Lisbon Facebook Marketing Workshop, part 1
Lisbon Facebook Marketing Workshop, part 1Lisbon Facebook Marketing Workshop, part 1
Lisbon Facebook Marketing Workshop, part 1Michael Leander
 
Taming the Social Media Beast - Social Media Analytics for Marketers
Taming the Social Media Beast - Social Media Analytics for MarketersTaming the Social Media Beast - Social Media Analytics for Marketers
Taming the Social Media Beast - Social Media Analytics for Marketersmariannemadden
 
5 Benefits of Using Reciprocity in Investigation Interviews
5 Benefits of Using Reciprocity in Investigation Interviews5 Benefits of Using Reciprocity in Investigation Interviews
5 Benefits of Using Reciprocity in Investigation InterviewsCase IQ
 
Swipe. Click. Binge. Repeat. What Happens When Performance Marketing Isn't En...
Swipe. Click. Binge. Repeat. What Happens When Performance Marketing Isn't En...Swipe. Click. Binge. Repeat. What Happens When Performance Marketing Isn't En...
Swipe. Click. Binge. Repeat. What Happens When Performance Marketing Isn't En...JD Prater
 
KEYNOTE: Swipe. Click. Binge. Repeat. What Happens When Performance Marketing...
KEYNOTE: Swipe. Click. Binge. Repeat. What Happens When Performance Marketing...KEYNOTE: Swipe. Click. Binge. Repeat. What Happens When Performance Marketing...
KEYNOTE: Swipe. Click. Binge. Repeat. What Happens When Performance Marketing...Hanapin Marketing
 
Brand Audio IVR Presentation
Brand Audio IVR PresentationBrand Audio IVR Presentation
Brand Audio IVR PresentationDaniel Graham
 
Using Public Social Media to Find Answers to Questions
Using Public Social Media to Find Answers to QuestionsUsing Public Social Media to Find Answers to Questions
Using Public Social Media to Find Answers to QuestionsJeffrey Nichols
 

Similaire à Vovici Vision 2011: New Design Better Surveys and Boost Response Rates b (20)

2010 Healthcare Exhibitors Association Annual Meeting: Mix-it-up Metricsl
2010 Healthcare Exhibitors Association Annual Meeting: Mix-it-up Metricsl2010 Healthcare Exhibitors Association Annual Meeting: Mix-it-up Metricsl
2010 Healthcare Exhibitors Association Annual Meeting: Mix-it-up Metricsl
 
Question2
Question2Question2
Question2
 
Tips to make you better at collecting customer feedback
Tips to make you better at collecting customer feedbackTips to make you better at collecting customer feedback
Tips to make you better at collecting customer feedback
 
InfoQuest Intro PowerPoint
InfoQuest Intro PowerPointInfoQuest Intro PowerPoint
InfoQuest Intro PowerPoint
 
2009 Market Research Dynamics
2009 Market Research Dynamics2009 Market Research Dynamics
2009 Market Research Dynamics
 
The complete guide to increasing quality and quantity of survey responses
The complete guide to increasing quality and quantity of survey responsesThe complete guide to increasing quality and quantity of survey responses
The complete guide to increasing quality and quantity of survey responses
 
Crowdsourced Probability Estimates: A Field Guide (FAIR Institute)
Crowdsourced Probability Estimates: A Field Guide (FAIR Institute)Crowdsourced Probability Estimates: A Field Guide (FAIR Institute)
Crowdsourced Probability Estimates: A Field Guide (FAIR Institute)
 
Vovici Vision 2011: Five Keys to Improving Customer Satisfaction
Vovici Vision 2011: Five Keys to Improving Customer SatisfactionVovici Vision 2011: Five Keys to Improving Customer Satisfaction
Vovici Vision 2011: Five Keys to Improving Customer Satisfaction
 
Are In-Context Respondents More "Honest"?
Are In-Context Respondents More "Honest"?Are In-Context Respondents More "Honest"?
Are In-Context Respondents More "Honest"?
 
Better Bidding for Facilities Management Contracts
Better Bidding for Facilities Management ContractsBetter Bidding for Facilities Management Contracts
Better Bidding for Facilities Management Contracts
 
Crowdsourced Probability Estimates: A Field Guide
Crowdsourced Probability Estimates: A Field GuideCrowdsourced Probability Estimates: A Field Guide
Crowdsourced Probability Estimates: A Field Guide
 
Survey Research
Survey Research Survey Research
Survey Research
 
Lisbon Facebook Marketing Workshop, part 1
Lisbon Facebook Marketing Workshop, part 1Lisbon Facebook Marketing Workshop, part 1
Lisbon Facebook Marketing Workshop, part 1
 
Taming the Social Media Beast - Social Media Analytics for Marketers
Taming the Social Media Beast - Social Media Analytics for MarketersTaming the Social Media Beast - Social Media Analytics for Marketers
Taming the Social Media Beast - Social Media Analytics for Marketers
 
5 Benefits of Using Reciprocity in Investigation Interviews
5 Benefits of Using Reciprocity in Investigation Interviews5 Benefits of Using Reciprocity in Investigation Interviews
5 Benefits of Using Reciprocity in Investigation Interviews
 
Hjc higher ed
Hjc higher edHjc higher ed
Hjc higher ed
 
Swipe. Click. Binge. Repeat. What Happens When Performance Marketing Isn't En...
Swipe. Click. Binge. Repeat. What Happens When Performance Marketing Isn't En...Swipe. Click. Binge. Repeat. What Happens When Performance Marketing Isn't En...
Swipe. Click. Binge. Repeat. What Happens When Performance Marketing Isn't En...
 
KEYNOTE: Swipe. Click. Binge. Repeat. What Happens When Performance Marketing...
KEYNOTE: Swipe. Click. Binge. Repeat. What Happens When Performance Marketing...KEYNOTE: Swipe. Click. Binge. Repeat. What Happens When Performance Marketing...
KEYNOTE: Swipe. Click. Binge. Repeat. What Happens When Performance Marketing...
 
Brand Audio IVR Presentation
Brand Audio IVR PresentationBrand Audio IVR Presentation
Brand Audio IVR Presentation
 
Using Public Social Media to Find Answers to Questions
Using Public Social Media to Find Answers to QuestionsUsing Public Social Media to Find Answers to Questions
Using Public Social Media to Find Answers to Questions
 

Dernier

AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraDeakin University
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDGMarianaLemus7
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Science&tech:THE INFORMATION AGE STS.pdf
Science&tech:THE INFORMATION AGE STS.pdfScience&tech:THE INFORMATION AGE STS.pdf
Science&tech:THE INFORMATION AGE STS.pdfjimielynbastida
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphNeo4j
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsAndrey Dotsenko
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Neo4j
 
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsHyundai Motor Group
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 

Dernier (20)

AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning era
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDG
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Science&tech:THE INFORMATION AGE STS.pdf
Science&tech:THE INFORMATION AGE STS.pdfScience&tech:THE INFORMATION AGE STS.pdf
Science&tech:THE INFORMATION AGE STS.pdf
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024
 
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 

Vovici Vision 2011: New Design Better Surveys and Boost Response Rates b

  • 1. Design Better Surveys and Boost Response Rates
  • 2. When It Comes to Surveys, Much to Complain About Too time consuming Ask too many personal questions Come at inconvenient times Are too long Can't be trusted 0% 5% 10% 15% 20% 25% Source: Vovici, n = 100 2
  • 3. What Makes a Successful Survey? Meets its original goal Produces accurate data that can be used to make decisions from Improves the satisfaction level of its respondents Used to improve the sponsoring organization 3
  • 4. The Seven Steps of Highly Successful Surveyors 1. Focus on a Goal 2. Survey the Right Group of People 3. Craft Your Invitation Carefully 4. Order Questions Logically 5. Write Objective Questions 6. Shorten the Survey 7. Close the Feedback Loop 4
  • 5. Agenda Response Rate Targets Drivers of Response Rates Jumping the Hurdles to Response Craft Your Invitation Carefully Reporting Charts Q&A Multi-Language Surveys 5
  • 7. Oversimplified Look at Market Research “Rates” General Population Invitees Incidence Rate = Invitees ÷ General Population Invitation Readers Participants Participation Rate = Participants ÷ Invitees Click-Thru Rate = Participants ÷ Invitation Readers Completes Response Rate 1 ~= Completes ÷ Invitees Completion Rate = Completes ÷ Participants 7
  • 8. Response Rate Goals Compliance Representative at Higher Confidence Representative by Cell Representative Overall 8
  • 9. High Response Rates Not Needed Higher response rates do not produce much more accurate results Fear is that low response rates indicate a potential non-response bias Pew Research Center found a 25% response rate did not reduce quality of survey estimates compared to a 50% response rate Source: “Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey” Holbrook, Krosnick and Pfent found: Little variance by response rate But aim for response rates of at least 20% (“Our findings might not generalize ... to declines in response rates below 20 percent.”) Source: “The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms”
  • 11. Key Drivers to Response Rates Target Quantity of Audience Reminders History of Invitation Frequency Empanelment Invitation Salience Recency Exclusivity Incentives 11
  • 12. Target Audience Matters Response Rates Proportional to Strength of Relationship 100% 80% 60% 40% 20% Low High 0% Average Source: Vovici 12
  • 13. Use Reminders to Improve Response Rate Cumulative Response Rate 35% 30% 25% 20% 15% Optimize rather than maximize response rates 10% 5% 0% Invitation 1st Reminder 2nd Reminder 3rd Reminder 13
  • 14. History of Invitation Frequency For customers who aren’t members of a panel: The Boy Who Cried “Survey!” – too frequent requests for feedback lead to a decline in response rates Rare requests to participate in a survey produce a higher response Response rates suffer if Marketing sends out a high volume of email (“spam”) 14
  • 15. Empanelment Respondents who have opted in to a panel expect to participate Engaged panelists will complain if they are not surveyed each month 15
  • 16. Invitation Salience: Effect of Topic & Text on Response • Participate and earn a free gift card from Home Depot • Participate and win one of 3 cash prizes • Participate and $5 will be credited to the credit card of your choice • We’d like to know your thoughts about gasoline prices •This survey is super-short! (we promise) • Shorter than the typical survey! Takes only 10 minutes… • We only need to ask you about 10 questions • Your answers will remain completely confidential • We only need to ask you about 25 questions • We’d like to ask you questions about home improvement • We’d like to ask you questions about Space Tourism • Get a free copy of the results after you participate! • The results of the survey will be published in the New York Times! • Earn frequent flyer miles on the credit card of your choice Source: CMOR, “Creating a Respondent Cooperation Playbook” 16
  • 17. Recency The longer the lag between the event and the survey the lower the response rate. The response rate for one transactional survey jumped from 15% to 63% by sending invite within 15 minutes of event instead of 24 hours later. 17
  • 18. Exclusivity Emphasizing the exclusivity of the invitation will improve response rates. Example: "You are one of a small, select group of customers that we have invited to provide us feedback." Of course, don't say it if it isn't true! 18
  • 19. Incentives & Response “It has been repeatedly shown that providing financial incentives as a persuasive motivator for reluctant respondents is a viable and effective technique for increasing response rates…” - “Demographics and Online Survey Response Rates”, Kurt Knapton and Steve Myers “....Goetz, Tyler and Cook…concluded that financial incentives increased response rates and showed no differences in the demographics of the incentive and non-incentive groups.” “....Mason, Lesser and Dillman (2003) found that non- response error may actually be reduced when financial incentives are used...” 19
  • 20. Meta Analysis “Factors affecting response rates of the web survey: A systematic review” Computers in Human Behavior (March 2010) Weimiao Fan and Zheng Yan Synthesizes results of 300 studies into a model of the contributing factors of response rates Response rate = completes / invites 20
  • 21. Factors Affecting Response Rates to Web Surveys Content of Target Invitation Technology web Survey delivery audience designs issues questionnaires Nature of the Type of sponsors of the Sampling Personalization population survey methods Survey software What the topic is Scarcity Contact delivery Demographics modes How long the survey takes to Access control complete Internet transmission Personality Contact quality capabilities Question writing Use of incentives Source: Fan and Yan, 2010 21
  • 22. The Science of Email Marketing – Applied to Surveys Businesses are consumers. Try sending survey invitations on the weekends. Send invites very early in the morning. Use lots of links in your survey invites. Include reference information in your survey invitation, providing recipients other ways to provide feedback or seek assistance after the survey is closed. Subject lines should use the word survey and should not emphasize incentives. Emphasize in the invite that this is an exclusive opportunity to provide feedback (for those studies for which it is true, of course). Source: Dan Zarrella, “The Science of Email Marketing”, 2011 22
  • 23. Jumping the Hurdles to Response
  • 24. The Third Step: Craft Your Invitation Carefully On • For research to house lists, make sure Email List that lists are double opt-in Not on • Unsubscribe List For general market research, do not Not Flagged as buy lists Spam • Strongly consider using external panel Message Opened from a reputable provider Link Clicked 24
  • 25. The Third Step: Craft Your Invitation Carefully On • Maintain a suppression list of Email List addresses that have opted-out Not on Unsubscribe • Provide different categories of List Not Flagged as Spam opt-out (e.g., marketing email vs. newsletters vs. surveys) Message Opened • Remove suppressed addresses Link prior to emailing Clicked 25
  • 26. The Third Step: Craft Your Invitation Carefully On CAN-SPAM Compliance Email List • Truthful subject lines Not on Unsubscribe List • Valid “From” e-mail Not Flagged as Spam address • Message Physical address Opened • Valid opt-out link Link Clicked • Avoid SPAM filter trigger words (“Free, “$”, “act now”) 26
  • 27. The Third Step: Craft Your Invitation Carefully On • Personal email address Email List rather than corporate Not on mail box Unsubscribe List Not Flagged as Spam Message Opened Link • Compelling subject line Clicked o “Help us improve our products” o “Share your opinion” 27
  • 28. Clearing the Final Hurdle: Craft Your Invitation Carefully Less than 8 seconds to make a first impression On Tell recipients Email List what you want them to do as soon Not on as possible in the invitation Unsubscribe List Not Flagged as Know your audience and write the invitation with Spam this knowledge in mind Message Opened For transactional surveys, keep it timely Link Clicked For random samples, mentioning the exclusivity of the invite improves response rates Let them know how long the survey will be Consider using HTML e-mails vs. text 28
  • 29. Hurdles in Race to Turn Recipients into Respondents On A well-crafted invitation is Email List essential to jumping response Not on hurdles. Unsubscribe List Not Flagged as Spam Message Opened Link Clicked 29
  • 32. Clearing the Final Hurdle: Six Points to Cover Intro/Basic Appeal Your You've been selected for a research OpinionCounts! study! Importance of Your participation in this survey Your participation in this survey will Participation will help improve products and help make improvements to a services. government program. Survey Subject We'd like to know your thoughts We'd like to know your thoughts Matter about gasoline prices. about dairy products. Flexibility/ We only need to ask you about This survey will only take a few Time Burden 10 questions. moments of your time. Incentive Offered Participate and $5 will be Get a free copy of the results after for Participation credited to the credit card of you participate! Source: CMOR your choice Privacy/Data As member of the Better Your privacy is important to us, your Confidentiality Business Bureau we take your answers will be combined with privacy seriously and will others, and will never be linked with respect your confidentiality. you personally. 32
  • 33. Common Elements of Email Invitations Incentive amount Expected length Contact info Opt-out/Decline Subject included Privacy Policy Deadline Panel reminder 0% 20% 40% 60% 80% 100% Source: Vovici review of 30 ESOMAR 26 documents 33
  • 34. Invitations: To Disclose or Not to Disclose 100% “The invitation text does not Exclude 90% include…the topic of the Include 80% survey, as this may affect the 70% panelist’s willingness to 60% participate and thus create 50% bias.” - Norstat 40% “Research shows that people 30% may be more likely to 20% respond when the invitation 10% references a local geography, 0% Normalized. a topic of interest, or an appealing reward. However, care must be taken not to Source: Vovici review of 30 ESOMAR 26 documents bias the responses.” - SSI
  • 35. Effect of Topic & Text of Invitation on Response Rate • Participate and earn a free gift card from Home Depot • Participate and win one of 3 cash prizes • Participate and $5 will be credited to the credit card of your choice • We’d like to know your thoughts about gasoline prices •This survey is super-short! (we promise) • Shorter than the typical survey! Takes only 10 minutes… • We only need to ask you about 10 questions • Your answers will remain completely confidential • We only need to ask you about 25 questions • We’d like to ask you questions about home improvement • We’d like to ask you questions about Space Tourism • Get a free copy of the results after you participate! • The results of the survey will be published in the New York Times! • Earn frequent flyer miles on the credit card of your choice Source: CMOR, “Creating a Respondent Cooperation Playbook” 35
  • 36. Different Constituencies Respond Differently American College of Cardiology Case Study Surveys sent between 10 am and 2 pm Tuesday through Thursday Discovered by analyzing response patterns to past surveys “Strike at the best time, even if it means waiting.” Timing of invitations What works well for consumers may work poorly for businesses “Is Sunday the new Tuesday?” for B2B surveys A staggered series of reminders makes optimizing delivery time less important A/B test Adapt the split tests used by marketing departments to pit invitations against one another Rinse and repeat to find what works best 36
  • 38. Good Reports Require Upfront Planning Sampling’s Impact on Reporting Creating questionnaires that facilitate results. Construct Reports Easily consumable reports that accurately detail trends and highlight conclusions. Present & Share Reports & Data Efficiently format and share your detailed reports and raw data 38
  • 39. Qualitative/Quantitative Research • Random samples • Convenience samples • Closed questions Quantitative Research • Polls with comments • Limited response options • Online Communities • Can’t get the story behind • Focus Groups the story • Provide the Story behind • Provides hard data that Qualitative Research the story can be extrapolated to a • Can’t extrapolate to a larger audience larger audience 39
  • 40. Ramifications for Reporting Sometimes lots of numbers are really qualitative research, because you can’t extrapolate from them Forget sophisticated reporting techniques if research is based on a convenience sample But… you don’t need a probability sample to learn that a single customer is unhappy 40
  • 41. Structure the Questionnaire for Good Data Preload as much respondent demographic data as possible Use conditional logic and piping to keep respondents from answering questions that aren’t applicable to them Use validation to ensure data is provided in the correct format 41
  • 42. Structure Questions to Fit Analytic Models Set up parallel questions if you want gap analysis between importance and satisfaction Include questions to support the quadrant analyses and segmentations you will want to report When designing recurring surveys, develop multi- question indices to report on 42
  • 44. Use Test Data Before Fielding Populate the survey with random test data Generate reports on the mock data to verify your questions will give you the information that you are seeking 44
  • 45. Understand Your Audience Who, What, When Who will be looking at your data? What are their expectations? Format, delivery, depth. How often? 45
  • 46. Building your Report Use a top-down approach Start by understanding the major themes, then drill down. Use filtering and cross-tabulation to find hidden patterns and trends. Preloaded historical and demographic data will provide essential context. Do others reach the same conclusions when they work with the data? Let the results shape the report Question order should not determine reporting order Abandon preconceived notions/self-fulfilled ‘realizations’/dramatic over- generalizations However, understand, explain and develop conclusions that seem contradictory or illogical 46
  • 47. Logically Consolidate & Summarize • Keep it simple and assume nothing. • Be precise; remove platitudes; use examples and draw logical parallels • Include: an executive summary, your methodology, any special notes. Always cite all sources. • Call out significant findings. Use the “tell them, then tell them again” approach. • Avoid unintentional conclusions. Reconcile easily misunderstood or contradictory results. • Provide links to secondary research and articles that: • Provide perspective • Reinforce conclusions 47
  • 48. Sharing Your Insight Build Reports that are Scalable & Repeatable Researcher’s quandary: People always want more. Provide a method for other business users to work with reports Real-time self service & Raw data (filtered or in its entirety) Choose The Right Format & Distribution Method Self-service vs. email vs. print Word, Excel, PowerPoint, PDF, etc. Consider the Future The most cost effective research is that which you have already done Integrate data into future research 48
  • 49. Create Hierarchical Reports Hierarchical reports eliminate manual processes and offer personalized data per report user Case Study: Corporate Climate Survey, reviewed by 1000 individuals, with different views: Branch manager and assistant branch manager: branch results compared to district, region, nation District manager: each branch in district compared to district, region, nation Regional VP: each branch in district, each district in region, compared to nation Corporate executives: national results; drilldown to any level 49
  • 50. Five-Part Essay vs. Inverted Pyramid Introduction Conclusion Essential Narration Information Important Information Affirmation Supplemental Information Negation Data Tables Conclusion 50
  • 51. Reports & Presentations in Opposition Reading a Report Attending a Presentation Not scheduled in advance Scheduled No agenda Follows agenda Private Social No interaction with author Interaction with presenter Competing demands on reader No competing demands on attendee Recommends first Recommendations last Source: Lawrence Gibson, Eric Marder Associates 51
  • 54. Bar Charts are Easier to Interpret than Pie Charts Question 1 Question 2 Question 3 A B C D E A B C D E A B C D E
  • 55. Bar Charts Often Inconsistent % Completely % Completely Likely to Satisfied Repurchase 30% Product B Product B 35% 43% Product D Product D 37% 0% 20% 40% 60% 0% 20% 40% 60% % Completely Likely to % Completely Likely to Recommend Expand Purchases Product B Product B Product D Product D 0% 10% 20% 30% 10% 12% 14% 16%
  • 56. Summary Statistics Lose Information Mean = 3.0 5 5 5 4 4 4 3 3 3 2 2 2 1 1 1 0 0 0 Top Box = 20% 5 5 5 4 4 4 3 3 3 2 2 2 1 1 1 0 0 0
  • 57. Common Mistakes with Trend Reports Showing one year of data Grouping data by short time periods Using trend reports at all Including recent responses in trend reports 57
  • 58. Random Samples Converging As Sample Size Increases Series 1 Series 2 Series 3 10.0 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 1 10 109 208 19 28 37 46 55 64 73 82 91 100 127 136 145 154 163 172 181 190 199 217 226 235 244 253 262 271 280 289 298 118 58
  • 60. Use of Faces in Surveys A face of an interviewer or researcher will increase the accuracy of reporting Accuracy is maximized when the gender of the respondent and the displayed face match Male panelists are more likely to report accurately when the introduction is formal, rather than friendly Accuracy differed depending on the interaction of gender of the respondent, presence or absence of a face, gender of the face and style of introduction Source: NPD Group 60
  • 61. Continuum of Rich Media Offerings More like More like face-to-face paper interviews surveys Source: Gina K. Walejko 61
  • 62. Continuum of Rich Media Offerings Avatars with speech Faces with recordings Faces Surveys routed to human More like interviewers face-to- Automating traditional F2F face probes interviews Reintroduces social desirability bias 62
  • 63. Rich Media Can Increase Completion Rates Source: Deborah Sleep & Jon Puleston, Quirks – December 2008 63
  • 64. Element54 Research Data Consistency Respondent Usability 64
  • 65. Balanced 5 Cell Design: 2000 Total Interviews Radio Button Anchor @ „1‟ Anchor @ „5‟ Anchor @ ‟10‟ Experimental 65
  • 66. Anchoring of Slider Position Matters Source: Element54 66
  • 67. Inference vs. Instruction 42% did not read the question 33% did not read the "traditional" question Source: Element54 67
  • 68. Equal Ratings for ‘Engagement’ Metrics 1.0 2.0 Survey Enjoyment 8.20 8.25 Ease of Use 8.86 8.85 Desire to Repeat 7.02 6.56 Source: Element54 68
  • 69. Tips for Migrating to Rich Media Discretion is the better part of valor Split ballot test rich media questions against traditional questions to identify differences Visual choices introduce bias, especially for more abstract choices Use rich media in a considered manner 69
  • 70. Q&A
  • 72. Benefits of Multi-Language Surveys Increase response rates Improve data quality Language-specific surveys Use country as a basis for reports and conditional branching Highly efficient means of conducting a single survey among a global customer base Allows you to consistently communicate with and gain feedback from global customers 72 72
  • 73. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design 73 73
  • 74. Study Design What languages do you need to translate to? What internal and external resources will you use for translation? Once the results come in, how will you translate open-ended comments? 74 74
  • 75. Remember: Countries are Not Monoliths Most countries have multiple languages There is no Belgian language There is no Chinese language Even when two countries share a common language, you may need to budget for “localization” 75 7
  • 76. Biggest Mistake in Multi-Language Studies: Unrealistic Schedules • Prepare a conservative schedule Study Design • Time-consuming translation phase after the master Survey Design questionnaire is complete • International holiday schedules can extend the time the Fielding survey is in the field • Translation of open-ended comments adds weeks to the Analysis analysis phase 76 76
  • 77. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design 77 77
  • 78. Write Master Questionnaire with Translation in Mind Using your typical questionnaire would Increase translation costs Reduce representativeness of results Instead, structure for translation Avoid jargon, slang and technical terms Rewrite for readability Use as few open-ended questions as possible Use language-specific skip patterns 78 78
  • 79. Some Questions Need Alternates, Not Translations Think localization, not just translation Country-specific questions are often called-for No sense translating the choices of the question, “Which hotel did you choose?” Choice lists may need to be tailored for major brands in different markets Entire sections of the questionnaire may differ for structure of industry in that country Graphics and icons may be inappropriate in some countries or for some cultures 79 79
  • 80. Translation vs. Localization English United States Spanish French English English Spanish Master Canada Questionnaire French Japanese Mexico Spanish Japan Japanese 80
  • 81. Areas of Special Interest Weights and measures Currency Time – 24 hour clock, GMT, Zulu, UTC, WET Use images without text to best use in many cultures May need to reduce questionnaire length to take into account overall budget for translations 81 81
  • 82. Generational Considerations Use Age Groupings rather than Generational Groupings for reporting across cultures Exception: Generational Groupings may be used for the US and Canada What is Generation X? Group born between 1964 and 1975 (US) Younger Westernized generation (India) Generation Golf (Germany) Génération Bof (France) Generation Niks (Netherlands) 82 82
  • 83. Scales Matter Some scales don’t translate well They become too wordy, too subjective Some scales exaggerate cultural issues Likert scales can be particularly difficult Safe choices ConstantSum MaxDiff 83 83
  • 84. Answers Patterns for Common Response Styles Completely Somewhat Neither agree Somewhat Completely Disagree Agree Response Style disagree disagree nor disagree agree agree Optimal Responding Extreme Response Style (ERS) Truncated Scales Response Range (RR) Mild Response Style Midpoint Response (MPR) Acquiescence Social/Anti-Social Styles Response Style (ARS) Disacquiescence Response Style (DARS) Socially Desirable Responding (SDR) Noncontingent Responding (NCR) 84
  • 85. Response Styles by Country: Informed by Culture Source: Johnson, Kulesa, Cho, Shavitt, 2003; Vovici 85
  • 86. Constant Sum Example “Please distribute 100 points amongst the Item Points following 5 items, based Optical zoom of 3X or 30 greater on their importance to you Resolution: 10 40 when buying digital Megapixels or greater Available in 5 cameras. An item that has multiplecolors no importance to you Battery Life 25 should be given no points. Weight 0 100 The total must equal 100 points.” 86
  • 88. Important Safety Tips for Multi-Language Surveys Translation is the last part of the survey cycle Surveys must be finalized before having them translated to avoid confusion and re-work Do not trust machine translations! 88 88
  • 89. A Survey Translation is Not Just a Translation of the Survey Invitation and reminder emails Introductions Button captions Validation messages End pages Thank you messages Alerts 89 89
  • 90. Choose Your Translator Well The best translators have excellent understanding of the originating language, the target culture, the industry and survey research Too often, translation is “insourced” to employees who do not have the requisite experience Survey translations are more demanding than traditional translations Use only native speaking professional translators (preferably certified translators) 90 90
  • 91. Best Practices for Working with Translators Brief the translators on the intent of the survey Create a glossary of English terms and the target language for consistency Use native speakers to review the translation Kathleen Bostick and provide feedback VP Global Marketing Kathleen.bostick@lionbridge.com @Lion_MktgVEEP Pilot test of translation of survey may be http://blog.lionbridge.com/ required and will lengthen the process www.lionbridge.com A backtranslation may be recommended Source: Lionbridge Choose a tool that saves the question and Technologies translations. 91 91
  • 92. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design 92 92
  • 93. Deadlines and Timing Timing can affect survey response rates Be knowledgeable of the holidays and cultural norms of your survey populations Avoid survey deadlines on holidays or weekends (in some cultures Friday/Saturday constitute the weekend, not Saturday/Sunday) Be mindful of the International Date Line and individual times zones within countries Be careful of time indicators – many cultures use a 24-hour clock 93 93
  • 94. Respondent Language Selection • Translated email invitations with links toappropriate translation • Or self-selection based on all available languages: 94 94
  • 95. Respondent Language Selection Auto-detect based on primary language set in browser Conditional logic, alerts and triggers based on language-specific surveys Respondents’ browser settings may not match their native language 95 9
  • 96. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design 96 96
  • 97. Analysis Considerations Some countries skew towards “positive” responses (Latin America, Japan) Others skew negative (Germany) Take special care with Product Concept Testing and Brand Perception Surveys Always analyze data by country, not just by region Even if you plan to aggregate by region for reporting purposes Within some countries, important regional differences are also worth investigating 97 97
  • 98. Optimal Data Storage for Survey Results Base Survey English 98
  • 99. Optimal Data Storage for Survey Results French French Italian Italian German Base German Survey Spanish Spanish English Chinese Chinese Japanese Japanese 99
  • 100. Optimal Data Storage for Survey Results French French Italian Italian German Base German Response Survey Database* Spanish Spanish English Chinese Chinese Japanese Japanese *Safe Harbor Certified 100
  • 101. Optimal Data Storage for Survey Results French French Data Export Italian Italian German Base German Response Survey Database Spanish Spanish English Chinese Chinese Japanese Japanese 101
  • 102. Optimal Data Storage for Survey Results French French Data Export Italian Italian German Base German Response Survey Database Spanish Spanish View English Chinese Report Chinese Japanese Japanese 102
  • 103. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design 103 103
  • 104. What is the Role of the Translator Seat? User role that provides access to translate specific surveys into specific languages Economically allows translators to provide value Provides security by limiting what the translator can see and do Option to export surveys as XLIFF file, an XML version of the survey which translation houses can rapidly use to translate a single survey into a variety of languages 104
  • 105. Bad Translations: More Than Just Bad Data Significant schedule slips Due to re-fielding, Significant budget overruns new translation Non-blind surveys: Your brand loses credibility 105 105
  • 106. Thank You For Your Time Feel free to email questions: Jeffrey Henning, PRC VP of Strategy & Founder Vovici jhenning@vovici.com http://blog.vovici.com 106

Notes de l'éditeur

  1. http://blog.vovici.com/blog/bid/20582/The-Seven-Steps-of-Highly-Successful-Surveys
  2. http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Ratehttp://blog.vovici.com/blog/bid/24732/Optimize-rather-than-Maximize-Response-Rates
  3. http://blog.vovici.com/blog/bid/26604/Response-Rates-Driven-by-16-Major-Factors
  4. http://blog.vovici.com/blog/bid/26604/Response-Rates-Driven-by-16-Major-Factors
  5. http://blog.vovici.com/blog/bid/53107/The-Science-of-Email-Marketing-Applied-to-Surveys
  6. http://blog.vovici.com/blog/bid/22595/E-mail-List-Rental-Guidelineshttp://blog.vovici.com/blog/bid/25131/Panel-Rental-Guidelineshttp://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers
  7. http://blog.vovici.com/blog/bid/24309/Unsubscription-Survey-Results-Best-Practices
  8. http://blog.vovici.com/Blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam
  9. http://blog.vovici.com/Blog/bid/18205/Getting-Your-Survey-Invitation-Opened
  10. http://blog.vovici.com/blog/bid/18192/Compelling-Survey-Invitations
  11. http://blog.vovici.com/blog/bid/18258/Hurdles-in-Race-to-Turn-Recipients-into-Respondents
  12. http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover
  13. http://blog.vovici.com/blog/bid/54538/You-re-the-Top-You-re-the-Coliseum-Top-Box-Scores-vs-Means
  14. http://blog.vovici.com/blog/bid/56330/Real-Time-Reports-Not-Always-Ready-for-Prime-Time
  15. It's the desire to better understand the factors which can drive our confidence in the data - which has driven 2 large scale R&D initiatives in 2009.The initial study - "Sexy Questions, Dangerous Results" was presented at the Jan 09 NetGain 3 conference in Toronto. I am extremely grateful to the ResearchNow team, led by John Visser and Rasheeda Qureshi for their partnership on that study. We learned that by moving from a radio button, to a slider or drop down format - we would see significant differences in how often people say they brush their teeth ... how important issues are to Canadians ... and when they last bought products.
  16. what level of inference would we be OK with on the "are you ..." question?
  17. In our Quali/Quant design of 50 interview per interface - respondents rated both experiences similarly across enjoyment, ease of use, and desire to repeat.There are questions which need to be answered - like what will happen over time as survey takers experience the evolution of panels, and overall web experiences.
  18. Generation Bof (Generation Whatever)Generation Niks (Generation Nothing)Baby BoomersUS – between 1946 and 1963Japan – between 1947 and 1949Russia – called Sputnik Generation because they grew up under communism, reachied middle age during the Gorbachev Revolution, and have lived through the transition to a Russian-style market economy.When working in an international environment, you should try and understand the research best practices of the regions you’re working with, when the local agencies usually don’t report generations neither should you probably. And take some history lessons: Understanding a coungry’s past and development phases will help you interpret respondent’s reactions better. Finally, nothing beats talking with locals: speak, whenever possible, to people from the countries you research to understand the nuances of their culture, tradition, and politics.
  19. AttributesA competent translator has the following qualities:familiarity with the subject matter of the text being translated;a very good knowledge of the language, written and spoken, from which he is translating (the source language);an excellent command of the language into which he is translating (the target language);a profound understanding of the etymological and idiomatic correlates between the two languages; anda finely tuned sense of when to metaphrase ("translate literally") and when to paraphrase, so as to assure true rather than spurious equivalents between the source- and target-language texts.[30][edit] MisconceptionsA common misconception is that anyone who can speak a second language will make a good translator. In the translation community, it is generally accepted that the best translations are produced by persons who are translating into their own native languages,[31] as it is rare for someone who has learned a second language to have total fluency in that language. A good translator understands the source language well, has specific experience in the subject matter of the text, and is a good writer in the target language. Moreover, he is not only bilingual but bicultural.
  20. Identify words you don’t want translatedBest to use native speakers in country since usage changesBack translation is especially important in medical or psychometric researchValidation = pilot test of the surveyMay need local Subject Matter Experts to reviewGallup does backtranslations of all their surveys: Gallup does this for all its surveys, having a different person, using a native English speaker to do itMore on backtranslation:
  21. Understand the time differences that may separate various language populations
  22. Skewing toward positive and negative - This is especially critical to plan for in Product Concept Testing and Brand Perception Surveys