Speech assistance in online surveys #GOR2018

Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 1
General Online Research 2018 - Cologne, 1st March 2018
Track A2: Innovations for Online Surveys
Holger Lütters – Hochschule für Technik und Wirtschaft Berlin
Malte Friedrich-Freksa - GapFish GmbH
Marc Egger - insius u.G.
Effects of Speech Assistance in
Online Questionnaires

Revolutionary “beyond literacy survey approach” of Ian Thomson
is the inspiring framework of the presentation
Georges Perec Ian Thomson

Digital Speech Assistance:
A new technology available for market research
1

2013: Text2Speech - Avatars in Interviews
2018: Speech2Text - Talking instead of typing
Question text read
out loud by an avatar
Research 2013
Text2Speech
Avoidance of READING
Research 2018
Speech2Text
Avoidance of TYPING
Lütters, Holger /Maree, Tania / Wiese, Melanie (2014): Selecting an interviewer: Considerations for Avatar use in online market research.
In: Proceedings of the 26th annual SAIMS Conference, S. 264-272, The Southern African Institute for Management Scientists (SAIMS),

Digital Assistants conquer the lives from different entry points

Idea:
Integration of Speech2Text technology in online survey process
During an
online survey
mobile or PC
one or several
special
Speech2Text
question are
implemented
Prerequisites:
● Modern
browser
● Microphone
● Browser
allowance
to use the
microphone
Respondent is talking
into the device`s
microphone during
the survey question
Spoken words
are transcripted
into written text
Bonus technology option:
Text is sent to a sentiment
API during the interview
to analyze the tonality of
the spoken content in an
ongoing interview.
Different interview routes
possible

Different Speech2Text options are available in SaaS tool questfox
Background Speech2Text*
without showing results of the
transcript to the respondent
Talk2Me*
Live Speech2Text where transcript is
shown to the respondent
*Both questfox question types can save the audio answer in addition to the transcript of the text. www.questfox.com

• More open answers
• More details in the open answers
• Lower barriers for people with
lower willingness to express
themselves in a written form
• Open answers with more content
• More personal expressions in a less
controlled setting
• More System1 than System2
answers
Digital Speech assistance could improve the interview interaction
• Technical drop-out
• Device drop-out
• Tech-savvy sampling only
• Mental reservation against new
forms of interaction
• German speech2text capabilities
not yet developed on a usable
level (fear of being misunderstood
with a dialect)
• Legal issues of dealing with audio
information
Research chances of the idea Risks of the approach

Study set-up and field report
Empirical study comparing classic text input with speech assistance
input in a survey approach
2

HTW Berlin project with the technical support from several partners
Concept and programming of the study Holger Lütters
Ian Thomson
Software sponsor questfox Software as-a-Service
with Speech2Text functionality
Dima Feller
Tarik Hennings
Sampling sponsor with n>600 cases Malte Friedrich-Freksa
Analytics sponsor with InMap specialized tool for text
analytics and API development
Marc Egger
André Lange

Browser approach avoiding an App development
is the opposite of device agnostic research
https://caniuse.com/#search=MediaRecorder

2036 contacts needed to complete n=622 interviews with an online
representative panel
• 13 Minutes questionnaire
• More Screen-outs than Drop-outs
• August 2017 in Germany
• 2036 contacts with EntscheiderClub by GapFish
• 622 Completes
n=127
n=181
n=37
n=277

Three different routes of interaction in the overall setup
Route A Voice Only
n=152
Route B Voice and Text
n=151
Route C Text Only
(control group) n=322
Answer
Mode

Three routes with different question and answer modes integrated into a
standard questionnaire
I. Refugee situation and chancellor Merkel
Bitte kommentieren Sie die folgende Aussage:
“Während des Höhepunkts der Flüchtlingskrise im
Jahr 2015 sagte Angela Merkel bezüglich dieses
Themas den Satz "Wir schaffen das".“
II. Gay Marriage
“Was ist Ihre Einstellung gegenüber der
kürzlichen Legalisierung der "Homo-Ehe" in
Deutschland?“
III. Women in the workplace
Bitte kommentieren Sie die folgende Aussage:
“Es wird genug für die Gleichberechtigung der
Frau am Arbeitsplatz getan.”
Three “delicate” questions to provoke
open answers (NO randomization)
Questionnaire example
Example Link

The automated transcript largely depends on the technical setup
nein wir schaffen das nicht das ist
hochverrat am deutschen volk und
dann muss national auf nationaler
ebene dafür gesorgt werden das
erst die menschen der
heimatländer in nicht mehr in
armut leben sondern gut leben das
ist was wichtiges wir schaffen das
nicht wir werden kaputt gehen das
finanzsystem das wird zerbrechen
und alles kaputt gehen hier
Automated transcriptExamples extended answers
Example 1
Question 1: Chancellor Merkel; Open answer recorded

A new phenomen of open answers pops-up: spoken bullshit answers
• lalalalala
• käsekuchen
• ja was ist du arschloch
• elfriede manchester
Spoken „asdf“ type answers
Question 2: Gay marriage; Open answer recorded an transcripted

Results of the comparison between typed-in and spoken answers
3

Voice technology comes at higher cost of recruiting
Route A Voice Only Route B Voice and Text Route C Text Only

Average Word Count per Response shows fatigue when typing more
than one open answer

Less response time for spoken answers with longer input
with immediate fatigue/satisficing after the first task

Estimated time in comparison to real time is in favor for speech.
Speech feels less time consuming than it actually is.
Self-estimated time as
indicated by respondent
Real time measured
Route A (n=152)
Real time measured
Route B (n=151)
Real time measured
Route C (n=106)
Under 10 mins 14.65 13.88 10.33
From 10 to under
15 mins
15.93 15.73 12.36
From 15 to under
20 mins
19.67 20.06 13.88
Above 20 mins 24.69 26.76 21.64

Check for concepts shows more detailed answerswith speech
Topic I Chancellor Merkel and the refugee situation
Analysis with insius InMap http://insius.com/de/produkte/inmap/

People express more in a speaking mode compared to writing
Topic II Gay marriage

The burden of answering is lowered through the microphone usage
Topic III Women in the workplace

The concepts can be compared in terms of content.
Overall result: More concepts occur in the speech version

Evaluation of the question type Speech2Text
4

Summary
Speech2Text opens a new door in the world of interview research
• Speech as an answer type is a thrilling new way of getting more open
ended answers … but not necessarily to the respondent :-(
• People leave more answers and more detailed concepts in less time
at the cost of 50% technical drop-out
• The technical issues will be sorted out. In a panel approach this
negative effect can be controlled. In river sampling approaches this
remains an issue with strong effects
• It will take some time before it will become a standard tool, but it will
definitely change the research landscape for more qualitative input
from less educated respondents
• Further automation in the field of voice analytics will again change the
rules of the game

Thanks to the supporters of the study
Authors
Prof. Dr. Holger Lütters – HTW Berlin
Malte Friedrich-Freksa - GapFish GmbH
Dr. Marc Egger - insius u.G.
Contact
Prof. Dr. Holger Lütters
Hochschule für Technik und Wirtschaft Berlin
Holger.Luetters@htw-berlin.de
www.luetters.com
Thanks to Ian Thomson for his support
Thanks to
questionnaire
www.gapfish.com
www.insius.com
with www.questfox.com
questfox® is a development of pangea labs
www.pangealabs.com

Speech assistance in online surveys #GOR2018

Recommandé

Recommandé

Contenu connexe

Similaire à Speech assistance in online surveys #GOR2018

Similaire à Speech assistance in online surveys #GOR2018 (20)

Plus de Holger Lütters

Plus de Holger Lütters (11)

Dernier

Dernier (20)

Speech assistance in online surveys #GOR2018