A Subjunctive Exploratory Search Interface to Support Media Studies Researchers

A SUBJUNCTIVE EXPLORATORY SEARCH INTERFACE
TO SUPPORT MEDIA STUDIES RESEARCHERS

M. Bron1, J. van Gorp2, F. Nack1, M. de Rijke1, A. Vishneuski1, S. de Leeuw2
(15-08-2012)

(1) University of Amsterdam (2) Utrecht University
vrijdag 10 augustus 2012

INTRODUCTION TO

MEDIA STUDIES RESEARCH

deﬁnition: media studies concerns the study of production,
content and/or reception of various types of media, e.g.,
newspapers, ﬁlms, and television programs


INTRODUCTION TO


definition: media studies concerns the study of production,
content and/or reception of various types of media, e.g.,
newspapers, films, and television programs

example: how does the representation of women smokers
change in magazines and fiction shows between 1900 and
1950?


INTRODUCTION TO



ANALYSIS OF THE

RESEARCH CYCLE

how can we support media studies researchers in gathering
material in electronic archives?

methodology: interviews; 30 mins; topic list
pick a recent research project?
where did you get the idea?
how did you search for data?
did you encounter any problems?
did your research questions change?
can you describe the various stages during the project?
can you write them down on index cards and order them
chronologically?


ANALYSIS OF THE

RESEARCH CYCLE

explore literature data
analysis writing report
domain study collection


ANALYSIS OF THE

RESEARCH CYCLE

exploration

literature
study
explore
domain
data
collection

developing the research question
determining available material
literature
broad gathering of material


ANALYSIS OF THE

RESEARCH CYCLE

exploration contextualization

literature
study
literature
study
explore
analysis
domain
data
collection
data
collection

analyzing material
refining the research question
finding context
finding additional material


ANALYSIS OF THE

RESEARCH CYCLE

exploration contextualization presentation

literature
study
literature
study
explore
domain
data
collection
data
collection


ANALYSIS OF THE

RESEARCH CYCLE

exploration contextualization presentation

literature
study
literature
study
explore
domain
data
collection
data
collection

analysis of data, e.g., watching and transcribing ﬁlms costs a
lot of time, money, and effort.


DEVELOPMENT OF AN

INTERFACE

our example research question:
how does the representation of women smokers change in
magazines and fiction shows between 1900 and 1950?

requirements of the material:
time and genre constraints, e.g., fiction shows
a specific group, e.g., women smokers
is it possible to contrast findings in the data, e.g., a change


DEVELOPMENT OF AN

INTERFACE

our example research question:
how does the representation of women smokers change in
magazines and fiction shows between 1900 and 1950?

requirements of the material:
time and genre constraints, e.g., fiction shows
a specific group, e.g., women smokers
is it possible to contrast findings in the data, e.g., a change

Major problem: whether the material satisfies all these criteria is
only known after everything has been analyzed!


DEVELOPMENT OF AN

INTERFACE

key question:
does the option to compare statistics of multiple results sets
support media studies researchers in ﬁnding material that
satisﬁes the contrast requirement in their research questions?


EXPERIMENTAL SETUP

EXPERIMENTAL DESIGN

Total of 36 media studies researchers and MSc. students
(experience MD = 3; IQR = 0-6.5); between subject design; remote
user study; data is 1.2 M tv programs with metadata from audio
visual archive.

pre search task survey
a complex search task (30 mins):
Collect material relevant to develop and answer a research question
on the topic of how minorities are described in the Dutch media over
the last 40 years. After collection, formulate a research question.
post search task: write down the research question developed
during searching
exit survey


EXPERIMENTAL SETUP

EVALUATION

subjunctive interface interaction statistics:
more query formulations
more documents bookmarked
wider variety of documents bookmarked (average pairwise similarity)

research questions formulated with the subjunctive interface:
a more reﬁned scope

usability of the interface
usefulness for research; interesting
difﬁcult to use; intuitive


RESULTS

INTERACTION STATISTICS

showing median and (inter quartile ranges)
for the query formulations
for the number of documents bookmarked
interaction baseline subjunctive
queries 3 (2-6) 5 (3-8)*
bookmarks 9 (6-22) 9 (3-13)

for wider variety of documents bookmarked indicated by the
average pairwise cosine similarity
bookmark variety .62 (.56-.69) .52 (.43-.63)*
* signiﬁcant at the alpha = 0.05 level using Wilcoxon rank sum test


ANALYSIS

INTERACTION PATTERNS

Query -> Inspect -> Query -> Inspect
is more often seen with the subjunctive interface
Query -> Inspect -> Filter -> Inspect
is seen more often with the standard exploratory search interface

Bookmark -> Paginate -> Bookmark -> Paginate
is a frequent pattern when using the baseline interface;
pagination does not occur in the top interaction patterns with
the subjunctive interface


ANALYSIS

INTERACTION PATTERNS

Query -> Inspect -> Query -> Inspect
is more often seen with the subjunctive interface
Query -> Inspect -> Filter -> Inspect
is seen more often with the standard exploratory search interface

Bookmark -> Paginate -> Bookmark -> Paginate
is a frequent pattern when using the baseline interface;
pagination does not occur in the top interaction patterns with
the subjunctive interface

comparison visualizations invite more queries
bookmarking form two result lists i.s.o. single long list
promotes more diverse documents


RESULTS

RESEARCH QUESTIONS

Do the research questions formulated with the subjunctive
interface have a more refined scope determined by a contrast or
comparison?
three judges; experts in the field of media and migration
assessed the research questions

showing median and (inter quartile ranges)/ 5 point Likert scale 1 is very bad; 5 is very good

judge 1 judge 2 judge 3
baseline subjunctive baseline subjunctive baseline subjunctive
3 (3-4) 4 (3-4) 2 (2-4) 2 (2-3) 4 (3-4) 4 (2-4)
It is hard too judge whether the scope of research questions
is more or less refined
what if we look at the research questions?


ANALYSIS

RESEARCH QUESTIONS
analysis of two research questions...
Investigation of the evolution of the term integration in news and
human interest programs broadcasted between 1992 and 2012
In 1987 we observe a diminishing in the use of the term migrant
worker and a rise in the use of the term immigrant. Is it possible
to identify a cause in the broadcasting schedule of that time?

interface genre group topics period technical
baseline 11 14 8 10 2
subjunctive 10 17 12 9 3


ANALYSIS

RESEARCH QUESTIONS
analysis of two research questions...
Investigation of the evolution of the term integration in news and
human interest programs broadcasted between 1992 and 2012
In 1987 we observe a diminishing in the use of the term migrant
worker and a rise in the use of the term immigrant. Is it possible
to identify a cause in the broadcasting schedule of that time?

interface genre group topics period technical
baseline 11 14 8 10 2
subjunctive 10 17 12 9 3

In research questions formulated with the subjunctive interface
the contrast is more concrete.


RESULTS

USABILITY
usability of the interfaces

median and (inter quartile ranges)/ 5 point Likert scale 1 is absolutely not; 5 is very much

intuitive 4 (4-4) 4 (3-4)
interesting 4 (4-5) 4 (4-5)
useful 4 (4-5) 5 (4-5)
difﬁcult to use 2 (2-2) 2 (2-4)

both interfaces are found to be (very) useful for media studies
research
surprisingly: the subjunctive interface is not considered to be
difﬁcult to use


CONCLUSIONS

the media studies research process is a dynamic and iterative
process, in which it is difﬁcult to make research questions
concrete until material has been collected and analyzed

a subjunctive exploratory interface supports media researchers in
exploration: it invites more query formulations and more
diverse documents to be gathered
formulating research questions in which a contrast is
considered based on patterns in the material observed during
exploration
moreover
the subjunctive interface, while more complex, is not
considered to be difﬁcult to use and is considered very useful
for media studies research


CONCLUSIONS

the media studies research process is a dynamic and iterative process, in
which it is difﬁcult to make research questions concrete until material has
been collected and analyzed

a subjunctive exploratory interface supports media researchers in
exploration: it invites more query formulations and more diverse
documents to be gathered
formulating research questions in which a contrast is considered based
on patterns in the material observed during exploration
moreover
the subjunctive interface, while more complex, is not considered to be
difﬁcult to use and is considered very useful for media studies research

Thank you; are there any questions?


A Subjunctive Exploratory Search Interface to Support Media Studies Researchers

Recommandé

Recommandé

Contenu connexe

Plus de Sound and Vision R&D

Plus de Sound and Vision R&D (20)

Dernier

Dernier (20)

A Subjunctive Exploratory Search Interface to Support Media Studies Researchers