Understanding film scholars' annotation behavior: Supporting scholarship by enhancing online film annotations Liliana Melgar Estrada

1. Understanding film
scholars’ annotation
behavior:
Supporting scholarship
by enhancing online film
annotations
Liliana Melgar-Estrada
PhD candidate in Information Sciences
Universidad Carlos III de Madrid
lmelgar@bib.uc3m.es
Digital Humanities Benelux Conference
June 12, 2014
1. Motivation and Research
framework
i. Context, motivation
ii. PhD thesis overview
iii. Background
research:
Perspectives in
(moving) image
indexing
2. Conceptual framework
i. Crowdsourcing
ii. Nichesourcing
3. Case study
i. Research questions
ii. Participants
iii. Method
iv. Findings and
discussion
4. Conclusions

2. PhD thesis in Information Science(s)
Information behavior studies
2
Wilson's 1999 nested model of information behaviour
“How and why can current indexing perspectives be used by film
archives to support film scholarship. ”

3. Context: Digital tools in film studies
3
Videana
Visualizing Vertov
(Ross et al., 2009)

4. Indexing and annotating (moving) images
Moving image indexing: time-based
4
© 1
Text indexing:
based on words and characters
© 2

5. 5
Moving image indexing perspectives
1. Concept-based retrieval
The second order of orders
(Weinberger, 2008)
© 3

6. • Analysis of the content instead of
the metadata
• Video segmentation
• Shot/sequence detection
• Key-frames extraction
• Low-level features such as object
motion, color, shape, texture,
loudness, power spectrum,
bandwidth, and pitch
• High-level features (e.g.,“finding
a picture of water” or “searching
for Mona Lisa Smile”).
• Automatic cast identification
“When combined, we find that content-based video retrieval incorporated into the
archive’s practice results in significant performance increases for shot retrieval and
for retrieving entire television programs. The time has come for audiovisual archives
to start accommodating content-based video retrieval methods into their daily
practice.” (Huurnink et al., 2012)
6
Moving image indexing perspectives
2. Content-based indexing (CBIR)
© 4

7. Crowdsourcing
• Spontaneus “indexing”
• Users as indexers
• Different backgrounds and
knowledge
• Lack of domain knowledge of
the user
• Lack of control
7
Nichesourcing
(De Boer et al, 2012; Accurator
project)
• Experts as annotators
Moving image indexing perspectives
3. Crowdsourcing and Nichesourcing
© 5
© 6

8. Time-based crowdsourced annotations
8Geisler, 2012
Cinecast. La ligne du temp, BNF

9. Study: Understanding film scholars’ annotation behavior
• RQ: How do film scholars behave as moving image
indexers?
• Theoretical framework: Information behavior
– cognitive theory for interactive information retrieval (IIR) put
forward by Ingwersen (1992, 1996, 2001)
• Method: Pia Borlund’s simulated work tasks
All participants got 2 different film clips (1 control clip), 1 short movie
and selected 1 movie themselves for each task in randomized order (in
total 2 clips and 2 movies)
1. Context independent and system-free task
2. Context dependent and system-free task
3. Context and system dependent
4. System dependent, context not clear
9

10. Participants
• Participants: 10 film scholars from 5 universities
in Madrid:
– Autónoma de Madrid (Art History dept.)
– Camilo José Cela University (Communication Sciences
dept.)
– Carlos III (Audiovisual Communication dept.)
– Complutense (Audiovisual Communication dept.)
– Rey Juan Carlos (Communication Sciences dept.)
• Film scholars with 10 or more years of experience in
publishing, teaching, editorial activities, festival
commitees, etc. 10

11. Finding I. Annotation styles
1. Descriptive texts (Para-texts) *
– Narrative structure + critical texts
– Interpretative clues for the reader
2. Combination of Descriptive/Critical texts and Keywords
3. Keywords / Tags / Descriptors
4. Plot synopsis
*(Gérard Genette, after commented by Nico de Klerk and studied in a recent book
edited by Pellatt, 2013)
11

12. Finding II. Unit of analysis and indexing
granularity
1. What do I have to describe? Clips vs entire movies
2. Annotation for access vs annotation for scholars’ own
research –Describing and searching tasks differ (Fidel,
1997)
12
RDA: Rules for Archival Description (Turner, 2009)

13. Finding III. Attributes
Sound, music
Narrative, story, plot, characters
Genre, movement, country
Influences, relations
Cinematographic, formal, stylistic features
CONTEXT, historical information
13

14. Finding IV. Willingness to participate
• 9 of the 10 participants said “Yes” to a proposal to
contribute to Nichesourcing initiatives by a film archive,
1 said “No”
• 1 of the scholars saying yes, hesitated of the success of
such idea: “Access is not about indexing or ordering, but
about selection and education”
• Scholars suggested: clarity, easiness, reward (not
necessary economic, but in the form of input and
feedback on the use of their voluntary work)
• The scholar who said “No” was very much aware of the
need of Indexing knowledge for any annotation task
14

15. Recommendations for Nichesourcing
• Tags or keywords alone are not enough for supporting
Film scholarship, different annotation styles should be
supported and “mined”
• Indexing knowledge should not be required from
domain experts. Any annotation system for scholars
has to clearly guide the indexing process
• The purpose and use of the annotations has to be
stated clearly
• Scholars should be provided with films they are familiar
with or are interested in (this is already done in
projects such as Naturalia/Accurator).
15

16. A new indexing perspective: Annotating in
Living labs
16
Media Ecology Project
LARM Audio Research Archive
© 7
© 8

17. Recommendations for Film archives
• Request scholars’ participation in
annotation activities
• Ask researchers to contribute with
their notes during research visits
• Provide digital tools for research,
both in situ and online
• Create events around annotation
practices (i.e. “Bird watching at
the museum”)
• Create online communities of
researchers (Martorell, Canet, Codina,
2014) 17
© 9

18. REFERENCES
De Boer, V., Hildebrand, M., Aroyo, L., De Leenheer, P., Dijkshoorn, C.,
Tesfa, B., & Schreiber, G. (2012). Nichesourcing: Harnessing the Power of
Crowds of Experts. Presented at the EKAW, Ireland. Retrieved from
http://ekaw2012.ekaw.org/node/101
Fidel, R. (1997). The image retrieval task: implications for the design and
evaluation of image databases. The New Review of Hypermedia and
Multimedia, 3. Retrieved from
http://www.ischool.washington.edu/fidelr/RayaPubs/ImageRetrieval%2
0Task.pdf
Geisler, G., Willard, G. & Ovalle, C. (2011). “A crowdsourcing framework
for the production and use of film and television data.” New Review of
Hypermedia and Multimedia, 17(1), 73-97.
Huurnink, B., Snoek, C. G. M., de Rijke, M., & Smeulders, A. W. M.
(2012). Content-Based Analysis Improves Audiovisual Archive Retrieval.
IEEE Transactions on Multimedia, 14(4), 1166–1178.
doi:10.1109/TMM.2012.2193561
Martorell, S., Canet, F., & Codina, L. (2014). Channelling academic
audiences: proposal for a social network for film studies researchers.
Hypertext.net, (12). doi:10.2436/20.8050.01.12 18

19. Patel, B. V., & Meshram, B. B. (2012). Content based video retrieval
systems. International Journal of UbiComp, 3(2), 13–30.
doi:10.5121/iju.2012.3202
Pellatt, V. (Ed.). (2013). Text, Extratext, Metatext and Paratext in
Translation - E-book | Krisostomus. Newcastle upon Tyne, UK:
Cambridge Scholars Publishing. Retrieved from
http://www.cambridgescholars.com/download/sample/60628
Ross, M., Grauer, M., & Freisleben, B. (2009). Digital Tools in Media
Studies: Analysis and Research : an Overview. Bielefeld: transcript
Verlag.
Turner, J. M. (2009). Moving Image Indexing. In Encyclopedia of
Library and Information Sciences, Third Edition. New York: Taylor &
Francis. Retrieved from
http://www.tandfonline.com/doi/abs/10.1081/E-ELIS3-120043268
Weinberger, D. (2008). Everything Is Miscellaneous: The Power of the
New Digital Disorder. New York: Henry Holt. 19
REFERENCES

20. Acknowledgements
This study was possible due to the participation
of 10 enthousiastic film scholars in Madrid,
specially to Marina Díaz López (Cervantes
Institute) who provided information and helped
in contacting most of them.
20

21. Image credits
1. http://social.technet.microsoft.com
2. Buster Keaton, Sherlok Jr. http://www.cinema.indiana.edu
3. Laaf, Meike. Internet Visionary Paul Otlet: Networked Knowledge, Decades
Before Google. http://www.spiegel.de
4. The Image Processing and Analysis Laboratory.
http://imag.pub.ro/en/lab/index_2.jpg
5. http://woordentikkertje.manbijthond.nl/
6. http://www.dutchgameawards.nl/2013/spotvogel/
7. The Media Ecology Project: http://sites.dartmouth.edu/mediaecology/
8. Larm: Audio Research Archive. http://larm.blogs.ku.dk/about-larm/
9. Bibliothéque National de France (Annotation Vidéo)
http://blog.bnf.fr/lecteurs/index.php/2013/09/annotation-video-un-mois-
dexperimentation-avec-les-usagers-en-salle-p/
21