Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Artificial Intelligence for the Film Industry
1. Artificial Intelligence
for the Film Industry
Georg Rehm
DFKI, Germany
Propellor FilmTech Meetup #1 – 25 July 2017 – Berlin, Germany
2. • Sites in Saarbrücken, Kaiserslautern,
Bremen, Berlin, Osnabrück, St. Wendel
• Intelligent software systems: robotics, agents,
image processing, language understanding,
augemented reality, 3D, knowledge management,
HMI, security, Industrie 4.0.
• 900+ staff – ca. 300 running projects
• CEO: Prof. Dr. Wolfgang Wahlster
Propellor FilmTech Meetup #1 – 25 July 2017
Deutschland-GmbH
2
German Research Centre for
Artificial Intelligence GmbH (founded in 1988)
7. Artificial Intelligence
• Strong AI: hypothetical machine with a consciousness
and behaviour at least as flexible as that of a human.
• Weak AI: software without consciousness, tailored to
one specific purpose and task.
• Machine Learning: give “computers the ability
to learn without being explicitly programmed”
(Arthur Samuel, 1959)
• Examples: pattern recognition (e.g., hand
writing), predictions (stock exchange),
recommendations (films!) etc.
Propellor FilmTech Meetup #1 – 25 July 2017 7
8. Propellor FilmTech Meetup #1 – 25 July 2017 8
Data Intelligence
Current breakthroughs with machine learning methods
(Deep Learning). Also still in use: symbolic, rule-based methods
9. Language Technology
• Language Technology makes use of theoretical results
in linguistics in marketable solutions and applications.
• Uses research results from:
– Artificial Intelligence
– Computer Science
– Computational Linguistics
• Natural Language Processing
• Natural Language Understanding
– Psychology, Psycholinguistics
– Cognitive Science
• Language: Next big thing for AI!
Propellor FilmTech Meetup #1 – 25 July 2017 9
Example Applications
• Spellchecker
• Dictation systems
• Translation systems
• Search engines
• Report generation
• Expert systems
• Dialogue systems
• Text summarisation
10. AI and the Film Industry
• AI and Language Technology:
Many breakthroughs in multiple
different application areas
• Focus: Film industry
• Massive potential!
Propellor FilmTech Meetup #1 – 25 July 2017 10
Film
Industry
Language
Technology
AI and
Deep
Learning
Big Data
Fast
machines
and
networks
Internet of
Things
! Editing Trailers
! Writing Scripts
! Recommenders
13. Propellor FilmTech Meetup #1 – 25 July 2017 13
• Simple Machine Learning
• Training data: 100 trailers
• Create model and apply it
(i.e., to the film “Morgan”)
• Watson selected scenes
• “A human editor was still
needed to patch the
scenes together to tell
a coherent story.”
14. Propellor FilmTech Meetup #1 – 25 July 2017 14
• Good example of using
tech in a curation setting
• With the machine you’re
faster but you arrive at the
same result as the human
• The “AI” part is attributed
to the technology by the
(astonished) human who’s
also been influenced by
clever marketing
• Note: an “AI” is only good
at one very specific task!
19. Propellor FilmTech Meetup #1 – 25 July 2017 19
• Simple ML again
• Training data: scripts
of sci-fi movies
• Neural network learns
patterns and is able to
generate a new script
• Deep Learning for
Natural Language
Generation (NLG)
• Can also be applied
to Shakespeare
20. Propellor FilmTech Meetup #1 – 25 July 2017 20
• Simple ML again
• Training data: scripts
of sci-fi movies
• Neural network learns
patterns in scripts and
is able to generate
new script
• Deep Learning for
Natural Language
Generation (NLG)
• Can also be applied
to Shakespeare
To me, fair, so you never be,
Each trifle, way, when bore your beauty when,
Such hence your can still,
O thou how much were your self the wrong chide.
Thy youth’s time and face his form shall cover?
Now all fresh beauty, my love there,
Will ever time to greet, forget each, like ever decease,
But in a best at worship his glory die.
Stanley Xie, Ruchir Rastogi, Max Chang: “Deep Poetry: Word-Level and Character-
Level Language Models for Shakespearean Sonnet Generation” (Stanford)
21. Propellor FilmTech Meetup #1 – 25 July 2017 21
• The automatically
generated script
doesn’t make any
sense whatsoever.
• “Sunspring” is an
interesting exercise
but, essentially,
unwatchable.
22. AI – Taking Stock
What AI is good at
• Identifying patterns
• Extracting structure
• Data analysis
• Mimicking regularities
• Important: training data
(ideally structured)
• Emulating smart
behaviour
What AI is really bad at
• Creativity
• Eloquence
• Curiosity
• Freshness
• Originality
• Poetry
• Out-of-the-box’ness
• Understanding of the
world that surrounds us
Propellor FilmTech Meetup #1 – 25 July 2017 22
26. Propellor FilmTech Meetup #1 – 25 July 2017 26
Even “automatic mockbuster generation”
required a level of creativity that is way beyond
anything Artificial Intelligence can achieve today.
28. Film AI Startups
• VaultML, ScriptBook, Pilot Movies: Project ticket sales
and box office performance (script or trailer analysis)
• Iris.tv: Better recommendations
• Qloo: Cultural AI, predicts the tastes for any target
audience and maps relationships (music, books, films)
• Valossa: Detects people, context, topics etc. in video
and audio streams (assist video content discovery)
• Cinuru: Customer Relationship Management
• Much more can be done …
Propellor FilmTech Meetup #1 – 25 July 2017 28
http://www.nanalyze.com/2017/07/6-startups-ai-movies-entertainment/
29. Data for Film AI
• Current AI methods can do a lot with interesting data.
• What is “interesting data” in the film industry?
• Could be anything from every part of the life cycle:
– Scripts – Preferences – List of scenes
– Reviews – Films watched – Credits
– Emails – Categories – Rankings
– Production notes – Genres – Relations
– Demographics – Lexicons – Databases
– Statistics – Focus groups – Marketing
– Box office results – Target audience – etc.
Propellor FilmTech Meetup #1 – 25 July 2017 29
30. Example Use Case
• Let’s have a look at a concrete use case and challenge
• Deep, context-aware recommendations that fit the
viewer’s mood, time constraints, interests, focus areas
• Example: you have ca. 60 minutes, you’re interested in
current politics in the US, have an upcoming trip to
Vancouver, like running, AI, languages and technology
• Recommender could suggest films or documentaries
that exactly fit this bill (using a deep user model)
• How? By pulling different sources of data together
• Calendar (upcoming trips and meetings), browsing and
search history, to do list, social media, IMDB profile etc.
Propellor FilmTech Meetup #1 – 25 July 2017 30
31. Example: Details
• Data sources:
– Calendar: upcoming trip to Vancouver
– To do list: prepare the trip (e.g., “find running route”)
– Email archive: hotel booking in Vancouver
• The smart recommender algorithm could examine these
data points and help the user get a few things done
• Upcoming trip + likes running + location of hotel = videos
of running routes or running clubs in Vancouver
• Upcoming trip + likes running = films about, or including,
running that are set in or that were shot in Vancouver
Propellor FilmTech Meetup #1 – 25 July 2017 31
32. Lifelogging and IoT
• Lifelogging = record your whole life
• Mobile phones and activity trackers
are getting closer (quantified self)
• Measuring heart-rate 24/7/365
• Advanced measurements like
VO2 max through several sensors
is consumer-grade technology!
• What about film-related data points?
• Measuring excitement, boredom, attention, repetition,
amazement, imitation, cringe-worthiness, disgust,
tenseness, eye-tracking etc.
Propellor FilmTech Meetup #1 – 25 July 2017 32
https://en.wikipedia.org/wiki/Lifelog
33. Film and Quantified Self
• Vision: create deep user models by pulling together a
user’s heterogeneous information and data streams
(calendar, contacts, to do lists, profiles, youtube etc.)
• Add lifelogging-related data by tapping into activity
trackers, smart watches, mobile phone sensors
• Endless possibilities would emerge … – and will!
• Measure the reactions of one viewer or a whole theater
by measuring their vital stats when watching a film
• Revolutionise film development and test screenings
• Adapt films dynamically (insert explosion when bored)
Propellor FilmTech Meetup #1 – 25 July 2017 33
34. • Propellor | Forum #1 created intriguing results
• Any Film, Anywhere – user model, watchlist, loc, reco
• Bubble Buster – user model, reco (safe & surprising)
• Super AI Brain – user model, reco
• Data of the Movie – user model, reco, biofeedback
• AI-based Storytelling – user model, audience
clustering, Big Data-based storytelling
Propellor FilmTech Meetup #1 – 25 July 2017 34
http://www.propellorfilmtech.com/forum
35. Challenges
• Integration of heterogeneous data sources (from silos!)
into a unified and homogeneous model as well as
making this model available to recommender algorithms.
• Getting the data is hard, so is mapping the data.
• How do we get – on a very large scale – the data out of
connected devices (smart phones, smart watches,
activity trackers, tv sets etc.) into our own applications?
• The typical, very hard, AI challenges: How can we really
model creativity, originality etc.?
Propellor FilmTech Meetup #1 – 25 July 2017 35
36. Thank you!
Propellor FilmTech Meetup #1 – 25 July 2017 36
DKT kick-off meeting – 25 September 2015
Digital Curation Technologies
• Support and optimise digital curation through language and
knowledge technologies
• Develop innovative prototypes together with the SME partners
• Further develop DFKI technologies and transfer them into
industry through platform for digital curation technologies
Georg Rehm und Felix Sasaki. “Digital Curation Technologies.” In Proceedings of the 19th Annual
Conference of the European Association for Machine Translation (EAMT 2016), Riga, Lettland, Mai 2016
Georg Rehm und Felix Sasaki. “Digitale Kuratierungstechnologien – Verfahren für die effiziente
Verarbeitung, Erstellung und Verteilung qualitativ hochwertiger Medieninhalte.” In Proceedings der
Frühjahrstagung der Gesellschaft für Sprachtechnologie und Computerlinguistik (GSCL 2015), S. 138-139,
Duisburg, 2015
Sprach- und Wissenstechnologien
Kuratierungstechnologien
Branchentechnologien
Plattformtechnologie
Branchenlösungen
http://digitale-kuratierung.de