People’s write-ups, such as online reviews and personal micro-blogs, often reflect their emotions, ranging from just-in-the-moment sentiment to long-lasting mood. In this talk, I will first give an overview on modeling human emotions encapsulated in people’s write-ups. I will then sample two visual analytic systems that use very different methods to automatically extract and visualize human emotions for two very different purposes. The first is an interactive visual analytic system that automatically summarizes human sentiment captured in online reviews and leverages the power of a crowd to rectify the imperfections in machine sentiment analysis. The second is a timeline-based visual analytic tool that extracts and visualizes a person’s moods over time based on the person’s tweets. Finally, I will discuss the challenges of inferring human emotional DNA in general and potential research directions.

1. Interac(ve
Visual
Analysis
of
Human
Emo(ons
Sen(ment,
Mood,
and
Emo(onal
DNA
Michelle
X.
Zhou
mzhou@juji-­‐inc.com

2. Modeling
Human
Emo(ons
Basic
Emo*ons:
Moment
of
feelings
“I
was
so
happy”
“I
am
very
sad”
Human
emo2ons
are
complex
and
mul2-­‐dimensional
[Frijda
1993;
Ekman
2007]

3. Modeling
Human
Emo(ons
Human
emo2ons
are
complex
and
mul2-­‐dimensional
“The
app
is
just
awesome”
“The
app
is
dreadful
and
hard
to
use”
Sen*ment:
A
feeling
toward
a
situa(on

4. Modeling
Human
Emo(ons
Human
emo2ons
are
complex
and
mul2-­‐dimensional
“Recently
I
have
been
in
a
good
mood”
Mood:
Longer-­‐las(ng
feelings
over
a
period
of
(me
[Frijda
1993;
Ekman
2007]

5. Modeling
Human
Emo(ons
– Emo(onal
outlook
“I
am
very
op2mis2c”
– Emo(onal
vola(lity
“I
am
quite
calm”
– Emo(onal
resilience
Human
emo2ons
are
complex
and
mul2-­‐dimensional
Emo*onal
DNA:
Unique
emo(onal
paXerns
and
characteris(cs
[Davidson
&
Begley
2012]

6. Modeling
Human
Emo(ons
Basic
Emo*ons
Sen*ment
Mood
Emo*onal
DNA
Thing
Outlook
Vola(lity
Resilience

7. PEARL:
Interac(ve
Visual
Analy(cs
of
Personal
Emo(onal
Style
from
Social
Media
[Zhao
et
al.
VAST
2014]

8. Personal
Emo(onal
Style
from
Social
Media
• What
are
your
basic
emo(ons
conveyed
by
your
current
posts?
• What
is
your
mood
over
the
(me
period
X?
• What
is
your
overall
emo(onal
style?

9. Live
Demo
hXp://www.cs.toronto.edu/~jianzhao/webapp/Pearl/pearl.html

10. Emo(onal
Psychology:
Types
of
Emo(on
• 8
basic
emo(ons
• Complex
emo(ons
composed
of
basic
ones
[Plutchik
2001]

11. Emo(onal
Psychology:
Dimensions
of
Emo(on
• Valence
– Posi(vity
• Arousal
– Intensity
• Dominance
– In-­‐control
Valence
Arousal
Happy
Rage
Dominance
Fear
Anger
Sad
[Mehrabian
1980]

12. Analyze
Personal
Emo(ons
from
Text:
Detect
Basic
Emo(ons
• Lexical
based
approach
• Aggregate
the
amount
of
evidence
+
VAD
scores
Valence
Arousal
Dominance
Anger
v1
a1
d1
An*cipa*on
v2
a2
d2
Disgust
.
.
.
Fear
.
.
.
Joy
.
.
.
Sadness
.
.
.
Surprise
.
.
.
Trust
v8
a8
d8
E(t):
[M4x8(t)]
[Calvo
&
Kim
2013]

13. Analyze
Personal
Emo(ons
from
Text:
Infer
Mood
Constraint-­‐based
co-­‐clustering
to
aggregate
basic
emo(ons
to
infer
mood
– Temporal
proximity
– Emo(on
proximity
– Seman(c
proximity
[Pan
et
al.
IUI
2013]

14. Visually
Iden(fying
Emo(onal
Style
Valence
Emo(on
Bubble
+
Band

15. Visually
Iden(fying
Emo(onal
Style
Resilience
Outlook
Vola(lity

16. Evalua(on
• How
good
is
our
emo(on
analysis?
• How
useful
is
our
analysis?
– Discovery
of
personal
emo(onal
style
– Discovery
of
others’
emo(onal
style

17. Ground
Truth
vs.
Derived
Emo(ons
10
people
over
1
month
of
tweets,
~60
segments
Cohen’s
Kappa
indica(ng
the
agreement
between
ground
truth
and
derived
emo(ons.

18. Discovering
Personal
Emo(ons
• 6
people,
each
analyzed
over
2-­‐
year
of
own
tweets
(600-­‐4000)
• Self
awareness
• Valida(on
Valence

19. Discovering
Others’
Emo(ons
• 50
Turkers,
44
valid
results
• 10
emo(on
analysis
tasks
on
user
data
w/
ground
truth
“in
which
2me
period,
the
person
is
most
emo2onally
submissive?”

20. OpinionBlocks:
Crowd-­‐Powered,
Self-­‐Improving
Visual
Analy(cs
of
Opinion
Text
[Hu
et
al.
INTERACT
2013]

21. The
Challenge
• User
challenge
– Much
content
and
liXle
(me
• 18000
reviews
for
Kindle
Fire
• Research
challenge
– Sen(ment
analysis
is
hard
– Feature-­‐based
sen(ment
analysis
is
harder
[Liu
2012]

22. OpinionBlocks

23. Crowd-­‐Powered,
Self-­‐Enhancing
Visual
Sen(ment
Analysis

24. How
Well
A
Crowd
Can
Help
88.8%
users
made
right
correc(ons
95%
users
were
willing
to
correct
machine
mistakes

25. Quality
of
Improvements
• Two
crowds
performed
same
set
of
tasks
– First
crowd
made
system
improvements
– Second
crowd
used
the
improvements
• A
set
of
non-­‐trivial
tasks
• Significantly
improved
task
comple(on
(me

26. Remaining
Challenges
• Limita(ons
in
current
emo(on
analysis
– Lexical
approach
– Full
NLP
• Visual
summariza(on
of
analy(c
results
– Need
to
take
into
account
of
analy(c
confidence/uncertainty
• Real-­‐(me,
incremental
emo(on
analysis
• Modeling
paXerns
of
emo(ons
– Temporal
– Individual
differences

27. Acknowledgement
• Jian
Zhao
• Mengdie
Hu
• Liang
Gou
• Huahai
Yang
• Yunyao
Li
• Fei
Wang
• Eben
Haber

28. Call
for
Submissions
hXp://(is.acm.org/

29. References
• J.
Zhao,
L.
Gou,
F.
Wang,
M.X.
Zhou:
PEARL:
An
interac(ve
visual
analy(c
tool
for
understanding
personal
emo(on
style
derived
from
social
media.IEEE
VAST
2014:
203-­‐212.
• M.Hu,
H.
Yang,
M.X.
Zhou,
L.
Gou,
Y.
Li,
E.
Haber:
OpinionBlock:
A
Crowd-­‐Powered,
Self-­‐
improving
Interac(ve
Visual
Analy(c
System
for
Understanding
Opinion
Text.
INTERACT
(2)
2013:
116-­‐134.
• S.
Pan,
M.X.
Zhou,
Y.
Song,
W.
Qian,
F.
Wang,
S.
Liu:
Op(mizing
temporal
topic
segmenta(on
for
intelligent
text
visualiza(on.
IUI
2013:
339-­‐350.
• N.
H.
Frijda.
Moods,
emo(on
episodes
and
emo(ons.
In
M.
Lewis
and
J.
M.
Haviland,
editors,
Handbook
of
Emo(ons,
pages
381–403.
Guilford
Press,
1993.
• P.
Ekman.
Emo(ons
Revealed.
Holt
Paperbacks,
2
edi(on,
2007.
• R.
J.
Davidson
and
S.
Begley.
The
Emo(onal
Life
of
Your
Brain:
How
Its
Unique
PaXerns
Affect
the
Way
You
Think,
Feel,
and
Live–and
How
You
Can
Change
Them.
Hudson
Street
Press,
2012.
• B.
Liu.
Sen(ment
Analysis
and
Opinion
Mining.
Morgan
&
Claypool
Publishers,
2012.
• A.
Mehrabian.
Basic
Dimensions
for
a
General
Psychological
Theory.
Oelgeschlager,
Gunn
&
Hain
Inc.,
1980.
• R.
A.
Calvo
and
S.
Mac
Kim.
Emo(ons
in
text:
Dimensional
and
categorical
models.
Computa(onal
Intelligence,
29(3):527–543,
2013.
• R.
Plutchik.
The
nature
of
emo(ons.
American
Scien(st,
89(4):344,
2001.