HCI 2014 (9 of 10): Affective Factors

Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – www.purl.org/net/busaco
affective factors
Human-Computer Interaction
http://lts5www.epfl.ch/

“Designers think, so people can feel.”
Juan-Carlos Fernandez

Emotion
Affective aspects of interaction (Rogers, 2007)
expressivity
how the interface look & feel could affect the user?

Emotion
frustration
how we can detect and reduce it?

Emotion
persuasive technologies
how we can change the user attitude & behavior
by using the actual technologies?

Emotion
Emotion is a reaction to events deemed relevant
to the needs, goals, or concerns of an individual
Scott Brave & Clifford Nass, 2008

Emotion
Emotion represents the experience in UX

Emotion
Emotion encompasses physiological, affective,
behavioral, and cognitive components

Emotion
emotion in the cognitive context – Moore (2005)

Emotion: classification
Emotions as discrete categories
basic emotions (Paul Ekman et al., 1972):
anger
disgust
fear
happiness
sadness
surprise

Emotions as discrete categories
additional emotions (Paul Ekman, 1990):
amusement contempt contentment
embarrassment excitement guilt
pride in achievement relief satisfaction
sensory pleasure shame

Emotions as multi-dimensional models
circumplex model (James Russell, 1980)
2 axes:
arousal (calm↔excitement) + valence (pleasantness)

circumplex model (James Russell, 1980)
used to test stimuli of emotion words,
emotional facial expressions, and affective states

Plutchik’s wheel of emotions (Robert Plutchik, 1980)
8 primary bipolar emotions:
joy vs. sadness; anger vs. fear;
trust vs. disgust; surprise vs. anticipation
+
more advanced emotions

Negative & forceful
Anger, Annoyance, Contempt,
Disgust, Irritation
Negative & not in control
Anxiety, Embarrassment,
Fear, Helplessness,
Powerlessness, Worry
Negative thoughts
Doubt, Envy, Frustration,
Guilt, Shame
Negative & passive
Boredom, Despair,
Disappointment, Hurt, Sadness
Agitation
Shock, Stress, Tension
Positive & lively
Amusement, Delight, Elation,
Excitement, Happiness,
Joy, Pleasure
Caring
Affection, Empathy,
Friendliness, Love
Positive thoughts
Courage, Hope, Pride,
Satisfaction, Trust
Quiet positive
Calm, Content, Relaxed,
Relieved, Serene
Reactive
Interest, Politeness, Surprise
Douglas-Cowie et al. (2015)

positive activation – negative activation (PANA)
Watson and Tellegan, 1985
PAD (Pleasure, Arousal & Dominance) emotional state
model – Mehrabian, 1997
used to study nonverbal communication

Lövheim cube of emotion (Hugo Lövheim, 2012)
a theoretical model considering the relationships
between monoamine neurotransmitters
(serotonin, dopamine, noradrenaline) and emotions

Parrotts’ classification (Gerrod Parrott, 2001)
primary emotions: love, joy, surprise, anger, sadness, fear

secondary emotions – e.g., love: affection, lust, longing

secondary emotions – e.g., love: affection, lust, longing
tertiary emotions – e.g., affection: adoration, fondness,
liking, attraction, caring, tenderness, compassion,…

discussion
http://uniquelang.peiyinglin.net/visualization/Parrott_Model_b.png

Emotion
Modeling languages & vocabularies
Emotion Annotation and Representation Language
HUMAINE Project, 2006
http://emotion-research.net/projects/humaine/earl/

Emotion
Modeling languages & vocabularies
Emotion Markup Language (EmotionML) 1.0
W3C Proposed Recommendation, 2013
http://www.w3.org/TR/emotionml/
Vocabularies for EmotionML
W3C Working Group Note, 2014
http://www.w3.org/TR/emotion-voc/

Emotion
neurological structure of emotion (LeDoux, 1996)

Emotion
limbic system – the “seat of emotion” – evaluates
the need/goal relevance of its inputs

Emotion
thalamus + limbic system mainly deal with
primitive (basic) emotions
e.g., startle-based fear, anger, sadness, joy, disgust

Emotion
Emotions that require more extensive cognitive
(i.e., knowledge-based) processing are called secondary

Emotion
Emotions that require more extensive cognitive
(i.e., knowledge-based) processing are called secondary
examples:
frustration
pride
satisfaction
in the cortex

Emotion
An emotion can result from a combination of both
thalamic-limbic and cortex-limbic mechanisms

Emotion
An emotion can result from a combination of both
thalamic-limbic and cortex-limbic mechanisms
an event causing an initial fear reaction
can be later recognized as harmless
by more extensive, rational evaluation
example: screensaver initialization

Emotion
Emotions could be learned from the social environment

Emotion
emotion versus mood
emotions are intentional

Emotion
emotion versus mood
emotions are intentional
“they imply and involve relationships
with a particular object”
Frijda, 1994

Emotion
emotion versus mood
moods are not directed at any object in particular and
are experienced as more diffuse, global, and general

Emotion
emotion versus mood
moods are not directed at any object in particular and
are experienced as more diffuse, global, and general
a person can be sad about something (an emotion)
or generally depressed (a mood)

Emotion
emotion versus mood
from a functional point of view,
emotions could determine the execution of task(s)

Emotion
emotion versus mood
emotions cause or contribute to moods

Emotion
emotion versus sentiment
sentiments are not states of an individual,
but assigned properties of an entity (person, object)

Emotion
sentiments can persist indefinitely, by contrast with
emotions (seconds) or moods (hours/days)

Emotion
sentiments are often generalizations about a class of
objects with a given recognizable property

Emotion
sentiments are often generalizations about a class of
objects with a given recognizable property
these generalizations must not necessarily be logical
“I hate spreadsheet applications!”

How about the role of emotion in HCI?

Emotion
Emotion has influences on
attention
memory
performance
opinion (assessment)

Emotion
Attention
an interface capable of detecting – or at least predicting –
a user’s emotional or mood state could similarly assume
an affect-regulation role, helping to guide attention away
from negative and toward more positive stimuli

Emotion
Memory
negative events, which tend to be highly arousing,
are typically remembered better than positive events

Emotion
Memory
negative events, which tend to be highly arousing,
are typically remembered better than positive events
avoid error messages

Emotion
Memory
emotionality “improves memory for central details,
while undermining memory for background details”
Heuer & Reisberg, 1992; Parrott & Spackman, 2000

Emotion
User performance
keeping a user happy may, therefore,
not only affect satisfaction,
but may also lead to efficiency and creativity

Emotion
the circumplex model in the context of user experience
P. Desmet & P. Hekkert, “Framework of Product Experience”,
International Journal of Design, Vol. 1, No. 1 (2007)
http://www.ijdesign.org/ojs/index.php/IJDesign/article/view/66/15

Emotion
User performance
negative example:
objects that appear or move unexpectedly
– e.g., pop-ups, animations – and loud or sharp noises
are likely to trigger startle-based fear

Emotion
Assessment
mood influences judgment and decision making
users in a good mood will likely judge
both the interface and their work more positively,
regardless of any direct emotional effects

Emotion
Causes:
needs & goals
Maslow’s hierarchy of basic needs (1968)
physiological, safety/security, social, esteem,
cognitive, aesthetic, self-actualization, transcendence

Emotion
Causes:
needs & goals
interfaces can also directly address a user’s basic needs

Emotion
Causes:
needs & goals
example (Fogg, 1998):
establishing a trusting and safe relationship with users

Emotion
Causes:
needs & goals
example (Brave & Nass, 2006):
educational software should address users’ emotional
needs, not only teaching the relevant content,
but also ensuring users believe that they are learning

Emotion

Emotion
Causes:
obstacles & control
Ellsworth (1994)
if user has the control over the software,
the negative affective consequences are least important

Emotion
Causes:
social context – “agency”
(Ellsworth, 1994) & (Friedman & Kahn, 1997)
“This system failed to understand your command.” – blaming itself
“The command was not understood.” – blaming no one
“You did not speak clearly enough for your command
to be understood.” – blaming the user

Emotion
blaming
itself

Emotion
Causes:
previous emotional state
repeated pleasurable affective states, therefore,
become expected and thus gradually lose intensity

Emotion
How emotion could be detected?
neurological response – via electroencephalogram (EEG)
or magneto-resonance imaging (MRI)

Emotion
autonomous activity of the human organism
breath
facial expression
voice

Emotion
categorization of facial responses to determine
which emotions they express
Facial Action Coding System – FACS
(Paul Ekman et al., 1978, 2002)

Emotion
interpreting micro-expressions
(e.g., pushed up cheeks, dropping upper eyelids,
eyebrows raised, nose wrinkling,…) – via FACS –
and body language to detect emotions
see also http://tinyurl.com/k47dhen
http://temasys.com.sg/vidyoedm22102013/

Emotion
behavioral changes

6 + 1 emotions with universal facial gestures
(N. Lazzaro, Why We Play: Affect and the Fun of Games, 2008)
Emotion Example
Frustration Figuring out how to get character off a roof in Tom Clancy’s Splinter
Cell (usability issues that detract from the player experience)
Fear Falling into boiling lava,
fast-moving projectiles aimed at the player in Doom
Surprise Using Myst’s linking books for the first time to transport to
a new world
Sadness When the young magician Aerith, in Final Fantasy VII is murdered
Amusement When two Sims get married in The Sims, or rolling
over and picking up sumo wrestlers in Katamari Damacy
Disgust Becoming a social outcast (social disgust)
after losing the dancing challenge in Sid Meier’s Pirates
Curiosity Wanting to know what happens by driving the race track
the wrong way in Project Gotham Racing 3

Emotion
Expressivity
induces an emotional state via UI elements:
color, icons, sound, graphics,…

Emotion
Expressivity
lack of expressivity in communication at a textual level
emoticons
:) :| >:-(

Emotion
Expressivity
abbreviations could have emotional connotations
see leet speak
I 12 CU 2NITE

Emotion
To stimulate emotions, products must have
attributed a personality based on appearance
and interaction means
Reeves & Nass, 1989

How to create emotions in a person?

Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
Ekman, 2004

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
a dialog window
appears

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
thinking about
previous message

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
remembering
last game

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
thinking about
what happens
now on Web

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
discussing UI
features with others

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
another person’s
facial expression
(video-chat)

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
another person’s
review of a task

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
deleting others’ files

Emotion
1. Auto appraisal
4. Imagination
5. Talking about
laughing after defeat
in front of friends makes
it feel more positive

Emotion: design solutions
metaphors
+
idioms

metaphors + idioms
“Metaphors create insight. But they also distort.
They have strengths. But they also have limitations.”
Gareth Morgan

metaphors + idioms
use metaphors to convey and/or create
mental associations
Dan Saffer, 2005

metaphors + idioms
proper metaphors used to denote UI objects/devices
contribute to the personification of the software,
increasing the usability

icons
denote a relation between a visual representation
and a certain concept

icons
denote a relation between a visual representation
and a certain concept
based on similarity – e.g., an empty page = new file
analogue – example: scissors = cut operation
arbitrary (based on an idiom) – e.g., X = close window

icons
most effective are the icons designed with respect to
similarity between visual representation and the concept
Rogers, 2007

arbitrary applications
system programs
remark: the orientation is different
see also GNOME Human Interface Guidelines

icons
beware of the cultural context

earcons
earcon = the aural equivalent of an icon

earcons
earcon = the aural equivalent of an icon
using sounds to denote a specific concept, event, or object
M. Blattner et al, “Earcons and Icons”,
Human-Computer Interaction, Vol. 4 (1989)
http://www.daimi.au.dk/~dsound/DigitalAudio.dir/Papers/Earcons_and_Icons.pdf

earcons
design guidelines (Karen Kaushansky, 2012):
choose the right type of audio
embed meaning in audio earcons
design in context
consider the “non-use cases”
decide between recorded prompts and text-to-speech
www.smashingmagazine.com/2012/09/14/guidelines-for-designing-with-audio/

earcons
examples:
mobile phones, operating system sound schemes,
instant messaging, electronic games, 3D environments,…
commonly, not used in the Web context

tactons
communicate non-visual information
by using the tactile sense
“structured tactile messages” (Brewster & Brown, 2004)
http://eprints.gla.ac.uk/3443/1/tactons_aussi.pdf

tactons
communicate non-visual information
by using the tactile sense
a common example:
coded tactile representations (Braille code)
text informationnon-visual form

tactons
context:
wearable computing

tactons
examples:
haptic I/O devices – e.g., sensors
case study:
L. Brown, Tactons: Structured Vibrotactile Messages for Non-Visual
Information Display, Microsoft Research (2006)
http://research.microsoft.com/apps/video/default.aspx?id=135935

abstract
representation
cognitive
dimension
usage
icon visual conveying information
earcon audio attention
tacton haptic appeal to a known touch
emoticon emotion conveying
an emotional state
gesticon dynamic indicating a state change
kineticon dynamic state change, progress
some are more suitable to (visually) impaired users

persuasive computing

Positive psychology

hedonomics
focusing on the pleasant or enjoyable aspects of HCI
Hancock, Pepe & Murphy, 2005

Persuasive (seductive) technologies
interactive computational systems deliberately designed
to change human attitudes and behaviors
B. J. Fogg (2003)

interactive computational systems deliberately designed
to change human attitudes and behaviors
captology
(computers as persuasive technologies)

persuasion in the context (Maria Miceli et al., 2011):
accidental vs. intentional
communicative vs. non-communicative
coercive vs. non-coercive

typical examples:
advertisement techniques,
warning boxes, reminders, prompts,
personalized messages, recommendations,…

case studies:
games with social impact – for people with special needs
applications helping the cure of certain phobias
infant simulators
virtual pets – to diminish “bad habits” of children
…

Emotion
Herman the bug (Lester et al., 1997)

interesting uses in the HCI context:
get understanding of user behaviour
get understanding of cognitive factors

multimodal factors (Guerini, 2011):
embodied conversational agents – ECAs
kinetic typography
music
use of images

multimodal factors (Guerini, 2011):
embodied conversational agents – ECAs
kinetic typography
music
use of images
virtual characters

Virtual characters
avatars – artificial (visual) representations of people

Virtual characters
user-facing agents
showing simulated human behavior
(knowledge, inferring, choices,…artificial intelligence)

Virtual characters
autonomy no human intervention
reactivity takes stimuli from environment
proactivity takes initiative to act
collaboration able to collaborate with other agents

Virtual characters
autonomy no human intervention
reactivity takes stimuli from environment
proactivity takes initiative to act
collaboration able to collaborate with other agents

multi-agent systems

intelligent virtual agents having human
appearance and the capability to teach
foreign language vocabulary
M. Macedonia et al., “Intelligent virtual agents as language trainers
facilitate multilingualism”, Frontiers in Psychology (2014)
http://journal.frontiersin.org/Journal/10.3389/fpsyg.2014.00295/full
case study

The use of anthropomorphism
adding human qualities to virtual things
reducing the user’s anxiety regarding computers
(computer and/or Internet phobia)

example:
“Hi, Daniel, welcome back! What we learnt last time?
Oh, yes, that experiment on Web navigation…
Let’s repeat it!”
versus
“User 33, start the experiment 7!”

warnings:
user might feel inferior, anxious, or stupid

Affective interaction
goal: a program/device able to register emotions and
other stimuli and to generate a correct response
to these stimuli

input:
sensors measuring (or learning) the user affective state
facial expression, gestures,
heartbeat rate, blood pressure, transpiration, speech

output:
expressing empathy using computational approaches
“intelligently” / “emotionally” reacting on a situation

iCAT – a robot hardware platform using a set
of logical rules to convey emotional states (Philips, 2005)
context: human-robot interaction (HRI)
www.research.philips.com/technologies/projects/robotics/
case study

Aspect of interest: computational humour
humour is a powerful generator of emotions

theories of humour applied in computer science
details in P. Petta et al. (Eds.),
Emotion-Oriented Systems, Springer, 2011

humour generation
building computational models
to generate humourous content (text)

humour recognition
identifying humour
in natural language and/or human behaviour

Emotion: abuse on software
Humans could have an offensive behavior
regarding the digital entities
www.agentabuse.org

example (Tatsuya Nomura, 2005):
impolite behavior of the patients,
when they discovered that the therapist is a virtual agent

example (Antonella De Angeli & Rollo Carpenter, 2005):
“injuries” on conversational – e.g., IRC – robots:
“liar”, “stupid”, “dumb” software

experiment (Christoph Bartneck et al., 2005):
humans tend to hurt more easily robots/devices
than people

User: What is your address?
Jabberwacky: I don't think I'll tell you.
You'd kill me if I told you. Wouldn't you!
User: Yes I would.
Jabberwacky: What is your favourite food?
User: I WILL KILL YOU! I WILL HUNT YOU DOWN!!!
Jabberwacky: Please don't... I really have to leave.
User: I WILL KILL YOU!
Jabberwacky: Good.

Emotion: open question
With which emotion
should HCI designers be most concerned?

When and how should interfaces attempt
to directly address users’ emotions and
basic needs (versus application-specific goals)?

When and how should users be informed
that their affective states are being monitored
and adapted to?

How does emotion play out
in computer-mediated communication?

affective factors
emotions, moods, sentiments
persuasive technologies
“Conclusion”

next lecture: ubiquitous computing

HCI 2014 (9 of 10): Affective Factors

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (20)

Similaire à HCI 2014 (9 of 10): Affective Factors

Similaire à HCI 2014 (9 of 10): Affective Factors (20)

Plus de Sabin Buraga

Plus de Sabin Buraga (20)

Dernier

Dernier (20)

HCI 2014 (9 of 10): Affective Factors