Science matters

M.A.Hurst@lboro.ac.ukwww.affectinghci.com@mikehurst
Imagine a World
Michael Hurst
Department of Information Science and Computer Science
Supervisors: Dr Tom W Jackson, Dr Mashhuda Glencross

Motivation
Imagine a world where a computer
can interact with you with the same
quality and fluidity with that as
another human being

The Current Situation is Very Different

Issues
• Can only work on what information they have
• Hand Location (Mouse)
• Written Text (Keyboard)
• Sight
• Smell
• Taste
• Hearing
• Touch

Addressing the Issues
• Historically, design was the answer
• However, one size fits all only goes so far
• Give computers the information people have
• Advancements mean technology is now capable
• Key question:
1. What is the key variable
involved in interaction?

The Research
Using a combination of readily available and
affordable devices that yield sensory data in the
hope to correctly predict the emotional state of a
user at any one time
• Why?
1. How can the Data can be Used (Motivation)

Motivation - Entertainment
Imagine a world:

Motivation – Productivity/Communication
Imagine a world:

Motivation
Imagine a world:

Imagine a World
• Do we really have to Imagine?

Research Questions
1. How do we collect the data needed to
understand emotional state?
2. What do we do with the sensory data once
collected?

How do we Collect the Data?
Give computers the information people have
• Facial Expressions
• Gestures
• Movement
• Vocal Tone
• Vocal Context
• Gaze Velocity
• Gaze Fixations
• Pupil Dilation
• Eye Movement

How do we Collect the Data?
Give computers the information people have
…And possibly further
• Heart Rate
• Heart Rate Variability
• Location of electrical
activity (EEG)

What to do with Sensory Data?
• Compare sensory data with participants self-
assessment of own emotional state to establish
relationships
• For each metric
• Find somatic marker, create a rule
• To combat participant subjectivity, deviation from mean
used
• Rules can be used by computers to assist user/alter
behaviour

• Development of an Emotional Salient Model
• Works in real time
• Determines emotional saliency at time t
• Based on likeliness of emotional state from somatic markers
• Researcher friendly
• Field Standard
• Expandable to incorporate new technologies, task data, other input
methods (twitter), personalisation etc.
• User friendly
• Model is dynamic using only technology that is available to user

Distressed
(11%)
Aroused
(11%)
Excited
(11%)
Displeased
(11%)
Neutral
(11%)
Pleased
(11%)
Depressed
(11%)
Sleepy
(11%)
Contented
(11%)
Arousal
Valence
Russell’s Circumplex Model of Affect
19
1 9
• Baseline
probabilities
• Presence of
somatic markers
makes
probabilities
fluctuate

Microsoft
Kinect
Heart Rate
Sensor
EEG
Sensor
Eye
Tracking
Time t

Microsoft
Kinect
Heart Rate
Sensor
EEG
Sensor
Eye
Tracking
Time t
• Increase in Pupil Dilation
• Rule Triggered
• Emotional Probabilities
recalculated

Microsoft
Kinect
Heart Rate
Sensor
EEG
Sensor
Eye
Tracking
Time t
• Pupil Dilation Returns to
Mean
• As t Progresses, Likeliness
Reverts to Baseline

40% 15% 5%
15% 10% 3%
5% 3% 2%
Time t
Predicted Emotion
(Salient Emotion)
Distressed
CPU Alters
Behaviour
(e.g. … )
• Tell user to take a break
• If showing video/image turn off
• Inform friends and/or family
• Pause game

To Conclude
• If your research involves people, could emotion
be used to further enhance your findings?
• Study Participation: bit.ly/AffectingHCI
• Questions?

Science matters

Recommandé

Recommandé

Contenu connexe

Similaire à Science matters

Similaire à Science matters (20)

Dernier

Dernier (20)

Science matters

Notes de l'éditeur