Twin tide skopje2012_june5

Measuring users’ experiences
or, the memory of them?

Trajectory reminders EmoSnaps Footprint tracker iScale

Evangelos Karapanos

Skopje, 5 June 2012
Wednesday, June 6, 12

My
background
BSc Physics / microelectronics, U Patras, Greece (2004)
Thesis: Model based design and evaluation of walk-up-and-use interfaces (HCI
Group, ECE department)

MSc HCI / UCL Interaction Centre, UK (2005)
Thesis: User acceptance of nomadic user interfaces (Philips Research,
Eindhoven)

PhD HCI / TU Eindhoven, NL (2010)
Title: Quantifying diversity in user experience

Assist. Prof. HCI / Madeira ITI (2010-)
Design for Experience in pervasive computing


Soft Reliability

48% of returned products are not attributed to a
violation of product speciﬁcations


problems rooted early
in (concept) design phase

failure to truly incorporate it in one’s life


436 Studies in Computational Intelligence 436
The series Studies in Computational Intelligence (SCI) publishes new developments

Karapanos
and advances in the various areas of computational intelligence – quickly and with
high quality. The intent is to cover the theory, applications, and design methods
of computational intelligence, as embedded in the fields of engineering, computer
science, physics and life sciences, as well as the methodologies behind them.
The series contains monographs, lecture notes and edited volumes in computational
intelligence spanning the areas of neural networks, connectionist systems, genetic
algorithms, evolutionary computation, artificial intelligence, cellular automata,
self-organizing systems, soft computing, fuzzy systems, hybrid intelligent, and
virtual reality systems. Of particular value to both the contributors and the

Coming!
readership are the short publication timeframe and the world-wide distribution,
which enable both wide and rapid dissemination of research output.

Over the past decade the field of Human-Computer Interaction has evolved from the
study of the usability of interactive products towards a more holistic understanding
Evangelos Karapanos
of how they may mediate desired human experiences.

This book identifies the notion of diversity in users? experiences with interactive
1
products and proposes methods and tools for modeling this along two levels:

Modeling Users'

Modeling Users' Experiences with Interactive Systems
(a) interpersonal diversity in users? responses to early conceptual designs, and
(b) the dynamics of users? experiences over time.

The Repertory Grid Technique is proposed as an alternative to standardized

June 2012
psychometric scales for modeling interpersonal diversity in users? responses to early
concepts in the design process, and new Multi-Dimensional Scaling procedures are
Experiences with
Interactive Systems
introduced for modeling such complex quantitative data.

iScale, a tool for the retrospective assessment of users? experiences over time is
proposed as an alternative to longitudinal field studies, and a semi-automated
technique for the analysis of the elicited experience narratives is introduced. Through

Foreword: Jean-Bernard Martens
these two methodological contributions, this book argues against averaging in the
subjective evaluation of interactive products. It proposes the development of
interactive tools that can assist designers in moving across multiple levels of

Closing note: Marc Hassenzahl
abstraction of empirical data, as design-relevant knowledge might be found on
all these levels.

Foreword by Jean-Bernard Martens and Closing Note by Marc Hassenzahl.

issn 1860-949X
isbn 978-3-642-30999-1

9 783642 309991

springer.com
13


Nuno
Nunes Vassilis
Kostakos Monchu
Chen

Laura Rodríguez Gonçalo
Gouveia Néstor
Catano

• 20
faculty Pedro
Campos Paulo
Sampaio Eduardo
Fermé

– 14
countries,
8
languages

• Areas: Larry
ConstanIne Jos
van
Leeuwen Barbara
Pizzileo

– 11
CS,
2
physics/electronics,

2
psychology,
2
architecture,

2
design,
2
art,
2
other
Ian
Oakley Luis
Gomes Ron
Salden

Leonel
Nóbrega ValenIna
Nisi Evangelos
Karapanos

David
Aveiro Luis
Gomes Yoram
Chisik


MSc HCI & Entertainment Technology


Industry Involvement


Design
for
Experience
in
pervasive
compu3ng

Socially translucent eco-feedback technologies
How do eco-feedback technologies:
a) raise mutual awareness of family members’ consumption behaviors
! b) induce feelings of accountability on individuals regarding their consumption behaviors.

Technologies for Social Inclusion in primary schools
a) Using sociometric technologies to assess the inclusiveness of school communities
b) Designing Persuasive technologies that challenge pupils’ perceptions of diversity

Citizen participation on the go
How can we motivate citizen participation through mobile technologies?
•Public transit: The role of psychological empowerment: self-efﬁcacy, sense of community,
and causal importance

Awareness technologies for parents, children and school
e Senseµ ( a) To support interpersonalinfer the physical,within family social activity of a pupil
b)
Using mobile sensors to
connectedness
verbal and

hat aims at supporting awareness in parent
c) To engage parents and school in ad-hoc communication

Location-aware narratives: Does locality matter?
Does the coupling between physical and virtual space result to increased immersion in
the narrative world?


Measuring users’ experiences
or, the memory of them?


Skopje, 5 June 2012

Technology Assisted Reconstruction


Skopje, 5 June 2012

User Experience deﬁned

a momentary,
primarily evaluative feeling (good-bad)
while interacting with a product or service
Hassenzahl, 2008


Most of our evaluation tasks
rely on memory


Heuristics and biases
• Peak-and-end phenomenon

• Summary judgments can be best predicted by a simple
average of the most extreme and the last experience
(Fredrickson and Kahneman, 1993)

• Replicated in HCI - Summarizing mental effort to
perceived usability: end matters (Hassenzahl and
Sandweg, 2004)

• Other biases (e.g., Rosy retrospection, Focusing illusion
etc.)


Why do memory biases exist?
Robison & Clore (2002)

“The emotional experience can neither
be stored nor retrieved”

it is reconstructed from recalled
contextual details


Why do memory biases exist?
Robison & Clore (2002)
event, but instead, every attempt to Type of Knowledge Source of information Type of Self-Report

often altered representation of the
“The emotional experience can neither
articipants to recall an unfamiliar Experiential Online emotion, e.g.
Episodic
be stored nor retrieved”
d 20 hours before. Recalled stories
Knowledge Experience Sampling

nal one in missing details, altering
ance is reconstructedin applying
it of events, or from recalled
Retrospective, e.g.
contextual details
erpretations to the original story.
Episodic Episodic memory
Day Reconstruction

r distorted through repeated
Situation-speciﬁc
Semantic Exit questionnaires
bering is an act of reconstruction belief

ction has received wide support. At
ction lies the distinction between
Identity-related
memory [69]. While episodic Semantic
belief
Exit questionnaires
a particular event from the past,
ot tied to any particular event but
Robinson & Clore (2002)
n generalizations (i.e. beliefs) that Figure 1. Four sources of information in emotional self-repor
935). These two types of memory according to Robinson and Clore [63]. Figure adapted from [6
uch as learning new information

bottom line...

If you want to know what the user really experiences,
ask her at that exact moment!


Experience Sampling Method
Prompts at random, or computationally
estimated times, to self-report on ongoing
behaviors and experiences.

– Where are you?
– What are you doing?
– How far is your mobile phone?
– How do you feel?

Karapanos, E. (2012) Experience Sampling, Day Reconstruction, what’s next?
Towards Technology-Assisted Reconstruction. M-ITI internal report.

Prompts at random, or computationally
estimated times, to self-report on ongoing

– Where are you?
– What are you doing?
– How far is your mobile phone?
– How dovenues would the ACM a substantial number
a few relevant
you used still miss Guide to Computing
of studies. We instead
feel? method while following a user-initiated diary approach.
The analysis of the remaining 49 studies is being reported
Literature querying for the term “experience sampling” below.
No of papers referring to
without constraining to particular venues. This query
returned 284 papers, published in more than ﬁfty venues. Study length, sampling frequency, and response rate
Experience Sampling The majority (80%) of the studies had a duration of several
60 days up to one month with 14 studies (34%) lasting
between four and seven days (see ﬁgure 3). Only two
studies had a duration of more than a month.
45

15
30

15
10

0
2001 2003 2005 2007 2009 2011
5

Figure 2. 243 papers referring to experience sampling over a
Karapanos, E. (2012) Experience Sampling, Day Reconstruction, what’s next?
ten-year period. Retrieval took place on August 26th, 2011.
Towards Technology-Assisted0 Reconstruction. M-ITI internal1m > 1m
≤1h < 24h ≤ 3d ≤ 1w ≤ 2w ≤
report.
Fourty-one papers were excluded from further processing.

What variables do ES studies measure? sampling method. Two of these provided no justifications

We distinguish below between self-reported measures of for their choice. Analyzing the remaining 19 papers resulted
behavior and experience (see Table 1). This distinction is to a total of 11 reasons for choosing alternative methods to
relevant as their reconstruction follows a different process ESM (see table 2).
whereas behavioral information may be directly accessible
through episodic memory while experiential information Table 2. Reasons for not selecting the Experience Sampling
has to be further inferred from recalled episodic cues [63]. Method along with frequency of occurence (No of papers).

Prompts at random, orES studies eliciting self-reported
Table 1. Number of computationally Reason No
measures of behavior, experience, or both.
estimated times, to self-report on ongoing Disrupts the activity 6
Type of measures that studies elicit No
Imposes high burden to participants 3
Self-reported measures of behavior 5
Requires high effort from researchers 3
– Where areSelf-reported measures of experience
you? 22
– What are you doing? of behavior & experience 18
Self-reported measures
Inappropriate for eliciting rich qualitative data 3

– How far is your mobile phone? Misses rare and brief events 3
– How dovenues would thethe participant Computing with analysisactivityremainingThe studiesshould be in control of when, what and
you used stillACM a substantial number method whileofbeing a user-initiated isdiary approach.
feel? miss Guide to was engaged The prior to following 49 user being reported
a few relevant Behavioral measures related most frequently to the
of studies. We instead that
(n=18) the 2
Literature querying for the term “experience sampling” (n=2) (e.g. [35]), the
interrupted (e.g. [30]), its duration below. how often to report
Noparticipant’s referringphysical location (n=15) (e.g.sampling frequency, and response rate
of papers current to
without constraining to particular venues. This query
returned 284 papers, published in more than fifty venues. Study length,
[17])
Experience Sampling (n=10), e.g. the number or nature ofof the studies hadsample size several
and the social context
The majority (80%) Limits a duration of 2
relationship of people that are in close proximity month with 14 studies (34%) lasting
days up to one or
60
participate with in a conversation (e.g. [34]). Other
between four and seven days (see figure 3). Only two
Depends on participants’ ability to articulate 2
measures of behavior related to modestudiestransitduration of moreongoing experience
of had a [25], than a month.
45
participants’ current physical engagement [18] and mode of
convrersation (e.g. f2f, fixed/mobile phone etc.) [32].
15
30 Poses privacy concerns 2
Experiential measures related to:
15
10
• Attitudes towards behaviors or events (n=15) such as Limits number of measured variables 1
0
being interrupted (e.g. [55]), disclosing information to
2001 2003 2005others (e.g. [17]), or being video recorded (e.g.
relevant 2007 2009 2011 Technology limitations 1
5
[58]).
Figure 2. 243 papers referring to experience sampling over a
• Measures of affectE. (2012) Experience Sampling, Day Reconstruction, the most frequent reason
Karapanos, and experience (n=18) such as mental
ten-year period. Retrieval took place on August 26th, 2011.
As expected, what’s next? for not selecting the
0
Towards [18]) and concentration [14], ≤ 3d ES 1w ≤ 2w was the interruptions that
method
engagement (e.g. Technology-Assisted Reconstruction. M-ITI internal1m > 1m
report. the method imposes
≤1h < 24h ≤ ≤
Fourty-one papers were excluded [27], further processing.
Wednesday, June 6, 12 satisfaction from mood and emotional states (e.g. [53], on the user’s activity (eg. [48], [5]). For instance, Lindley

Day Reconstruction Method
Kahneman et al. (2004)

Can a retrospective method help participants in
recalling more accurately their experiences?

ry attempt to Type of Knowledge Source of information Type of Self-Report

ntation of the
an unfamiliar Episodic
Experiential
Knowledge
Online emotion, e.g.
Experience Sampling
called stories
tails, altering
in applying Retrospective, e.g.
Episodic Episodic memory
riginal story. Day Reconstruction

h repeated
Situation-speciﬁc
Semantic Exit questionnaires
econstruction belief

e support. At
ion between
Identity-related
hile episodic Semantic
belief
Exit questionnaires
om the past,
lar event but Robinson & Clore (2002)
beliefs) that
Figure 1. Four sources of information in emotional self-report

Day Reconstruction Method
Kahneman et al. (2004)

Can a retrospective method help participants in
recalling more accurately their experiences?


Technology
Assisted
ReconstrucIon

Can mobile sensors assist participants in reconstructing
their daily experiences and whereabouts?



Trajectory reminders in location-based preferences

Do trajectory reminders (locations visited before and after) increase the test-retest
reliability of the reconstruction process?

Control condition With trajectory reminders

! !


Emosnaps - inferring emotion from self-face pics
Can self-face pictures assist in recalling momentary emotions?
If so, is it through a recognition or a reconstruction process?


Emosnaps - inferring emotion from self-face pics
Can self-face pictures assist in recalling momentary emotions?
If so, is it through a recognition or a reconstruction process?

Experience Sampling (Ground truth)

78% of pictures
could be used for
inferring emotions

Time-Day Photo-day Photo-week
control condition Emotion reconstruction Emotion recognition


Footprint tracker
How do visual cues (i.e., Sensecam), location cues, and context cues (SMS and calls
made or received) assist in reconstructing daily behaviors and experiences?

1. Sensecam

2. Location logging
3. Context logging
(SMS/calls made or received)


Karapanos, E., Martens, J.-B., Hassenzahl, M. (2009) Reconstructing Experiences through Sketching. Arxiv preprint, arXiv:0912.5343.


Cross-‐sec8onal Repeated
sampling

“Longitudinal”
paradigms
in
HCI
Longitudinal Retrospec8ve


Constructive Value-Account Control (no-graphing)

Constructive iScale, but not the Value-Account, performed better than
control condition

•More experience reports
•With more details (references to temporal information, discrete events)
•Higher test-retest consistency of time estimation (i.e., when did
an experience take place)
•Higher test-retest consistency of graphed patterns (over Value-
Account)
33
Karapanos, E., Martens, J.-B., Hassenzahl, M. (2009) Reconstructing Experiences through Sketching. Arxiv preprint, arXiv:0912.5343.


Can we really trust 6-month old
memories?

• Validity? i.e. do memories reﬂect what we really
experienced?

• Reliability? i.e. in a second trial, will we recall
the same experiences?


Memories are (sometimes)
more important than experiences

• Memories deﬁne how you evaluate your
past and how you decide on your future

• What do we measure for?
• Why do people drive irresponsibly?
• Why do people recommend their
products


Twin tide skopje2012_june5

Recommended

Recommended

More Related Content

Similar to Twin tide skopje2012_june5

Similar to Twin tide skopje2012_june5 (20)

More from Evan Karapanos

More from Evan Karapanos (6)

Recently uploaded

Recently uploaded (20)

Twin tide skopje2012_june5