Evaluating User Experience and User Engagement for Design
1. Evaluating User Experience
And
User Engagement for Design
Jennefer Hart
PhD Year One
Supervisors: Professor Alistair Sutcliffe & Dr Antonella De Angeli
Manchester Business School
July 2011
2. Research Contribution
Extend (and Verify) User Experience Theories
Explore the Nature of Interactivity
Develop and Test New Research Methodologies
Inform Design Practice
Improve Product Design
3. Research Aims & Objectives
➤ To Investigate Key Constructs
Extend the Cognitive Models (e.g., Interaction, Immersion, Flow,
within Presence, Aesthetics, Emotion &
User Experience Individual Dispositions).
Develop Research ➤ To Capture User Experiences (as
Methodologies they occur).
Explore
Prolonged Patterns of ➤ To Capture UX Patterns (over a
Interaction prolonged time frame)
Inform ➤ To Inform Design Guidelines (by
Design determining what interactive
Guidelines features elicit positive UX)
4. Research Questions
RQ1: What are the key constructs that contribute
to user engagement and user experience?
RQ2: What design protocols or features lead to
positive user experiences?
RQ3: What are the patterns of interaction over
prolonged use?
5. Research Plan & Methodology
STUDY 1 STUDY 2 STUDY 3
Quasi- Controlled Longitudinal
Experiment Experiment Study
Diary X-Section
Test Adapted Applied Study Survey
Method Survey Method Method
METHODS
METHODS Retrospect
Interview
Observation Interview
6. Research Plan (1)
Study One: Quasi Experiment
Compare 3 different levels of interactive protocols
Aesthetics & Usability
Interaction & Engagement (Flow & Presence)
Individual Aesthetic Disposition
Develop Research Methodology
Survey Instrument
Observation – Critical Incidents
Interval Mood Scales (iScale*)
Video - Facial Expression
RSQ 1: What are the key constructs that contribute to user experience?
*[Karapanos, Martens & Hassenzahl, 2009]
10. Research Plan (2)
Study Two: Controlled Experiment
Compare 2-3 different levels of interactive protocols
Bespoke Technologies
Same Content V Different Interactive Features (e.g. video, presence,
virtual, etc)
Research Methodology (from Study One)
Tested Survey Instrument
Observation – Critical Incidents
Interval Mood Scales (iScale)
Video – Facial Expression
Focus on Key Design Features
Interactive Guide (Presence)
Virtual Environment (Flow & Engagement)
RSQ 2: What design ‘features’ lead to positive user experiences?
11. Research Plan (3)
Study Three: Longitudinal Study
Compare 2-3 different levels of interactive protocols
Existing technologies (within same domain)
Prolonged Time Frame
Capturing patterns of prolonged user experiences
Adapted Research Methodology
Diary Based Study
Experience Probes
Adapted Mood Scales (iScale)
Two Phased Survey
Staged Interviews
Selected Observation
RSQ 3: What are the patterns of interaction over prolonged use?
12. Research Technologies: Personal Fitness Tools
Classic Text Based Style Video Based Interaction Style
Interactive Customization Facility Active (Haptic) Interactive Style
14. References
Hassenzahl, M. (2003) The thing and I: understanding the relationship between user and product. In M.Blythe, C.
Overbeeke, A. F Monk, & P C. Wright (Eds.), Funology: From Usability to Enjoyment, Dordrecht: Kluwer Academic
. .
Publishers, pp. 31-42.
Csikszentmihalyi, M. (2002) Flow: The Classic Work on How to Achieve Happiness, London: Rider.
Hartmann, Sutcliffe, & De Angeli, (2008) Towards a Theory of User Judgement of Aesthetics and User Interface
Quality. ACM Transactions on Computer-Human Interaction, 15(4), 15-30.
Hassenzahl, M. (2003) The thing and I: understanding the relationship between user and product. In M.Blythe, C.
Overbeeke, A. F Monk, & P C. Wright (Eds.), Funology: From Usability to Enjoyment, Dordrecht: Kluwer Academic
. .
Publishers, pp. 31-42.
Hassenzahl, M. (2004) The Interplay of Beauty, Goodness, and Usability in Interactive Products, Dordrecht, Human
Computer Interaction (19) 4, 319-349.
Jordan, P W. (2002) Designing Pleasurable Products: An Introduction to the New Human Factors, CRC Press.
.
Karapanos, E, Martens, J, B., & Hassenzahl, M. (2009) Reconstructing Experiences through Sketching, ACM TOCHI.
Law, E. L., Roto, V., Hassenzahl, M., Vermeeren, A. P O., & Kort, J. (2009) Understanding, scoping and defining user
.
experience: A survey approach, Proceedings: CHI 2009, Botson MA, pp. 710-728.
Leong, T. W., Vetere, F. & Howard, S. (2005) The Serendipity Shuffle Proceedings of the 19th conference of the computer-
interaction special group (CHISIG), Citizens online: considerations for today and the future, OZCHI 2005, Canberra,
Australia.
Nielson, J. (1993) Usability Engineering, Morgan Kaufmann, San Francisco.
Norman, D. A. (2004) Emotional Design: Why we love (or hate) everyday things, New York: Basic Books.
Shen, K. N., & Khalifa, M. (2008) Exploring Multidimensional Conceptualizations of Social Presence in the Context of
Online Communities, Intl. Journal of Human-Computer Interaction, 24(7), 722-748.
Sutcliffe, (2010), Designing for User Engagement: Aesthetic and Attractive User Interfaces. Edited by Carroll, J.M,
Synthesis Lectures on Human-Centred informatics, Morgan & Claypool Publishers, Chapter 1.
Tractinsky, N., Katz, A. S. & Ikar, D. (2000) What is beautiful is usable, Interacting with Computers, 13 (2), 127-145.
Tractinscky and Zmiri, (2006) Exploring Attributes of Skins as Potential Antecedents of Emotion in HCI, In: Fishwick,
P (ed.) Aesthetic Computing, MIT Press. Cambridge.
,
Wright, P & McCarthy, J. (2009) Experience-Centered Design: Designers, Users, and Communities in Dialogue,
.,
Synthesis Lecturers on Human-centered Informatics, Morgan & Claypool Publishers.