NeuroIS Research Progress and Future Directions

PIERRE-MAJORIQUE LÉGER, Ph.D.
NSERC-Prompt Industrial Research Chair in User Experience
Co-director Tech3Lab, Director ERPsim Lab
HEC Montréal
Smith School of Business, Queen’s University, March 2021.
NECESSITY IS THE MOTHER OF INVENTION :
Enabling remote measurement
of user psychophysiological
reactions in HCI research

© Copyright Léger (2021)
Pierre-Majorique Léger, Ph.D.
Lab Serious games to learn
enterprise systems and
business analytics
https://www.researchgate.net/profile/Pierre-Majorique_Leger
NSERC-Prompt
Industrial Research Chair
in User Experience
Director ERPsim Lab
Co-direct0r Tech3Lab
pml@hec.ca
GRADUATE STUDIES
POST-DOCTORATE
LABORATORIES FINANCIAL SUPPORT
GUEST PROFESSOR

Marc Fredette, Ph.D.
Data Sciences
Sylvain Sénécal, Ph.D.
Marketing
Information technologies

▪ Electroencephalography

▪ Functional near-infrared spectroscopy

▪ Electrocardiogram

▪ Electrodermal activity

▪ Eye tracking

▪ Facial expression analysis
STATE OF THE ART
NEUROPHYSIOLOGICAL TOOLS

Industrial research partners
from a wide range of industries
INSURANCE
ONLINE GROCERY
MEDIA
RAILWAYS
BANKING
AERONAUTICS
LOGISTICS
FINANCE
r partenaire de
PLUSIEURS BOURSES DE RECHERCHE
À TOUS LES NIVEAUX :
chaire_ux.hec.ca
INSCRIVEZ-VOUS COMME
PARTICIPANTS À NOS ÉTUDES :
panel.hec.ca
Organismes subventionnaires :
partenaire de

est fier partenaire de
PLUSIEURS BOURSES DE RECHERCHE
À TOUS LES NIVEAUX :
chaire_ux.hec.ca
INSCRIVEZ-VOUS COMME
PARTICIPANTS À NOS ÉTUDES :
panel.hec.ca
a proud partner of
Powered by: Financial support:

UX Chair
& Tech3Lab :
Scientific staff

UX Chair Summer
school 2020
Lab Serious games to learn
enterprise systems and
business analytics

KEEPING HUMANS IN THE AI LOOP
Self regulating sustained
attention helps keep
operators engaged over
long periods
We report results of a study that utilizes a neuroadap-
tative system to drive an interactive interface counter-
measure that allows users to self-regulate sustained
attention while performing an ecologically valid, long-
duration business logistics task. The results suggest
that providing a means to self-regulate sustained
attention has the potential to keep operators engaged
over long periods, and moderately increase on-task
performance while decreasing on-task error.
Karran, A. J., Demazure, T., Leger, P. M., Labonte-LeMoyne, E., Senecal,
S., Fredette, M., & Babin, G. (2019). Towards a hybrid passive BCI for the
modulation of sustained attention using EEG and fNIRS. Frontiers in hu-
man neuroscience, 13, 393.
Demazure, Karran, Léger, Labonte-LeMoyne, Senecal, Fredette, Babin
(2021). « Enhancing sustained attention: A pilot study on the integration
of a brain-computer interface with an enterprise information system »,
Business & Information Systems Engineering.
TEAM:
Alexander J. Karran, Ph.D.
Théophile Demazure
Pierre-Majorique Leger, Ph.D.
Elise Labonte-LeMoyne, Ph.D.
Sylvain Senecal, Ph.D.
Gilbert Babin, Ph.D.

In the land of zombies:
Your mind stays in the
phone even when
you take your eyes off it.
AN IMPORTANT COGNITIVE EFFORT IS
NECESSARY TO SHARE ATTENTION BETWEEN
THE APPLICATION AND THE EXTERNAL STIMULI.
THE GREATER THE EFFORT, THE MORE THE
PEDESTRIAN MAKES ERRORS OF JUDGMENT.
Courtemanche, Francois, Elise Labonté-LeMoyne, Pierre-Majorique Léger,
Marc Fredette, Sylvain Senecal, Ann-Frances Cameron, Jocelyn Faubert, and
François Bellavance. “Texting while walking: an expensive switch cost.” Acci-
dent Analysis & Prevention 127 (2019): 1-8.
TEAM:
Pierre-Majorique Léger, Ph.D. (PI)
Ann-Frances Cameron, Ph.D.
Francois Courtemanche, Ph.D.
Jocelyn Faubert,Ph.D.
Elise Labonté-Lemoyne, Ph.D.
Franco Lepore, Ph.D.

ÉQUIPE :
Pierre-Majorique Léger, Ph.D. (PI)
Ann-Frances Cameron, Ph.D.
Jocelyn Faubert,Ph.D.
Elise Labonté-Lemoyne, Ph.D.
Franco Lepore, Ph.D.
Gabrielle Mourra, M. Sc.
Using your smartphone
while walking: Users who
are more dependent on
the phone have a higher
risk of accident.
USING YOUR PHONE WHILE WALKING IS
UNDENIABLE DANGEROUS, IT'S EVEN MORE
FOR ABUSIVE USERS!
Mourra, G. N., Sénécal, S., Fredette, M., Lepore, F., Faubert, J.,
Bellavance, F., ... & Léger, P. M. (2020). Using a smartphone while
walking: The cost of smartphone-addiction proneness. Addictive
Behaviors, 106346.
Gabrielle Mourra
Boursière CRSH

Using an active desktop
does not affect performance
cognitive users
OUR RESULTS SUGGEST THAT NO MATTER THE
LEVEL OF DIFFICULTY OF THE TASK, THE ACTIVE
OFFICE DOES NOT AFFECT NEGATIVELY THE
PERFORMANCE OF THE USER.
Labonte-LeMoyne, E., Jutras, M.-A., Senecal, S., Léger, P.-M., Fredette, M., Begon,
M., and Mathieu, M.-E. (2019). Does reducing sedentarity with active desks hinder
cognitive performance? - A study by the FIT24 NETWORK. Human Factors:
The Journal of the Human Factors and Ergonomics Society.
TEAM:
Marie-Eve Mathieu (PI) (UdeM)
Marc Fredette, Ph.D. (PI)
Elise Labonte-Lemoyne, Ph.D.
Michael Begon, Ph.D. (UdeM)
Marc-Antoine Jutras (M.Sc.)
Marc-Antoine Jutras
Boursier CRSNG

1 2 3 4
Why
UX evaluation?
Why
UX evaluation
with neurosciences ?
How to evaluate UX
with neurosciences
during COVID ?
What are the
new opportunities ?
A B
?

Immobilier.ca
Please
test!

Immobilier.ca

Immobilier.ca
Testez-le
svp!
Ease
of use?

Immobilier.ca
3 of 5
Testez-le
svp!
Ease
of use?

Immobilier.ca
Testez-le
svp!
?
?
?
?
?
?
?
?

1 2 3 4
Why
UX evaluation?
Why
UX evaluation
How to evaluate UX
with neurosciences
during COVID ?
What are the
new opportunities ?
A B
?
2
Why
UX evaluation

?
Conclusion?

NeuroIS:
The Basic Idea
IT Behavior
BIOLOGY

▪ Body physiology

▪ Brain anatomy & functionality

▪ Hormones

▪ Genes

Traditional Approach
EXAMPLE: SHOPPING BEHAVIOR
INDEPENDENT VARIABLE DEPENDENT VARIABLE
e.g., perceived
trustworthiness of ebay offer
or
purchase intention

NeuroIS Approach
EXAMPLE: SHOPPING BEHAVIOR
IT ARTIFACT
(e.g., GUI)
MEDIATOR VARIABLE
IS RELEVANT VARIABLES

“NeuroIS is a subfield in the IS literature that relies on neuroscience and neurophysiological
theories and tools to better understand the development, use, and impact of information
technologies (IT). NeuroIS seeks to contribute to (i) the development of new theories
that make possible accurate predictions of IT-related behaviors,
and (ii) the design of IT artifacts that positively affect economic
and non-economic variables (e.g., productivity, satisfaction,
adoption, well being).”
What is NeuroIS?
Riedl, R. et al. (2010): On the foundations of NeuroIS: Reflections on the Gmunden Retreat 2009. Communications of the Association for Information Systems, 27, 243-264.

MAIN DATES AND DEADLINES

▪ Submission deadline:
March 1, 2021

▪ Notification about acceptance:
April 1, 2021

▪ Submission of final paper version:
April 15, 2021

▪ Retreat:
June 1-3, 2021 (Virtual)

A decade of NeuroIS
research: progress,
challenges, and future
directions
NEUROIS RESEARCH AGENDA
RIEDL, René, FISCHER, Thomas, LÉGER, Pierre-Majorique, et al.
A decade of NeuroIS research: progress, challenges, and future
directions. ACM SIGMIS Database: the DATABASE for Advances
in Information Systems, 2020, vol. 51, no 3, p. 13-54.
A Decade of NeuroIS
Research: Progress,
Challenges, and
Future Directions
René Riedl
University of Applied Sciences Upper Austria
& University of Linz
Thomas Fischer
University of Applied Sciences Upper Austria
Pierre-Majorique Léger
HEC Montréal
Fred D. Davis
Texas Tech University
Acknowledgments
The authors thank the editors, Stacie Petter and Tom
Stafford, for their excellent work in providing guidance
on ways to improve the paper during the review
process. Also, we acknowledge the support of Shirley
Gregor, Benjamin Mueller, and Anand Gopal who
served as chairs of the track “Research Methods,
Theorizing, and Philosophy” at the 2017 International
Conference on Information Systems (ICIS, Seoul),
where an earlier version of this paper was presented,
and we also thank the ICIS conference participants
who provided useful comments. Finally, we thank the
participants of the NeuroIS Retreat 2018 (Vienna),
who also provided useful comments on an earlier
version of this paper.
Abstract
NeuroIS is a field in Information Systems (IS) that
makes use of neuroscience and neurophysiological
tools and knowledge to better understand the
development, adoption, and impact of information and
communication technologies. The fact that NeuroIS
now exists for more than a decade motivated us to
comprehensively review the academic literature.
Investigation of the field’s development provides
insights into the status of NeuroIS, thereby
contributing to identity development in the NeuroIS
field. Based on a review of N=200 papers published in
55 journals and 13 conference proceedings in the
period 2008-2017, we addressed the following four
research questions: Which NeuroIS topics were
investigated? What kind of NeuroIS research was
published? How was the empirical NeuroIS research
conducted? Who published NeuroIS research? Based
on a discussion of the findings and their implications
for future research, which considers results of a recent
NeuroIS survey (N=60 NeuroIS scholars), we
conclude that today NeuroIS can be considered an
established research field in the IS discipline. However,
our review also indicates that further efforts are
necessary to advance the field, both from a theoretical
and methodological perspective.
Keywords: Brain; Information Systems; Literature
Review; NeuroIS; Neuroscience.
Introduction
NeuroIS is a field in Information Systems (IS) that
makes use of tools and knowledge that are developed
and/or applied in neuroscience to better understand
the development, adoption, and impact of information
and communication technologies (Dimoka et al., 2012;
Dimoka, Pavlou, & Davis, 2007, 2011; Riedl, Banker
et al., 2010; Riedl, Davis, Banker, & Kenning, 2017).
Neuroscience is a field that deals with the study of the
brain and the nervous system, using measures of
neurophysiological processes such as functional
magnetic resonance imaging, fMRI, or
electroencephalography, EEG, or other tools related
to measurement of nervous system activity such as
electrodermal activity, EDA, or heart rate, HR (Society
for Neuroscience, 2019). It follows that NeuroIS
research comprises not only studies that apply
neurophysiological tools but also research that utilizes
theories and concepts from neuroscience to support
the four main goals of IS research: namely, to
describe, explain, and predict IS phenomena and to
design IS artifacts (Riedl & Léger, 2016). The
complementary aspect of neuroscience tools and
knowledge is neatly summarized by Riedl et al. (2018):
“The advent of neuroscience tools and theories allows
IS research to integrate biological factors, in particular
The DATA BASE for Advances in Information Systems 13 Volume 51, Number 3, August 2020

Advancing a NeuroIS
research agenda with
four areas of societal
contributions
NEUROIS RESEARCH AGENDA
VOM BROCKE, Jan, HEVNER, Alan, LÉGER, Pierre Majorique, et al.
Advancing a NeuroIS research agenda with four areas of societal
contributions. European Journal of Information Systems, 2020,
vol. 29, no 1, p. 9-24.
ISSUES AND OPINION
Advancing a NeuroIS research agenda with four areas of societal contributions
Jan vom Brockea
, Alan Hevnerb
, Pierre Majorique Légerc
, Peter Wallad
and René Riedle
a
Institute of Information Systems, University of Liechtenstein, Vaduz, Liechtenstein; b
Information Systems and Decision Sciences
Department, University of South Florida, Tampa, FL, USA; c
Department of Information Technologies, HEC Montréal, Montreal, Canada;
d
Institute for Information Engineering, Webster Vienna Private University, Vienna, Austria; e
University of Applied Sciences Upper Austria &
Johannes Kepler University Linz, Austria
ABSTRACT
On the 10th anniversary of the NeuroIS field, we reflect on accomplishments but, more
importantly, on the future of the field. This commentary presents our thoughts on a future
NeuroIS research agenda with the potential for high impact societal contributions. Four key
areas for future information systems (IS) research are: (1) IS design, (2) IS use, (3) emotion
research, and (4) neuro-adaptive systems. We reflect on the challenges of each area and
provide specific research questions that serve as important directions for advancing the
NeuroIS field. The research agenda supports fellow researchers in planning, conducting,
publishing, and reviewing high impact studies that leverage the potential of neuroscience
knowledge and tools to further information systems research.
ARTICLE HISTORY
Received 13 July 2018
Accepted 25 November 2019
ACCEPTING EDITOR
Pär Ågerfalk
ASSOCIATE EDITOR
Michel Avital
KEYWORDS
NeuroIS; Research Agenda;
Emotion Sensing Technology;
Design Science Research;
Neuro-adaptive Systems
1. Introduction
At the 2007 ICIS (International Conference on
Information Systems), Dimoka, Pavlou, and Davis
(2007) coined the term “NeuroIS” and initiated –
together with other scholars who presented research at
the nexus of IS and neurobiology in the context of the
2007 ICIS conference – a new subfield by “applying
cognitive neuroscience theories, methods, and tools in
Information Systems (IS) research.” (For a description
of the genesis of NeuroIS, see Riedl and Léger (2016, pp.
73–74)). In NeuroIS studies, neurophysiological data
are typically collected in combination with self-
reported data to study existing systems’ use and impact,
as well as to inform the design of new systems; hence
contributing to both behavioural and design-oriented
IS research (Dimoka et al., 2012; Loos et al., 2010; Riedl
et al., 2010). In this new strategy of inquiry, researchers
use data from the human body to measure the effects of
human interactions with technology more directly;
revealing the mechanisms underlying human beha-
viour, particularly affective and other non-conscious
processes (Dimoka, Pavlou, & Davis, 2011; Riedl &
Léger, 2016; vom Brocke & Liang, 2014). The influence
of non-conscious processes on human beliefs, attitudes,
intentions, and behaviour is well documented in the
field of neuroscience where empirical research has
shown the effectiveness of physiological measurement
tools (e.g. Rugg et al., 1998). Even human self-awareness
has been shown to consist of non-conscious aspects
(Walla, Greiner, Duregger, Deecke, & Thurner, 2007)
and, thus, it seems logical to apply neuroscience tools to
investigate constructs relevant to IS research.
Early studies in NeuroIS demonstrate both beha-
vioural science and design science research objectives.
Dimoka (2010) investigates the concepts of trust and
distrust in IS. The study shows that trust and distrust
are associated with separate brain areas (trust with the
striatum and distrust with the amygdala and the insula),
challenging the previous understanding of trust and dis-
trust as the two ends of one construct. Based on neu-
roscience evidence, today it is an established fact that
trust and distrust are two separate constructs (Riedl &
Javor, 2012). To improve IS design, vom Brocke, Riedl,
and Léger (2013) identify three strategies with which to
apply neuroscience in design science research (DSR) by
adapting existing neuroscience theory to inform IS design
(without using neuroscience tools), using neuroscience
tools to evaluate IS design, and applying neuroscience
theory and tools to develop neuro-adaptive IS (i.e. sys-
tems that automatically adapt in real time based on users’
neurophysiological states to improve human-computer
interactions). For a concrete example, see Astor, Adam,
Jerčić, Schaaff, and Weinhardt (2013).
Over the past decade, the field of NeuroIS has devel-
oped rapidly and has made major achievements.
Foundational papers, such as those of Riedl et al. (2010)
and Dimoka et al. (2012), provide important conceptual
groundwork and help conceptualise NeuroIS as
a discipline. Annual events attract eager researchers,
such as the NeuroIS Retreat, which began in 2009 as an
CONTACT Jan Vom Brocke jan.vom.brocke@uni.li
This article has been republished with minor changes. These changes do not impact the academic content of the article.
EUROPEAN JOURNAL OF INFORMATION SYSTEMS
2020, VOL. 29, NO. 1, 9–24
https://doi.org/10.1080/0960085X.2019.1708218
© 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-
nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or
built upon in any way.

Immobilier.ca
COGNITION
ATTENTION
EMOTIONS
1 gig/hour
ENCYCLOPEDIA
ENCYCLOPEDIA
ENCYCLOPEDIA
ENCYCLOPEDIA
ENCYCLOPEDIA
ENCYCLOPEDIA
ENCYCLOPEDIA
ENCYCLOPEDIA

COGNITION
ATTENTION
EMOTIONS

Tech3Lab
Louis-Collin
Tech3Lab
Côte-Ste-Catherine
Tech3Lab
Beaver Hall (2022)
1 1
1
3 3
5 5
2 2
2
4 4
6

EMOTIONAL AROUSAL
Calm Excited

Mesuring emotional
arousal Calm Excited

Mesuring emotional
arousal
Calm Excitd

Mesuring emotional
arousal
Calm Excited

EMOTIONAL VALENCE
Sad Happy

Mesuring emotional
valence Sad Happy

Happy
AU 6. Cheek Raiser
Contributes to the emotion
happiness. Orbicularis oculi
(pars orbitalis) is the underlying
facial muscle.
AU 12. Lip Corner Puller
Contributes to the emotion hap-
piness and contempt when the
action appears unilateraly.
Muscular basis: zygomaticus major.
AU 6 - 12
Contributes to happiness. Notice
the wrinkles around the eyes
caused by cheek raising, also know
as the "Duchenne Marker".
+ =

Surprised
AU 1. Inner Brow Raiser
Contributes to the emotions sad-
ness, surprise, and fear, and to the
affective attitude interest. Muscu-
lar basis: frontalis (pars medialis).
AU 2. Outer Brow Raiser
Contributes to the emotions sur-
prise and fear, and to the affective
attitude interest. Frontalis (pars
lateralis) is the underlying facial
muscle.
AU 1 - 2
Contributes to the emotion sur-
prise and can be recognized by a
smooth line formed by the wrinkles
across the forehead.
+ =

NEUTRAL
HAPPY
SAD
DISCUSTED
ANGRY
SURPRISED
SCARED

NEUTRAL
HAPPY
SAD
ANGRY
SURPRISED
SCARED
DISCUSTED

VALENCE
ÉMOTIONNELLE
Ne sais pas
quoi inscrire
dans barre de
recherche
Inscription
adresse
courriel
Ajout de filtres
pour choix
produits Résultats de
Produits
sans photos
Clic sur
“achat”
Inscription
du No de tel.
recherche non
Aroursal
Arousal
Valence
Identification
des points
de friction
par étape

0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400
Time (sec)
-0,7
-0,6
-0,5
-0,4
-0,3
-0,2
-0,1
0,0
0,1
Activation
Valence
-1,0000
-0,5000
0,0000
0,5000
1,0000
Tâche
Pain Point
Création de compte
Magasinage
Paiement
Sélection du lieu de collecte
Sélection du moment de collecte
Null
1
« Aucun stress pour moi.
J'ai entré ma carte de crédit
et c'était bien. »
AROUSAL
VALENCE
ÉMOTIONNELLE

0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 3200 3400
Time (sec)
-1,0
-0,8
-0,6
-0,4
-0,2
0,0
0,2
0,4
0,6
0,8
1,0
Activation
Valence
-1,000
-0,500
0,000
0,500
1,000
Tâche
Pain Point
Création de compte
Magasinage
Paiement
Sélection du lieu de collecte
Sélection du moment de collecte
Null
1
« La recherche,
ça j’ai aimé ça »
Arousal
VALENCE
ÉMOTIONNELLE

DENSITY
OF FIXATION

Method of a system
for processing
UNITED STATES PATENT
Courtemanche, F., Fredette, M., Senecal, S., Leger, P.M., Dufresne,
A., Georges, V. and Labonte-lemoyne, E., Valorisation Gestion LP and
Valorisation-Recherche LP, 2019. Method of and system for processing
signals sensed from a user. U.S. Patent 10,368,741.
PATENT PROTECTED : U.S. Patent No. 10,368,741

PATENT PROTECTED : U.S. Patent No. 10,368,741

POSITIVES EMOTIONS
NEGATIVE EMOTIONS

Physiological heatmaps: a tool for visualizing
users’ emotional reactions
François Courtemanche1
& Pierre-Majorique Léger2
& Aude Dufresne3
&
Marc Fredette4
& Élise Labonté-LeMoyne1
& Sylvain Sénécal5
Received: 26 January 2017 /Revised: 4 August 2017 /Accepted: 8 August 2017 /
Published online: 22 August 2017
# The Author(s) 2017. This article is an open access publication, Corrected publication September/2017
Multimed Tools Appl (2018) 77:11547–11574
DOI 10.1007/s11042-017-5091-1
* François Courtemanche
francois.courtemanche@hec.ca
1
Tech3lab, HEC Montréal, 3000 Côte Sainte-Catherine, H3T 2A7, Montréal, QC, Canada
2
Department of Information Science, HEC Montréal, 3000 Côte Sainte-Catherine, H3T 2A7 Montréal,
QC, Canada
3
Department of Communication, University of Montréal, 2900 Edouard-Montpetit, H3T 1J4,
Montréal, QC, Canada
4
Departement of Statistics, HEC Montréal, 3000 Côte Sainte-Catherine, H3T 2A7, Montréal, QC,
Canada
5
Departement of Marketing, HEC Montréal, 3000 Côte Sainte-Catherine, H3T 2A7, Montréal, QC,
Canada
Abstract Practitioners in many fields of human-computer interaction are now using physio-
logical data to measure different aspects of user experience. The dynamic nature of physio-
logical data offers a continuous window to the users and allows a better understanding of their
experience while interacting with a system. However, in order to be truly informative,
physiological signals need to be closely linked to users’ behaviors and interaction states. This
paper presents an analysis method that provides a direct visual interpretation of users’
physiological signals when interacting with an interface. The proposed physiological heatmap
tool uses eyetracking data along with physiological signals to identify regions where users are
experiencing different emotional and cognitive states with a higher frequency. The method was
evaluated in an experiment with 44 participants. Results show that physiological heatmaps are
able to identify emotionally significant regions within an interface better than standard gaze
heatmaps. Applications of the method to different fields of HCI research are also discussed.
Keywords Physiological computing . Eyetracking . Affective computing . Heatmap . User
experience
A tool for
visualizing
users’ emotional
reactions
PHYSIOLOGICAL HEATMAPS:
Courtemanche, F., Léger, P. M., Dufresne, A.,
Fredette, M., Labonté-LeMoyne, É., & Sénécal, S.
(2018). Physiological heatmaps: a tool for visual-
izing users’ emotional reactions. Multimedia Tools
and Applications, 77(9), 11547-11574.

The Arithmetic
Complexity of Online
Grocery Shopping
THE SUITABILITY BETWEEN THE PHOTO
OF PRODUCT AND UNIT QUANTITY HELPS
TO REDUCE COMPLEXITY MATHEMATICS
OF ONLINE PURCHASE OF AGRI-FOOD
PRODUCTS.
Desrochers, C., Léger, P., Fredette, M., Mirhoseini, S. and Sénécal, S.
(2019), "The arithmetic complexity of online grocery shopping: the mod-
erating role of product pictures", Industrial Management & Data Systems,
Vol. 119 No. 6, pp. 1206-1222.
TEAM:
Sylvain Sénécal, Ph.D. (PI)
Camilles Desrocher (M.Sc.)
Seyedmohammadmahdi
Mirhoseini (Doctorat)
Seyedmohammadmahdi
Mirhoseini
Boursier
CRSNG-PROMPT-Sobeys

1 2 3 4
Why
UX evaluation?
Why
UX evaluation
How to evaluate UX
with neurosciences
during COVID ?
What are the
new opportunities ?
A B
?
3
How to evaluate UX
with neurosciences
during COVID ?

Remote UX testing
PARTICIPANT'S SCREEN MODERATOR PARTICIPANT

Remote UX testing
PARTICIPANT SCREEN INSTRUCTION PARTICIPANT

Participant recruitment Préparation d’étude Tech3Lab
Prérequis de participation
Navigateur internet
Chrome
Peut être téléchargé et installé
à l’aide de ce lien :
https://www.google.com/chrome/
1
Microphone connecté
ou intégré à l’ordinateur
2
Webcam connectée
et intégrée à l'ordinateur
3
4
Espace de travail isolé
Espace de travail isolé
(pour réduire le bruit et la présence
d’autres personnes pendant le test)
Une bonne luminosité
(permettant de voir votre visage
lors d’enregistrement vidéo)
Pas de lunettes
Ne pas avoir besoin de lunettes
de vue pour travailler à l’ordinateur
5 6
Accès Wi-Fi
Connectez-vous
sur le réseau 5Ghz de votre Wifi,
si cela est possible.

Participant set-up Préparation d’étude Tech3Lab
À faire la veille du test
Vérifier votre WebCam et microphone
3
Lire et compléter le formulaire de consentement
1
Vérifier votre connection à internet ou au Wi-Fi
2
Installer l’extension Lookback Google Chrome
4
1. Dans le courriel de confirmation
de participation, ouvrir le PDF
“Formulaire de Consentement”
2. Lire attentivement le document
au complet, et le compléter en
répondant à toutes les questions.
3. Envoyer le formulaire de
consentement complété
à panel.admin@hec.ca
1. Si vous êtes connecté au WIFI,
et que vous avez 2 réseaux
disponibles, sélectionner le 5Ghz.
2. Aller à l'adresse suivante :
http://www.speedtest.netà
et cliquer sur GO
1. Faire un enregistrement vidéo
de 15 sec. dans lequel vous
parlez : https://webcamera.io/fr/
2. Écouter l’enregistrement en vous
assurant que l’image et le son
soient de bonne qualité
3. Valider la qualité du son et
de l'image
1. Ouvrir le navigateur
Google Chrome
2. Installer l'extension Lookback
dans Google Chrome, en cliquant
le lien "Extention LookBack" inclu
dans le courriel de confirmation
3. Accepter l’utilisation de votre
microphone, caméra vidéo, et
du partage d’écran pour participer
3. Vérifier les résultats du test.
Si “Download” et/ou “Upload” est
inférieur à 5 Mbps, contactez les
chercheurs ( panel@hec.ca ).
INSTALL
ACCEPT
2.4Ghz 5Ghz
GO
DOWNLOAD UPLOAD
Mbps Mbps
30.3 10.4
DOWNLOAD UPLOAD
Mbps Mbps
4.7 3.5

Preparation for the test
1
Accès Wi-Fi
Connectez-vous
sur le réseau 5Ghz de votre Wifi,
si cela est possible.
Préparation d’étude Tech3Lab
À faire 15 min. avant le début du test
Fermer les
fenêtres inutiles
Fermer tous les logiciels et
pages web sur votre ordinateur,
sauf ce qui vous donne
accès à vos courriels
2
Attention aux
informations personnelles
Assurez-vous qu’aucune
information personnelle est visible
sur votre bureau d’ordinateur
(par exemple, photos ou documents)
3
Copier le lien LookBack
Copier et coller le lien LookBack
dans Chrome. Le lien est fourni dans
le courriel de confirmation de votre
participation
4
Set up du test
Cliquer
Débuter la session
Restez sur cette page en attendant l’heure prévue de votre participation.
5
6
Ouverture du test dans LookBack
GET STARTED!
1. Entrer votre prénom seulement
2. Entrer votre adresse courriel
( la même qu’utilisée sur le
Panel HEC )
3. Accepter l’utilisation du microphone
4. Accepter l’utilisation de la caméra
5. Accepter l’utilisation de l’enregis-
trement d’écran
pour débuter le set up du test :

Procedure
1 2 3
SET-UP TASK AND QUESTIONNAIRE INTERVIEW

Giroux, F., Léger, P. M., Brieugne, D., Courtemanche, F., Bouvier, F., Chen, S.L., Tazi,
Guidelines for collecting automatic facial expression
detection data synchronized with a dynamic stimulus in
remote moderated user tests
Félix Giroux1
, Pierre-Majorique Léger1,2
, David Brieugne1
, François Courtemanche1
,
Frédérique Bouvier1
, Shang-Lin Chen1
, Salima Tazi1
, Emma Rucco1
, Marc
Fredette1,4
, Constantinos Coursaris1,2
and Sylvain Sénécal1,3
1 Tech3Lab, HEC Montréal, Montréal, Québec, Canada
2
Department of Information Technologies, HEC Montréal, Montréal, Québec, Canada
3
Department of Marketing, HEC Montréal, Montréal, Québec, Canada
4
Department of Decision Sciences, HEC Montréal, Montréal, Québec, Canada
Abstract. Because of the COVID-19 pandemic, telework policies have required many user
experience (UX) labs to restrict their research activities to remote user testing. Automatic Facial
Expression Analysis (AFEA) is an accessible psychophysiological measurement that can be
easily implemented in remote user tests. However, to date, the literature on Human Computer
Interaction (HCI) has provided no guidelines for remote moderated user tests that collect facial
expression data and synchronize them with the state of a dynamic stimulus such as a webpage.
To address this research gap, this article offers guidelines for effective AFEA data collection that
are based on a methodology developed in a concrete research context and on the lessons learned
from applying it in four remote moderated user testing projects. Since researchers have less
control over test environment settings, we maintain that they should pay greater attention to
factors that can affect face detection andor emotion classification prior, during, and after remote
moderated user tests. Our study contributes to the development of methods for including
psychophysiological and neurophysiological measurements in remote user tests that offer
promising opportunities for information systems (IS) research, UX design, and even digital health
research.
Keywords: NeuroIS, User Experience, Remote User Test, Automatic Facial Expression
Analysis, Psychophysiological Data, Human-Computer Interaction
1 Introduction
In Human Computer Interaction (HCI) research, conducting a user experience (UX)
study remotely can improve access to participants and provide an ecologically valid
environment for tests in remote environments such as a person’s living room [1].
Nonetheless, remote user tests are limited in terms of the equipment and measurement
tools that can be used and installed during this distributed setup, preventing scholars
from typically collecting psychophysiological data. Therefore, lab-based user tests are
still preferable as they allow researchers to enrich their understanding of the user’s
experience by triangulating traditional self-reported measures via survey scales and
interviews with psychophysiological measurements such as automatic facial expression
analysis (AFEA) [2, 3]. However, due to the COVID-19 pandemic and the telework
policies that have been introduced as part of the public health response, user experience
Guidelines for
collecting automatic
facial expression in
remote moderated user
test
REMOTE NEUROPHYSIOLOGICAL UX TESTING
Giroux, F., Léger, P. M., Brieugne, D., Courtemanche, F., Bouvier, F.,
Chen, S.L., Tazi, S., Rucco, E., Fredette, M., Coursaris, C., Sénécal,
S.: Guidelines for collecting automatic facial expression detection
data synchronized with a dynamic stimulus in remote moderated
user tests. In: International Conference on Human-Computer Inter-

Distributed Remote Psychophysiological Data Collection
for UX Evaluation: A Pilot Project
Aurélie Vasseur1
, Pierre-Majorique Léger1
, François Courtemanche1
, Elise Labonte-
Lemoyne1, Vanessa Georges1, Audrey Valiquette1, David Brieugne1, Emma Rucco1,
Constantinos Coursaris1, Marc Fredette1, Sylvain Sénécal1
1
HEC Montréal, Montréal QC, Canada
Abstract. User experience (UX) research has been critically impacted by the
recent COVID-19 pandemic and the sanitary restrictions put in place.
Observational or perceptual studies can be adapted remotely with participants
using their own computer and internet access. However, studies based on the
unconscious and automatic physiological states of participants use
neurophysiological measurements that requires highly specific hardware.
Electrodermal activity (EDA) or electrocardiogram (ECG) based studies are
complex to transpose to a remote environment since researchers have no physical
contact with the participants. To address this concern, our research team
previously developed a remote instrument that can collect the EDA and the ECG
activity at the participants’ location through a moderated self-installation of
sensors. We developed a protocol for remote physiological data collection that
we pilot tested with 2 UX studies. After each study, we administered an open-
ended questionnaire regarding the full experience of remote data-collection from
both the moderator’s and the participant’s side. We collected 92 responses total
which provided us with a rich dataset that we analyzed through a thematic
analysis lens in order to uncover the success factors of remote
psychophysiological data collection. Operational support, moderator-participant
collaboration, individual characteristics, and technological capabilities clearly
emerged as drivers for success. This project aimed to develop a rigorous and
contextually relevant protocol for remote physiological data collection in UX
evaluations, train our research team on the developed protocol, and provide
guidance regarding remote physiological data collection activities.
Keywords: User Experience, NeuroIS, Electrodermal Activity, Physiological
Data, Remote Research Methods
1 Introduction
User research in human-centered design is critical because each step of the design
process revolves around the feedback received by potential users (Alvarez et al., 2019).
There are multiple ways to collect data related to the users’ experience during their
interaction with a technology, ranging from quantitative methods (e.g., clickstream
Distributed remote
phychophysiological
data collection :
A Pilot Project
REMOTE NEUROPHYSIOLOGICAL UX TESTING
Vasseur, A., Léger, P.M, Courtemanche, F., Labonte-Lemoyne, E.,
Georges, V., Valiquette, A., Brieugne, D., Rucco, E., Coursaris, C.,
Fredette, M., Sénécal, S. : Distributed Remote Psychophysiological
Data Collection for UX Evaluation : A Pilot Project. In: International
Conference on Human-Computer Interaction (forthcoming).

1 2 3 4
Why
UX evaluation?
Why
UX evaluation
How to evaluate UX
with neurosciences
during COVID ?
What are the
new opportunities ?
A B
?
4
What are the
new opportunities ?

Tech3Lab Bluepanel
20 PARTICIPANT PAR MOIS 4 TESTS PER MONTH = 200$

Day 1
Day 5
Day 9
Day 2
Day 6
Day 10
Day 3
Day 7
Day 11
Day 4
Day 8
Day 12

9h 10h 11h 12h 13h 14h 15h 16h 17h

10h 11h 12h 13h 14h 15h 16h 17h

Guidelines for
remote EEG in teaching
REMOTE NEUROPHYSIOLOGY
Demazure, T., Karran, A., Boasen, J., Léger, P.-M., & Sénécal, S.
(2021). Distributed remote EEG data collection for NeuroIS research:
A methodological framework, International Conference on Human-
Computer Interaction (Forthcoming)
Demazure, T., Karran, A., Léger, P.-M. (2021). Continuing doctoral
student training for NeuroIS and EEG during a pandemic: A distance
hands-on learning syllabus, Information Systems and Neuroscience
[Forthcoming]action (forthcoming)
Distributed remote EEG data collection for NeuroIS
research: A methodological framework
Théophile Demazure1
, Alexander J. Karran1
, Jared Boasen1,2
, Pierre-Majorique Lé-
ger1
, Sylvain Sénécal1
1
HEC Montréal, Montréal, QC, Canada
2 Hokkaido University, Sapporo, Hokkaido, Japan
{theophile.demazure}@hec.ca
Abstract. Remote electroencephalography (EEG) studies offers the exciting op-
portunity to gather data within a participants’ home environment. However, re-
mote EEG data collection trades some internal validity for ecological validity.
When interacting with interfaces or other artifacts in remote settings, neurophys-
iological responses and behaviour may display distinct differences compared to
laboratory studies. We propose a methodological approach composed of several
recommendations and an iterative process framework to support this new avenue
of research. The framework was developed during workshops composed of a di-
verse panel and a literature review of relevant research to complement our dis-
coveries. We highlight and discuss the significant challenges associated with re-
mote EEG data collection, and propose recommendations. We introduce the con-
cept of self-applicability and propose a set of measures to guarantee good signal
quality. Additionally, we offer specific recommendations for research design,
training, and data collection strategies. We offer the iterative process framework
to provide support rigorous data collection, innovative research questions, and
the construction of large-scale datasets from remote EEG studies.
Keywords: electroencephalography, remote experiment, real-world, methodol-
ogy, human-computer interaction.
1 Introduction
The current global pandemic has necessitated developing novel methodologies to sus-
tain ongoing research programs relying on neurophysiological measurements, such as
in the field of NeuroIS and HCI [1, 2]. The critical requirement of social distancing to
reduce COVID-19 transmission severely complicates conventional neurophysiological
experimental methodologies where the researcher is responsible for the installation of
neuro-psychophysiological measurement tools on the research subject. A potential so-
lution to this problem is for researchers to conduct experiments remotely. This type of
solution involves using the Internet to connect with a distributed pool of experimental
subjects who participate from their homes, and who are not technically trained to use

NeuroIS Research Progress and Future Directions

Recommandé

Recommandé

Contenu connexe

Similaire à NeuroIS Research Progress and Future Directions

Similaire à NeuroIS Research Progress and Future Directions (20)

Plus de Pierre-Majorique Léger

Plus de Pierre-Majorique Léger (20)

Dernier

Dernier (20)

NeuroIS Research Progress and Future Directions