Users often access and re-access more than one site during an online session, effectively engaging in multitasking. In this paper, we study the effect of online multitasking on two widely used engagement metrics designed to capture users browsing behavior with a site. Our study is based on browsing data of 2.5M users across 760 sites encompassing diverse types of services such as social media, news and mail. To account for multitasking we need to redefine how user sessions are represented and we need to adapt the metrics under study. We introduce a new representation of user sessions: tree-streams - as opposed to the commonly used click-streams - present a more accurate picture of the browsing behavior of a user that includes how users switch between sites (e.g., hyperlinking, teleporting, backpaging). We then discuss a number of insights on multitasking patterns, and show how these help to better understand how users engage with sites. Finally, we define metrics that characterize multitasking during online sessions and show how they provide additional insights to standard engagement metrics.

1. ONLINE MULTITASKING
AND USER ENGAGEMENT
CIKM 2013
Jane%e
Lehmann
In
collabora*on
with:
Mounia
Lalmas,
Ricardo
Baeza-­‐Yates,
George
Dupret

2. OUTLINE
1. Mo%va%on
2. Characteris%cs
of
online
mul%tasking
Ac2vity
during
and
between
visits
3. Measuring
online
mul%tasking
Defini2on
of
new
metrics,
case
study
Lights
on
by
JC*+A!
How
do
users
browse
the
web
today?

3. leC
by
[
embr
]
How
do
users
browse
the
Web
today?

4. ONLINE MULTITASKING
Browsing
the
“old
way”
1min
facebook
news
2min
news
1min
news
3min
news
mail
news
site
Dwell
2me
during
a
visit
on
a
news
site:
7min
on
average
JaneGe
Lehmann
Mo2va2on
4

5. ONLINE MULTITASKING
Nowadays
1min
news
2min
facebook
news
3min
1min
news
mail
news
Dwell
2me
during
a
visit
on
a
news
site:
2.33min
on
average
(1min
|
3min
|
3min)
JaneGe
Lehmann
Mo2va2on
5

6. ONLINE MULTITASKING
• Users
switch
between
sites,
to
do
related
or
totally
unrelated
tasks
• E.
Herder
[1]:
» 75%
of
sites
are
visited
more
than
once
» 74%
of
revisits
are
performed
within
a
session
Measuring
browsing
behavior
can
lead
to
incorrect
conclusions.
[1]
E.
Herder.
Characteriza*ons
of
user
web
revisit
behavior.
In
LWA,
2005.
JaneGe
Lehmann
Mo2va2on
6

7. Danboard's
Messy
Home
by
Mullenkedheim
Characteris%cs
of
online
mul%tasking

8. DATA SET
Interac%on
data
• July
2012
• 2.5M
users
• 785M
page
views
• We
defined
a
new
naviga2on
model
(see
paper
for
detail)
• Categoriza2on
of
the
most
frequent
accessed
sites
(e.g.
mail,
news,
shopping)
» 11
categories
(news),
33
subcategories
(e.g.
news
finance,
news
society)
» 760
sites
from
70
countries/regions
JaneGe
Lehmann
Characteris2cs
8

9. Visit activity
Visit
frequency
#Visits
(avg sd)
news (finance)
news (tech)
social media
mail
JaneGe
Lehmann
2.09
1.76
2.28
2.09
4.65
1.59
4.78
4.61
Mul%tasking
depends
on
the
site
under
considera%on
• Social
media
sites
are
revisited
the
most
• News
(tech)
sites
are
the
least
revisited
sites
Characteris2cs
9

10. Visit activity
Ac%vity
between
visits
Cumulative probability
Differences
in
the
absence
%me
• 50%
of
sites
are
revisited
aCer
less
than
1min
-­‐
Interrup*on
of
a
task
1.00
0.75
0.50
news (finance)
news (tech)
social media
mail
0.25
0.00
10
2
10
1
10 0
10 1
10 2
• There
are
revisits
aCer
a
long
break
-­‐
Returning
to
a
site
to
perform
a
new
task
Absence time [min]
v1
*
v2
*
v3
*
-­‐
absence
2me
JaneGe
Lehmann
Characteris2cs
10

11. Visit activity
Ac%vity
paLern
Proportion of total
dwell time on site
decreasing attention
mail sites
0.33 p-value = 0.09
m = -0.01
social media sites
• Four
types
of
"aGen2on
shiCs”
p-value = 0.07
m = -0.02
0.28
0.23
constant attention
news (finance) sites
Proportion of total
dwell time on site
increasing attention
complex attention
• Complex
cases
refer
to
no
specific
paGern
or
repeated
paGern
news (tech) sites
0.33 p-value = 0.79
m = 0.00
• Successive
visits
can
belong
together
(i.e.,
to
the
same
task)
0.28
0.23
JaneGe
Lehmann
Characteris2cs
11

12. Danboard
by
sⓘndy°
Measuring
online
mul%tasking

13. Cumulative activity
Cumula%ve
ac%vity
CumActm,k
n
= log10 (v1 + ∑ ivik • vi )
i=2
v1
iv2
v2
iv3
vi
v
3
ivi
Browsing
ac2vity
between
the
(i-­‐1)th
and
ith
visit
k=3
Rescaling
factor
for
ivi
m
Browsing
ac2vity
(e.g.
dwell
2me,
page
views)
Browsing
ac2vity
during
the
ith
visit
Assump%on:
If
users
return
aCer
short
2me,
they
return
to
con2nue
with
same
task.
If
users
return
aCer
longer
2me,
they
return
to
perform
a
new
task
-­‐
an
indica2on
of
loyalty
to
the
site.
JaneGe
Lehmann
Metrics
13

14. Activity pattern
ALen%on
shiN
and
range
invm,n − min Invm,n
AttShiftm,n =
| max Invm,n | − | min Invm,n
|
σ (Vm,n )
AttRangem,n =
µ (Vm,n )
n=4
Number
of
visits
in
session
σ
μ
inv
Variance
in
the
visit
ac2vity
Average
of
the
visit
ac2vity
Modifica2on
of
the
“Inversion
number”
Descrip%on:
AGShiC
models
the
shiC
of
aGen2on
in
the
browsing
ac2vity
AGRange
describes
fluctua2ons
in
the
browsing
ac2vity
JaneGe
Lehmann
Metrics
14

15. Activity pattern
ALen%on
shiN
and
range
ARen*on
shiS
ARen*on
range
-­‐1
JaneGe
Lehmann
0
1
constant
constant
constant
decreasing
complex
increasing
0
>
0
Metrics
15

16. Comparing metrics
Comparing
the
ranking
of
the
sites
• Visitdt
–
Dwell
2me
during
a
visit
• Sessiondt
–
Dwell
2me
during
a
session
Visitdt
Sessiondt
CumActdt
ALShiNdt
Sessiondt
0.57
CumActdt
-­‐0.04
0.24
ALShiNdt
0.09
0.22
0.02
ALRangedt
-­‐0.01
-­‐0.01
-­‐0.26
0.19
Ø Visitdt
and
Sessiondt
correlate
Ø Otherwise
no
correla2on
à
the
other
metrics
capture
different
aspects
of
browsing
behavior
JaneGe
Lehmann
Metrics
16

17. Models of browsing behavior
“Models”
of
browsing
behavior
•
Clustering
of
sites
using
mul2tasking
and
standard
engagement
metrics:
• CumActdt,
AGShiCdt,
AGRangedt
• Visitdt,
Sessiondt
•
We
iden2fied
five
cluster:
C1: 172 sites
C2: 108 sites
C3: 156 sites
C4: 74 sites
C5: 166 sites
0.75
0.75
0.75
0.75
0.75
0.25
0.25
0.25
0.25
0.25
-0.25
-0.25
-0.25
-0.25
-0.25
-0.75
-0.75
-0.75
-0.75
-0.75
JaneGe
Lehmann
Visitdt [min]
Sessiondt [min]
CumActdt,3
Metrics
AttShiftdt,4
AttRangedt,4
17

18. Models of browsing behavior
C1: 172 sites
C2: 108 sites
mail, maps, news,
news (soc.)
auctions, front page,
shopping, dating
0.75
0.75
0.25
0.25
-0.25
-0.25
-0.75
-0.75
Visitdt [min]
JaneGe
Lehmann
Sessiondt [min]
One
task
during
a
session
§ High
dwell
2me
per
visit
and
during
the
whole
session
§ Users
return
to
con2nue
a
task
(short
absence
2me)
§ C1:
aGen2on
is
shiCing
to
another
site
§ C2:
aGen2on
is
shiCing
slowly
towards
the
site
CumActdt,3
Metrics
AttShiftdt,4
AttRangedt,4
18

19. Models of browsing behavior
C3: 156 sites
C4: 74 sites
auctions, search,
front page, shopping
front page, search,
download
Several
tasks
during
a
session
§ Users
perform
several
tasks
on
these
sites
during
a
session
0.75
0.75
§ No
simple
ac2vity
paGern
0.25
0.25
-0.25
-0.25
-0.75
-0.75
Visitdt [min]
JaneGe
Lehmann
Sessiondt [min]
§ C3:
Dwell
2me
per
visit
is
low,
but
the
dwell
2me
per
session
is
high
CumActdt,3
Metrics
AttShiftdt,4
AttRangedt,4
19

20. Models of browsing behavior
C5: 166 sites
service, download,
blogging, news (soc.)
0.75
0.25
-0.25
Sites
with
low
ac%vity
§ Users
do
not
spend
a
lot
of
2me
on
these
sites
§ Time
between
visits
is
short
§ AGen2on
is
shiCing
towards
the
site
-0.75
Visitdt [min]
JaneGe
Lehmann
Sessiondt [min]
CumActdt,3
Metrics
AttShiftdt,4
AttRangedt,4
20

21. Models of browsing behavior
C2: 108 sites
auctions, front page,
shopping, dating
C3: 156 sites
auctions, search,
front page, shopping
0.75
0.75
0.25
0.25
-0.25
-0.25
-0.75
Browsing
behavior
can
differ
between
sites
of
the
same
category
§ C2:
users
visit
site
once
to
perform
their
task
-0.75
Visitdt [min]
JaneGe
Lehmann
Sessiondt [min]
§ C3:
users
visit
site
several
2mes
to
perform
task(s)
CumActdt,3
Metrics
AttShiftdt,4
AttRangedt,4
21

22. SUMMARY and Future Work
• Online
mul2tasking
affects
the
way
users
access
sites
–
Standard
metrics
do
not
capture
this!!!
• We
defined
metrics
that
describe
different
aspects
of
mul2tasking
• CumAct
accounts
for
the
2me
between
visits
• AGShiC,
AGRange
describe
aGen2on
shiCs
• We
showed
that
mul2tasking
depends
on
the
site
under
considera2on
Future
work:
• Can
we
improve
the
defini2on
of
a
task?
• How
does
mul2tasking
affect
other
metrics,
such
as
bounce
rate
and
click-­‐
through
rate?
• Does
mul2tasking
differ
in
different
countries?
JaneGe
Lehmann
Summary
22

23. Ques%ons?
Online
Multitasking
+
User
Engagement
JaneGe
Lehmann
Universitat
Pompeu
Fabra,
Spain
lehmannj@acm.org
Mounia
Lalmas
Yahoo
Labs
London
mounia@acm.org
George
Dupret
Yahoo
Labs
Sunnyvale
gdupret@yahoo-­‐inc.com
Ricardo
Baeza-­‐Yates
Yahoo
Labs
Barcelona
rbaeza@acm.org