Slides from my session for the marketing students at Windesheim College. About why performance matters to your end user, how to measure performance and what to look for when optimizing performance of your website...

1. @jeroentjepkema
Windesheim Almere
Why Performance Matters?

3. Slides: bit.ly/MW-WHA15

4. Performance? Why should we care?

5. #mobilegeddon

6. April 2015

7. December 2015?

8. Slow impacts design

10. 0
25
50
75
100
1 2 3 4 5
Slow Average Fast
%ofrespondentsthatratedtheactualwebsitespeed
Design score (1=bad - 5=beautiful)

11. When users experienced a slow
website, 60% of them rated
design lower

12. (bad) Performance triggers emotions

13. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour/day
Performance complaints patterns per branche/day
Retail100 Travel30
Research from MeasureWork & Social and More. Twitter mentions collected for Twinkle100
webshops between July 1 2014 - July 1 2015
Average:
179 tweets p/hour

14. 21% of social complaints are about
user experience
Research from MeasureWork & Social and More. Twitter mentions collected for Twinkle100
webshops between July 1 2014 - July 1 2015

15. 30 33 36 39 42 45 48 51 2 5 8 11 14 17 20 23 26
Week/Year (2014-2015)
Performance complaints pattern per branche/day
Retail100 Travel30
Normal average:
2398 tweets p/day
Research from MeasureWork & Social and More. Twitter mentions collected for Twinkle100
webshops between July 1 2014 - July 1 2015

16. 30 33 36 39 42 45 48 51 2 5 8 11 14 17 20 23 26
Week/Year (2014-2015)
Performance complaints pattern per branche/day
Retail100 Travel30
Research from MeasureWork & Social and More. Twitter mentions collected for Twinkle100
webshops between July 1 2014 - July 1 2015
Normal average:
2398 tweets p/day
Holiday average:
2897 tweets p/day

17. 18% increase in
performance complaints
during holiday season
Research from MeasureWork & Social and More. Twitter mentions collected for Twinkle100
webshops between July 1 2014 - July 1 2015

18. 30 33 36 39 42 45 48 51 2 5 8 11 14 17 20 23 26
Week/Year (2014-2015)
Performance complaints pattern per branche/day
Retail100 Travel30
Research from MeasureWork & Social and More. Twitter mentions collected for Twinkle100
webshops between July 1 2014 - July 1 2015
Holiday average:
2897 tweets p/day
Complaint top 3:
1. Downtime (36%)
2. Mobile readiness (21%)
3. Slow (19%)
Complaint top 3:
1. Downtime (39%)
2. Slow (29%)
3. Mobile readiness (11%)

19. ...Conversion Impact Score

20. 0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Conversionrate(%)
Category Title
Real User Monitoring: True Conversion Rate
Data collected by MeasureWorks at various e-commerce websites using real user monitoring, reference: http://bit.ly/MW-VEUrum
0%
10%
Competitive Custom B2B

21. Real User Monitoring: True Conversion Rate
Data collected by MeasureWorks at various e-commerce websites using real user monitoring, reference: http://bit.ly/MW-VEUrum
0%
10%
Optimal Conversion: 1,8s
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Conversionrate(%)
Category Title
Competitive Custom B2B

22. Real User Monitoring: True Conversion Rate
Data collected by MeasureWorks at various e-commerce websites using real user monitoring, reference: http://bit.ly/MW-VEUrum
0%
10%
Optimal Conversion: 1,8s
LD50: 4,3s
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Conversionrate(%)
Category Title
Competitive Custom B2B

23. Real User Monitoring: True Conversion Rate
Data collected by MeasureWorks at various e-commerce websites using real user monitoring, reference: http://bit.ly/MW-VEUrum
0%
10%
Optimal Conversion: 1,8s
LD50: 4,3s
Poverty Line: 7,6s
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Conversionrate(%)
Category Title
Competitive Custom B2B

24. On average a fast experience
converts up to 70% higher...

25. Yet, in reality...

26. ...this is a familiar sight ;-(

27. Presented by MeasureWorks & Philips at ShoppingToday 2013, reference: http://bit.ly/MW-ST2013

28. “You’re more likely to miss stuff just
because it takes a long time to scroll
down the page”
~ User 56A on the MeetHue.com website
Presented by MeasureWorks & Philips at ShoppingToday 2013, reference: http://bit.ly/MW-ST2013

29. Presented by MeasureWorks & Philips at ShoppingToday 2013, reference: http://bit.ly/MW-ST2013

30. “Your getting blamed for
things that are not your fault”
Presented by MeasureWorks & Philips at ShoppingToday 2013, reference: http://bit.ly/MW-ST2013

31. WHAT happened here?

32. the WAITING experience
Source: Jakob Nielsen

33. 0 0,5
Visual
confirmation
Source: Jakob Nielsen
the WAITING experience

34. 0 0,5 1,5
Visual
confirmation
Understand
Navigation
Source: Jakob Nielsen
the WAITING experience

35. 0 0,5 1,5 3
Visual
confirmation
Understand
Navigation
Relevant
Content?
Source: Jakob Nielsen
the WAITING experience

36. 0 0,5 1,5 3 10
Visual
confirmation
Understand
Navigation
Relevant
Content?
Bye
Bye!
Source: Jakob Nielsen
the WAITING experience

37. Offline vs. Online

41. “Both offline and online consumers
associate slow performance with
poor customer service”

42. Click away slidePerformance tolerance?

43. 0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8 9 10
Bouncerate per pagetype/session
Bouncerate(%)
Page load time (sec.)
Median Campaign Product search
Data collected by MeasureWorks at various e-commerce websites using real user monitoring, reference: http://bit.ly/MW-VEUrum

44. 0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8 9 10
Bouncerate(%)
Page load time (sec.)
Median Campaign Product search
Bouncerate per pagetype/session
Data collected by MeasureWorks at various e-commerce websites using real user monitoring, reference: http://bit.ly/MW-VEUrum

45. Queuing for iPhone6 launch - London 2014
Purpose vs. Context

46. Online users often lack context for delays...

47. ...and see no other option than to click away

48. For many, Performance is like the gym...

49. There’s always an excuse ;-(

50. Measuring Performance?

51. Three simple performance checks

52. actionable,
non-technical,
End User Focused

53. Speed

54. How fast am I?

55. 55
go to http://webpagetest.org

56. 56
www.example.com
select location
select browser
Set connection speed
Create video

57. 1: Configure test
2: Key Optimizations
3: Performance metrics
Use Akamai State of the Internet quarterly reports to select right bandwith per country: http://www.akamai.com/stateoftheinternet/

58. 58
http://www.digitalmarketinglive.nl

59. 0
5.0
10.0
15.0
20.0
IE10 Chrome Firefox Safari iOS Android
Page load time per browser type (sec.)
Backend First Paint Page Load time
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

60. Visual Comparison

61. 61

62. >
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

63. Compared to the competition?

64. Performance influence cycle
Search/Orientation phase
Perception of experience
1
http://benchmark.measureworks.nl

65. Delivered
experience
Your website
3
5
2
Stimulate content/
conversion
4
Performance influence cycle
Search/Orientation phase
Perception of experience
1
http://benchmark.measureworks.nl

66. No permission needed...

67. 9.76 sec. 4.75 sec. 7.09 sec. 2.31 sec.
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

68. Or the easy way: http://benchmark.measureworks.nl

69. Responsive web?

70. http://whatismyscreenresolution.com

74. Do you trust your third parties?

77. 4.3 sec
22.1 sec
Source: webpagetest.org

79. Source: requestmap.webperf.tools

80. Source: requestmap.webperf.tools

81. + conversion

82. 3 types of data:
Which pages are clicked on?
How fast is it on every device?
Which elements do they click on?

83. Real User monitoring

84. 1
Insert tag (.js file)
into (mobile) web
pages
Pages are requested
from browser/device
As pages execute,
tag collects
performance metrics
After onload tag
send detailed report
for further analysis
tag.js
tag.js
tag.js
tag.js
2 3 4

86. Relies on navigation timing API, custom variables can be added
Turn on
“Site Speed”
in your GA
account!
http://bit.ly/1ivGWTt

87. Or try
Soasta “Lite”
http://bit.ly/I5ngs8

88. Holy performance, Batman...

89. Why is (nigerian) spam so badly
written?

90. Aunshul Rege of Rutgers University, USA in 2009
1000 emails
1-2 responses
1 fool and their money, parted.
Bad language (0.1% conversion)
Gullible (70% conversion)
1000 emails
100 responses
1 fool and their money, parted.
Good language (10% conversion)
Not-gullible (.07% conversion)

91. Nigerian spammers
really understand their target
market and ‘customers’

92. One Metric That Matters

93. Every website has goals

94. Your Analytics measures the
movement towards these goals

95. Complete Web Monitoring

96. Complete Web Monitoring
Web Analytics
(what did they
do on the site?)

97. Complete Web Monitoring
Web Analytics
(what did they
do on the site?)
Usability
(how did they
interact with it?)

98. Complete Web Monitoring
Web Analytics
(what did they
do on the site?)
Performance
(could they do
what they wanted
to?)
Usability
(how did they
interact with it?)

99. Complete Web Monitoring
Web Analytics
(what did they
do on the site?)
Performance
(could they do
what they wanted
to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)

100. Complete Web Monitoring
Web Analytics
(what did they
do on the site?)
Competition
(what are they up
to?)
Performance
(could they do
what they wanted
to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)

101. Complete Web Monitoring
Web Analytics
(what did they
do on the site?)
Competition
(what are they up
to?)
Performance
(could they do
what they wanted
to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)
Social Media
(what were they
saying?)

102. Complete Web Monitoring
Web Analytics
(what did they
do on the site?)
Competition
(what are they up
to?)
Performance
(could they do
what they wanted
to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)
Social Media
(what were they
saying?)
“Soft” data
“Hard” data

103. Some fundamentals

104. X
A simple online business model:
Marketing Conversion Optimization Revenue
New
visitors
Bounce
rate
Conversion
rate
Order
value
Growth
Loss
Time on
site
Pages per
visit
Number of
visits
Search
Tweets
Mentions
ADs seen

105. Context!

106. Checkout
Visits
Shopping cart
Payment options

107. Amplify your message

108. Checkout
Visits
Shopping cart
Payment options
Unpaid search Social mentions Email
campaign Banner ads
Generate traffic to website

109. Don’t trust your data, analyse it

110. 9 followers
9 reach
4 followers
16 reach

111. 9 followers
9 reach
4 followers
16 reach

112. Checkout
Visits
Shopping cart
Payment options
Unpaid search Social mentions Email
campaign Banner ads
Generate traffic to website

113. Just imagine if data tells you how to
influence your visitors?

114. Spread your message...

115. ...influence your followers...

116. ...go viral

117. Conversions
Visits
Shopping cart
Payment options
Impressions
Spread
Seed
Higher likelihood
of conversion
Generate traffic to website

118. Importance of
correlation vs. causation

119. 1
10
100
1000
10000
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Relation between Ice cream and drownings?
Ice cream consumption Drownings

120. http://www.flickr.com/photos/25159787@N07/3766111564

121. http://www.flickr.com/photos/25159787@N07/3766111564
1
10
100
1000
10000
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
True causality
Ice cream consumption Drownings Temperature

124. A leading, causal, connection is
the analytics superpower...

125. That’s why we pick one at at time...

126. ‣ A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
‣ A Dropbox user who puts at least one file in one folder on one
device (ChenLi Wang)
‣ Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh
Elman)
‣ A LinkedIn user getting to X connections in Y days (Elliot
Schmukler)
Some examples
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)

128. A wealth of information creates a
poverty of attention...
(Computers, Communications and the Public Interest, pages 40-41,
Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert
Simon

129. Focus on the desired behavior, not
just the information.
http://www.psychologytoday.com/blog/yes/200808/changing-minds-and-changing-towels
26% increase in towel re-use with an appeal to social
norms; 33% increase when tied to the specific room.

130. Validating experiments:
B.R.A.S.S. framework

131. Blink Relevance Availability Scalability Sum

132. Blink Relevance Availability Scalability Sum
5-second rule
Gut feeling
Instinct score
Don’t overthink it

133. Blink Relevance Availability Scalability Sum
5-second rule
Gut feeling
Instinct score
Don’t overthink it
Do we have the
available
resources?
Do we have the
skills?
Do we have the
tools?
Is it expensive?

134. Blink Relevance Availability Scalability Sum
5-second rule
Gut feeling
Instinct score
Don’t overthink it
Is this Channel
relevant to your
product/service?
Does your
audience hang out
on this channel?
Can you target
them effectively?
Do we have the
available
resources?
Do we have the
skills?
Do we have the
tools?
Is it expensive?

135. Blink Relevance Availability Scalability Sum
5-second rule
Gut feeling
Instinct score
Don’t overthink it
Is this Channel
relevant to your
product/service?
Does your
audience hang out
on this channel?
Can you target
them effectively?
Do we have the
available
resources?
Do we have the
skills?
Do we have the
tools?
Is it expensive?
How scalable is
this channel?
Can we easily
increase it?
Law of diminishing
returns

136. Blink Relevance Availability Scalability Sum
5-second rule
Gut feeling
Instinct score
Don’t overthink it
Is this Channel
relevant to your
product/service?
Does your
audience hang out
on this channel?
Can you target
them effectively?
Do we have the
available
resources?
Do we have the
skills?
Do we have the
tools?
Is it expensive?
How scalable is
this channel?
Can we easily
increase it?
Law of diminishing
returns
1-5 1-5 1-5 1-5 BRASS score

138. bit.ly/brassframework

139. Optimizing Performance?

140. The Skip: http://musicmachinery.com/2014/05/02/the-skip/

141. “Hit them with a beat! The decision to continue
listening to a song is often determined in the first 10
seconds” - Dr. Dre, Source Magazine
The Skip: http://musicmachinery.com/2014/05/02/the-skip/

142. Current state of responsive web ;-(

143. WiFi 3G
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

144. 6 must haves
for happy customers (again)...

145. 1.
Perception is KEY!

146. http://www.nytimes.com/2012/08/19/opinion/sunday/why-waiting-in-line-is-torture.html?pagewanted=all&_r=0

147. http://answers.yahoo.com/question/index?qid=1005081200005

148. nytimes.com/2004/02/27/nyregion/27BUTT.html

150. Which is faster?
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

151. Which is faster?
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

152. Which is faster?
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

153. Which is faster?
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

154. “The faster you can visualize your content,
the more engaged your visitors will become”

155. 2.
Understand & Prioritize
your feature content

156. Defer non-feature content

157. A: Load carousel first B: Delayed download
20% time spent on priority content 1% time spent on priority content
http://www.nngroup.com/articles/website-response-times/

158. Yet, 83% of all Twinkle100 sites
loads menu before feature
content...

161. “The faster you can
paint your feature content,
the more engaged
your visitors will become”

162. Avoid (too much) Sequencing

163. “Provide your users the time to
understand navigation as they often
overlook things that change too fast and/or
often”

164. www.example.com

165. Homepage Search
Product
overview
Product
Detail
Basket

166. Homepage Search
Product
overview
Product
Detail
Basket

167. Homepage Search
Product
overview
Product
Detail
Basket
Understand
navigation
(1-2 sec.)
Consistent
navigation
(0,5-1 sec.)
Renew your
orientation
(1-2 sec.)
renew
your navigation
(1-2 sec.)
Consistent
navigation
(0,5-1 sec.)

168. Consistent, simple &
user generated navigation

169. Use speed to increase perceived value

170. Homepage Search
Product
overview
Product
Detail
Payment
Tolerated
speed

171. Fast response = Fast Experience?

172. The kayak effect: http://bit.ly/UgTneD
People prefer to wait for up
to a minute to get what they
want from an app rather
than get it instantly – if, and
it’s an important if, they
believe the app is working
for them

173. http://www.fastcodesign.com/1669788/the-3-white-lies-behind-instagrams-lightning-speed

175. http://blog.placeit.net/ux-tactics-make-slow-things-seem-faster/

176. Show effort,
provide specific content,
build trust

177. 3.
Set your performance budget

178. Setting a performance budget:
A pre-defined set of metrics
that describe normal behavior
in order to detect variances
and to be comparable in historical context

179. Example performance budget...

180. Perception
Actionable
Flow
TtFB
Page Size
Time to first Byte 0.0 - 0.2 sec.
Speed Index 0.5 - 1 sec.
Document complete: 1-3 sec
Variation < 20% between pages in funnel
Page size < 1Mb (75 requests)
Example performance budget...

181. A practical (and easy DIY) example:

182. Service Window
Purchasing a bed,
must be completed (speed),
where every page loads under 3 sec.,
from any location in the Netherlands,
for 95% of all users,
every 5 minutes between 6am and 12pm,
using IE9 and higher,
Customer journey
Metric: Speed
Target: Sec
User scenario
User locations
Percentile
measured with Synthetic Monitoring. Monitoring type
Business
Performance

183. 4.
How fast is your (front-end) code?

184. Anatomy of a
web page
Anatomy of a web page

185. A set of resources to be fetched from a server...
The browser renders the page...
This results in the page being displayed on your
screen
1
2
3

186. Introducing the waterfall chart

188. See also: 90 minute optimization cycle, Strangeloop, Velocity 2012

189. HTML
See also: 90 minute optimization cycle, Strangeloop, Velocity 2012

190. HTML
Resources
See also: 90 minute optimization cycle, Strangeloop, Velocity 2012

191. HTML
Resources
Start
Render
See also: 90 minute optimization cycle, Strangeloop, Velocity 2012

192. HTML
Resources
Start
Render
Document
Complete
See also: 90 minute optimization cycle, Strangeloop, Velocity 2012

193. Waterfall technique
a.k.a. show me where it hurts

194. Code Green:
Time to first byte?

196. Code Orange:
Too many connections

197. 106 Connections
(almost one per request)

198. Code Blue:
Too many bytes (page bloat)

199. 199

200. Code Grey:
Bad repeat view

201. First view Repeat view

202. Why is this important?

203. Why is this important?
85-90% of all slowdowns are
caused by your front-end code...

204. 5.
Fast enough?
Then test for peak traffic

205. Load/Performance testing:
Measure performance under both normal and
peak volumes to discover how your application
behaves under stress

206. Find your ceilings in time to fix them...
The End
Current state

207. 0
250
500
750
1000
January-14 April-14 July-14 October-14 January-15 April-15 July-15 October-15
#Pageviews(x1000)
Forecast pageviews Actual pageviews
Traffic realized
Traffic forecast
Max infra capacity (90%)
Safety infra capacity (80%)
# pageviews at peak
hour of peak day per
week/month
Build your spreadsheet...

208. Your marketing campaign must never be a
suprise for your site...

209. 6.
Don’t forget?
Optimize for Mobile

210. Let the device work...

212. Make life easy, insert touch events...

213. Search

214. Touch gestures

215. Native gestures

216. Scrolling

217. 7.
Repeat, repeat, repeat...

218. It’s a process. Integrate the
performance budget in your
continuous delivery cycle

219. Analyze the performance as your
users experience it and
compare against your budget to
identify what areas need
improvement
Build faster websites
and optimize the
rendering of your
application and the
perception of speed
Automate & Update
performance testing and
integrate into your delivery
processes
Repeat
Measure Build Run

220. Performance is a team sport...

221. Developer
Makes it real,
builds &
deploys code.
Marketer
BizDev,
validates
customers
demand and
continuously
generates new
ideas
Designer
Designs
compelling
customer
experience
with
performance
in mind
Analyst
Tracks
performance
budget, drives
conversion
experiments
and keeps
you honest.

222. ...Use performance as a business
metrics to make it stick

223. Questions so far?

224. Always remember this...

225. Performance = Happy users

226. Performance = Reducing Complexity

227. Frictionless = Recognizable navigation

228. Frictionless = Fast delivery

229. You owe it to your customers...

230. Something to read...

232. Thanks! More questions?
M: jeroen@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl
View slides:
bit.ly/MW-WHA15