The document discusses the importance of website performance and the impact of speed on key metrics like customer satisfaction, conversion rates, and revenue. It outlines challenges to performance from increasing traffic levels, mobile usage, different browsers, and third-party components. The presentation argues for a holistic approach to web performance management to identify and address bottlenecks across infrastructure, applications, and real user experience.

2. Performability, Could they do it?
Complete
Web
Monitoring
@jeroentjepkema
#WAC2010

3. The Internet is getting complex….
"

4. The End-User:
Everywhere!
!

5. Technology:
Enterprise/Virtualization/Cloud!

6. Weʼre getting mobile"

7. The end-user is in control
"

9. Back to the fundamentals:
Quality of Service"

10. Performance matters!
"

11. Google loses 1% of traffic for every
100 milliseconds of delay "

13. Impact of additional delay on business metrics !

15. Shopzilla had another angle"
Big, High-traffic site" 16 Month re-engineering"
 100M impressions a day!  Page load from 6 seconds to 1.2!
 8000 searches a second!  Uptime from 99.65% to 99.97%!
 20-29M unique visitors a month!  10% of previous hardware
 100M products! needs!

16. 12% increase in revenue,"
25% increase in page views for"
every 5 seconds of gain"

18. As of April 9th 2010:!
Site speed co-determines "
Google page ranking!"

19. Google Pagerank"

20. Customer Experience"

21. Evolving Customer Expectations"
8 seconds
4 seconds
2 or less
?
Response Time
Source: Forrester Consulting, 2009

22. Availability and Stability are!
KEY ISSUES!!

23. Performance impacts Customer Loyalty!

24. Average Impact of 1 second delay in
response time for web users"
Source: Aberdeen Group, 2009
24

25. Impact of Poor Experience"
Source: Gartner, 2008!

26. A/B Performance Testing!

27. Objective: Influence visitor engagement"
-
Visitor engagement
10
ms
+

28. Objective: Influence visitor engagement"
-
Visitor engagement
100
ms
10
ms
+

29. Objective: Influence visitor engagement"
-
Visitor engagement
1
sec
100
ms
10
ms
+

30. Objective: Influence visitor engagement"
-
10
sec
Visitor engagement
1
sec
100
ms
10
ms
+

31. A/B Performance Test Setup"
Visitor Strangeloop Website
Decide whether
to optimize
Normal
Receive Accelerated
content
page
Insert
Process segment
scripts Optimize?
marker
Send
analytics Unaccelerated
Google Analytics

32. Technical Problem Outlined - Before"
32

33. Technical Problem Outlined - Before"
HTML (BackEnd)
63 Round Trips (Front End)
1.975sec 9.537sec
769KB 33

34. Strangeloop Solution - After
"
Before" After"
 Round trips: 63!  Round trips: 8!
 Start Render: 1.975 Sec!  Start Render: 1.198 Sec!
 Load Time: 9.537 Sec"  Load Time: 3.69 Sec"
 Payload : 769 KB!  Payload: 456 KB!
34"

35. So hereʼs what weʼve discovered…."

36. Traffic Levels"
18!
9000
8.505!
16!
14!
6750
12!
Total number of visits"
4.740!
10!
4500
8!
6!
2250
4!
2!
00!
Optimized! Unoptimized!

37. Bounce rate"
14,60%!
14,40%! 14,35%!
Bounce rate"
14,20%!
14,00%!
13,80%!
13,60%!
13,38%!
13,40%!
13,20%!
13,00%!
12,80%!
Optimized! Unoptimized!
37

38. Average Time on Site"
00:36:00!
00:30:10!
Time (seconds)"
00:28:48!
00:23:50!
00:21:36!
00:14:24!
00:07:12!
00:00:00!
Optimized! Unoptimized!
38

39. Pages per Visit
"
18!
15,64!
16!
14!
12! 11,04!
Pages"
10!
8!
6!
4!
2!
0!
Optimized! Unoptimized!
39

40. Conversion rate and order value"
20,00%!
16,07%!
Change in shopping cart metric"
15,00%!
10,00%!
5,51%!
5,00%!
0,00%!
Conversion rate! Order value!
-5,00%!
-10,00%!
-15,00%!
-20,00%!
40

41. Customers want more and they want it faster
"

42. 42 Entire contents © 2009 Forrester Research, Inc. All rights reserved.

43. Time to speed up!"

44. Compared to the competition!

45. ABN Amro 1.81
sec
Bijenkorf 0.76
sec
D-reizen 1.79
sec
Alex 3.43
sec
Bol.com 1.20
sec
Neckermann 0.61
sec
ING 0.86
sec
Hema 1.19
sec
NS 0.86
sec
Rabobank 1.42
sec
Mediamarkt 2.39
sec
Sunweb 0.39
sec
SNS Reaal 1.64
sec
TomTom 1.46
sec
Vliegwinkel 5.04
sec

46. ABN Amro 1.81
sec
Bijenkorf 0.76
sec
D-reizen 1.79
sec
Alex 3.43
sec
Bol.com 1.20
sec
Neckermann 0.61
sec
ING 0.86
sec
Hema 1.19
sec
NS 0.86
sec
Rabobank 1.42
sec
Mediamarkt 2.39
sec
Sunweb 0.39
sec
SNS Reaal 1.64
sec
TomTom 1.46
sec
Vliegwinkel 5.04
sec

47. Retail 2.31
sec
Average speed per sector,
Finance 3.30
sec
Gomez Benchmark 2009
Travel 3.55
sec
ABN Amro 1.81
sec
Bijenkorf 0.76
sec
D-reizen 1.79
sec
Alex 3.43
sec
Bol.com 1.20
sec
Neckermann 0.61
sec
ING 0.86
sec
Hema 1.19
sec
NS 0.86
sec
Rabobank 1.42
sec
Mediamarkt 2.39
sec
Sunweb 0.39
sec
SNS Reaal 1.64
sec
TomTom 1.46
sec
Vliegwinkel 5.04
sec

48. Retail 2.31
sec
Average speed per sector,
Finance 3.30
sec
Gomez Benchmark 2009
Travel 3.55
sec
ABN Amro 1.81
sec
Bijenkorf 0.76
sec
D-reizen 1.79
sec
Alex 3.43
sec
Bol.com 1.20
sec
Neckermann 0.61
sec
ING 0.86
sec
Hema 1.19
sec
NS 0.86
sec
Rabobank 1.42
sec
Mediamarkt 2.39
sec
Sunweb 0.39
sec
SNS Reaal 1.64
sec
TomTom 1.46
sec
Vliegwinkel 5.04
sec

49. Performance by region!

51. The userʼs distance from the content matters"

52.
6.54
sec
4.89
sec
4.07
sec
5.49
sec
6.09
sec
7.19
sec

53. Forrester
research
shows
that
over
40%
of
consumers
will
wait
no
more
than
3
seconds
for
a
Web
page
to
load
before
clicking
away
to
a
compeHtor
6.54
sec
4.89
sec
4.07
sec
5.49
sec
6.09
sec
7.19
sec

54. Mobile site performance!

55. 58%
of
mobile
phone
users
expect
websites
to
load
as
quickly,
almost
as
quickly
or
faster
on
their
mobile
phone,
compared
to
the
computer
they
use
at
home
Among adults who accessed the internet with a mobile phone in the past 12 months, Gomez Mobile Web Experience Survey

56. desktop
1.17
sec
Iphone

57. desktop
1.17
sec
20.19
sec
Iphone

58. desktop
1.17
sec
20.19
sec
Iphone
http://m.bol.com
5.02
sec

59. desktop
1.94
sec
Iphone

60. desktop
1.94
sec
15.60
sec
Iphone

61. desktop
1.17
sec
20.19
sec
http://m.nu.nl
4.64
sec

63. Performance differs per browser
"

64. Each browser acts different"
Firefox
Internet
Explorer

65. Raw page load time"

66. Raw page load time vs. Perceived Performance"

67. 0.95
sec
1.82
sec
1.00
sec
1.19
sec
2.637
sec
2.59
sec

68. Availability & performance issues can also be browser specific

69. Assembly within the browser: it all has to work
!

71. ?

72. ?

73. Real User Experience
!

80. Congestion!

81. Watch out for Peak Traffic!"

82. Watch out for Peak Traffic!"

83. Conclusion: Bottlenecks are everywhere!

84. Bottlenecks are everywhere"
How
fast
is
fast
enough?
Performance
by
region
Mobile
Performance
Browser
Experience
3d
Party
Performance
Real
User
Experience
CongesHon

85. How to make your website faster?"

86. Introducing:
Web Performance Management!

87. How does the performance of
Infrastructure and Application….
Management Tool!
Load Web App DB!
balancer! server! server!
Internet!
Client!

88. …relate to the customer experience
Management Tool!
Load Web App DB!
balancer! server! server!
Internet!
Client!

89. Introducing the synthetic user
Management Tool!
Active
Monitoring!
Load Web App DB!
balancer! server! server! DB!
Internet!
Client!

90. Active Monitoring
Continous
Measurements

91. Active Monitoring
Continous Check Performance
Measurements & Availability

92. Active Monitoring
Continous Check Performance In case of failure:
Measurements & Availability Alerting

93. Active Monitoring
Continous Check Performance In case of failure: Root Cause Analysis /
Measurements & Availability Alerting SLA reporting

94. Simulate Business Transactions"

95. Response time and Availability"

96. Page breakdown"

97. From where your users are
"

98. But synthetic isnʼt enough"

99. But synthetic isnʼt enough"
Gomez Asia
Gomez US
Gomez UK

100. Measure all your users
Client!
Client! Management Tool!
Active
Monitoring! Load Web App DB!
balancer! server! server! DB!
Internet!
Real User!
Monitoring!
Client!
Client! Client!

101. Collected data: Active
Client"
Monitoring!
 Client-­‐ip,
Server-­‐ip,
Gateway-­‐ip
Page
 Requested
URL
(Page,Object)
request
 Cookie/Session-­‐id
Network Response Time
 Referrer
Networkresponse time
 GET
&
POST
requests
 User
agents
(phone
type)
Real User! Load
Monitoring! balancer"
Server Response Time
Web server"
 Response?
 Server
error,
web
site
error,
Hit
OK
 Content
of
page
(i.e
order
details,
App server"
funcHonal
errors)
DB"
 Delivered?
 Aborted?
 Network
Hmeout?

102. Business Impact per user
Satisfaction Reports Business Transaction Funnel
Usage Reports:
• Data Browser
• Sales by Country/State

103. Session Replay
User SessionTracking"
Session Replay"
Business Service Monitoring (e.g.
Order Entry, Buy Stock)!

104. Root Cause Analysis per user
"
KPI configuration,
monitoring, & alerting!
Server, network, &
object
performance
diagnostics!
Error reports &
analysis!

105. Active Monitoring:!
Active Monitoring"  Simulate users by scripting pages and/or
business transactions!
 Continuous measurements from outside the
firewall!
 Best used for incident, problem and SLA
management.!
Real User Monitoring!
Real User"
Monitoring"  Measures traffic of ALL users, transactions
and pages!
 Passive solution which gives information
about the user experience!
 Best used for performance and capacity
management!

106. But, how to become the performance leader?!

107. 5 Web Performance Best Practices
"

108. 1: Create Shared Performance Objectives"

109. Business
vs.
IT

110. Business

Invest
to
aeract
more
and
new
customers

Quick
Time
to
market

Conversion
rates

Realize
Sales
targets
IT

Quality
of
Service
 Be
Cost
efficient

Follow
procedures

Standardize

111. …..How to work
together?"

112. Key Business Metrics"
AJenKon
Engagement
Conversion
NEW
VISITORS
GROWTH
CONVERSION
SEARCHES RATE
PAGES TIME
TWEETS
MENTIONS
NUMBER
OF VISITS
PER
VISIT
ON
SITE
x
ADS SEEN ORDER
LOSS VALUE
BOUNCE
RATE
112

113. Key Business Metrics"
AJenKon
Engagement
Conversion
NEW
VISITORS
GROWTH
CONVERSION
SEARCHES RATE
PAGES TIME
NUMBER
TWEETS
MENTIONS
OF VISITS
PER
VISIT
ON
SITE
x
ADS SEEN ORDER
LOSS VALUE
BOUNCE
RATE
Performance
maeers!
Web
Quality
directly
impact
business
metrics
113

115. Business oriented Key Performance objectives
!

116. Business oriented Key Performance objectives
!
System
Management
Business
TransacHons

117. Business oriented Key Performance objectives
!
System
Management
Business
TransacHons
System
Availability
End
User
Availability

118. Business oriented Key Performance objectives
!
System
Management
Business
TransacHons
System
Availability
End
User
Availability
Build
Quality
Web
Stability

119. Business oriented Key Performance objectives
!
System
Management
Business
TransacHons
System
Availability
End
User
Availability
Build
Quality
Web
Stability
Capacity
(Peak)
Traffic

120. Business oriented Key Performance objectives
!
System
Management
Business
TransacHons
System
Availability
End
User
Availability
Build
Quality
Web
Stability
Capacity
(Peak)
Traffic
Network
Page
Speed

121. 2: Measure from End User Perspective
!

122. Monitor Key Transactions from where your End-Users are!

123. Benchmark against the competition!

124. Measure every detail of your traffic"
Customers: Global! Access: Getting Mobile!
Technologies: Enterprise/Virtualization/Cloud!
12
4!

125. 3. Plan for Peak Crowds"

126. 1.
Compare
Actual
Results

127. 1.
Compare
Actual
Results
2.
With
Actual
Performance

128. 1.
Compare
Actual
Results
2.
With
Actual
Performance
3.
To
forecast
Capacity
&
Performance

129. Peak Traffic: Itʼs a Business KPI!
"
Capacity
forecast,
actuals
and
maximum
Max.
number
of
pageviews
per
hour
1.200.000
1.000.000
800.000
Christmas/EOY
Pageviews
600.000
400.000
Traffic
Realized
Traffic
Forecast
200.000 MAX
capacity
Safety
capacity
0
nov-11
Jan-11
apr110
sep-11
jan-10
jul-10
sep-10
dec-10
mrt-10
jun-10
okt-10
aug-10
jul-09
sep-09
dec-09
nov-10
jun-09
okt-09
aug-09
jul-11
Mei-09
mei-11
okt-11
nov-09
feb-10
jun-11
aug-11
apr-11
dec-11
mrt-11
feb-11
mei10

130. 4. Test and Simulate performance"

131. The performance challenge"

132. Make sure the view your customers are getting
is the view you want them to see!

133. Make sure the view your customers are getting
is the view you want them to see!

134. Measure performance for normal traffic….
"

135. …. AND Peak Traffic"

136. Find your ceilings
"

137. Source: John Allspaw, Flickr.com
what you have left
The End
Know what your limits are…"

138. Plan for the unexpected
Measure performance under normal and peak
conditions to find all the problems and optimization
opportunities

139. 5. Tying performance to conversion:"
Web Performance Analytics!

140. Is it marketing?"

141. Or the WEB Experience?"
 Why
do
customers
exit?
 Offer?
 Performance?
 FuncHonal
errors?
 Technical
errors?
 Design
errors?
 Business
impact?
 In
affected
users?
 In
missed
transacKons?
 In
€?

142. Online Success requires a complete picture"
IT
OperaKons:
? Web
AnalyKcs:

System
Management

Conversion

CPU,
Memory

Pageviews

Quality
of
Service

Visitors

143. Online Success requires a complete picture"
IT
OperaKons:
Web
AnalyKcs:

System
Management

Conversion

CPU,
Memory

Pageviews

Quality
of
Service

Visitors
Web
Performance
AnalyKcs:

Assess
impact
of
performance
on
user
behaviour

Tie
customer
saHsfacHon
to
quality
of
delivery

QuanHfy
revenue
risked

144. The Web Performance Challenge:
Balancing Cost vs. Performance
Revenue"
Conversion
Optimization"
Operations" Development"

145. The Web Performance Challenge:
Balancing Cost vs. Performance
Revenue"
Conversion
Optimization"
Operations" Development"
Am
I
opHmizing
for
performance
AND
cost?

146. The Web Performance Challenge:
Balancing Cost vs. Performance
Revenue"
Are
these
the
most
cost
effecHve
improvements?
Conversion
Optimization"
Operations" Development"
Am
I
opHmizing
for
performance
AND
cost?

147. The Web Performance Challenge:
Balancing Cost vs. Performance
Is
my
performance
improvement
generaHng
conversion?
Revenue"
Are
these
the
most
cost
effecHve
improvements?
Conversion
Optimization"
Operations" Development"
Am
I
opHmizing
for
performance
AND
cost?

148. Creating the Big Picture:
Break down the Web Application Delivery Chain…!
Management Tool!
Internet! Load Web App DB!
Client! balancer! server! server!

149. Creating the Big Picture:
….and relate Customer Experience…..
!
Type of visitors Browser type Application Infrastructure
Performance Performance
Management Tool!
Internet! Load Web App DB!
Client! balancer! server! server!

150. Creating the Big Picture:
…to revenue risked!
Type of visitors Browser type Application Infrastructure
Performance Performance
Management Tool!
Internet! Load Web App DB!
Client! balancer! server! server!
Conversion Speed/Availability Functional Technical
Quality Quality & Capacity

151. Make your own business case"
Current
daily orders" (!
*!
Increased"
conversions" +!
Increased
order value" )!
ROI
=! (days)"
Cost of performance
enhancement"
151!

152. Make your own business case"
(! )!
$10,000! *! 0.1607! +! 0.0551! $2,158! 23.17
=! =! days"
$50,000! $50,000!
152!

153. Actionable reporting:"
Monetize your performance "
Availability
Speed
Capacity
User Experience
SLA IT suppliers
Shared KPI’s
+/-
Conversion

154. Summary"

155. 1. Create Shared Performance Objectives
2. Measure from End User perspective
3. Plan for Peak Crowds
4. Test and simulate performance
5. Create the BIG picture:
Web Performance Analytics

156. Questions?!

157. Analytics
Usability
Performability
(What did they do (How did they (Could they do
on the site?)! interact with it?)! what they wanted
to do?)!
VOC
Communilytics
Competition
(What were their (What were they (What are they
motivations?)! saying?)! up to?)!

158. Thanks! More questions?
M: jtjepkema@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl