TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
Boomerang: How fast do users think your site is?
1. Introduction
How does it work?
Using boomerang
Data
Boomerang: How fast do users think your site
is?
Philip Tellis / philip@bluesmoon.info
ConFoo.ca/2011 – 2011.03.09-11
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
2. Introduction
How does it work?
Using boomerang
Data
$ finger philip
Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paranoid - speedfreak
yahoo
http://bluesmoon.info/
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
3. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
4. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
5. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
6. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
Only 10–20% of page load time is spent on systems we control
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
7. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
It’s what we can’t control that users notice
http://twitter.com/stoyanstefanov/status/43747252317077504
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
8. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
browsers
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
9. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
plugins
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
10. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
OSes
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
11. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
viruses
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
12. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
antiviruses
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
13. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
microwaves
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
14. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
baby monitors
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
15. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
naughty neighbours
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
16. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
file shares
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
17. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
governments
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
18. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
rodents
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
19. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We can’t control
Try simulating all that in the lab!
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
20. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We need to measure real end-user performance
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
21. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
We need to measure real end-user performance from the real
end-user’s perspective
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
22. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
23. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
While this might work, it isn’t necessarily representative
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
24. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
What about JavaScript?
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
25. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
26. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
boomerang is...
A piece of javascript that you add to your web page where it
measures and beacons back to you, the end user’s perceived
performance of your page
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
27. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
How?
<script src="boomerang.js" type="text/javascript">
</script>
<script type="text/javascript">
BOOMR.init({
user_ip: "<network ident>",
beacon_url: "http://mysite.com/beacon.php"
});
</script>
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
28. Introduction Time
How does it work? The adversary
Using boomerang Measure twice
Data boomerang
What does it do?
Measures user’s network throughput and latency to your
server
Measures the current page’s load time
Beacons these results back to your server
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
29. Introduction Latency
How does it work? Bandwidth/Throughput
Using boomerang Load time
Data Accuracy
How does boomerang work?
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
30. Introduction Latency
How does it work? Bandwidth/Throughput
Using boomerang Load time
Data Accuracy
Let’s take that one at a time
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
31. Introduction Latency
How does it work? Bandwidth/Throughput
Using boomerang Load time
Data Accuracy
Measuring latency
Download a 32 byte gif 10 times in sequence
Measure the time to download each
Discard the first measurement because it’s overpriced9
Calculate the arithmetic mean, standard deviation and
margin of error of the rest
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
32. Introduction Latency
How does it work? Bandwidth/Throughput
Using boomerang Load time
Data Accuracy
Measuring throughput
After the latency test is done, we download progressively
larger images
Stop at the first image that times out
Redownload that image a few more times
Calculate the median, standard deviation and margin of
error of the largest image
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
33. Introduction Latency
How does it work? Bandwidth/Throughput
Using boomerang Load time
Data Accuracy
Measuring latency before throughput helps here
Those 10 latency images do a lot to widen the TCP
window size
The bandwidth images make much better use of available
bandwidth
The image we end with makes the best use of bandwidth
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
34. Introduction Latency
How does it work? Bandwidth/Throughput
Using boomerang Load time
Data Accuracy
How do we measure page load time?
In the onbeforeunload event, measure the time and
store it in a cookie
In the onload event, check the cookie, and measure the
difference with the current time
We also make sure that the page that set the cookie is the
referrer of the current page
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
35. Introduction Latency
How does it work? Bandwidth/Throughput
Using boomerang Load time
Data Accuracy
What? Two pages?
Yes, this needs two pages and cookies. If those aren’t
supported, we try to use the WebTiming API10 .
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
36. Introduction Latency
How does it work? Bandwidth/Throughput
Using boomerang Load time
Data Accuracy
How accurate is it?
Latency measurements are very accurate (±1%)
Bandwidth is to an order of magnitude. For bad
connections can be ±30%
Page load time sometimes has outliers, you need
post-filtering
The margin of error tells you how good your data is
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
37. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
Include it on your page
<script src="boomerang.js" type="text/javascript">
</script>
<script type="text/javascript">
BOOMR.init({
user_ip: "<network ident>",
beacon_url: "http://mysite.com/beacon.php"
});
</script>
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
38. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
For most sites, that’s about it
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
39. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
You probably want to do more
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
40. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
Measure more than just load time
<html><head>
<script>
var t_pagestart=new Date().getTime();
</script>
...
<script>
var th=new Date().getTime();
</script>
</head>
<body>
...
<script>
var tj=new Date().getTime();
</script>
... ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
41. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
...
<script src="boomerang.js"></script>
...
var te=new Date().getTime();
BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart).
setTimer("t_body", te-th).
setTimer("t_js", te-tj);
</script></body></html>
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
42. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
This adds the t_head, t_body and t_js fields to the beacon
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
43. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
Loading dynamic content
BOOMR.init({
user_ip: "<network ident>",
beacon_url: "http://mysite.com/beacon.php",
auto_run: false
});
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
44. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
Loading dynamic content
// Just before download starts
BOOMR.plugins.RT.startTimer("t_done");
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
45. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
Loading dynamic content
// Just after download finishes
BOOMR.plugins.RT.done();
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
46. Introduction Basic
How does it work? In-page timers
Using boomerang AJAX
Data
Much more
http://yahoo.github.com/boomerang/doc/howtos/
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
47. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
The beacon
GET request to the beacon URL (response ignored)
All parameters passed in the query string
Extra timers are passed in as a comma separated list in
t_other
before_beacon JavaScript event fired just before the
beacon is sent
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
48. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
What should we do with the data?
Sanity checking to:
Remove fake data
Remove abusive data
Maybe just rate limiting
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
49. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
What can we do with the data?
Statistical analysis to:
Remove outliers
Aggregate based on bandwidth blocks
Measure trends over time and correlate them with code
changes
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
50. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
Bandwidth blocks
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
51. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
Bandwidth blocks
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
52. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
Bandwidth blocks
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
53. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
Bandwidth blocks
Data points from some countries may require narrower bands
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
54. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
Geographic data
Looking at latency from different geographic locations can tell
you where your next mirror should be
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
55. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
ISPs
Grouping data by ISP can tell you who’s behaving badly
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
56. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
More data
Write plugins to get more performance data
We already have a DNS plugin
I’ve just completed a plugin to measure IPv6 support and
latency
What about a full WebTiming plugin?
I’m also looking at measuring connection setup time
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
57. Introduction
shoulda
How does it work?
coulda
Using boomerang
woulda
Data
You decide
Once you have the data, you can do anything with it
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
58. Introduction
How does it work?
Using boomerang
Data
Boomerang: It (almost) always comes back
http://github.com/yahoo/boomerang
http://yahoo.github.com/boomerang/doc/
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
59. Introduction
How does it work?
Using boomerang
Data
Photo credits
flickr.com/photos/21233184@N02/4389412851
http://xkcd.com/445/
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
60. Introduction
How does it work?
Using boomerang
Data
Contact me
Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paranoid - speedfreak
yahoo
http://bluesmoon.info/
slideshare.net/bluesmoon
joind.in/talk/view/2913
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
61. Introduction
How does it work?
Using boomerang
Data
References
github.com/yahoo/boomerang
More bandwidth doesn’t matter (much) – Mike Belshe
Analysing Bandwidth & Latency – YUI Blog
It’s the latency, stupid – Stuart Cheshire
The statistics of web performance
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
62. Introduction
How does it work?
Using boomerang
Data
Overpriced images
The first image might require a DNS lookup and TCP
handshake
Slow start is not an issue since 32 bytes fit in 1 packet
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?
63. Introduction
How does it work?
Using boomerang
Data
WebTiming API
JavaScript interface to network timing information including
DNS, TCP connect, download, etc. Supported by:
Microsoft Internet Explorer 9
Google Chrome (WebKit)
W3C specification
ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?