You're probably familiar with the well-known performance success stories from companies like Amazon, Google, Microsoft and Shopzilla. But how relevant are these megasites to "mortal companies" that don't make billions of dollars per year or have teams of in-house performance engineers to do their bidding?
Strangeloop president Joshua Bixby walks through case studies of Strangeloop customers like AutoAnything.com and Artbeads.com to show how mortal companies have improved performance and achieved measurable success, including:
· Increased revenue by 13%
· Increased cart size by 6%
· Increased conversions by 9%
Joshua offers practical tips for successfully evangelizing performance within your organization. He also gives a snapshot of the current performance landscape in North America, as well as a sense of where the industry is headed.
36. A Few Points of Clarification We’ll use it to describe where performance pain points are, but that doesn’t mean the page actually has these problems What we’re going to do: Improve performance incrementally Not so good (slow) Awesome (fast) * The real Velocity site is somewhere in the middle!
53. Performance Problems Too many connections (too much orange) Too many bytes (too much blue) Concurrency Bad Caching for repeat views No CDN (the greens are too big)
54. The Green Problem #1: No CDN TTFB
55. Performance Problems Too many connections (too much orange) Too many bytes (too much blue) Concurrency Bad Caching for repeat views No CDN (the greens are too big) Too Many Roundtrips (too many greens)
56. The Green Problem #2: Roundtrips Repeat View First View 80 Requests 78 Requests 27 Requests 14 Requests
57. The Green Problem #2: Roundtrips Every fetch still pays the HTTP overhead penalty TTFB is still a problem Exacerbated by concurrency issues Getting worse as number of objects per page grows Generally, the hardest problem to solve
58. Performance Problems Too many connections Too many bytes (too much blue) Concurrency Bad Caching for repeat views No CDN (the greens are too big) Too Many Roundtrips (too many greens) Others
59. Examples of Other Problems Blocking Javascript 3rd party calls (http://stevesouders.com/p3pc/)
65. Stepwise Acceleration Start from the beginning and fix the easy stuff Step by step acceleration of the page Apply techniques/methods/etc and see the result Try to make it as fast as possible
67. Keep-Alive Solves the too-many connection problem (Less Orange!) Will help alleviate the TCP connection setup overhead 97 Connections
68. Compression Addresses the too-many-bytes problem (Less Blue!) We’ll compress textual content (html/css/etc) Not the only solution to less blue, but the easiest
73. How Did We Do? Original KA+Comp Improvement First View Repeat View 52% 71% 34% 94% 31% 51% 23% 75% 40% 62%
74. Before and after: Keep-alives & compression http://www.youtube.com/watch?v=KPYBF41yiFw
75. Pros and Cons Pros Really easy to do Single configuration switches in servers, proxies, or load balancers Good benefit seen right away Cons Compression has processing overhead On their own they’re just not enough
79. How Do We Get Better Caching RFC 2616, Section 13 Caching headers should be used on static (non-changing) objects, so they can be cached browser-side And by intermediate caching proxies Validators are not enough
82. How Did We Do? KA+Comp With Good Caching Improvement Repeat View 70% 42% 67%
83.
84. Pros and Cons Pros Good caching can have a major performance impact on repeat visits to a page Sometimes it’s easy to do Browsers generally pay attention (although interpretation may vary slightly) Cons The spec appears scary Invalidation and stale content
91. How Did We Do? KA+Comp +CDN Improvement First View 21% 17% 22% 0.7 sec 2.3 sec 2.7 sec Seconds Gained
92. Before and after: Adding a CDN http://www.youtube.com/watch?v=BR5hO5rL8lE
93. Pros and Cons Pro Good mitigation of the TTFB problem Established industry: lots of vendors to choose from Cons Sometimes costly May require code change (CDN’ed objects should be written to the CDN domain)
95. We Can Get Better! Still too many roundtrips Still too many bytes Not Fast Enough!!
96. What to do Next? Reduce Roundtrips Combine images Combine JavaScript Combine CSS Reduce Payload even more Minify CSS and JavaScript Add Image Compression Increase Concurrency Add a couple of domains to the mix
99. How Did We Do? +CDN 81 107 +Strangeloop 11 37 Improvement First View 19% 30% 54% 45% 31% 0.5 sec 4.6 sec 4.1 sec Seconds Gained
100. Before and after: The final, fastest version http://www.youtube.com/watch?v=IPn0T1UacIA
101. Pros and Cons Pros Most significant benefit for the hardest part of the acceleration lifecycle Address multiple performance points (somtimes multiple ones with the same technique) Cons It’s not easy Regression
120. Step 5Test your site to get a sense of how much faster it could be. strangeloopnetworks.com/test-your-website
121. Step 6Compare these gains to your graphs from step 4. What lift can you anticipate in value per visit?
122. Caveats Correlation does not imply causation. Browser and connection speed might imply something about the buyer (i.e. s/he is more affluent) that is unrelated to the effects of speed.
125. How to be your company’s in-house performance evangelist
126. The average exec wants to know 3 things What’s in it for the company? What’s in it for me? How do we compare to our competitors?
127. When talking to an exec… Tell the time, not how the watch works. Most important, urgent point first. Keep it short. Keep it simple. Make it visual. Be ready to give context.
128. What to say: #1 “Our site is slower than our competitors. We’re losing money.”
129.
130.
131. What to say: #2 “We’ve proven that, when our site is faster for users, they spend more.”
132. What to say: #3 “This is where we should be aiming.”
133. What to say: #4 “Consumers expect EVERY website to load fast.”
136. History of performance automation “I will deliver what the server gives me as efficiently as possible to the browser.” “I will transform what the server gives me to optimize it for the user’s browser” Delivery Transformation
If you’re here, you’re a performance convert. But one of your biggest problems may be trying to explain the urgency of performance to non-techies in your company… and getting higher-ups to commit to long-term investing in performance. Sure, there are lots of high-profile stories of how speeding up pages has been successful for mega-sites…
I talk to a lot of execs, and I hear this a lot. I get why. It’s difficult for mortal companies to see themselves in relation to ecommerce mega-giants thatmake billions of dollars a year and have teams of in-house performance engineers to do their bidding.
Again, performed 50/50 test to see the impact of faster pages on performance:Conversions increased by 9%Cartsize by 11%Sales by 13%
Again, 50/50 test of the site after implementing the Site Optimizer service. Wetracked three metrics among both test groups: revenue per visitor, revenue per visit, and overall revenue. The goal was a baseline increase of 2.5% across all three metrics. The accelerated site dramatically outperformed this goal:Revenue per visit +8%Overall revenue +6%
I talk to a lot of execs, and I hear this a lot. I get why. It’s difficult for mortal companies to see themselves in relation to ecommerce mega-giants thatmake billions of dollars a year and have teams of in-house performance engineers to do their bidding.
The obvious answer is to just implement Site Optimizer, speed things up, then check out conversion rate changes using a segmentation test, but this isn’t an option for everyone. So we developed a hack. It lets you use two tools you’re probably already familiar with – Google Analytics and WebPagetest – to slice your own data a bunch of different ways, and create decent proxies for performance.
Last fall, we performed a study which showed that IE8 is about 25% faster than IE7 across almost 200 websites. Based on this, we felt that browser version is a solid performance proxy for exploring conversion behaviour. Reference original case study: “The first thing we did was perform a Webpagetest in IE7 and IE8. We found that his site was 30% faster in IE8.”
Explore different connection speeds within IE8. First, perform Webpagetests on the different connection speeds, and then compare them to the results in Google Analytics.If you’re using Google Analytics, I think the easiest way to get this data is with a custom report that looks something like this. (reference Google Analytics screen grab)From case study: “Again we found a remarkable relationship between connection speed and order value. On average, online shoppers using T1 connections spent about 11% more than shoppers with DSL connections. And shoppers with T1 connections spent roughly 32% more than those using dialup.”
To pass any hardcore statistical muster, a much more in-depth regression test would need to occur. But these early proof points are enough to convince many non-believers that performance matters and they should invest in it.
In a real-world application of this approach, with all variables accounted for, the optimized site still outperformed the unoptimized site.
From case study:Before I released this hack into the wild, I needed to apply this methodology to one last test to determine if it had any validity in the real world. I took a Strangeloop customer who had been through a rigorous month-long 50/50 test. In this particular case, with all other variables accounted for, the optimized site outperformed the unoptimized site. On average, order value for the optimized site was 20% greater than for the unoptimized site
This is what you need to start with.You may have your own variation on this. What’s the best way of showing this?
With a table?This is a table showing the load time and start render time for the top 20 ecommerce sites of the 2009 holiday shopping season. There’s some interesting data here, but if you were a performance newbie, you wouldn’t know it.Tables show that you’ve done your homework and tabulated your data. Okayin the appendix of a performance report, but they’re not goingto light fires under any butts.
Better…Side by side comparison graphic. (It’s easy to create using Webpagetest’s visual comparison feature. First, run your side-by-side test, then click the “Export filmstrip as an image” text link on the bottom of the results page.)You can see how your site loads, frame by frame, compared to your competitors.Interesting, but requires a bit of scrutiny to understand.Good to include in a competitive analysis section of your performance audit, but not a showstopper.
This is when you share the findings of your 5-minute speed/benefit analysis. It can be as simple as Google Analytics screenshots like this.
After you’ve grabbed attention, then start providing big-picture context.Use concrete benchmarks to create goals… and introduce competitiveness.You can get this data from companies like Gomez, which updates benchmarks every week.This index may focus on larger companies, but these numbers are relevant to all companies. Here’s why…
Users don’t care if your company is large or small. They expect all sites to load quickly.
Organizations that recognize the need to take their website’s performance to the next level need to change their basic assumption about acceleration. This change is not a 180-degree turn, however – it’s an evolutionary change. Delivery-based solutions such as CDNs and network devices still form a solid foundation for a total acceleration solution. Transformation-based solutions complement this foundation.
Here we see three waterfall graphs showing how web page objects are delivered from the server to the browser.OriginalThis is an unaccelerated site, with 63 objects making 63 roundtrips between server and browser. The total page load time is 9.5 seconds.DeliveryThis graph shows how a delivery solution – comprised of a content delivery network (CDN) and an application delivery controller (ADC) – shortened these roundtrips by bringing content closer to the user’s browser. There were still 63 roundtrips, but the total page load time was 5.7 seconds.TransformationThis graph shows how Strangeloop worked in conjunction with the CDN and ADC to not just shorten the roundtrips, but reduce the number of roundtrips required – from 63 to just 9. The result: The same page loads in just 2.1 seconds. It is important to remember that 2 seconds is the goal that every site should be aiming for, based on current user expectations. It’s also worth remembering that only one company out of the Fortune 500 actually meets this standard.