Convincing an organization that performance matters and is worth investing in is often a tough thing to sell. This was no different at Intuit, who operated many sites built in the pre “web standards” era. Then, one day, one test changed everything – an A/B comparison successfully demonstrated that faster page loads increased conversion and SEO. And the conversation quickly changed from “Not interested” to “How quickly can you make the rest of our pages faster?”
A performance team was formed, and optimization began across multiple properties in a phased approach with each release delivering incremental performance gains. As we iterated through the core performance principles, the team introduced additional techniques that led us to exceed our original performance goals. Techniques such as lazyloading, prefetching, smarter image optimization/spriting, and module rewrites enabled us to successfully shave off additional time.
This session will cover the steps that we took, lessons learned including what worked well or didn’t work well, as well as the performance improvements that were realized, and their impact on business metrics. Some of the topics include:
* How we went from 15s web pages to 2s web pages
* How combining CSS/JS files and image sprites had both positive as well as negative impact
* How lazy loading of resources and JavaScript rewrites improved our page render times (including our experiments with Control.js)
* How we addressed blocking as well as high-latency third-party components
* How we solved for issues/constraints arising from shared code across multiple sites
* How we optimized for user flows spanning multiple pages with positive results
* How automated benchmarking enabled us to continuously monitor our performance health
* How we succeeded in making “performance” a common theme among developers, marketers, and stakeholders
DevEX - reference for building teams, processes, and platforms
Velocity NY 2013 - From Slow to Fast: Improving Performance on Intuit Websites by up to 5x
1. From Slow To Fast:
Improving Performance On Intuit
Web Sites By Up To 5x
Velocity Conference NY / Oct 16, 2013
Jay Hung (WebMocha) / Norberth Danson (Intuit)
3. Who we are
Jay Hung / @jayhung
Chief Architect, WebMocha LLC
- Ex-Yahoo engineer
- 16+ yrs in consumer web + digital media
- Performance consultant; frontend
optimization since 2007
Norberth Danson
Staff Software Engineer, Intuit Inc.
- Full stack engineer
- 9+ yrs web development
- Member of Intuit core performance team
12. Engineering presented a case:
• cited experts like Steve Souders
• research/examples from Google / Bing / Yahoo
• blog posts on performance lifting conversions
But people didn’t get it.
Couldn’t get buy-in.
Image source: DJ Octoon
13. Engineering presented a case:
• cited experts like Steve Souders
• research/examples from Google / Bing / Yahoo
• blog posts on performance lifting conversions
But people didn’t get it.
Couldn’t get buy-in.
Image source: DJ Octoon
15. A/B test showed a faster page
increased conversion, improved
SEO & lifted organic traffic.
people
16. Conversation went from “not interested”
to “how soon can we make
all of our pages faster?”
people
Image source: onlinemarketingwithvince.com
17. 80-90% of the enduser response time
is spent on the
frontend.
Start there.
people
Steve Souders - Performance Golden Rule
18. But before we could start, we needed:
Image source: selfmadefitness.com
19. Common Ground
• Pageload times are frontend (client-side) only
• Common tool (webpagetest.org)
• DSL connection speed
• Doc Complete load time
• Average from 9 iterations
Image source: selfmadefitness.com
25. Performance Rule #1
Minimize HTTP requests
General Problem:
–Each http request has overhead
• Connection setup/teardown
• DNS lookup times
Our Problem:
–Excessive
–Excessive
–Excessive
–Excessive
CSS files (10+)
JS files (30+)
images (50-70+)
beacons (20+)
26. Merge + Minify CSS/JS
NOT SO EASY!
– Many shared files & embedded snippets
– Numerous copy-n-paste variations
– document.write() statements around every corner
– Expensive DOM traversals
– Non-contained CSS
– Global JS vars & functions everywhere
Bleeding and collisions galore!
27. Merge + Minify CSS/JS
HOW SOLVED
• Cleaned & refactored all CSS/JS
• Optimized handful of JS DOM Traversals
• Created global-global mergeset (shared across all sites)
• Created site-global mergesets (shared across all pages of same-site)
A BIG WIN, BUT WITH SOME COST
• CSS + JS containment extremely tedious
• 2 uber mergesets == lots of unused code on most pages
• Other pages aside from our top-50 included these CSS/JS files
• Other sites (outside of our 6 sites) also included these CSS/JS files
– all hell broke loose
• Seems every commit, more things broke
Took FAR longer than expected, but we got it done.
28. Merge + Minify CSS/JS
HOW SOLVED
• Cleaned & refactored all CSS/JS
• Optimized JS DOM Traversals
• Created global-global mergeset (shared across all sites)
• Created site-global mergesets (shared across all pages of same-site)
A BIG WIN, BUT WITH SOME COST
• CSS + JS containment extremely tedious
• 2 uber mergesets == lots of unused code on most pages
• Other pages aside from our top-50 included these CSS/JS files
• Other sites (outside of our 6 sites) also included these CSS/JS files
– all hell broke loose
• Seems every commit, more things broke
Took FAR longer than expected, but we got it done.
30. Spriting Woes – Take 1
PROBLEM
• This image is a whopping 306k as a 24-bit PNG
alpha-transparent gradients
HOW SOLVED
• Worked with design, removed alpha-transparency
• Saved as a JPG (came in < 50K)
• Saved > 250k
31. Spriting Woes – Take 1
PROBLEM
• This image is a whopping 306k as a 24-bit PNG
alpha-transparent gradients
HOW SOLVED
• Worked with design, removed alpha-transparency
• Saved as a JPG (came in < 50K)
• Saved > 250k
32. Spriting Woes – Take 2
PROBLEM
• This image is 179K as a 24-bit PNG
HOW SOLVED
alpha-transparent
only used if no Flash
installed, < 1%!
• If get rid of third image, can save as JPG (< 50k)
• Instead, we rewrote player code to show play button as icon (< 20k)
• Saved > 150k
33. Spriting Woes – Take 3
Each screen: 1024x768
Dimensions: 4096x9216
Filesize: 5MB!
34. Spriting Woes – Take 3
Each screen: 1024x768
Dimensions: 4096x9216
Filesize: 5MB!
35. Spriting Woes – Take 3
Each screen: 1024x768
Dimensions: 4096x9216
Filesize: 5MB!
36. CDN (Content Delivery Network)
THE GOOD THING
• Already using Akamai
PROBLEM
• Wasn’t configured properly
– Gzip, Expires, etc.
• Assets loaded from multiple sources
– Some from CDN
– Others from app servers
• All sites shared same CDN domain
– images.smallbusiness.intuit.com
37. Angry Cookie Monster
PROBLEM
• Domain cookies are passed with every http request for that
domain
• CDN domain: images.smallbusiness.intuit.com
• Avg cookie size: ~800b-1k
• Total cookie bytes/page = ~100k+
– ~1k * ~100 reqs
Image source: cheezburger.com
38. Happy Cookie Monster
HOW SOLVED
• Reconfigured Akamai
• Load ALL static assets from CDN
• Cookie-less CDN domain:
– {sitename}-s{#}.intuitstatic.com
Image source: birdonacake.blogspot.com
39. Beacons + Redirects
PROBLEM
• Marketing sites (we tracked everything)
• Excessive beacons (20-30+)
• Many 302-redirects (adds http requests)
• Over the years, many beacons added
– but hardly any removed
Image credit: GunGodTheGreat via deviantart.com
40. Beacons + Redirects
HOW SOLVED
• Inventoried all beacons
• Benchmarked all beacons
• Asked marketers to review
– Removed many legacy beacons
• Pinged vendors for current beacons that
were sub-optimal
– replaced with faster versions
Image source: wikipedia.org
41. Beacons + Redirects
HOW SOLVED
• Inventoried all beacons
• Benchmarked all beacons
• Asked marketers to review
– Removed many legacy beacons
• Pinged vendors for current beacons that
were sub-optimal
– replaced with faster versions
Image source: wikipedia.org
42. Other Optimizations
• Recompressed all images
– low hanging fruit + a big win
• Lazyloaded images below fold
– another big win
• Removed custom fonts
• Removed duplicate code
• Removed more 30x redirects
• Remaining usual suspects
43. Eventually, we hit 6s… yay!
But then the
question came…
Can we go faster?
51. Blocking Calls
• Notice staircase pattern
• Milliseconds all add up
• These specific calls returned no content
– Calls w/ content take even longer
• These + 2 calls not shown == ~750ms
– Nearly 3/4 of a second to show nothing different
52. Redundant Content, Multiple Payloads
Default
content
Default
content
Test A
Default
content
Test B
• all users download default content
• test case users download both default + test content
53. Poor Performing Architecture
blocking call – if loads content,
traverses DOM and inserts here
• Inherent high latencies
• Code that blocks the UI thread
• Lots of these calls on a page
• All adds up!
54. A/B Testing Software
HOW SOLVED
✓ Seek & destroy
✓ Removed expired tests
✓ Comment out inactive tests
✓ Switched test software
59. New Global Header – aka QuickNav
✓ Full replacement
✓ Utilizes web standards HTML/CSS/JS
✓ Minimizes DOM traversals
✓ Minimizes image utilization
✓ Uses event delegation
✓ Across the board, every page had 1s-1.5s faster load time
60. JavaScript & Control.js
• created by Steve Souders
• designed to give control over how JS downloads and executes
• separates download and execution phases
• downloads scripts as either IMAGE or OBJECT elements
• scripts subsequently fetched from CACHE
Read more about Control.js:
http://stevesouders.com/controljs/
61. Prefetch assets
• prefetching assets across a flow of pages vs. single page
• examples: Sign-up flow or conversion funnel
– At start of flow, as user idles, reads or types in form fields, we
prefetch assets in the background
– As user hits subsequent page needing this asset, it’s already
available in the cache resulting in a speedier load time
62. Other Phase 2 Optimizations
• JavaScript rewrites
• Advanced lazyloading
– components, functionality, etc.
• Internal WebPageTest
– behind firewall testing
• Automated merge/minify
• Automated performance
monitoring
• Changed technology
infrastructure
– for some products (faster
stack)
• Performance Leaderboard
– glass half-full interpretation of
Wall-of-Shame
• Performance KPIs added to
Stakeholder dashboards
• Bunch of other stuff
71. Tough to measure!
• Lots of chatter about positive lifts in metrics
• Hard to credit results solely to performance
• Best if no other changes besides performance
• (Uh, good luck with that in fast-paced business!)
72. lookups
When
* 2012week14 (week/week improvement)
only change was performance
pageload
pageload
9-12s
3-4s
5-6s
+14%*
then
conversions
conversions
upgrade
Payroll In-product Purchase
73. Performance & Shop Cart Checkouts
Slow cart == more abandonment
Source: Intuit, Inc. (shopping cart completions)
74. Performance & Shop Cart Checkouts
Sped up cart == more completions
Source: Intuit, Inc. (shopping cart completions)
75. QuickBooks Online Signup Flow
-2s total pageload == +10% conversion*
~6s
QBO
Marketing Page
~6s
Signup:
Account Info
~6s
Signup:
Credit Card Info
Only change this period was performance
* FY2012Q4, Source: Jeremy Goodrich (Intuit, Inc.)
~8s
~8-10s
QBO
App Home
76. Generally
For every 1s of performance gain:
Pageload Time
Conversion
Pageload >= 7s
+3%
7s > Pageload > 5s
+2%
4s >= Pageload
+1%
(as long as final pageload time is 5s or faster)
Diminishing business ROI as pages get faster.
78. Getting Buy-in
• Stick with it!
• Evangelize
• Get a champion
• Do experiments
squeeky wheel gets the oil
Photo credit: Denis De Mesmaeker via flickr.com
79. Start with:
• It does work
• Measurable
improvements
• Code clean-up prior
is good idea
Photo credit: Gregory Mancuso
80. One size does not fit all
• Figure out your issues
• If you don’t know what’s
slow, you can’t fix it
• Deep analysis, find
bottlenecks, fix each one
• Learn to read waterfall
charts!
• Best results
Photo credit: spiritwomen.blogspot.com
81. Dedicated performance team?
• If you can afford it, YES!
• No competing priorities
• Highest chance of success
• (That’s what we did)
• (Steve Souders advocates
for this)
Image source: bearportpublishing.com
82. Maintaining Performance
• Many sites regress when stop
watching
• Pay attention!
• Setup monitoring/alerts
– Jenkins + WPT Private Instance
• Weekly dashboards
• Wall of Shame (or
Leaderboard Wall)
• Talk performance. Often.
stay on top of it!
Photo credit: Tobin Copley via flickr.com
83. Our Humble Opinion
(sample size of 2)
• 99% of web sites can be made faster
• Most of us are not in that
• Our chances for a faster web site
are very good!
84. Photo credit: Nathan Kowald via flickr.com
Questions?
people
Slides: webmocha.com/2013velocityny
jayhung@webmocha.com
@jayhung
norberth_danson@intuit.com
Notes de l'éditeur
Hello, and welcome.Today, we’re really excited to share with you some of the successes we’ve had at Intuit around performance.
This is our story: We spent a year, optimizing the top 50 pages (by traffic) from 6 of our marketing web sites, and hit pageload speeds as fast as 2 seconds in some cases.There’s a lot we’ll be covering in a short amount of time, so we might talk a little fast. We’ll have time for questions at the end, or stop by our office hours immediately following this presentation.
This is what our web site looked like back in the day.The first version came online in mid 90s. This screenshot is from 1999.As you can see, it’s quite basic compared to what we’re used to today.
Any of you remember this phone?I’m probably dating myself, but I had one of these bad boys back then and thought it was the coolest thing ever.
It was built on dated technology.And it was built before web standards.
Over the years, lots of “stuff” was added or changed.Single site grew and became many sites.All on same codebase.
Codebase big and bloated.LOTS of stuff going on.Everything plugged into everything else.Tough to maintain.
Something had to be done.Engineering presented a case… People just didn’t get it. Nobody understood value.Everyone too busy and no time.
A frustrated engineer, in his own time, beganquietly doing performance optimizations.
It successfully showed improved metrics as a result of a faster loading page.
What? Higher conversions? Higher SEO rankings? Morepageviews?I get it now!
Everyone familiar with this quote?Steve first coined it back in 2006 or 2007. Still holds true today.
Apples to apples.So what do I mean?Get everyone on common ground.Lots of chatter about performance – a lot sounded the same - but not everyone was saying the same thing.Numbers cited from different tools (which had wide variances).Server response times unintentionally mixed with frontend render times.We just needed to get everyone comparing apples to apples.
We start with the usual suspects.
The usual suspects are the original rules of high performance from Steve Souder’s first book on high performing web sites.These are general rules, in order of priority, that applies to the majority of sites out there.We added a few bonus items of our own.We won’t have time to go over everything, but we’ll cover a number of our biggest wins (and mistakes).
So, we started out by merging and minifying our CSS and JS files.The codebase was not very clean, and had all of “this stuff” going on with it.Extremely difficult to combine everything together and still have the site working properly.
… we kind of went at this blindly and jumped in headfirst and started to combine files left and right.Knowing what we know now, we may have spent an initial period getting our files into shape for merging and minifying.
Pretty simple here.A good thing, right? Well, let’s take a look.
Describe problem. Then how solved:In our case, design was flexible, they modified the design to remove the alpha-transparency.Saved over 250k. Let’s think about that number for a second – 250k. That’s a lot of overhead, for a single image!
That’s over a quarter of a megabyte on one image alone.
3rd sprite on right uses alpha transparency which causes the entire image to be saved as 24-bit when sprited together.But this 3rd sprite is only used for users that do not have Flash installed – this is < 1% of Internet users!This 3rd image was incorrectly sprited with the other images.Between this image and the last image, we saved 400k – close to half a meg of payload size.The funny thing here is, it actually gets worse.
(image displays): I’m just going to pause here for a moment, to let you take in the full glory of this image - and let the absurdity sink in.(click for bullets)
Only thing I can say about this, is it’s 5 megs of wrong.
These examples just goes to show how we took something good likespriting images and reducing http requests – and executed wrong, even hurting our performance in the process.Anyways, we fixed all of this and moved on.
I’ll circle back to this shortly, but before I do, I want to talk about something we call “the angry cookie monster”.
Common for marketers to request pixels to be added when new campaigns began.Very rare to request pixels be removed once a campaign has finished.
After 6 months of work, we managed to get all 50 pages into that ~6s rangeBut it wasn’t enough. The company had tasted the forbidden fruit, and wanted more.Kept coming up:How can we go faster?Perhaps car images should be moved to next slide.Perhaps we don’t mention 2 seconds here, but just “can we go faster?” and we can talk about how we got rationale for 2s
Thank you Jay! Hello everyone, thank you for stopping by!My name is Norberth Danson and I’m a full stack engineer currently working on front end and performance at Intuit. ! I will take this opportunity to come out clean and confess that I do not have a Twitter account. ! Now that I’ve got that off my chest, let’s get straight to business.As Jay mentioned, it took us roughly six months of optimization effort to bring the top 50 URLs down to approximately 6 seconds.And then two things happened.At first, good times rolled in. We were rock stars. It was pretty awesome to be part of the performance team.
As I mentioned before, there were 3 video players, with various levels of inefficiency.Let’s look at one of them.In theory, a video player is supposed to play video.In practice, especially these days, when everyone wants analytics around every corner, most video players come with an Entourage.Allow me to introduce the cast members:SWF file (actual player) takes 6-8s on avg to downloadLoad of external calls for tracking, beacons, and 3rd party elements (23 requests, 9 domains, 7 swf files) – even the NSA would have trouble keeping track of everything that’s going on!!!Poster frame still image displayed on page load takes >3s on avg to loadPage BLOCKS until poster frame image loaded; rest of page elements have to wait before downloadingEntourages are expensive and this example is not different – the parent page has to pay the price for loading this video player.
To recap – not all video players are created equal – make sure that the one you are using is not accompanied by a vast entourage.When a video player does 19 redirects before playing its first byte of video, it’s time to look for alternatives.And that’s exactly what we did – we replaced the 3 video players with one player from a different vendor. If replacing the vendor is not an option, you may look into lazy loading the video player. If you have to pay the price, pay the price after the browser’s onload event fires.
Launched a new platform, pageload of landing page went from 3-4s to 9-12sConversions took a hit.After improving pages back to 5-6s, conversions improved by 14%In this case, conversion == # of new activations
In the summer during our year of optimization, we launched a new technology platform that the shopping cart was on.This version of the platform had forked before our phase 1 optimizations had launched, and performance immediately suffered.At launch, we had some irregular tracking that was over-counting our shopping cart completions, and we fixed that a month in.From then on, this section in red showed severely degraded performance, and we saw a reduced number of people completing their checkout flow.
When we finally fixed this and was able to re-optimize the shopping cart, we see the number of people completing their checkout improving significantly.Performance was the only thing changed on the cart.
Finally, we’d like to leave you with our humble opinion.Unfortunately, it’s a samplesize of 2.