1. From Slow To Fast:
Improving Performance On Intuit
Web Sites By Up To 5x
Velocity Conference NY / Oct 16, 2013
Jay Hung (WebMocha) / Norberth Danson (Intuit)

2. Our Story
1 year. 50 pages. 6 sites. 2 seconds.

3. Who we are
Jay Hung / @jayhung
Chief Architect, WebMocha LLC
- Ex-Yahoo engineer
- 16+ yrs in consumer web + digital media
- Performance consultant; frontend
optimization since 2007
Norberth Danson
Staff Software Engineer, Intuit Inc.
- Full stack engineer
- 9+ yrs web development
- Member of Intuit core performance team

4. The early
days…
Image source: Internet Archive (edited)

5. Built when this was
top of the line…
Image source: Eliot Phillips (aka RobotSkirts) via Flickr

7. Over the years…
www.intuit.com
quickbooks.intuit.com
quickbooksonline.intuit.com
payroll.intuit.com
payments.intuit.com
websites.intuit.com
and more…
1996-1999
2012

8. …it became this:
+
Image source: Jesse Hernandez Photography
Image source: vibrant.com

9. …it also became this:
+
Image source: Jesse Hernandez
Image source: vibrant.com

10. Negatively affecting:
• Users’ Experience
• SEO
• Pageviews
• Conversions/Sign-ups

11. Downright embarrassing!
Image source: Alex E. Proimos via Flickr

12. Engineering presented a case:
• cited experts like Steve Souders
• research/examples from Google / Bing / Yahoo
• blog posts on performance lifting conversions
But people didn’t get it.
Couldn’t get buy-in.
Image source: DJ Octoon

13. Engineering presented a case:
• cited experts like Steve Souders
• research/examples from Google / Bing / Yahoo
• blog posts on performance lifting conversions
But people didn’t get it.
Couldn’t get buy-in.
Image source: DJ Octoon

14. Then one experiment
changed everything.
people

15. A/B test showed a faster page
increased conversion, improved
SEO & lifted organic traffic.
people

16. Conversation went from “not interested”
to “how soon can we make
all of our pages faster?”
people
Image source: onlinemarketingwithvince.com

17. 80-90% of the enduser response time
is spent on the
frontend.
Start there.
people
Steve Souders - Performance Golden Rule

18. But before we could start, we needed:
Image source: selfmadefitness.com

19. Common Ground
• Pageload times are frontend (client-side) only
• Common tool (webpagetest.org)
• DSL connection speed
• Doc Complete load time
• Average from 9 iterations
Image source: selfmadefitness.com

20. Our Mission
Reduce page load times
for top 50 pages of
6 marketing sites by ~50%
people

21. Phase 1:
people

22. Original Rules of High Performance
people
Image credit: Gregory Mancuso

23. Original Rules of High Performance
1.
2.
3.
4.
5.
6.
7.
Minimize HTTP requests
Use CDN
Add expires header
Gzip components
Put stylesheets at top
Put scripts at bottom
Avoid CSS expressions
Bonus 1:
• Compress images
• Lazyload images
• Removed custom fonts
8. Make JS & CSS external
9. Reduce DNS lookups
10. Minify JS & CSS
11. Avoid redirects
12. Remove duplicate scripts
13. Configure Etags
14. Make Ajax cacheable
Bonus 2:
• Cookie-less domains
• Reduce beacons
• Optimize JS/DOM

24. A Typical Page
• Total Bytes: 1.5M-2M+
• Images: 50-70+ (1.2M)
• External CSS/JS: 30-40+
• JavaScript: 400k+
• Redirects: 20+
• HTTP Requests: 120+

25. Performance Rule #1
Minimize HTTP requests
General Problem:
–Each http request has overhead
• Connection setup/teardown
• DNS lookup times
Our Problem:
–Excessive
–Excessive
–Excessive
–Excessive
CSS files (10+)
JS files (30+)
images (50-70+)
beacons (20+)

26. Merge + Minify CSS/JS
NOT SO EASY!
– Many shared files & embedded snippets
– Numerous copy-n-paste variations
– document.write() statements around every corner
– Expensive DOM traversals
– Non-contained CSS
– Global JS vars & functions everywhere
Bleeding and collisions galore!

27. Merge + Minify CSS/JS
HOW SOLVED
• Cleaned & refactored all CSS/JS
• Optimized handful of JS DOM Traversals
• Created global-global mergeset (shared across all sites)
• Created site-global mergesets (shared across all pages of same-site)
A BIG WIN, BUT WITH SOME COST
• CSS + JS containment extremely tedious
• 2 uber mergesets == lots of unused code on most pages
• Other pages aside from our top-50 included these CSS/JS files
• Other sites (outside of our 6 sites) also included these CSS/JS files
– all hell broke loose
• Seems every commit, more things broke
Took FAR longer than expected, but we got it done.

28. Merge + Minify CSS/JS
HOW SOLVED
• Cleaned & refactored all CSS/JS
• Optimized JS DOM Traversals
• Created global-global mergeset (shared across all sites)
• Created site-global mergesets (shared across all pages of same-site)
A BIG WIN, BUT WITH SOME COST
• CSS + JS containment extremely tedious
• 2 uber mergesets == lots of unused code on most pages
• Other pages aside from our top-50 included these CSS/JS files
• Other sites (outside of our 6 sites) also included these CSS/JS files
– all hell broke loose
• Seems every commit, more things broke
Took FAR longer than expected, but we got it done.

29. Spriting Images
• Combines images
• Reduces HTTP requests
A good thing, right?
…

30. Spriting Woes – Take 1
PROBLEM
• This image is a whopping 306k as a 24-bit PNG
alpha-transparent gradients
HOW SOLVED
• Worked with design, removed alpha-transparency
• Saved as a JPG (came in < 50K)
• Saved > 250k

31. Spriting Woes – Take 1
PROBLEM
• This image is a whopping 306k as a 24-bit PNG
alpha-transparent gradients
HOW SOLVED
• Worked with design, removed alpha-transparency
• Saved as a JPG (came in < 50K)
• Saved > 250k

32. Spriting Woes – Take 2
PROBLEM
• This image is 179K as a 24-bit PNG
HOW SOLVED
alpha-transparent
only used if no Flash
installed, < 1%!
• If get rid of third image, can save as JPG (< 50k)
• Instead, we rewrote player code to show play button as icon (< 20k)
• Saved > 150k

33. Spriting Woes – Take 3
Each screen: 1024x768
Dimensions: 4096x9216
Filesize: 5MB!

34. Spriting Woes – Take 3
Each screen: 1024x768
Dimensions: 4096x9216
Filesize: 5MB!

35. Spriting Woes – Take 3
Each screen: 1024x768
Dimensions: 4096x9216
Filesize: 5MB!

36. CDN (Content Delivery Network)
THE GOOD THING
• Already using Akamai
PROBLEM
• Wasn’t configured properly
– Gzip, Expires, etc.
• Assets loaded from multiple sources
– Some from CDN
– Others from app servers
• All sites shared same CDN domain
– images.smallbusiness.intuit.com

37. Angry Cookie Monster
PROBLEM
• Domain cookies are passed with every http request for that
domain
• CDN domain: images.smallbusiness.intuit.com
• Avg cookie size: ~800b-1k
• Total cookie bytes/page = ~100k+
– ~1k * ~100 reqs
Image source: cheezburger.com

38. Happy Cookie Monster
HOW SOLVED
• Reconfigured Akamai
• Load ALL static assets from CDN
• Cookie-less CDN domain:
– {sitename}-s{#}.intuitstatic.com
Image source: birdonacake.blogspot.com

39. Beacons + Redirects
PROBLEM
• Marketing sites (we tracked everything)
• Excessive beacons (20-30+)
• Many 302-redirects (adds http requests)
• Over the years, many beacons added
– but hardly any removed
Image credit: GunGodTheGreat via deviantart.com

40. Beacons + Redirects
HOW SOLVED
• Inventoried all beacons
• Benchmarked all beacons
• Asked marketers to review
– Removed many legacy beacons
• Pinged vendors for current beacons that
were sub-optimal
– replaced with faster versions
Image source: wikipedia.org

41. Beacons + Redirects
HOW SOLVED
• Inventoried all beacons
• Benchmarked all beacons
• Asked marketers to review
– Removed many legacy beacons
• Pinged vendors for current beacons that
were sub-optimal
– replaced with faster versions
Image source: wikipedia.org

42. Other Optimizations
• Recompressed all images
– low hanging fruit + a big win
• Lazyloaded images below fold
– another big win
• Removed custom fonts
• Removed duplicate code
• Removed more 30x redirects
• Remaining usual suspects

43. Eventually, we hit 6s… yay!
But then the
question came…
Can we go faster?

44. Phase 2:
6s to 2s
people

45. 6
seconds?!
Very nice!
Photo credit: http://merionminor.deviantart.com/

46. Photo credit: David Endelman, NetCarShow.com

47. • NEW RULE: go beyond the rules

48. • Deep analysis to find Intuit bottlenecks!

49. Our Biggest Bottlenecks
• A/B Testing Software
• Inefficient video players
• Slooow global header full of
surprises
• … good ole JavaScript

50. A/B Testing Software

51. Blocking Calls
• Notice staircase pattern
• Milliseconds all add up
• These specific calls returned no content
– Calls w/ content take even longer
• These + 2 calls not shown == ~750ms
– Nearly 3/4 of a second to show nothing different

52. Redundant Content, Multiple Payloads
Default
content
Default
content
Test A
Default
content
Test B
• all users download default content
• test case users download both default + test content

53. Poor Performing Architecture
blocking call – if loads content,
traverses DOM and inserts here
• Inherent high latencies
• Code that blocks the UI thread
• Lots of these calls on a page
• All adds up!

54. A/B Testing Software
HOW SOLVED
✓ Seek & destroy
✓ Removed expired tests
✓ Comment out inactive tests
✓ Switched test software

55. Video Player

56. Video Player
PROBLEM
• Poor for performance
• Callout smorgasbord
HOW SOLVED
✓ Replace vendors & consolidate
✓ Lazy load video player

57. Global header – aka World Class Nav
•
•
•
•
•
•

58. Global header – aka World Worst Class Nav
•
•
•
•
•
•

59. New Global Header – aka QuickNav
✓ Full replacement
✓ Utilizes web standards HTML/CSS/JS
✓ Minimizes DOM traversals
✓ Minimizes image utilization
✓ Uses event delegation
✓ Across the board, every page had 1s-1.5s faster load time

60. JavaScript & Control.js
• created by Steve Souders
• designed to give control over how JS downloads and executes
• separates download and execution phases
• downloads scripts as either IMAGE or OBJECT elements
• scripts subsequently fetched from CACHE
Read more about Control.js:
http://stevesouders.com/controljs/

61. Prefetch assets
• prefetching assets across a flow of pages vs. single page
• examples: Sign-up flow or conversion funnel
– At start of flow, as user idles, reads or types in form fields, we
prefetch assets in the background
– As user hits subsequent page needing this asset, it’s already
available in the cache resulting in a speedier load time

62. Other Phase 2 Optimizations
• JavaScript rewrites
• Advanced lazyloading
– components, functionality, etc.
• Internal WebPageTest
– behind firewall testing
• Automated merge/minify
• Automated performance
monitoring
• Changed technology
infrastructure
– for some products (faster
stack)
• Performance Leaderboard
– glass half-full interpretation of
Wall-of-Shame
• Performance KPIs added to
Stakeholder dashboards
• Bunch of other stuff

63. So… how did we do?
people

64. www.intuit.com
Feb 15, 2012
Apr 15, 2013
• 15 seconds
• 3.6 seconds
• 2.3s start render
• 1259kb payload
• 1.7s start render
• 407kb payload
Source: httparchive.org

65. www.intuit.com
Feb 15, 2012
Apr 15, 2013
• 15 seconds
• 3.6 seconds
• 2.3s start render
• 1259kb payload
• 1.7s start render
• 407kb payload
Source: httparchive.org

66. www.intuit.com
Feb 15, 2012
Apr 15, 2013
Source: webpagetest.org

67. payments.intuit.com
Mar 14, 2012
Feb 15, 2013
Source: webpagetest.org

68. payments.intuit.com
Mar 14, 2012
Feb 15, 2013
Source: webpagetest.org

69. All 6 Sites
Source: Intuit, Inc.

70. Business Impact
people

71. Tough to measure!
• Lots of chatter about positive lifts in metrics
• Hard to credit results solely to performance
• Best if no other changes besides performance
• (Uh, good luck with that in fast-paced business!)

72. lookups
When
* 2012week14 (week/week improvement)
only change was performance
pageload
pageload
9-12s
3-4s
5-6s
+14%*
then
conversions
conversions
upgrade
Payroll In-product Purchase

73. Performance & Shop Cart Checkouts
Slow cart == more abandonment
Source: Intuit, Inc. (shopping cart completions)

74. Performance & Shop Cart Checkouts
Sped up cart == more completions
Source: Intuit, Inc. (shopping cart completions)

75. QuickBooks Online Signup Flow
-2s total pageload == +10% conversion*
~6s
QBO
Marketing Page
~6s
Signup:
Account Info
~6s
Signup:
Credit Card Info
Only change this period was performance
* FY2012Q4, Source: Jeremy Goodrich (Intuit, Inc.)
~8s
~8-10s
QBO
App Home

76. Generally
For every 1s of performance gain:
Pageload Time
Conversion
Pageload >= 7s
+3%
7s > Pageload > 5s
+2%
4s >= Pageload
+1%
(as long as final pageload time is 5s or faster)
Diminishing business ROI as pages get faster.

77. Takeaways
people

78. Getting Buy-in
• Stick with it!
• Evangelize
• Get a champion
• Do experiments
squeeky wheel gets the oil
Photo credit: Denis De Mesmaeker via flickr.com

79. Start with:
• It does work
• Measurable
improvements
• Code clean-up prior
is good idea
Photo credit: Gregory Mancuso

80. One size does not fit all
• Figure out your issues
• If you don’t know what’s
slow, you can’t fix it
• Deep analysis, find
bottlenecks, fix each one
• Learn to read waterfall
charts!
• Best results
Photo credit: spiritwomen.blogspot.com

81. Dedicated performance team?
• If you can afford it, YES!
• No competing priorities
• Highest chance of success
• (That’s what we did)
• (Steve Souders advocates
for this)
Image source: bearportpublishing.com

82. Maintaining Performance
• Many sites regress when stop
watching
• Pay attention!
• Setup monitoring/alerts
– Jenkins + WPT Private Instance
• Weekly dashboards
• Wall of Shame (or
Leaderboard Wall)
• Talk performance. Often.
stay on top of it!
Photo credit: Tobin Copley via flickr.com

83. Our Humble Opinion
(sample size of 2)
• 99% of web sites can be made faster
• Most of us are not in that
• Our chances for a faster web site
are very good!

84. Photo credit: Nathan Kowald via flickr.com
Questions?
people
Slides: webmocha.com/2013velocityny
jayhung@webmocha.com
@jayhung
norberth_danson@intuit.com