Raiders of the Fast Start: Frontend Performance Archeology There are a lot of books, articles, and online tutorials out there with fantastic advice on how to make your websites performant. It all seems easy in theory, but applying best practices to real-world code is anything but straightforward. Diagnosing and fixing frontend performance issues on a large legacy codebase is like being an archaeologist excavating the remains of a lost civilization. You don’t know what you will find until you start digging! Pick up your trowels and come along with Etsy’s Frontend Systems team as we become archaeologists digging into frontend performance on our large, legacy mobile codebase. I’ll share real-life lessons you can use to guide your own excavations into legacy code: What tools and metrics we used to diagnose issues and track progress. How we went beyond server-driven best practices to focus on the client. Which fixes successfully increased conversion, and which didn’t. Our work, like all good archaeology, went beyond artifacts and unearthed new insights into our culture. We at Etsy pride ourselves on our culture of performance, but, like all cultures, it needs to adapt and reinvent itself to account for changes to the landscape. Based on what we’ve learned, we are making the case for a new, organization-wide, frontend-focused performance culture that will solve the problems we face today.

1. STARRING KATIE SYLOR-MILLER AND ETSY FRONTEND SYSTEMS

2. STAFF SOFTWARE ENGINEER
ETSY FRONTEND SYSTEMS
AND OHSHITGIT.COM
Findme@ksylor and@ohshitgit

3. gum.co/ohshit-git/perfnow
gum.co/dangit-git/perfnow
5€!

4. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

6. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

7. Archaeology is the study
of human culture through
it’s physical remains

8. Performance archaeology
uncovers insights into
your development culture

9. Survey
Excavate
Hypothesis
Interpret
Archaeology 101

10. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

11. Our Site: The Mobile Listing Page

12. Hypothesis
Image by HeritageDaily on Wikipedia

13. Mobile Visits
67% 52%
Mobile GMS

14. RUM Dom Content Loaded Times
1000 2000 3000 4000 50000
DOMContentLoaded ms
Numberofvisits

15. Dom Content Loaded Times
15%
37%
24%
13%
6%
1000 2000 3000 4000 50000
DOMContentLoaded ms
Numberofvisits

16. Conversion Vs. Dom Content Loaded
1000 2000 3000 4000 5000
DOMContentLoaded ms
Conversionrate

17. 1000 2000 3000 4000 5000
Conversion Vs. Population
15%
37%
24%
13%
6%
DOMContentLoaded ms
Conversionrate Percentage of visits

18. Improving the performance
of the listing page will
increase conversion
Hypothesis

19. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

20. Survey
Hypothesis

21. Performance Best Practices
Great server response times
Minify and gzip static JS and CSS assets
Resize and compress images
Responsive images (mostly)
Use proper headers to take advantage of browser caching
Use dns-prefetch resource hints
Use HTTP/2 when available
In-house RUM monitoring and alerting
3rd party synthetic monitoring service

22. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

23. Initial Load Performance
How quickly users:
• receive confirmation that the page has started loading?
• see the most important information on the page?
• can interact with the page?

24. Optimize the loading
and rendering of
above-the-fold content
Critical Rendering Path

25. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

26. Ground Penetrating Radar
Image by Tapatio via Wikipedia

27. WebPageTest.org
Ground Penetrating Radar

28. The Mobile Listing Page

29. WebPageTest Results
iPhone 6 iOS9 - 3GSlow
(this is much worse than our typical user experience)
METRIC BEFORE AFTER CHANGE
TTFB 1.8s
Start Render 8.5s
DOM Content Loaded 12.1s
Speed Index 13,193
Time to First Interactive >19s
CSS size 98KB/0.62MB
JS size 386KB/1.46MB
Images 37
Cost $$$$-

30. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

31. Waterfall (before)

32. Waterfall (before)
Five CSS files
Block CSSOM construction
and Start Render

33. Waterfall (before)
Related listings images
(below the fold)

34. Waterfall (before)
Two different sizes of shop avatar image
(1 above, 1 below the fold)

35. Waterfall (before)
Three JS files
Global base (XL), page-specific (L), category nav(S)
Block DOM Content Loaded

36. Waterfall (before)
Similar listing images
(waaaayyyyy below the fold)

37. Five CSS bg-images
One icon font

38. Main listing image is 36th!!!
Followed by more carousel images

39. Reviews images (from async content)
Logging
Logging
Logging

40. $(‘document’).ready()

41. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

42. Areas For Improvement
• Lazyload images:
Defer loading of “below the fold” images
• Reduce CSS File Size:
Combine CSS files and remove unused rules
• Switch to SVGs:
Replace CSS bg-images and icon fonts with SVGs
• Reduce JS File Size:
Remove unused Javascript

43. Survey
Excavate

44. Test Pits

45. Lazyload
Images
Test Pit #1

46. 37 Images
814KB

48. lazy loading synthetic tests
DOMContentLoadedms
1750
3500
5250
7000
Network Speed
Wifi 4G 3G
Before
After Lazy Loading

49. Reduce CSS
File Size
Test Pit #2

50. 5 CSS files
98KB/0.62MB

51. Automation to the Rescue?
• Selenium script opens the page(s) in a headless browser
• Run uncss (https://github.com/uncss/uncss)
• Output a new file with only the CSS in use
… But, we didn’t capture *all* states and had to keep adding
more every time we found a bug.

52. CSS Reduction Synthetic Tests
DOMContentLoaded(ms)
1750
3500
5250
7000
Network Speed
Wifi 4G 3G
Before
After CSS Reduction
After Lazy Loading

53. Lazyloading + CSS Reduction
Experiment Results
User-facing experiment #1

54. RUM Results - Lazyloading + CSS Reduction
METRIC BEFORE AFTER CHANGE
TTFB 1.16s 1.18s 1.2%
DOM Content Loaded 3.3s 3.1s 6% ➡
Page Load 6.0s 5.2s 13.2% ➡

55. RUM Results - Lazyloading + CSS Reduction

56. Conversion

57. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

58. Switch to
SVGs
Test Pit #3

59. 5 bg-images
1 icon font

60. SVG Switch Synthetic Tests
Logged Out
DOMContentLoadedms
1500
3000
4500
6000
Wifi 4G 3G
Before
After SVGs
Logged In
1500
3000
4500
6000
Wifi 4G 3G
Before
After SVGs

61. SVG Switch Results
User-facing Experiment #2

62. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

63. RUM Results - SVG Switch
METRIC BEFORE AFTER CHANGE
TTFB 1.17 1.17s ⬌
DOM Content Loaded 3.0s 3.0s ⬌
Page Load 5.6s 5.5s 1%➡

64. RUM Results - SVG Switch

65. Conversion ⬌

66. WebPageTest.org again
How are we doing?

67. METRIC BEFORE AFTER CHANGE
TTFB 1.8s 2.0s 11%
Start Render 8.5s 4.8s 44% ➡
DOM Content Loaded 12.1s 12.6s ?? 4%
Speed Index 13,193 13,071 ⬌
Time to First Interactive >19s >17s 10%➡
CSS size 98KB/0.62MB 38KB/0.24MB 60%➡
JS size 386KB/1.46MB 375KB/1.59MB 3%
Images 37 25 32%➡
Cost $$$$- $$$-- $➡
WebPageTest Results
iPhone 6 iOS9 - 3GSlow

68. Waterfall - After
Down to 2 CSS Files!

69. Waterfall - After
Listing carousel images!

70. Waterfall - After
One shop avatar image!

71. Waterfall - After
Still two large JS files

72. Waterfall - After
CSS Background image
added back in by experiment

73. Logging
Reviews images (added by new experiment)
Async content

74. Stratigraphy

75. A Brief History of JS at Etsy
2010 - jQuery w/Homegrown system to
concatenate and serve JS files
2011 - Dependencies get inlined using rails-style
//=require ‘path/to/dependencies’
window.Etsy.module;
2012 - RequireJS - AMD dependencies
require([‘path/to/dependencies’],
function(dep) { });
2017 - still a mix of everything :(

77. Reduce JS
File Size
Test Pit #4

78. “Load only what you need”
- Yoav Weiss, today

79. before afterbefore after
121 modules 124 modules
GLOBAL BASE FILE
142kB
712kB
PAGE-SPECIFIC FILE
56kB
264kB

80. Indiana Jones ™ & © 2018 Lucasfilm Ltd.
TYPES

81. Manual JS
Reduction
Test Pit #4a

82. Javascript Excavation
JS dependency
$(‘selector’)
Smarty/Mustache
templates
PHP View
PHP Controller
Experiment flags

83. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

84. before afterbefore after
25% ➡ 33% ➡
GLOBAL BASE FILE
56kB
264kB 42kB
177kB
142kB
712kB
94kB
469kB
PAGE-SPECIFIC FILE

85. WebPageTest Results
METRIC BEFORE AFTER CHANGE
TTFB 2.0s 1.9s 5%➡
Start Render 4.8s 4.87s 1%
DOM Content Loaded 12.6s 9.8s 22%➡
Speed Index 13,071 10,903 16%➡
Time to First Interactive >17s >15s 12%➡
CSS size 38KB/0.24MB 38KB/0.24MB ⬌
JS size 375KB/1.59MB 299KB/1.15MB 20%➡
Images 25 23 8%➡
Cost $$$-- $$$-- ⬌
iPhone 6 iOS9 - 3GSlow

86. Manual JS Reduction Results
User-facing Experiment #3

87. RUM Results - Manual JS Reduction
METRIC BEFORE AFTER CHANGE
TTFB 1.2s 1.2s ⬌
DOM Content Loaded 2.9s 2.8s 2.9%➡
Page Load 5.6s 5.5s 1.6%➡

88. RUM Results - Manual JS Reduction

89. Conversion ➡

90. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

91. Javascript
Instrumentation
Test Pit #4b

92. Code Coverage?

93. window.Etsy =
window.Etsy || {};
window.Etsy.Vimes.register
(“vb383”);
function vb383(id,fn) {
return function() {
window.Etsy.Vimes.start('vb383'
, id);
return fn.apply(this,
arguments);
};
}
return vb383(1, function(args)
{
return stuff;
}
Re-write JS to log
function calls
Capture real user
actions
Send logs onLoad
and onUnLoad
Map # of calls back
to source files
Introducing Vimes

95. before afterbefore after
PAGE-SPECIFIC FILE
28% ➡ 37% ➡
GLOBAL BASE FILE
56kB
264kB 40kB
173kB
142kB
712kB
89kB
448kB

96. Instrumented JS Reduction
Results
Fourth Experiment

97. RUM Results - Instrumented JS
METRIC BEFORE AFTER CHANGE
TTFB 1.2s 1.2s ⬌
DOM Content Loaded 3.0s 2.9s 2.7%➡
Page Load 5.2s 5.1s 1.2%➡

98. RUM Results - Instrumented JS

99. Conversion

101. Excavate
Interpret

102. Performance directly
affects conversion

103. Frontend performance
is just as important as
backend, if not more.

104. Indiana Jones ™ & © 2018 Lucasfilm Ltd.

105. Our experiences are not
our users experiences
(especially on mobile)

106. Small-scale
performance tweaks
don’t always pan out

107. Your architecture
needs to match your
development culture

108. Static top-level
CSS or JS
dependency
managed at the
page level
Dynamic FE
dependency
management at
the component
level
WHAT WE HAVE WHAT WE NEED

109. Architect for deletion

110. Performance culture is
not self-sustaining

111. Thank you!
Indiana Jones ™ & © 2018 Lucasfilm Ltd.
@KSYLOR