In this presentation I go through why a wide range of data is useful in understanding the performance optimisation competitive landscape. It helps us know how fast is fast... helps us add context and boundaries to our projects, as well as helps us know when we're doing a good job or when we need to push harder to get faster.

1. THE GIANT MOBILE SITE
SPEED BENCHMARK
Your Emotional Support Dog
@NICHOLASTOTT

2. “It is a capital mistake to theorize before one
has data. Insensibly one begins to twist facts
to suit theories, instead of theories to suit
facts”.
Sir Arthur Conan Doyle

3. WE BELIEVE THAT…
#SEARCHELITE @NICHOLASTOTTSOURCE LINK
Users hate
slow sites

4. WE BELIEVE THAT…
#SEARCHELITE @NICHOLASTOTTSOURCE LINK
Faster sites
convert better

5. WE BELIEVE THAT…
#SEARCHELITE @NICHOLASTOTT
Speed - More
important to mobile
ranking

6. FASTER THAN THE
COMPETITION

7. VISIT THE BENCHMARK

8. HOW DID WE DO IT, WHAT DID
WE LEARN AND HOW CAN WE
USE IT?
#SEARCHELITE#SEARCHELITE @NICHOLASTOTT

9. RESEARCH & DEVELOPMENT
#SEARCHELITE @NICHOLASTOTT

10. #SEARCHELITE @NICHOLASTOTT
√
App Manifest
Service Worker Registration
Satisfy speed criteria
Use HTTPS
√
√
√
√

11. 2VISIT THE STUDY
Mobile Readiness & Progressive Web
Apps in the UK
Are Digital Leaders Adapting to Mobile Audiences?

12. 5 LIGHTHOUSE PWA AUDIT CRITERIA
1. HTTPS – is the site served over HTTPS?
2. Service Worker – does the site register a Service Worker?
3. Speed – <10,000 TTFI/CI
4. Add to Home Screen – user prompt ‘‘Add to Home Screen’’?
5. Content Sized for Mobile Viewport – responsively or adaptively?
#SEARCHELITE @NICHOLASTOTT

16. Using Node CLI to
Automate at Scale…
run: lighthouse <url>
#SEARCHELITE @NICHOLASTOTT

17. PWA-SPECIFIC FEATURES?
#SEARCHELITE @NICHOLASTOTT

18. Hi! Love this analysis! Did
you know that httparchive
has lighthouse results for all
of it's URLs in BigQuery?
Oh…
#SEARCHELITE @NICHOLASTOTT

19. OVER 5,000 WEBSITES WITH
SEVEN DISTINCT
PERFORMANCE- RELATED
COMPARABLE CRITERIA…
#SEARCHELITE @NICHOLASTOTT

20. #SEARCHELITE @NICHOLASTOTT
HTTP ARCHIVE
SOURCE: RICK VISCOMI
500K
Alexa URLS
Pulls Lighthouse Audit Data Queryable Database

21. #SEARCHELITE @NICHOLASTOTT
Dev-facing
presentation

22. @NicholaStott
Thinking about Performance
Optimisation less in terms of
developer metrics and more about
delivering on experience and
improving perception…
Speed, Perception & Lighthouse – Weyl and Ahammad
MINDSET SHIFT

23. @NicholaStottLighthouse: More on PSI Measures
User
results-
facing
NEW LIGHTHOUSE METRICS

24. #SEARCHELITE @NICHOLASTOTT
No TTFB there is…
Evolved, we have...

25. LIGHTHOUSE MOBILE CONFIG
User agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/65.0.3325.181 Safari/537.36
Device Emulation Nexus 5X: Enabled
Network Throttling 562.5ms RTT, 1.4Mbps down,
0.7Mbps up: Enabled
CPU Throttling 4x slowdown: Enabled
#SEARCHELITE @NICHOLASTOTT

26. STEP 1
#SEARCHELITE @NICHOLASTOTT
Data Hygiene
5,239 URLS
• Checked all still return 200
• Removed couple of counterfeit
sites
• Removed handful of seriously
anomalous outliers

27. STEP 2
#SEARCHELITE @NICHOLASTOTT
Classify 5,010
sites

28. STEP 2… IN NUMBERS
#SEARCHELITE @NICHOLASTOTT
Classify 5,010
sites

29. HOW DID WE DO IT, WHAT DID
WE LEARN AND HOW CAN WE
USE IT?
#SEARCHELITE @NICHOLASTOTT

30. #SEARCHELITE @NICHOLASTOTT
FASTEST TTFB
Other, News &
Media Fastest
TTFB
All <2 second

31. #SEARCHELITE @NICHOLASTOTT
VC/CI
News & Media
now slowest
The big 3 -
slowest
SLOWEST SITES

32. The most popular consumer
websites offer the slowest
mobile experience.
#SEARCHELITE @NICHOLASTOTT

33. renderStart = FCP
onContentLoaded =
FMP
visualComplete = CI
RETAIL: DEEP-DIVE OVERVIEW
#SEARCHELITE @NICHOLASTOTT

34. METRICS EQUIVALENT
WebPagetest Lighthouse
TTFB Gone
renderStart FCP
onContentLoaded FMP
visualComplete TTFI/CI
fullyLoaded Same
#SEARCHELITE @NICHOLASTOTT

35. #SEARCHELITE @NICHOLASTOTT
FCP: MEDIAN & QUARTILES (S)
Top Quartile: 2.06
Median: 2.87
Bottom Quartile: 3.90

36. #SEARCHELITE @NICHOLASTOTT
FMP: MEDIAN & QUARTILES (S)
Top Quartile: 2.71
Median: 3.75
Bottom Quartile: 5.31

37. #SEARCHELITE @NICHOLASTOTT
TTI/CI: MEDIAN & QUARTILES (S)
Top Quartile: 7.45
Median: 11.17
Bottom Quartile: 16.23

38. #SEARCHELITE @NICHOLASTOTT
FL: MEDIAN & QUARTILES (S)
Top Quartile: 6.00
Median: 9.40
Bottom Quartile: 14.20

39. RETAIL: SPEED INDEX
Top Quartile
3,475
Median
4,862
Bottom Quartile
7,222
#SEARCHELITE @NICHOLASTOTT
THIS
ONE!

40. HOW DID WE DO IT, WHAT DID
WE LEARN AND HOW CAN WE
USE IT?
#SEARCHELITE @NICHOLASTOTT

41. #SEARCHELITE @NICHOLASTOTT
“Ensure 3G CI is
consistently below 7
seconds when audited via
Lighthouse, by the end of
Q1.”
SET GOALS THAT ARE SMART

42. #SEARCHELITE @NICHOLASTOTT
GET BUDGET AND TALENT
SOURCE LINK

43. EVIDENCE COMPETITOR PERFORMANCE
#SEARCHELITE @NICHOLASTOTT
Top
Quartile
3,475
Median
4,862
Bottom
Quartile
7,222
2,840

44. RESOURCE AND PROJECT PRIORITISATION
There’s so much we could be doing…
How fast does fast need to be?
How can we take priority in the dev queue?
#SEARCHELITE @NICHOLASTOTT

45. THANKS FOR LISTENING
GET THE SLIDES:
SLIDESHARE/NICHOLASTOTT