Over the years, multitude of metrics were developed, from Time To First Byte and Page Load Time all the way to more modern and sophisticated ones like Time to First Paint, ATF, Speed Index and Frames per Second. One challenge with current performance metrics is that they all are proxies for user experience and do not translate directly into user satisfaction and business gains, but with each new metric we are getting closer to measuring what matters. Sergey will talk about pros and cons of technical metrics historically used to monitor performance, describe two core methods or measurement/monitoring (synthetic and RUM) and introduce latest developments in tracking user-centric metrics. Slides updated from similar talks since original presentation.

1. T R A C K I N G
PA G E
S P E E D
S E R G E Y C H E R N Y S H E V

2. S C A L A B I L I T Y & P E R F O R M A N C E
number of users
timetoload

3. number of users
timetoload
S C A L A B I L I T Y & P E R F O R M A N C E
0

4. S C A L A B I L I T Y & P E R F O R M A N C E
One User
timetoload
0
S1

5. W H Y D O W E M E A S U R E S P E E D ?
• Monitor for degradations (Ops)
• Analyze code for perf issues (Devs)
• Verify improvements (Devs)
• Prioritize improvements (Business)
• Budget for WPO initiatives (Business)

6. S TAT S
• Pinterest: -40% wait time => +15% SEO, +15% conversion (2017)
• DoubleClick: 5s vs 19s on mobile => 2x ad revenue (2016)
• Trainline: -300ms latency => +$11.5M / year revenue (2016)
• Etsy: +160Kb => +12% bounce rate (2014)
• Edmunds: -77% load time => +20% page views (2011)
• Mozilla: -2.2s => +15.4% Downloads (2010)
• Google: +400ms => -0.21% searches after experiment! (2009)
• Netflix: +GZip => -43% Traffic cost (2008)
• Amazon: +100ms => -1% revenue (2008)
• Google: +500ms => -25% searches (2006) WPOStats.com

7. WHAT DOES MY SITE COST? WHATDOESMYSITECOST.COM
http://www.disney.com/

8. % OF DAILY BUDGET WHATDOESMYSITECOST.COM
http://www.disney.com/

9. http://www.slownesskills.com

10. S Y N T H E T I C M O N I T O R I N G ,
T E S T I N G & A N A LY S I S
• From particular location
• Tester controls instrumentation
• One metric value
• Data can have lots of details for analysis

11. R E A L U S E R M E A S U R E M E N T
• Real users (a lot of them)
• A lot of data (need to store it)
• All noise you can get, requires filtering
• Metrics are distributions
• Can correlate to business KPIs

12. T O D AY ' S M E T R I C S
• DNS, SSL/TLS, Time To First Byte (TTFB)
• Page Load, Document Complete, Fully Loaded
• Size of DOM, Frames per Second
• First Paint, First “Contentful” or Meaningful Paint
• Above the Fold Time (AFT)
• SpeedIndex

13. T I M E T O F I R S T PA I N T
• Shows when completely useless part is over
TTFP
3.5s

14. A F T ( A B O V E T H E F O L D T I M E )
• When everything is finally visible
AFT
15.3s

15. S P E E D I N D E X
• Unifies rendering progress in one number
Speed Index
8618

16. M E A S U R E U S E R E X P E R I E N C E
• Great experience for users
• "Fast" is just a component
• Correlate what you measure
to business KPIs
• Do not measure what's easy,
measure what matters
N O T T E C H N O L O G Y

17. I N T E R A C T I O N O N T H E W E B
1. Verify destination
2. Provide primary content
3. Allow interaction
4. Show secondary content
5. Invisible tasks
✦ Acknowledge action

18. D E S I G N C O M P O N E N T S
1. Verify destination
2. Primary content
…

19. E X T E N D E D M E T R I C S
• Time To First Paint (TTFP) - 3.5s
• Verify Destination - 8s
• Primary content - 11s
• ...
• Above The Fold (AFT) - 15.3s

20. U S E R T I M I N G A P I
• Records custom JS timings on the page
• Recorded by both RUM and Synthetic tools
• Can be hard to match with user's reality
• Uses JavaScript instrumentation
https://github.com/sergeychernyshev/ux-capture

21. H E R O E L E M E N T T I M I N G A P I
• Use elementtiming=“name” HTML attribute
• Currently recorded only by

22. T H A N K Y O U !
sergey.chernyshev@gmail.com
@SergeyChe