Many websites use real user measurement data to analyze their performance, as well as to validate that optimizations are having the desired effect on end user experience and business metrics. Akamai's mPulse service is a popular commercial RUM solution that is used by thousands of websites, and it gives us a unique panoptic view of web performance. This is not your typical RUM talk. I won't be talking about implementations, product features, etc. Instead we are going to dive deep into the data and try to answer some interesting questions about the web.

1. ©2016 AKAMAI | FASTER FORWARDTM
Real User Measurement Insights
August 9, 2018
Paul Calvano
Principal Web Performance Architect
pacalvan@akamai.com
@paulcalvano

2. ©2018 AKAMAI | FASTER FORWARDTM
Synthetic
Real User (RUM)

3. ©2018 AKAMAI | FASTER FORWARDTM

4. ©2018 AKAMAI
Implementing RUM without Impacting Performance
Source: https://gist.github.com/bluesmoon/351ea5b2815606863a2de2c4889e8016/
https://calendar.perfplanet.com/2017/an-audit-of-boomerangs-performance/

5. ©2018 AKAMAI | FASTER FORWARDTM

6. ©2018 AKAMAI | FASTER FORWARDTM
What’s In An mPulse Beacon?
● Device and Browser Information
● Geographic Information
● Performance Timing Information
● Feature Flags
● Custom Metrics and Timers
Source: https://developer.akamai.com/mpulse/whats-in-a-beacon

7. ©2016 AKAMAI | FASTER FORWARDTM
Device Types

8. ©2018 AKAMAI | FASTER FORWARDTM

9. ©2018 AKAMAI | FASTER FORWARDTM

10. ©2018 AKAMAI | FASTER FORWARDTM

11. ©2018 AKAMAI | FASTER FORWARDTM

12. ©2018 AKAMAI | FASTER FORWARDTM
Source: https://meyerweb.com/eric/thoughts/2018/08/07/securing-sites-made-them-less-accessible/
Akamai mPulse data from August 7th, 2018
Median:
US: 3.7 seconds
UG: 7.8 seconds
75th Percentile:
US: 6.5 seconds
UG: 25.4 seconds
95th Percentile:
US: 20 seconds
UG: 98 seconds

13. ©2018 AKAMAI | FASTER FORWARDTM

14. ©2018 AKAMAI | FASTER FORWARDTM

15. ©2018 AKAMAI | FASTER FORWARDTM
Time to Interactive
https://calendar.perfplanet.com/2017/time-to-interactive-measuring-more-of-the-user-experience/
https://developer.akamai.com/blog/2017/04/12/gauge-user-experience-time-interactive/

16. ©2018 AKAMAI | FASTER FORWARDTM
How mPulse Measures Time to Interactive: https://blogs.akamai.com/2018/04/perception-matters-measure-perceived-
performance.html

17. ©2018 AKAMAI | FASTER FORWARDTM

18. ©2018 AKAMAI
Device Identification
Mozilla/5.0 (iPhone; CPU iPhone OS 11_4_1 like Mac OS X)
AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15G77
https://www.paintcodeapp.com/news/ultimate-guide-to-iphone-resolutions

19. ©2018 AKAMAI
iOS: ~15% of Pages Served to Older iPhones

20. ©2018 AKAMAI
Analyzing Performance of iPhone Devices

21. ©2018 AKAMAI
Identifying Android Model Numbers

22. ©2018 AKAMAI
Android Device Fragmentation: ~ 11,500 devices!

23. ©2018 AKAMAI | FASTER FORWARDTM
Long Tasks Per Android Device
● LongTasks API - implemented in
Chrome browsers:
○ https://www.w3.org/TR/longtasks/
● Whenever a task takes longer than
50ms it is considered a long task.
● mPulse is collecting the total durations
of all long tasks during a page load.

24. ©2018 AKAMAI
Comparing Two Popular Android Phones
Source: https://www.sammobile.com

25. ©2018 AKAMAI
Galaxy J3 is Much Slower Compared to S7

26. ©2018 AKAMAI | FASTER FORWARDTM
Popular Devices per Geo
● iPhone: Most Popular
● Samsung Galaxy S8:
○ Most popular Android device
in most but not all countries.
● Other Popular Phones
○ Redmi Note 4 in India and
Russia
○ Huawei in Italy
○ Xperia in Japan
● Device Fragmentation is global
○ Most popular Android
phones account for 2-6% of
traffic

27. ©2016 AKAMAI | FASTER FORWARDTM
Web Browsers

28. ©2018 AKAMAI
Browser Distributions

29. ©2018 AKAMAI
Analyzing Browser Performance - Firefox Quantum
Source: https://developer.akamai.com/blog/2017/11/29/measuring-performance-firefox-quantum-rum/

30. ©2018 AKAMAI
Firefox: 7-10 Days to Reach 70% After Release
Controlled Deployment Details https://wiki.mozilla.org/Balrog

31. ©2018 AKAMAI
Chrome: ~3 Weeks to Reach 80% After Release
Controlled Deployment Details - https://chromium.googlesource.com/chromium/src/+/lkcr/docs/process/release_cycle.md#stable-release

32. ©2018 AKAMAI | FASTER FORWARDTM
Chrome 68 Update…
v67
v68

33. ©2018 AKAMAI
Edge Accounts for Almost 40% of Microsoft Browser Traffic

34. ©2018 AKAMAI
Weekly Fluctuations: Enterprise Users on IE11

35. ©2018 AKAMAI
Safari v11 Reached 40% Before High Sierra Release

36. ©2018 AKAMAI
iOS Safari - 1 Month to Reach 50%

37. ©2018 AKAMAI
20% of Chrome Mobile Users Are One Year Behind Stable
Samsung Browser uses Chromium v59 rendering engine - https://developer.samsung.com/internet

38. ©2016 AKAMAI | FASTER FORWARDTM
Additional Insights

39. ©2018 AKAMAI | FASTER FORWARDTM
Outages = Sudden Drop of Real Users
https://blogs.oracle.com/internetintelligence/power-failure-leaves-brazilian-internet-in-the-dark

40. ©2018 AKAMAI | FASTER FORWARDTM
Performance by Mobile Network
Based on US Traffic from mPulse, July 2018

41. ©2018 AKAMAI | FASTER FORWARDTM
How’s that InFlight Wifi?

42. ©2018 AKAMAI | FASTER FORWARDTM
Performance by Battery Level
Source: https://developer.akamai.com/blog/2018/05/29/does-androids-battery-saver-degrade-the-mobile-web-experience/

43. ©2018 AKAMAI | FASTER FORWARDTM
Long Task Time Impacted by Battery Level
Source: https://developer.akamai.com/blog/2018/05/29/does-androids-battery-saver-degrade-the-mobile-web-experience/

44. ©2018 AKAMAI | FASTER FORWARDTM
Analyzing the Long Tail…
Median - 3.7 seconds
75th Percentile - 7.5 seconds
95th Percentile - 25.1 seconds
Loooooooooooooooooong Tail! (5% = 100s of millions of pages)

45. ©2018 AKAMAI | FASTER FORWARDTM
Dissecting the Slowest 5% of Page Views - Aug 8, 2018

46. ©2018 AKAMAI | FASTER FORWARDTM
Dissecting the Slowest 5% of Page Views - Aug 8, 2018
70% under
500ms
72% under
700ms
85% under
1000ms
85% under
4800ms 90% OVER
24700ms

47. ©2018 AKAMAI | FASTER FORWARDTM
What Did We Learn?
● Mobile Now Exceeds Desktop During Weekends
● Tablet usage varys by Geo, but is consistently slower than Desktop and Mobile
● iPhone models have a measurable performance delta, and 15% are older devices
● Android Device Fragmentation continues to be a very large performance problem
● Most browsers are carefully deployed to users in a multi-stage process
● Safari, Edge and IE browsers are closely aligned to OS Upgrades
● The performance impact of new browser features are limited by end-user upgrades
● Battery life may impact your performance
● Analyze the long tail of your performance and you may be surprised what you find!

48. ©2018 AKAMAI | FASTER FORWARDTM
mPulse Lite = Free RUM
Akamai Customers - Sign up via the Akamai MarketPlace
Not Using Akamai? - Sign up for free at http://www.akamai.com/trympulse

49. ©2018 AKAMAI | FASTER FORWARDTM
Akamai - State of the Internet Reboot!
● Akamai State of the Internet Report
originally published between 2008 - 2017
● Currently on hiatus, but coming back
soon!
● Will include mPulse stats and trends
● We’re open to suggestions on what you
would like to see included!
○ stateoftheinternet@akamai.com
○ pacalvan@akamai.com

50. ©2016 AKAMAI | FASTER FORWARDTM
Q&A
Paul Calvano
Principal Web Performance Architect
pacalvan@akamai.com
@paulcalvano