Recommandé
Recommandé
Contenu connexe
En vedette
Similaire à Rum first london web perf meetup
Similaire à Rum first london web perf meetup (20)
Dernier
Dernier (20)
Rum first london web perf meetup
- 2. Who I Am Cliff Crocker VP Product SOASTA, Inc. @cliffcrocker
- 6. RUM 101 Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user. • Involves instrumentation of the page with JavaScript • Measurements are fired across the network to a collection point through a small request object (beacon) <JS> MySite.com <beacon>
- 7. Origins of RUM 2005-6: Steve S. builds JS tool to measure page load from the browser during a Yahoo! hackday 2008: Steve open sources ‘Episodes’. White pages open sources ‘Jiffy’ 2010: Yahoo! Open sources ‘boomerang.js’ the same day Navigation Timing is introduced in IE9 beta • http://stevesouders.com/episodes/ • https://code.google.com/p/jiffy-web/ • http://lognormal.github.com/boomerang/doc/ • https://dvcs.w3.org/hg/webperf/raw- file/tip/specs/NavigationTiming/Overview.html
- 8. This helped…
- 9. Navigation Timing API • Exposes network level timing details to JavaScript • DNS lookup, TCP connect, SSL handshake, First Byte time, and more • For a given web request, you can tell exactly where the time was spent • Supported by IE >= 9, Chrome >=6, Firefox >=7, Android Browser .=4.0, others • No exposure into rendering times
- 10. C’mon Safari, be cool… http://caniuse.com/nav-timing ~70% Browser Support
- 11. Why RUM is Awesome • Reduces the level of convincing we have to do within our organization that ‘faster is better’ • The only way to measure perceived performance from your end user • Easy to maintain • Only getting better… – Check out:http://blog.patrickmeenan.com/2013/07/measuring- performance-of-user-experience.html • User Timing for RUM and Synthetic – Fun with resource timing: https://github.com/andydavies/waterfall
- 12. What gives?? No RUM RUM • According to a study done in 2012* only 1 in 5 of the top ecommerce sites are using some form of RUM *http://www.webperformancetoday.com/2012/11/13/real-user-monitoring-rum-ecommerce-sites/
- 13. Challenges with RUM 1. It’s a “data” problem 2. People don’t like change 3. Hard to “sell” internally
- 14. For a modest sized site, in a given day 1-3 Million Page Views Perf Timing User Agent OS/Device Geography Page Segmentation Bandwidth Classification A/B Bucket ‘Other’
- 16. Many have advocated aggressive sampling of data…
- 17. ~1% sample, looking at a period of about 3 hours for a given page segment (Checkout) 1% > checkout
- 18. Filtered to the United States… 1% > >checkout
- 19. Looking specifically at California… 1% > > >checkout
- 20. IE9 Browsers only. 1% > > >checkout
- 21. Looking at all traffic. 100% > checkout
- 22. All filters applied, identified bimodal distribution driven by backend time. 100% > > >checkout > backend time
- 23. Sampled at 1% Not sampled
- 25. Many use a batch solution…
- 26. • Batch processing large amounts of data has been a traditional approach for RUM – Great for looking at historical data, trending, etc. – Big Data is not necessarily Fast Data • Not ‘Operational’
- 27. In order for RUM to be operationally relevant, you need the data in real-time
- 29. It is Not Either Or • Synthetic and RUM coexist • Important to understand: – Geographic performance – Object level detail (waterfall) – Browser performance – Competitive positioning – Site coverage
- 32. Understanding Performance by Geography • RUM provides better visibility into geographic variability (simply more data points from more locations) RUM 1 Synthetic 0
- 33. Object Level Detail RUM 1 Synthetic 1 Waterfall available for most major browsers available for synthetic monitoring (IE, FF (Commercial), Chrome, Safari) Limited support for waterfall using resource timing (IE10 & Chrome stable) – using waterfall.js
- 34. Browser Coverage RUM 2 Synthetic 1 Synthetic Support for major browsers Commercial (mainstream synthetic) WebPagetest support RUM Support for any browser that executes JS
- 35. Competitive Positioning RUM 2 Synthetic 2 Synthetic Ability to measure competitors easily RUM More complex, no standard benchmarks (yet!) Commercial offerings WebPagetest.org
- 36. Site Coverage RUM 3 Synthetic 2 Synthetic Choose a ‘happy path’ or spot check RUM No blind spots! http://scoutmastercg.com/wp- content/uploads/typimg/green_laser.png http://www.flickr.com/photos/areese/with/4002609608/
- 37. Synthetic Monitoring • Pros – Technology is great – Real Browsers (IE, Chrome, FF) – Built in Alerting – Waterfall analysis – Screen shots & headers – Object level detail • Cons – False Positives – Impossible/impractical to measure everything – Fixed number of browser/OS combinations – Simulated bandwidth constraints at best – Too few data points for statistical relevance
- 38. RUM Cast a wide net • Identify key areas of concern • Understand real user impact • Tie this back to $$ (next) Synthetic Diagnostic tool • Identify issues in a ‘lab’ • Reproduce a problem found with RUM http://www.flickr.com/photos/84338444@N00/with/3780079044/ http://www.flickr.com/photos/ezioman/
- 40. Studies show…
- 41. However, you have to look at your own data
- 42. Understand your KPIs and How to Track Them • Retail: Order Conversion, Add to Cart Conversion, Revenue • Media: In Line Display Ad Views, Comments, • Other: Facebook likes/Social Media sharing
- 43. Bottom Line, if you want your business to care – then make it relevant for them (speak their language)
- 44. Strategy for Success – RUM First RUM First • All the data (or statistically relevant sample) • Real-time and historical Supplement with Synthetic • Reduce measurements based on coverage with RUM • Use for adhoc diagnostic & competitive benchmarking Map to business KPIs • YOUR goals • YOUR data • Single source of truth