Presentation from 17/3/2011 at the NY Web Performance Chapter about the iPhone/Android Comparison Study by Blaze.io (http://www.blaze.io), presented by Guy Podjarny

1. State of Mobile Web Performance Guy Podjarny, CTO

2. Agenda Why Mobile Web Performance Matters Challenges in Mobile WPO Intro to Mobitest State of Mobile Web Performance Study Anecdotes & Insights Coming Soon Summary Q&A 2

3. Why Mobile Web Perf Matters Because Web Performance matters Many studies show rev/perf correlation Because Mobile users are impatient 58% of users expect desktop speed on mobile Tablets take away the “Mobile Site” solution Because Mobile browsing is growing quickly Mobile Browsing will pass Desktop by 2014 31% of US Mobile Users have a Smartphone

4. Japan is leading the charge 4

5. iPhone & Android Growing Fast 5

6. Challenges in Mobile WPO Slower Clients Less Powerful Devices, Slower Networks Fragmentation OS Fragmentation Browser Fragmentation Device Fragmentation Form-Factor Fragmentation Network Fragmentation Non-Smartphones (not in this presentation) Lack of Tools Emphasizes challenges above

7. Current Mobile Performance Tools Blaze Mobitest Details to follow Pcapperf (pcapperf.appspot.com) Upload captured mobile network dump Analysis Bookmarklets Mobile Perf, DOM Monster, etc. New: YSlowBookmarklet (Alpha) JDrop Collect on Device, Analyze Online 7

8. Blaze Mobitest

9. Mobitest - Overview Free service for measuring mobile page load Measures on real devices, using custom apps Available devices include: iPhone 4.3 iPhone 4.2 Android 2.3 (Nexus S) Android 2.2 (Galaxy S) Currently only measures over WiFi 3G works, but not yet included in service 9

10. How does it work? Mobile WebPageTest Agents Custom apps, created for each platform Poll WPT to receive jobs Load pages while monitoring activities Package and upload results Supports multiple runs, first/repeat view & video Private Modified WPT Instance Supports different results format 10

11. Getting the Data Mobile Devices are locked down Getting the info out is hard Each OS required different approaches And presented different limitations iPhone: Embedded Browser Callbacks Android: Browser Callbacks + On-device sniffer + pcap2har Blackberry (Alpha): Networking API 11

12. State of Mobile Web Performance

13. State of Mobile Web Perf Study Goal: Statistically Strong Performance Bakeoff What’s unique about it? Scale: Measured 45,000 Page Loads Automation: More accurate than manual timing Real Devices: Not measuring on simulators Questions Answered: Who’s Faster: Android or iPhone? What’s the impact of the latest versions? How does WiFi compare to 3G? 13

14. Methdology Measuring Software: Mobitest Test Sites: Fortune 1,000 Devices: iPhone 4, Nexus S, Galaxy S Network: Mostly WiFi, one 3G test Process: Measure sites on a device 3 times, save median Repeat 3 times on different days Use median results of medians 14

15. iPhone vs. Android - Average Android 52% Faster On Mobile Sites, Android only 3% Faster Effectively tied On Desktop Sites, Android is 59% Faster Will manifest even more in Tablets 15

16. iPhone vs. Android – Races Android Faster on 84% of Sites iPhone won 16% of the races On Mobile Sites, Android won 71% of the time Average gap decreased to 1.1 seconds 16

17. Android Version Comparison Android 2.3 is 10% Faster Than 2.2 2.144 Seconds vs. 2.370 Android 2.2 was still faster on 42% of sites Average gap merely 65 milliseconds JavaScript in 2.3 is 37% Faster Based on SunSpider JavaScript Benchmark Conclusions: JavaScript Speed != Browsing Speed We Optimize What We Measure 17

18. iPhone Version Comparison iPhone 4.3 Not Faster than 4.2 4.3 is 2% Slower: 3.253 vs. 3.182 4.3 was faster on 51% of Sites Average gap is an even zero JavaScript Engine is 2.5x Faster! And yet, no browsing speed improvement Theory: Nitro not always applied Doesn’t manifest on full-screen websites, home page web apps, embedded browsers 18

19. WiFi vs. 3G Compared iPhone 4.2 on WiFi & 3G Results: WiFi 13% Faster (3.6 vs. 3.2) WiFi faster on 82% of Sites Tests done under great 3G conditions: Bell’s HSPA Network, Night time, good reception Measured Mbps: 6 Download, 1.6 Upload 100 mid-day loads increase gap to 60% 19

20. Mobile vs. Regular Site Definition of a Mobile Site (in this study): iPhone requires 30+ less resources than IE8 Result: Mobile Sites Loaded 39% Faster 66% Faster on iPhone, 8% Faster on Android Simple Sites 2xFaster Than Complex Simple = 30- Requests 20

21. Anecdotes & Insights Preliminary (use at your own risk)

22. WSJ – iPhone 4.3 Load Time 17 Secs Start Render 8.4 Secs Requests 159 Size 1,252 KB 22

23. WSJ – Android 2.3 23 Load Time 7.5 Secs Start Render 8.4 Secs Requests 204 Size 1,313 KB

24. WSJ – Requests 20-35 iPhone 5 Domains Android 1 Domain 24

25. WSJ - Theories Android downloads resources faster Seen on other sites, more efficient network? Speedtest.net shows the same up/down speeds Android reorders image requests? Consistently seen in the WSJ waterfall Has some constraints – domains do repeat later Complex sites make room for optimizations 25

26. Velocity Conf – iPhone 4.3 26 Load Time 8.7 Secs Start Render 6 Secs Requests 75 Size 1,147 KB

27. Velocity Conf – Android 2.3 27 Load Time 4.6 Secs Start Render 2.1 Secs Requests 78 Size 1,075 KB

28. Velocity Conf - # Connections iPhone 4.3 More connections 6 per host, 35 max More than 4.2 Android 4.2 Less connections 4 per host, 4 max 28

29. Velocity Conf – JS/CSS In Parallel iPhone CSS & JS block each other Android CSS & JS don’t appear to block 29

30. Velocity Conf – Conclusions More connections doesn’t mean faster Apparently even more on mobile devices Doesn’t match with current desktop wisdom Smart parallel actions matter CSS & JS often mix together Blocking resources are a major slowdown Browser speed has many moving parts… 30

31. Coming Soon

32. iPad Agent 32

33. iPad & iPhone – Same OS, Different Site 33 2.3 Seconds 9 seconds

34. Blackberry 34

35. Q&A