Slow pages hurt mobile user metrics, from bounce rate to online revenues and long-term user retention. At Radware, we wanted to understand the science behind this, so we engaged in the first documented study of the neurological impact of poor performance on mobile users. Your takeaway from this presentation is hard data that you can use to make a case for investing in mobile performance in your organization. Based on similar research performed on desktop users, our study involved using a groundbreaking combination of eyetracking and electroencephalography (EEG) technologies to monitor brain wave activity in a group of mobile users who were asked to perform a series of online transactions via mobile devices. In our study, participants were asked to complete standardized shopping tasks on four ecommerce sites while using a smartphone. We studied participants during these tasks, both at the normal speed over Wifi and also at a consistently slowed-down speed (using software that allowed us to create a 500ms network delay). The participants did not know that speed was a factor in the tests; rather, they believed that they were participating in a generic usability/brand perception study. From the data, we were able to extract measures of frustration and emotional engagement for the browsing and checkout stages of both the normal and slowed-down versions of all four sites. This presentation, shared by Radware Web Performance Evangelist Tammy Everts at the 2014 Velocity Conference and the CMG Performance and Capacity 2014 Conference, provides a deeper understanding of the impact of performance on mobile users. For even more on the research, you can also download it here: http://www.radware.com/mobile-eeg2013/

1. Mobile Web Stress
Understanding the neurological impact of poor web performance
Tammy Everts
@tameverts

2. Slide 2

3. Slide 3

4. Slide 4
1Why care about mobile web performance?
2Why neuroscientific mobile testing?
3What is emotional engagement research?
4How did we perform our study?
5What kinds of insights did we gain?

5. It’s a mobile-first world.

6. Slide 6
eMarketer, June 2014

7. Slide 7
55% of all time spent on retail sites takes place on a mobile device.
comScore, October 2013

8. Slide 8

9. Slide 9
Stuart McMillan, Schuh’s Journey to RWD (Conversion Conference 2013)

10. Slide 10
By 2017, retail mcommerce is expected to hit $113 billion – 26% of total ecommerce sales.
eMarketer, September 2013

11. Keynote, 2012 Mobile User Survey
Slide 11

12. Slide 12

13. Mobile shopping cart abandonment rate is 39% greater than desktop rate.
2013 Google I/O
Slide 13

14. Slide 14
Case study: The impact of HTML delay on mobile business metrics

15. Skava/Harris Interactive, 2013
Slide 15

16. Two things are slowing down your site for mobile (and they’re completely beyond your control).
Latency – can range from 35 milliseconds to 350+ milliseconds per resource (e.g. images, CSS files)
Connection – 3G can be up to 15 times slower than broadband
Slide 16

17. RWD is awesome, but it *can* come with performance penalties.
Slide 17

18. Why neuroscientific mobile testing?
Slide 18

19. Slide 19
•2010 EEG study of desktop users
•Throttled connection from 5MB to 2MB
•Found that participants had to concentrate up to 50% harder
•Afterward, participants reported negative brand associations

20. “95% of the consumer’s decisions are made at the subconscious level.”
Dr. Gerald Zaltman, Harvard University Executive Committee of Harvard University’s Mind, Brain and Behavior Interfaculty Initiative
Slide 20

21. Slide 21

22. Slide 22

23. The problem with surveys…
Traditional research relies on eliciting post-cognitive responses.
But thinking and talking about emotions changes and distorts them.
Slide 23

24. Slide 24
Simplified cognitive timeline

25. Five benefits of neuroscientific testing
1 Evaluates think/feel (not say)
2 Quantified data
3 Moment-by-moment interaction
4 Cause-and-effect triggers
5 Fresh, deeper insights
Slide 25

26. EEG Emotional Engagement Study: How We Did It

27. Our research team
Seren – leaders in customer experience & service design
NeuroStrata – expert consultants in blending neuromarketing applications
Neurosense – global leader in implicit methodologies
Slide 27

28. The brands we tested
Slide 28

29. Our test subjects
•24 participants (12 male and 12 female)
•Pre-screened to ensure normal cognitive functioning
•Experienced mobile device users
•Did not know they were part of a performance study
Slide 29

30. Methodology
•Standardized set of shopping tasks (browsing and checkout)
•Testers served sites over one of two speeds:
–normal Wifi
–artificial 500ms delay
•Using EEG headset and eyetracker, measured moment- by-moment responses
Slide 30

31. Slide 31

32. Slide 32
Why test a 500ms delay?
Case study: The impact of HTML delay on mobile business metrics

33. 500ms delay: Peak frustration results
Slide 33

34. 500ms delay: Average engagement results
Slide 34

35. Impact of site speed on post-test brand association
Slide 35

36. If pages aren’t fast, everything suffers.
Content “boring”
Visual design “tacky” and “confusing”
Navigation “frustrating” and “hard-to-navigate”
Slide 36

37. Slide 37

38. Bonus Study: Implicit Response Test The effect of loading speed on brand perception

39. We react faster to congruent stimuli than incongruent stimuli.
Slide 39

40. Slide 40
https://www.projectimplicit.net/index.html

41. Two-part methodology
1Pre-test
2Test
Slide 41

42. Slide 42
Implicit pre-test
Two brands – Easyjet and Ryanair – were measured against 24 attributes.

43. Slide 43

44. Slide 44

45. Slide 45

46. Implicit pre-test: Brand mapping
Slide 46

47. Implicit pre-test: Purchase intent
Slide 47

48. Implicit test
1Participants then viewed one of two videos depicting a flight selection/booking process:
•Normal
•Slow (500ms delay per page)
2Each brand is measured again against the same 24 attributes.
3Difference between normal and slow indicates effect of speed on brand perception.
Slide 48

49. EasyJet: Impact of 500ms slowdown on each dimension
Slide 49

50. EasyJet: Impact of 500ms slowdown on each attribute
Slide 50

51. Ryanair: Impact of 500ms slowdown on each dimension
Slide 51

52. Ryanair: Impact of 500ms slowdown on each attribute
Slide 52

53. Slide 53
Brand perception summary
•500ms delay triggered downward shift in perception for both companies.
•Overall, EasyJet enjoyed a more positive brand perception. Therefore the impact on EasyJet was greater than on Ryanair.
•Impact varied across attributes for each brand:
–EasyJet site suffered more in Purchase Intent and Functional attributes.
–Ryanair suffered more in Warmth/Friendliness dimensions.

54. Slide 54

55. Takeaways

56. 1Slower web performance has a clear, measurable impact on people at a neurological level.
2People feel “web stress” even when shopping under ideal conditions.
3Slow sites can seriously undermine overall brand health.
4The nature and scale of impact varies, depending on a number of factors (e.g. inherent strength/weakness of brand).
5Greatest risk is to purchase intent.
6Excellent opportunities to strengthen overall brand by investing in performance optimization.
Slide 56

57. Slide 57

58. http://www.radware.com/mobile-eeg2013/
Slide 58

59. Sources
Web Stress: A Wake-Up Call for European Business (Foviance, 2010) http://www.ca.com/us/~/media/files/supportingpieces/final_webstress_survey_report_229296.aspx
2013 Social & Mobile Commerce Consumer Report (Shop.org / comScore)
http://shop.org/research/original/2013-social-mobile-commerce-consumer-report
2012 Mobile User Survey (Keynote)
http://www.keynote.com/docs/reports/Keynote-2012-Mobile-User-Survey.pdf
2013 State of the Union: Mobile Ecommerce Performance (Radware)
http://www.radware.com/mobile-sotu2013/
The Danger of a Poor Mobile Shopping Experience [INFOGRAPHIC]
http://www.getelastic.com/the-danger-of-a-poor-mobile-shopping-experience-infographic/
Case study: The impact of HTML delay on mobile business metrics (Web Performance Today, November 2011)
http://www.webperformancetoday.com/2011/11/23/case-study-slow-page-load-mobile-business-metrics/
Slide 59

60. Slide 60
Questions?
Tammy Everts
@tameverts tammye@radware.com webperformancetoday.com