This document summarizes a webinar presented by Courtney Ball, Antonio Contreras, and Whelan Boyd from 23andMe on how the company improved user experience through A/B testing and implemented Performance Edge to optimize page load times. The webinar discussed how website performance directly impacts business metrics like conversion rates, highlighted best practices for measuring performance, and explained how standard A/B testing can negatively affect page speeds. It then introduced Performance Edge as a solution that delivers only necessary code variations to visitors, significantly reducing file sizes and browser execution times. 23andMe's case study showed how implementing Performance Edge through Cloudflare Workers dramatically improved their page load performance metrics and allowed them to scale up testing without slowing down

1. 1
Ahead of the curve
How 23andMe improved UX
with Performance Edge
+
Courtney Ball, Antonio Contreras,
and Whelan Boyd

2. 2
Introductions
Courtney Ball
Sr. Manager, Web Marketing
Antonio Contreras
Sr. Web Developer
Whelan Boyd
Sr. Product Manager

3. ● Customize the widgets on your page
to your preference
● This webinar is recorded and you will
receive the link with the slides in the
next few days
● We will have time for questions! Please
submit them in the Q&A box on the left
side of the screen
Housekeeping

4. 4
Agenda
1. Why performance matters
2. How to think about measuring performance
3. How experimentation affects performance
4. Performance Edge
5. Case study: 23andMe
6. Q&A

5. 5
Why
Performance
Matters

6. 6
The business impact of speed
13% increase in sales
when page load time
was cut by half
Found that 53% of mobile
visits were abandoned
if page took > 3 seconds
to load
7% increase in
conversion when page
load time reduced by
850 ms
Source: https://developers.google.com/web/fundamentals/performance/why-performance-matters/

7. 7
How to measure
performance

8. 8
Measure impact to user experience
Source: Google’s Web Fundamentals
First Paint
(FP)
First Contentful
Paint (FCP)
First Meaningful
Paint (FMP)
Time to Interactive
(TTI)

9. 9
Best practices for measuring
Free Online Tools
Synthetic Traffic
Easy to run
Actionable Insights
Real User Monitoring
Production Traffic
Segment by Device, Location, etc.
Robust Analytics

10. 10
How website
experimentation
affects performance

11. 11
Standard Web Experiments
1. Snippet size increases with
more experiments
2. For specific visitors, the
downloaded snippet includes all
project variations vs just active
variations
3. Larger snippet results in more
time to download and execute
4. Resource-constrained browsers
have to execute several tasks
Performance Challenges
for Web Experiments
Standard
CDN
Script
Experiment A
variation 1: CTA: Get Started
variation 2: CTA: Try it out
Experiment B
variation 1: CTA: Promo Code
variation 2: CTA: No Promo Code Task load:
URL Targeting
Audience
Targeting
Traffic Splitting
Apply Variations
Event Tracking

12. 12
Introducing
Performance
Edge

13. 13
Only returns
visitor-specific variations
Standard Web Experiments
Task load:
Experiment
Delivery
Network
Apply Variations
Event Tracking
Key experiment decisions
are moved to the Edge
Much smaller file to download
and execute in the browser
with less tasks to perform
Performance Edge Experiments
Standard
CDN
Script
Experiment A
variation 1: CTA: Get Started
variation 2: CTA: Try it out
Experiment B
variation 1: CTA: Promo Code
variation 2: CTA: No Promo Code
URL Targeting
Audience
Targeting
Traffic Splitting
Apply Variations
Event Tracking
CTA: Get Started
Script
URL Targeting
Audience
Traffic Splitting

14. 14
Why is it so fast?
Less data
Far smaller file
loading over the wire
Reduce download time
Less code
Only code necessary
for active variations
Fraction of the execution
time in browser
Faster machines
Powerful Cloudflare Workers
execute expensive tasks
Located extremely
close to end visitor

15. 15
Proper Measurement Make things smaller Performance Edge
● User-focused Metrics
● Synthetic and real traffic
● Lighter download
● Shorter execution
● Quick implementation
with any CDN
● Push logic to the Edge
Summary

16. 16
23andMe
Case Study

17. 17
23andMe: Intro
Courtney Ball
Sr. Manager, Web Marketing
Antonio Contreras
Sr. Web Developer

18. 18
23andMe: Overview
Company size: 500+ employees
Location: Sunnyvale, California
Mission: Help people access, understand and benefit
from the human genome.
What we offer: 23andMe gives individuals easy and
affordable access to their genetic information, offering
a growing list of reports on traits, wellness, carrier
status and genetic ancestry.
Consumer Research Therapeutics

19. 23andMe: Testing Overview
TV
AUDIO
DISPLAY
SOCIAL
Testing focus for 2020:
● Optimize kit mix.
● Optimize mobile experience.
● Optimize personalization
through audience targeting.
23andMe Customer Acquisition / Marketing Funnel
SEARCH
SITE
Consideration
Conversion
Awareness

20. 20
23andMe: Approach to improving user experience
through a site redesign
Create new designs
(multiple options)
Consumer insights
conducts qualitative
research
Identify key areas of
opportunity through a
site audit
A/B test against
existing page via
Optimizely
Iterative testing
(if needed)

21. 21
Test: Redesigned Product Page
Test Variant #1Control
Round 1

22. 22
Test: Redesigned Product Page
Test Variant #1Control
Round 1
Winner

23. 23
Test: Redesigned Product Page
Control Test Variant #1
Round 2
Test Variant #2

24. 24
Test: Redesigned Product Page
Control Test Variant #1
Round 2
Test Variant #2: Winner

25. 25
Why we needed performance edge as we scaled
A growing point of internal tension…
1. More testing = slower page speed
2. More emphasis placed on performance
Eagerness to try new tactics to improve site
performance ahead of the holiday season.
+
Frequency A/B
Tests
2016 2017 2018 2019 2020
Year
NumberofTestsLaunched
20
40
60
0

26. 26
How we implemented
Performance Edge at 23andMe
1. We had an existing Worker on Cloudflare
that we were using to load our Optimizely
JavaScript
2. We updated the existing Worker for
Optimizely Edge and created a separate
Worker for the old Optimizely JavaScript
3. We used an integration environment to
test performance and root out errors
Cloudflare
CDN / Worker
Client / Browser 23andMe
Web
Server

27. 27
Speed improvements with Performance Edge
Our performance statistics improved across the board:
Standard Snippet ⚡Optimizely Edge Snippet
First Contentful Paint Mobile: 2.7s
Desktop: 2.5s
Mobile: 1.2s
Desktop: 1.1s
First Meaningful Paint Mobile: 3.0s
Desktop: 2.7s
Mobile: 1.6s
Desktop: 1.5s
Time to Interactive Mobile: 6.7s
Desktop: 6.2s
Mobile: 4.4s
Desktop: 5.4s
Key Benefits:
dramatic performance improvements on mobile and we can continue scaling our testing program without slowing down our site!

28. 28
Questions?
Ask them
in the webinar console