My slides from Emerce Conversion where I share Euroflorists first steps into an optimization process with evolutionary artificial intelligence algorithms.
17. Who is Euroflorist?
● We sell flowers & others gifts
● We’ve been doing this since 1947
● We do B2C & B2B, online and offline
● We are a Swedish company
● Our Dutch office is located in Amsterdam
18. Euroflorist in numbers (1/2)
● > 21 years in e-commerce
● > 60% revenue from online
● > 200 employees: 50% in CS, 35
working in online
● > 2M bouquets sent every year
● 19 sites in 11 countries and 8 languages
19. Euroflorist in numbers (2/2)
● 10M+ users on our sites in 2017
● Sites with up to 20% conversion rate
● Market share ranging from 5-45%
● 8+ score on Trustpilot, NPS > 50
● Growing revenue, teams and taking
market share
20. Guido Jansen
● Chief Psychology Officer @ Euroflorist
● UX/CX Team Lead
● Full Customer Journey optimization
● Wannabee Data Scientist
x@gui.do | @guido
28. “[Booking.com’s] utilization of A/B testing …
drives higher conversion across it’s entire
platform, resulting in conversion levels 2-3x
the industry average”
Source: Evercore Equity Research
29. Sadly, when you test more,
the % of tests with an uplift will go down.
30. When A/B testing a fully optimized site,
the % of tests with an uplift is low(er).
31. Euroflorist: 1 in 5 tests with uplift
Booking.com: 1 in 10 tests with uplift
Booking.com Source: Stuart Frisby, Booking.com - Conversions@Google 2017
41. Different applications of AI in E-commerce
● Catalog optimization ← already tried this
● Personalization
● RTB / Advertising
● Chatbots
● Virtual personal shopper
● Improved optimization process
● ...
43. Previous experience:
● “Black box”
● No (perceived) control for E-commerce managers
● No proven uplift in CR
● Hard to do for (emotional) consumers buying for
someone else?
44. The AI promise...
● Faster & more efficient
● Time-saving
● Test more
● Adaptive
● Greater impact
● ...
58. $143M
in total funding
9 YEARS
platform &
technology
development
35 PATENTS
submitted to date; 11 patents
issued
100+
EMPLOYEES
HQ in San Francisco with offices in San
Jose and Hong Kong
Founded by the team that developed
the core technology behind Siri
65. 8 elements on PDP, 2 variants each
● Header - less content
● Image position - left to right
● Add progress bar - feed forward
● Social sentence (under CTA, Hobson’s choice +1)
● Product info - full width / less prominent
● Price partitioning
● USP Bar icons/text - different content
● USP Bar position - to the top
80. Best variants
NL
Variant 8, generation 1
● USP Bar Icons/Text
SE
Variant 26, generation 4
● USP Bar Icons/Text
● Price perception
● Product info
● Call to action
● Progress Bar
NO
Variant 8, generation 1
● USP Bar Icons/Text
83. A/B test the winner to validate
Significant (frequentist; 90% confidence; 1-sided)
● Add-to-cart uplift: 3.9% (p-value 0.0161, Power: 95,9%)
● Conversion rate uplift: 5,3% (p-value 0.0361, Power 89,6%)
Not significant
● Average Order Value
84. How much does it add to our
bottom line?
€ € € € € € € € € €
85. Business case:
● Costs: 165K (subscription + support + salary)
● Expected uplift: 500K-1M profit
● Minimal CR uplift needed for break even: CR
increase of 1.25%
87. ● Steered on CRT, not on RPU
● GA data issues
● Sentient dashboard UX issues
● Calculator to determine ideal values would be very helpful
Specific Sentient caveats/ needed improvements to ßeta
88. Regular A/B/MvT testing
Pro:
● More control
● Stricter hypothesis testing
● ...
Con:
● Slow
● Less variants
● Needs a lot of traffic
● Limited test capacity
● ...
89. Testing with AI
Pro:
● “Time per tested variant” much
shorter
● End up with “uncommon”
variation
● Requires less traffic (around
75% less)
● Higher ROI
● ...
Con:
● More extensive setup process
● Black box (?)
● Don’t know exactly how fast
generations will follow...
● Garbage in, Garbage out
● Completely freaks out our local
Ecom managers.
● ...
93. Did Sentient deliver on their promise...?
MORE CAPABLE
• Test your entire funnel in a
single experiment
ADAPTIVE
• Evolves with your audience
• Always-on optimization
GREATER IMPACT
• More tests = better results
• Focus on marketing, not test
administration
FASTER AND
MORE EFFICIENT
• Tests dozens of ideas
• Test 1,000,000s of
combinations
• Get to results faster
TIME-SAVING
• Fully automated
• Eliminates tedious test
administration
• Handles the data science
95. We’re hiring!
Optimization team:
● Web Analysts
● UX designers
● Growth hackers
● Interns
● AI/CKI background
● (Cognitive/Behavioral) Psychologists
Other online:
● SEA Specialist
● Marketing & communications
specialist
● CRM specialist
● Social Media Manager
● Interns
● ...
Are you smart & curious? Looking for a growing international company where you can
experiment & learn a lot? Join Euroflorist!
euroflorist.com