These are the slides from Paul Rouke's talk 'From Iterative to Innovative Testing' which he gave at Elite Camp 2015 in Talinn, Estonia. How progressive are your tests? Is radical testing part of your company culture? How do you use testing as part of a website redesign? Do you test business level hypotheses? Gain a first hand understanding of when to do innovative testing, how you can develop innovative test hypotheses, and see how companies are changing their culture and whole approach to testing for growth. Paul Rouke is the Founder & Director of Optimisation at PRWD. Elite Camp is a highly practical event. No fluff, no theoretical debates, just an event which provides 100% practical, actionable content, delivered by highly-sought after experts, and infused with hardcore networking.

1. @paulrouke #elitecamp2015
FROM ITERATIVE TO INNOVATIVE TESTING
EXPLOITING THE FULL SPECTRUM
OF TESTING OPPORTUNITIES
PAUL ROUKE
Founder & Director of Optimisation at PRWD

2. @paulrouke #elitecamp2015
ITERATIVE
TESTING
INNOVATIVE
TESTING
OPPORTUNITY

3. @paulrouke #elitecamp2015
OUT-GROW YOUR
COMPETITION
By embracing the full spectrum of
research & insight driven
testing opportunities

4. @paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you best choosing each type?
What business impact does each type deliver?
8 steps to out-grow your competition
FROM ITERATIVE
TO INNOVATIVE
TESTING

5. @paulrouke #elitecamp2015
A full service conversion optimisation agency based in the UK
Optimising online experiences since 2004
Represents the UK in the Global Conversion Alliance, alongside WiderFunnel, WebArts & Conversionista
Conducted over 600 hours of in-depth user research across multiple sectors
540% ROI delivered on average through PRWD’s Six Step Optimisation Method
ABOUT PRWD

6. @paulrouke #elitecamp2015

7. @paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you best choosing each type?
What business impact does each type deliver?
8 steps to out-grow your competition
FROM ITERATIVE
TO INNOVATIVE
TESTING

8. @paulrouke #elitecamp2015
WHAT IS
THE DIFFERENCE?

9. @paulrouke #elitecamp2015
ITERATIVE TESTING
Headline changes
Copy changes
Image changes
Button size changes
Button text changes
Content removal
Page layout changes
Removing or optimising form fields
Go on then if you must… button colour changes
IN SUMMARY
Almost all the testing that
currently takes place

10. @paulrouke #elitecamp2015
Control
ITERATIVE TESTING

11. @paulrouke #elitecamp2015
PRWD Test Variation
ITERATIVE TESTING

12. @paulrouke #elitecamp2015
Result
+ 4.7%Reply Conversion Rate
ITERATIVE TESTING

13. @paulrouke #elitecamp2015
Control
ITERATIVE TESTING

14. @paulrouke #elitecamp2015
PRWD Variation
ITERATIVE TESTING

15. @paulrouke #elitecamp2015
Result
+ 20%Brochure Requests
CONSISTENTLY COMMUNICATE HOW GOOD YOU ARE
ITERATIVE TESTING

16. @paulrouke #elitecamp2015
Ad-hoc testing with weak hypotheses
will destroy buy-in
Intelligent, insight driven iterative testing
will build your test & learn culture

17. @paulrouke #elitecamp2015
INNOVATIVE TESTING
Full redesigns of homepage
Full redesigns of primary template pages
Landing page redesigns
Customer journey changes
Testing changes to a business proposition
Tests prior to changing back-end systems
Reducing steps within a flow ie. Checkout, application, registration
Even full website redesigns
IN SUMMARY
Very few businesses are
bold enough to invest in this
type of testing

18. @paulrouke #elitecamp2015
Control
INNOVATIVE TESTING

19. @paulrouke #elitecamp2015
PRWD Test Variation
INNOVATIVE TESTING

20. @paulrouke #elitecamp2015
Result
+ 78%New visitor sign-ups
BE BOLD & PROVIDE EARLY & EASY OPPORTUNITIES TO CONVERT
INNOVATIVE TESTING

21. @paulrouke #elitecamp2015
Poorly planned innovative testing will waste
valuable resources & damage testing momentum
Intelligent, insight driven innovative testing
can help transform & grow your business

22. @paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you best choosing each type?
What business impact does each type deliver?
8 steps to out-grow your competition
FROM ITERATIVE
TO INNOVATIVE
TESTING

23. @paulrouke #elitecamp2015
WHAT ARE THE
INFLUENCING FACTORS?

24. @paulrouke #elitecamp2015
ITERATIVE TESTING

25. @paulrouke #elitecamp2015
INNOVATIVE TESTING

26. @paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you best choosing each type?
What business impact does each type deliver?
8 steps to out-grow your competition
FROM ITERATIVE
TO INNOVATIVE
TESTING

27. @paulrouke #elitecamp2015
WHY CHOOSE
EACH TYPE?

28. @paulrouke #elitecamp2015
Quick wins
Start to learn
Use your tool
Limited resources
Political business
Build momentum
Demonstrate ROI
Develop a culture
IN SUMMARY
You want to walk before you
start running with testing
WHY CHOOSE ITERATIVE TESTING?

29. @paulrouke #elitecamp2015
Control
ITERATIVE TESTING

30. @paulrouke #elitecamp2015
PRWD Test Variation
ITERATIVE TESTING

31. @paulrouke #elitecamp2015
Result
+ 4.3%Sitewide Purchase Conversion Rate
Over £250k in additional revenue
ITERATIVE TESTING

32. @paulrouke #elitecamp2015
Make significant increases
Really shift user behaviour
Quantifiable evidence for a major change
Prior to back-end system changes
Business proposition evaluation
Reached local maximum
Value proposition learning
Simplify a stepped process
Data driven website redesign
WHY CHOOSE INNOVATIVE TESTING?
IN SUMMARY
You want grow your business
rather than just optimise your
website performance

33. @paulrouke #elitecamp2015
BUSINESS CHALLENGE
Identify the 1 most influential feature why
visitors will choose Premium over FREE

34. @paulrouke #elitecamp2015
WHY CHOOSE INNOVATIVE TESTING?

35. @paulrouke #elitecamp2015
GET REVISING
FREE OR PREMIUM DECISION
Price doesn’t put me off, although the
basic package is more than enough for
me.
I think the site has already got
everything I need.
Moderated user research participant
WHY CHOOSE INNOVATIVE TESTING?

36. @paulrouke #elitecamp2015
WHY CHOOSE INNOVATIVE TESTING?

37. @paulrouke #elitecamp2015
GET REVISING
SIGN-UP TABLE
+ 185%Premium Subscriptions
- 10%
Free Sign-ups
WHY CHOOSE INNOVATIVE TESTING?

38. @paulrouke #elitecamp2015
WHY CHOOSE INNOVATIVE TESTING?

39. @paulrouke #elitecamp2015
Intelligent, innovative testing is essential
in exploiting your growth potential

40. @paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you best choosing each type?
What business impact does each type deliver?
8 steps to out-grow your competition
FROM ITERATIVE
TO INNOVATIVE
TESTING

41. @paulrouke #elitecamp2015
WHEN ARE YOU BEST
CHOOSING EACH TYPE?

42. @paulrouke #elitecamp2015
Quick win test opportunities
Primary proposition headlines
Primary CTA’s in to sign-up process
When imagery is key
Sitewide value proposition messages
Landing pages with forms
Form tests
WHEN TO CHOOSE ITERATIVE TESTING
IN SUMMARY
When you have non-optimised
primary pages in your user journey

43. @paulrouke #elitecamp2015
When you have bold and/or innovative ideas built off previous test insights & deep user understanding
If you have run a range of iterative tests on a key page and are now struggling to impact page performance
After you have run tests aimed at understanding what USP’s most resonate with your target audience
When there is an opportunity to deliver a radically different user experience aimed at simplifying a process
ie. Registration, checkout, application
When there will be a considerable development cost to fully implementing a proposed solution
When you have quite low traffic and want to reduce your test length
When data driven optimisation is truly part of your company culture
WHEN TO CHOOSE INNOVATIVE TESTING
When you are ready to grow your business and
not just optimise your website performance

44. @paulrouke #elitecamp2015
Control
INNOVATIVE TESTING

45. @paulrouke #elitecamp2015
PRWD Test
Variation
INNOVATIVE TESTING

46. @paulrouke #elitecamp2015
Result
+ 18%Brochure Requests
INNOVATIVE TESTING

47. @paulrouke #elitecamp2015
Control
INNOVATIVE TESTING

48. @paulrouke #elitecamp2015
PRWD Test
Variation
INNOVATIVE TESTING

49. @paulrouke #elitecamp2015
Result
+ 31%Brochure Requests
INNOVATIVE TESTING

50. @paulrouke #elitecamp2015
What are the 2 biggest influencers to
running successful innovative tests?
1) In-depth user understanding
2) Meaningful insights from prior tests

51. @paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you best choosing each type?
What business impact does each type deliver?
8 steps to out-grow your competition
FROM ITERATIVE
TO INNOVATIVE
TESTING

52. @paulrouke #elitecamp2015
WHAT BUSINESS IMPACT
DOES EACH TYPE HAVE?

53. @paulrouke #elitecamp2015
BUSINESS IMPACT THROUGH ITERATIVE TESTING

54. @paulrouke #elitecamp2015
BUSINESS IMPACT THROUGH INNOVATIVE TESTING

55. @paulrouke #elitecamp2015
Innovative Testing
Iterative Testing
User Research
High Quality & Intelligent
Optimisation Strategy
=

56. @paulrouke #elitecamp2015
Innovative Testing
Iterative Testing
User Research
=High Quality & Intelligent
Optimisation Strategy

57. @paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you best choosing each type?
What business impact does each type deliver?
8 steps to out-grow your competition
FROM ITERATIVE
TO INNOVATIVE
TESTING

58. @paulrouke #elitecamp2015
8 STEPS TO OUT-GROW
YOUR COMPETITION

59. @paulrouke #elitecamp2015
Foundations - get the right foundations in place for optimisation
Quality – focus on quality of tests, then quantity
Intelligence - start with intelligent, research driven simple tests
Identification – use simple test insights & research to identify radical test opportunities
Momentum - build up testing momentum, buy-in and awareness of learnings
Business case – use ROI from simple/iterative tests to provide the business case for radical testing
Go innovative - begin planning and delivering bold, innovative tests alongside iterative tests
Growth - grow faster than your competition by exploiting the full spectrum of testing opportunities
1
2
3
4
5
6
7
8 STEPS TO EXPLOIT THE FULL SPECTRUM OF TESTING OPPORTUNITIES
8

60. PAUL ROUKE
Founder & Director of
Optimisation
@paulrouke
THANK YOU FOR LISTENING
HERE’S MY FINAL CTA…
OVER 75 CRO ARTICLES
bit.ly/CROresources