3 TED style talks of 15-20 minutes, featuring: (1) Conversion methodologies, Lean UX and Agile? What gives? (2) #Measurecamp and my Top Analytics Tips of 2013 (3) Conversion tools of the CRO masters

1. #WAWCPH
Meet @OptimiseOrDie
Reason for being : I Optimise Stuff
Past : John Lewis, Lovefilm, Belron, PwC
Now : Director of Optimisation, Rush
Email : sullivac@gmail.com
Twitter : @OptimiseOrDie
: linkd.in/pvrg14
Slideshare : slidesha.re/nlCDm6

2. "If you go to the men's washrooms at
the Schiphol airport in
Amsterdam, you may notice there's a
fly in the urinals. It’s screen printed .
So what do you think most men do?
That's right, they aim at the fly when
they urinate.
They don't even think about it, and
they don't need to read a user's
manual; it's just an instinctive
reaction that means 85% less spillage!
The interesting feature of these
urinals is that they're deliberately
designed to take advantage of this
inherent human male tendency.“
This is my job.

3. Is there a way to fix this then?
24 Jan 2012

4. Three Presentations
#1 Conversion Methodologies
How does CRO fit in with Lean UX, Agile, User Centred, Waterfall?
#3 Analytics Genius Tips
Top tips from #Measurecamp
+ my 2013 roundup of cool stuff
#2 The Optimisers Toolkit
The best tools recommended by CRO practitioners

5. #1 Conversion Methodologies

6. What’s the problem?
• #1 User Experience and Conversion
Optimisation are not a checkbox or a
step in the process – it needs to be
integrated in everything you do.
• #2 This work isn’t a one off exercise
either – it’s an ongoing continuous
improvement process – like Kaizen
• #3 Usability testing isn’t enough –
other UX factors like the visceral and
behavioural emotional responses we
have to products need tuning too.
• #4 It’s not just about the user!
• #5 It’s usually Self Centred Design
driven by Ego, Opinion, Assumption

8. Also, the dial won’t turn anymore
PPC SEO
With thanks to @morys

9. Why is this happening?
• PPC changes
• Advertising models flattening – i.e. mobile Google costs
• Comprehensive SEO changes
• Competition increasing
• Fleetness of foot – Asos in Australia
• Entry costs are lower now
• New entrants compete without the cruft
• Startups are using better ‘build and optimise’ methodologies
than nearly all corporates
• The old way of doing things is going to die
• The new way of doing things is your only survival ticket

10. So what do people do?
• They throw tools at the problem
• They try usability testing and research
• They generate more data to look at
• They make changes without measuring or testing
• They hope to randomly create the optimal system
• They get an expensive agency to help them
• They push their team harder, like galley slaves
• They experiment with riskier advertising models
• They wonder why they’re burning rubber
• They try more random things
• Then they call a CRO person and say
“We’ve tried everything. It isn’t working. Help!”

11. Skinner’s Pigeon Experiment
• Participants invited into a room with objects
• Told to score 100 points within 30 minutes
• Participants moved objects around, made
noise, jumped around, tried anything to make a
counter increase the points score.
• They got horribly confused
• They then created convincing lies for
themselves, to explain what they thought was
working. They became superstitious and made
rituals.
• The points allocation was made randomly by a
goldfish, swimming back and forward in a tank.
• Know any marketing departments like this?
• I’ve seen this a lot – and it paralyses companies
• We need a better way. A methodology?

13. Lean UX
“The application of UX design methods into product
development, tailored to fit Build-Measure-Learn cycles.”
Positive
– Lightweight and very fast methods
– Realtime or rapid improvements
– Documentation light, value high
– Low on wastage and frippery
– Fast time to market, then optimise
– Allows you to pivot into new areas
Negative
– Often needs user test feedback to
steer the development, as data not
enough
– Bosses distrust stuff where the
outcome isn’t known

14. Agile UX / UCD / Collaborative Design
“An integration of User Experience Design and Agile*
Software Development Methodologies”
*Sometimes
Research
Positive
– User centric
– Goals met substantially
– Rapid time to market (especially when
using Agile iterations) Wireframe
Negative
Concept Prototype
– Without quant data, user goals can
drive the show – missing the business
sweet spot
– Some people find it hard to integrate
with siloed teams Analyse Test
– Doesn’t’ work with waterfall IMHO

15. CRO

16. Lean Conversion Optimisation
“A blend of User Experience Design, Agile PM, Rapid Lean
UX Build-Measure-Learn cycles, triangulated data
sources, triage and prioritisation.”
Positive
– A blend of several techniques
– Multiple sources of Qual and Quant data aids triangulation
– CRO analytics focus drives unearned value inside all
products
Negative
– Needs a one team approach with a strong PM who is a
Polymath (Commercial, Analytics, UX, Technical)
– Only works if your teams can take the pace – you might be
surprised though!

17. Lean CRO
Inspection
Instrument Immersion
Learn Identify
Triage &
Measure
Triangulate
Outcome
Streams

18. Triage and Triangulation
“This is where the smarts of CRO are – in identifying the
easiest stuff to test or fix that will drive the largest uplift.”
• Starts with the analytics data
• Then UX and user journey walkthrough from SERPS -> key paths
• Then back to analytics data for a whole range of reports:
• Segmented reporting, Traffic sources, Device viewport and
browser, Platform (tablet, mobile, desktop) and many more
• We use other tools or insight sources to help form hypotheses
• We triangulate with other data where possible
• We estimate the potential uplift of fixing/improving something
as well as the difficulty (time/resource/complexity/risk)
• A simple quadrant shows the value clusters
• We then WORK the highest and easiest scores by…
• Turning every opportunity spotted into an OUTCOME

19. The Bucket Methodology
“Helps you to stream actions from the insights and prioritisation work.
Forces an action for every issue, a counter for every opportunity being lost.”
 Test
If there is an obvious opportunity to shift behaviour, expose insight or
increase conversion – this bucket is where you place stuff for testing. If
you have traffic and leakage, this is the bucket for that issue.
If an issue is placed in this bucket, it means we need to beef up the
 Instrument analytics reporting. This can involve fixing, adding or improving tag or
event handling on the analytics configuration. We instrument both
structurally and for insight in the pain points we’ve found.
 Hypothesise This is where we’ve found a page, widget or process that’s just not working
well but we don’t see a clear single solution. Since we need to really shift
the behaviour at this crux point, we’ll brainstorm hypotheses. Driven by
evidence and data, we’ll create test plans to find the answers to the
questions and change the conversion or KPI figure in the desired direction.
 Just Do It
JFDI (Just Do It) – is a bucket for issues where a fix is easy to identify or the
change is a no-brainer. Items marked with this flag can either be deployed
in a batch or as part of a controlled test. Stuff in here requires low effort
or are micro-opportunities to increase conversion and should be fixed.
 Investigate You need to do some testing with particular devices or need more
information to triangulate a problem you spotted. If an item is in this
bucket, you need to ask questions or do further digging.

20. How is it working out?
• Methodologies are not Real Life ™
• It’s mainly about the mindset of the team and
managers, not the tools or methodologies they
play with
• Not all my clients have all the working parts
• You should not be a methodology slave
• Feel free to make your own or flexibly adapt
• Use some, any techniques instead of ‘guessing’
• Blending lean and agile with conversion
optimisation outcomes is my critical learning of
the last 5 years
• Doing rapid cycles of this outcome driven work
for Belron:
• World Conversion Rate Increase:
2009 +5%, 2010 +10%, 2011 +15%, 2012 +25%
• If you’d like to develop a good one for your
company, talk to me first!
• Don’t over complicate it.

21. Questions ?

22. #2 2013 Tips roundup!!!!!!!

23. Tips Roundup 2013
#1 Performance tune-ups
#2 Browser money
#3 Keyboard shortcuts
#4 Ranking data
#5 Content engagement
#6 Enhanced In page
#7 Duplicate transactions
#8 Event tracking
#9 Google API + more

24. #1 : Performance Tune-ups
• With thanks to @Danbarker
• Add “_gaq.push(['_setSiteSpeedSampleRate', 100]);”
• Amps up sampling for small & medium websites
• Use the distribution report - % of pages < 3 seconds
• DOM timings vital – let me explain
• Avg. Document Content Loaded Time (sec)
• This data is very accurate and helps conversion – pretty vital for
landing pages, where I find lots of stuff
• Make yourself a pageview*content loaded time report
• Work your way down from the top
• Mobile speed doesn’t count Safari – please be careful!
• Read more at:
http://p.barker.dj/sitespeedtips

25. #2 : Browser money
• Create a desktop only segment [Exclude mobile inc. tablet ]
• Create a mobile only segment [Include Mobile (including tablet) & Exclude Tablet
containing yes]. Apply 3 segments desktop, tablet, mobile ONLY.
• Now start segmenting like this:
Browser Segment Conv rate Quote the
Safari Mobile Traffic 0.79%
Internet Explorer Desktop only 1.34%
opportunity!
Chrome Desktop only 1.01% IE8 = 1.41% revenue
Safari Tablet Traffic 1.00%
Safari Desktop only 1.28% IE8 converts at 20%
Firefox Desktop only 1.20% of IE9 and IE10
Android Browser Mobile Traffic 0.31%
Some nasty bugs
Safari (in-app) Mobile Traffic 0.69%
cause the problem
Chrome Mobile Traffic 0.62%
Safari (in-app) Tablet Traffic 0.89% Worth fixing?
Chrome Tablet Traffic 0.84%
Opera Desktop only 0.10% That 1.41 is worth
Android Browser Tablet Traffic 0.71% nearly 6% more
Mozilla Compatible Agent Mobile Traffic 0.51% revenue

26. #2 : Browser money
• You should see something like this:

27. #3 : GA Keyboard Shortcuts
• Thanks to Farid Alhadi and @fastbloke
dt Set date range to TODAY
dy Set date range to YESTERDAY
dw Set date range to LAST WEEK
dm Set date range to LAST MONTH
dc
Toggle date comparison mode (to the previous period of whatever you are looking at.
Example, if you’re looking at 6 days, this will compare it to the 6 days before it)
dx Toggle date comparison mode (to the previous year of the period you are looking at)
? Open keyboard shortcut help
h Search help center
a open account panel
shift + a Go to account list
s/ Search reports
shift + d Go to the default dashboard of the current profile

28. #4 : Ranking data
• Someone searches on Google for ‘Term’
• Clicks on a link to your site
• What was the keyword rank for that term?
• Reverse engineer the actual rankings from users machines
• More accurate than some SEO tools (IMHO)
• Two articles to read:
http://bit.ly/Vaisno
http://bit.ly/13lmYF2

29. #5 : Content Engagement
• Get a better bounce rate metric
• See more detailed engagement metrics
• Measure scrolling and reading activity
• Came from @fastbloke but originally Justin Cutroni
• Measure your scrolling and exit points from long form
content
• Very nice technique – read more at:
http://bit.ly/13lmYF2
http://goo.gl/1AZZb

30. #6 : Enhanced Page Analytics
var _gaq = _gaq || [];
var pluginUrl =
'//www.google-analytics.com/plugins/ga/inpage_linkid.js';
_gaq.push(['_require', 'inpage_linkid', pluginUrl]);
_gaq.push(['_setAccount', 'UA-XXXXXX-Y']);
_gaq.push(['_trackPageview']);
• Correct link attribution for in-page Analytics
• Very nice

31. #7 : Duplicate transactions in GA
• Thanks to Matt Clarke and @timlb
• Stop skewing the data with duplicates/reloads
• Custom report to check if you’re affected
http://techpad.co.uk/content.php?sid=247
• I’ve seen this a few places so worth
checking, particularly if figures don’t tally!
• Read more at : http://bit.ly/13lmYF2

32. #8 : Event tracking
• Thanks to #Measurecamp – check the stream
• A beginners guide : http://bit.ly/13RFoJs
• Some great ideas here : http://bit.ly/UCcptx
• Don’t go for ‘Event Blizzard’
• Focus on specific areas where insight is needed
• Choose your naming structure carefully:
http://bit.ly/WJ4R4c
• Read this complete guide : http://bit.ly/VmFSJ4

33. #9 : Google API and more
• Use the Google API to get super custom reports
• You can fetch different data types (on the fly as
well as pre-calculated)
• Automate a lot of Excel work – more later…
• @timlb recorded all the tips for #Measurecamp
• We then had an unconference
• He recorded a special session for us
• Listen to great tips and more:
www.youtube.com/watch?v=JWXg1_4quwU
www.youtube.com/watch?v=z41Cdg9xbk0
• Roundup : www.measurecamp.org/aftermath/

34. #10 : Microdata
• Which of these is most appealing?
24 Jan 2012

35. #10 : Microdata – SERPS UX
• Reviews – huge increases in CTR and Conversion
• People
• Products
• Businesses and Organisations
• Recipes
• Events
• Music
• Local
• Video
Helps to:
• Dominate the page
• Push other stuff down
• Makes it more persuasive
• The conversion journey starts here!

36. #11 : Measure viewport size
• Thanks to @Beantin and others!
• Measure the viewport size, not the resolution
• Why?
• Toolbars, chrome and setup varies
• UK average is 2.2 toolbars installed
• Code example here : http://bit.ly/4xaNYK
• A common conversion issue
• Your desk vs. Users = different
• Turn off the wifi, reduce the viewport
• Or keep the inside of toilet rolls

37. Questions
?

38. #3 The Optimisers Toolkit

39. The Optimisers Toolkit
#1 Session Replay
#2 Browser & Email testing
#3 VOC & Feedback tools
#4 Guerrilla Usability
#5 Productivity tools
#6 Split testing
#7 Performance
#8 Crowdsourcing
#9 Analytics Love

40. #1 : Session Replay
• 3 kinds of tool :
Client side
• Normally Javascript based
• Pros : Rich mouse and click data,
errors, forms analytics, UI interactions.
• Cons : Dynamic content issue, Performance hit
Server side
• Black Box -> Proxy, Sniffer, Port copying device
• Pros : Gets all dynamic content, fast, legally tight
• Cons : No client side interactions, Ajax, HTML5 etc.
Hybrid
• Clientside and Sniffing with central data store

41. #1 : Session Replay
• Vital for optimisers & fills in a ‘missing link’ for insight
• Rich source of data on visitor experiences
• Segment by browser, visitor type, behaviour, errors
• Forms Analytics (when instrumented) are awesome
• Can be used to optimise in real time!
Session replay tools
• Clicktale (Client) www.clicktale.com
• Mouseflow (Client) www.mouseflow.com
• Ghostrec (Client) www.ghostrec.com
• Tealeaf (Hybrid) www.tealeaf.com
• Usabilla (Client) www.usabilla.com
• UserReplay (Server) www.userreplay.com
• SessionCam (Client) www.sessioncam.com

45. #2 : Feedback / VOC tools
• Anything that allows immediate realtime onpage feedback
• Comments on elements, pages and overall site & service
• Can be used for behavioural triggered feedback
• Tip! : Take the Call Centre for beers
• Kampyle
www.kampyle.com
• Qualaroo
www.qualaroo.com
• 4Q
4q.iperceptions.com
• Usabilla
www.usabilla.com
Any more?

46. #3 – Testing tools
Email testing www.litmus.com
www.returnpath.com
www.lyris.com
Browser testing www.browsercam.com
www.crossbrowsertesting.com
www.cloudtesting.com
www.multibrowserviewer.com
www.saucelabs.com
Mobile devices www.perfectomobile.com
www.deviceanywhere.com
www.mobilexweb.com/emulators
www.opendevicelab.com

47. #2a : Survey Tools
I’ve used
• Surveymonkey www.surveymonkey.com (1/5)
• Zoomerang www.zoomerang.com (3/5)
• SurveyGizmo www.surveygizmo.com (5/5)
• Read anything by and follow @cjforms
• Any others?

48. #4 : Guerrilla Usability Testing
• All you need is a device, time and people!
• Use one of these tools for session recording:
Mediacam AV (cheap)
www.netu2.co.uk
Camtasia
www.techsmith.com
Silverback (Mac)
www.silverbackapp.com
Screenflow (Mac)
www.telestream.net
UX Recorder (iOS)
www.uxrecorder.com

49. #5 : Productivity tools

50. #5 Join.me
24 Jan 2012

51. #5 Pivotal Tracker
24 Jan 2012

52. #5 Google Docs
• Seriously wasting your time doing manual Excel?
• Fed up doing stuff that takes hours?
• Use the Google API to roll your own reports straight into Big G
• Lots of good articles but ask for advice from:
@danbarker
@timlb
#measurecamp
• Google Analytics + API + Google docs integration = A BETTER LIFE!
• Hack your way to having more productive weeks
• Learn how to do this and to have fun with GA custom reports
• Ask me about the importance of training
24 Jan 2012

53. #5 Conceptshare
24 Jan 2012

54. #5 Basecamp
24 Jan 2012

55. #6 – Split testing tools – Cheap!
• Google Content Experiments
bit.ly/Ljg7Ds
• Multi Armed Bandit Explanation
bit.ly/Xa80O8
• Optimizely
www.optimizely.com
• Visual Website Optimizer
www.visualwebsiteoptimizer.com

56. #7 Performance
• Google Site Speed
• Webpagetest.org
• Mobitest.akamai.org

57. Remote tests : iPhone
Site Size Requests
www.coop.se 1200k 63
m.ikea.com/se/sv/ 684k 14
High St Retailer 307k 43
Department Store 100k 18
Newspaper 195k 35
Supermarket 125k 14
Auto Sales 151k 47
Autoglass 25k 10

58. Show your boss this split test:
Slides : slidesha.re/PDpTPD

59. Show the e-com director this one:

60. #7 – UX and Crowd tools
Som, feedback
Remote UX tools (P=Panel, S=Site recruited, B=Both)
Usertesting (B) www.usertesting.com
Userlytics (B) www.userlytics.com
Userzoom (S) www.userzoom.com
Intuition HQ (S) www.intuitionhq.com
Mechanical turk (S) www.mechanicalturk.com
Loop11 (S) www.loop11.com
Open Hallway (S) www.openhallway.com
What Users Do (P) www.whatusersdo.com
Feedback army (P) www.feedbackarmy.com
User feel (P) www.userfeel.com
Ethnio (For Recruiting) www.ethnio.com
Feedback on Prototypes / Mockups
Pidoco www.pidoco.com
Verify from Zurb www.verifyapp.com
Five second test www.fivesecondtest.com
Conceptshare www.conceptshare.com
Usabilla www.usabilla.com

61. #8 : Web Analytics Love
• Properly instrumented analytics
• Investment of 5-10% of developer time
• Add more than you need
• Events insights
• Segmentation
• Call tracking love!

62. #8 : Tap 2 Call tracking
Step 1 : Add a unique phone number on ALL channels
(or insert your own dynamic number)
Step 2 : For phones, add “Tap to Call” or “Click to Call”
• Add event or PV trigger for any phone interaction
• Incredibly useful keyword level call data
• Very reliable data, easy & cheap to do
• What did they do before calling?
• Which page did they call you from?
• What PPC or SEO keyword did they use?
• What are you over or underbidding for?
• Will help you shave 10, 20%+ off PPC
• Which online marketing really sucks?

63. What about desktop?
Step 1 : Add ‘Click to reveal’
• Can be a link, button or a collapsed section
• Add to your analytics software
• This is a great budget option!
Step 2 : Invest in call analytics
• Unique visitor tracking for desktop
• Gives you that detailed marketing data
• Easy to implement
• Integrates with your web analytics
• Let me explain…

64. 10.0
15.0
20.0
25.0
0.0
5.0
safe lit
windshield chip repair
safelite windshield
autoglass
auto window replacement
safelight auto
auto glass repair
windshield replacement costs
safelite auto glass
safe auto glass
safelite repair
windshield
autoglass replacement
car glass
safelite locations
auto glass repair quotes
Phone to Booking Ratio
windshield repair
mobile windshield replacement
replace windshield
car windshield repair
new windshield cost
auto glass windshield replacement
car window repair cost
safe gl
auto glass repair houston tx
windshield crack

66. What about desktop?
Onbound Call analytics:
Split testing
Priority routing
Wait time
VIP teams
Call centre
Abandon
A 222 1234 Call length music
On hold
0800 222 1234 0800 Operator outcome
Menu
Sales value
0800 222 1245
Messages
Ratings
Call recording
B 222 1263
0800
Offline promotions!
0800 222 1299 Profit line!
C
Web
Analytics Telephony Cloud

67. So what does phone tracking get you?
• You can do it for free on your online channels
• If you’ve got any phone sales or contact operation, this will
change the game for you
• For the first time, analytics for PHONE for web to claim
• Optimise your PPC spend
• Track and Test stuff on phones, using web technology
• The two best tests?
• Talk to Freespee today!
• Other countries, see the next slide

68. Company Website Coverage
Who?
Mongoose Metrics* www.mongoosemetrics.com UK, USA, Canada
Ifbyphone* www.ifbyphone.com USA
TheCallR* www.thecallr.com USA, Canada, UK, IT, FR, BE, ES, NL
Call tracking metrics www.calltrackingmetrics.com USA
Hosted Numbers www.hostednumbers.com USA
Callcap www.callcap.com USA
UK, SE, FI, NO, DK, LT, PL, IE, CZ,
Freespee* www.freespee.com
SI, AT, NL, DE
Adinsight* www.adinsight.co.uk UK
Infinity tracking* www.infinity-tracking.com UK
Optilead* www.optilead.co.uk UK
Switchboard free www.switchboardfree.co.uk UK
Freshegg www.freshegg.co.uk UK
Avanser www.avanser.com.au AUS
Jet Interactive* www.jetinteractive.com.au AUS
* I read up on these or talked to them. These are my picks .

69. #9 : Web Analytics Love
• UX of web analytics tools and reports
• People, Process, Human problems
• Make the UI force decisions!
• Playability and exploration
• Skunkworks project time (5-10%)
• Give it love, time, money and iteration
• Spend to automate, get more time

70. Evil CRO
• Optimisation can be used for evil!
• Either by accident (just stupid)
• Or by design, to trick people
wiki.darkpatterns.org/wiki/Home

71. So how do I become an Optimiser?
• It’s a state of mind, not a course!
• You need to read lots, test loads and watch customers even more
• Ask someone to mentor you
• Go to conferences, events, find people online
• Don’t say “I think” – say “I’ll find out” or “I’ll test that”
• Volunteer for shit jobs and do them well
• Be the glue between people – volunteer for cross team work
• Fix bad things so your team loves you
• Always connect others with outcomes
• Hack your job to a better life
• Make yourself more valuable, not less
• Challenge fucking everything
• Be annoying and never give up!

72. Don’t give up!
• Start with the customer at the heart of everything
• It’s liberating!
• Use low budget solutions, cheap or free tools to get business cases
made or gain insight
• Money talks so even micro testing will illustrate ROI
• Invest in people who care about customers, as well as conversion
• Invest in analytics talent and instrumentation
• If you can make a 10% shift in a key metric (CTR, conversion), just
from smart thinking, then what would a team of 5 get you?
• If you want to hire people, ask me for advice!

73. “We always
had a plan for
pulling out”

74. “We always
had a plan for
pulling out”

75. More reading. Slides and resources on slideshare.net
Email : sullivac@gmail.com
Twitter : @OptimiseOrDie
: linkd.in/pvrg14
Slideshare : slidesha.re/nlCDm6