Website Testing & Optimization Framework

1. Sumo Logic
Framework & Journey of
Website Testing/Optimization
Saad Hameed & Jithesh Ambalapad
Marketing, Operations, Analytics & technology team

2. LinkedIn: http://j.mp/SaadHameed
Twitter: @SaadVirtus https://twitter.com/SaadVirtus

3. LinkedIn: https://www.linkedin.com/in/apjithesh

4. ○ Improve Navigation (IA)
○ Align Messaging
○ Improve Bounce Rate
Website Journey Phase 1: Create
Baseline of Web Visitor Engagement

5. ○ Build taxonomy & foundational tools to measure &
optimize
• Google Tag Manager
• Google Analytics
• User Testing
• Optimizely
• CrazyEgg
○ Website Redesign & Continuous Improvement
• Major revamp of our website UI/UX
• Updated Content/Messaging
The How of Improving Baseline
Engagement for Website

6. Sumologic.com Site
Pre-Phase 1 (Aug 2015)
https://web.archive.org/web/20160405124254/https://www.sumologic.com/

7. Sumologic.com
Post-Phase 1 Refreshes:
May 2016
https://web.archive.org/web/20150814044927/https://www.sumologic.com/

8. • Results
○ Continued refreshes improved Bounce Rates from 65% to 40%
○ Improved free conversions + 10%
○ Improved Alexa Ranking (15% Improvement)
Nov 2015 May 2016
Results From The Phase 1 Refresh

9. ○ Ongoing major initiative to optimize for engagement and conversion
○ Started with detailing a testing framework
○ Framework and testing is geared towards
• Improving Conversion
• Improving Engagement
Journey Phase 2: Web Optimization to
Improve Conversions

10. Improve Conversions & Engagement:
• Platinum (Free/Trial, Contact Us, Click to chat)
• Gold (Platinum + Gated content resources)
• Silver (Gold + Non-Gated content resources)
Define the different elements that we would need to test and have
a categorization (A/B Testing taxonomy)
Detailing out test stories, execute the tests, measure, &
communicate
Why?
Why to test
What?
What do we need
to test
How?
Process to test
The Why, What, and How of Web
Optimization & Testing

11. Information
Architecture
Content
● Components
○ Main Navigation
○ Utility Navigation
○ Footer
● Attributes
○ Size
○ Position
○ Verbiage
Images Content Modules Call to ActionsOffers
● Components
○ Spotlight
○ Logos
○ Icons
○ Product
● Attributes
○ Size
○ Position
○ Image types
(Human /Product)
● Components
○ Company Pitch
○ Product Pitch
○ Product Validators
○ Product Demos
○ Product Benefit
○ Customer Testimonial
○ Customer Logos
○ Pricing
● Attributes
○ Tone
○ Voice
○ Size
○ Position
● Components
○ Gated
○ Non-Gated
● Attributes
○ Title
○ Type (Blog, Podcast,
video, PDF, etc.)
○ Perceived Value of
offer
○ Relevance to phase
in buyer lifecycle
● Components
○ Free CTA
○ Video CTA
○ Download
● Attributes
○ Size
○ Position
○ Action/Verbiage
○ Color
○ Font
○ Image
Web Optimization & Testing Framework

12. Information
Architecture
● Components
○ Main Navigation
○ Utility Navigation
○ Footer
● Attributes
○ Size
○ Position
○ Verbiage
Size Position Verbiage
Main Nav Large or Small Change Position Nav Item Verbiage
Utility Nav Large or Small Top or Next to Main Item Verbiage
Footer Fat footer or Thin Footer Left or Center Item Verbiage
How to Leverage the Framework? Example:

13. • Definition and detailing of “What” becomes the essence
of testing ideas
• Map the ideas based on Impact to business & Ease of
implementation
• Prioritize and execute ideas
• Document & take action on Results
Creating a Funnel for Web Optimization &
Testing Experiments

14. Ease of Implementation
BusinessImpact
● New Offers
● Nav position
● Related content
position
● Offers A/B testing
with Audience data
● Contact us position
● Footer Link sizes
● Media download
positioning
● Positioning of Share
button
● Link out positioning
for external
references
● Position of Mobile
nav
● Enable/Disable link
to product demo
● Elevator pitch
message
● Hide/enable related
links
● Multivariate Testing
with customer quotes
● Spotlight size
● Demo Multi page
● CTA positioning
● Nav element
positioning
● Free CTA
● Nav text verbiage
● Login position
High
Medium
Low
Low
(Hard)
Medium High
(easy)
THIS IS A SAMPLE
NOT ACTUAL IDEAS
How to Prioritize the Web Optimization &
Testing Experiments?

15. Example of Web Experiment
Test Click and Conversion rate when button verbiage is changed

16. Example of Web Experiment

17. Web Experiment Results

18. The How (Process) of Web Optimization &
Testing