A joint presentation at DMDays09 New York, showcasing a case study between Widemile and Microsoft Office Live Small Business, which through Widemile's multivariate testing platform improved Microsoft's online conversions by 40%.

1. Microsoft Case Study: 40% Conversion Rate Increase of SEM Traffic using Multivariate Testing Michael Schultz Director Product Management & Marketing, Microsoft Office Unmanaged Services Frans Keylard Director of Optimization, Widemile

4. Testing Options – Approach Overview Ideal use: > Validates traffic and conversion rates Limitation : > Simple measurement; no optimization Basic Analytics A/B Split Testing Ideal use: > Best for testing two or more radically different pages <i.e. template testing> Limitation : > No insight into why one page is better, just which won among traffic tested Multivariate Testing (MVT ) Ideal use: > Test different content, simultaneously determining most effective combination > Segmentation can be applied for more granular optimization in follow-on tests Limitation : > Requires sufficient traffic & conversions

5. MVT Testing Options - Full & Fractional Factorial Fractional Factorial Full Factorial

6. Requirements for Success Expertise User Interface Proven Architecture Reporting Interface Analyst Creative Web Developer Technology

7. 2009 – Year for Conversion Optimization

11. Testing Considerations What is the best hero shot? Does the headline grab the customer’s attention? Do these resource links belong here? Does the button maximize conversions? Does this copy help or hurt?

12. Indentifying Factors to Test 1 2 3 4 5

13. Defining a Test Matrix No. Factor (Variable) Levels (versions) 1 Hero Shot 4 2 Title 4 3 Description 4 4 Call-To-Action Button 4 5 Resource Links 2

14. Audience Participation – Best CTA Button Level 2 Level 3 Level 4 Level 1

15. Efficiency Through Technology 1 of 16 2 of 16 3 of 16 4 of 16 16

16. And the Winner is… 40% Improvement

17. Which Factors Made the Biggest Difference Hero Shot Title Description Call-To-Action Resource Link Location High Influence Medium Influence Low Influence