SlideShare une entreprise Scribd logo
1  sur  24
Télécharger pour lire hors ligne
#opticon2015
Scaling The Next Web’s Testing
Program to Maximum Impact
Martijn Scheijbeler
Lead Growth, SEO & Analytics - The Next Web
@martijnsch
martijn@thenextweb.com
Yearly 5.000 attendees, 75.000 products sold.

Monthly 6 million unique users, 12 million pageviews.
The Next Web: 2006 > 2015
What’s happened in the last 12 months?
Scaling our testing program
That clearly wasn’t good enough
1 test a month :(
Currently 5-6 tests a week, how did we get there?
15-20 tests a month :)
What did we learn along the way?
• Swim lanes, KPIs, Goals.
• Segments: traffic sources, etc.
• Idea creation: backlog + prioritisation.
• Scaling > Analysing results.
• Do it live!
Unique Users, Page Views, Emails & Revenue.
Goals: UU, PV, Emails & $
Related stories, pagination, sharing, email signups.
KPI: CTR, Engagement, Funnels
New v.s. Returning - Europe v.s. USA - Facebook v.s. SEO - Mobile v.s. Desktop
Type, Regions, Source, Device
Idea creation v.s. Planning
• Different templates + segments.
• How much time per test? Run tests for a fixed predefined time.
• The maximum of your program.
• Take away: create a backlog!
The part that we can’t scale yet
Analysing > Scaling!
Well, this is both the most boring and most exciting part!
1-2 main KPIs, 15 secondary KPIs
Rails, Google Analytics API

Built with love by the TNW Marketing team.
TNW Analytics
TNW Analytics was born!
Adding a new idea
Schedule
Test
Saving 20% of time per test!
Take away: Be able to push code!
What’s next?
Scaling our testing program to the next level?
• Even more templates, across more projects.
• 25+ tests a month with current growth of traffic.
• Bigger team? We can’t keep up with coding ;-)
• More automation? Push test variant to GTM.
• Mix it up with personalisation/ behaviour.
• Push our experiments to Slack for sharing.
• Create a better style guide + Bootstrap guidelines.
Summary & Take aways
• Know your KPI framework, expectations and limitations.
• Know your segments, CR% so you know how to prioritise.
• Follow your planning, priorities.
• Scale = 40% - 50% automation: analyse fast!
• Be able to push code live!
#opticon2015
Have fun & Thank you!
Martijn Scheijbeler
Lead Growth , SEO & Analytics - TNW
@martijnsch
martijn@thenextweb.com

Contenu connexe

En vedette

Google_Controlled Experimentation_Panel_The Hive
Google_Controlled Experimentation_Panel_The HiveGoogle_Controlled Experimentation_Panel_The Hive
Google_Controlled Experimentation_Panel_The Hive
The Hive
 

En vedette (11)

Google_Controlled Experimentation_Panel_The Hive
Google_Controlled Experimentation_Panel_The HiveGoogle_Controlled Experimentation_Panel_The Hive
Google_Controlled Experimentation_Panel_The Hive
 
LinkedIn Breakfast: Generating more opportunities with LinkedIn Sales Navigator
LinkedIn Breakfast: Generating more opportunities with LinkedIn Sales NavigatorLinkedIn Breakfast: Generating more opportunities with LinkedIn Sales Navigator
LinkedIn Breakfast: Generating more opportunities with LinkedIn Sales Navigator
 
4 Questions to Ask to Determine Which Social Channels Are Right for Your Busi...
4 Questions to Ask to Determine Which Social Channels Are Right for Your Busi...4 Questions to Ask to Determine Which Social Channels Are Right for Your Busi...
4 Questions to Ask to Determine Which Social Channels Are Right for Your Busi...
 
LinkedIn Executive Playbook
LinkedIn Executive PlaybookLinkedIn Executive Playbook
LinkedIn Executive Playbook
 
Työnhakijan LinkedIn
Työnhakijan LinkedInTyönhakijan LinkedIn
Työnhakijan LinkedIn
 
LinkedIn Publishing Playbook
LinkedIn Publishing PlaybookLinkedIn Publishing Playbook
LinkedIn Publishing Playbook
 
LinkedIn 5 things to know in 2016 Infographic
LinkedIn 5 things to know in 2016 InfographicLinkedIn 5 things to know in 2016 Infographic
LinkedIn 5 things to know in 2016 Infographic
 
LinkedIn Profile Guide for Government Professionals
LinkedIn Profile Guide for Government ProfessionalsLinkedIn Profile Guide for Government Professionals
LinkedIn Profile Guide for Government Professionals
 
Aatif Awan, Head of Growth LinkedIn - Growth Hacking is Dead. Long Live Growth.
Aatif Awan, Head of Growth LinkedIn - Growth Hacking is Dead. Long Live Growth. Aatif Awan, Head of Growth LinkedIn - Growth Hacking is Dead. Long Live Growth.
Aatif Awan, Head of Growth LinkedIn - Growth Hacking is Dead. Long Live Growth.
 
Social Selling 101: The Fundamentals of Social Selling
Social Selling 101: The Fundamentals of Social SellingSocial Selling 101: The Fundamentals of Social Selling
Social Selling 101: The Fundamentals of Social Selling
 
LinkedIn Military Recruitment Playbook
LinkedIn Military Recruitment PlaybookLinkedIn Military Recruitment Playbook
LinkedIn Military Recruitment Playbook
 

Plus de Martijn Scheijbeler

Not Provided - Search Marketing Thursday 7 November 2013
Not Provided - Search Marketing Thursday 7 November 2013Not Provided - Search Marketing Thursday 7 November 2013
Not Provided - Search Marketing Thursday 7 November 2013
Martijn Scheijbeler
 

Plus de Martijn Scheijbeler (12)

Digital Analytics Summit 2022
Digital Analytics Summit 2022Digital Analytics Summit 2022
Digital Analytics Summit 2022
 
BrightonSEO October 2022 - Martijn Scheybeler - SEO Testing: Find Out What Wo...
BrightonSEO October 2022 - Martijn Scheybeler - SEO Testing: Find Out What Wo...BrightonSEO October 2022 - Martijn Scheybeler - SEO Testing: Find Out What Wo...
BrightonSEO October 2022 - Martijn Scheybeler - SEO Testing: Find Out What Wo...
 
Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.
Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.
Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.
 
Conversion Jam
Conversion JamConversion Jam
Conversion Jam
 
Brighton SEO - September 2016
Brighton SEO - September 2016Brighton SEO - September 2016
Brighton SEO - September 2016
 
Emerce Conversion
Emerce ConversionEmerce Conversion
Emerce Conversion
 
Friends of Search 2016
Friends of Search 2016Friends of Search 2016
Friends of Search 2016
 
SEO Meetup Utrecht
SEO Meetup UtrechtSEO Meetup Utrecht
SEO Meetup Utrecht
 
MeasureFest
MeasureFestMeasureFest
MeasureFest
 
Digital Data Tips Tuesday
Digital Data Tips TuesdayDigital Data Tips Tuesday
Digital Data Tips Tuesday
 
ConversionSummit 2015 - Testing culture: making testing look easy!?
ConversionSummit 2015 - Testing culture: making testing look easy!?ConversionSummit 2015 - Testing culture: making testing look easy!?
ConversionSummit 2015 - Testing culture: making testing look easy!?
 
Not Provided - Search Marketing Thursday 7 November 2013
Not Provided - Search Marketing Thursday 7 November 2013Not Provided - Search Marketing Thursday 7 November 2013
Not Provided - Search Marketing Thursday 7 November 2013
 

Dernier

Brand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdfBrand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdf
tbatkhuu1
 

Dernier (20)

Kraft Mac and Cheese campaign presentation
Kraft Mac and Cheese campaign presentationKraft Mac and Cheese campaign presentation
Kraft Mac and Cheese campaign presentation
 
Brand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdfBrand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdf
 
How to utilize calculated properties in your HubSpot setups
How to utilize calculated properties in your HubSpot setupsHow to utilize calculated properties in your HubSpot setups
How to utilize calculated properties in your HubSpot setups
 
Unraveling the Mystery of the Hinterkaifeck Murders.pptx
Unraveling the Mystery of the Hinterkaifeck Murders.pptxUnraveling the Mystery of the Hinterkaifeck Murders.pptx
Unraveling the Mystery of the Hinterkaifeck Murders.pptx
 
How to Leverage Behavioral Science Insights for Direct Mail Success
How to Leverage Behavioral Science Insights for Direct Mail SuccessHow to Leverage Behavioral Science Insights for Direct Mail Success
How to Leverage Behavioral Science Insights for Direct Mail Success
 
Situation Analysis | Management Company.
Situation Analysis | Management Company.Situation Analysis | Management Company.
Situation Analysis | Management Company.
 
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15
 
Instant Digital Issuance: An Overview With Critical First Touch Best Practices
Instant Digital Issuance: An Overview With Critical First Touch Best PracticesInstant Digital Issuance: An Overview With Critical First Touch Best Practices
Instant Digital Issuance: An Overview With Critical First Touch Best Practices
 
Enjoy Night⚡Call Girls Dlf City Phase 4 Gurgaon >༒8448380779 Escort Service
Enjoy Night⚡Call Girls Dlf City Phase 4 Gurgaon >༒8448380779 Escort ServiceEnjoy Night⚡Call Girls Dlf City Phase 4 Gurgaon >༒8448380779 Escort Service
Enjoy Night⚡Call Girls Dlf City Phase 4 Gurgaon >༒8448380779 Escort Service
 
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
 
SEO for Revenue, Grow Your Business, Not Just Your Rankings - Dale Bertrand
SEO for Revenue, Grow Your Business, Not Just Your Rankings - Dale BertrandSEO for Revenue, Grow Your Business, Not Just Your Rankings - Dale Bertrand
SEO for Revenue, Grow Your Business, Not Just Your Rankings - Dale Bertrand
 
Pillar-Based Marketing Master Class - Ryan Brock
Pillar-Based Marketing Master Class - Ryan BrockPillar-Based Marketing Master Class - Ryan Brock
Pillar-Based Marketing Master Class - Ryan Brock
 
Generative AI Content Creation - Andrew Jenkins
Generative AI Content Creation - Andrew JenkinsGenerative AI Content Creation - Andrew Jenkins
Generative AI Content Creation - Andrew Jenkins
 
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort Service
 
Google 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best StrategiesGoogle 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
 
Digital Strategy Master Class - Andrew Rupert
Digital Strategy Master Class - Andrew RupertDigital Strategy Master Class - Andrew Rupert
Digital Strategy Master Class - Andrew Rupert
 
BDSM⚡Call Girls in Sector 128 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 128 Noida Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Sector 128 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 128 Noida Escorts >༒8448380779 Escort Service
 
No Cookies No Problem - Steve Krull, Be Found Online
No Cookies No Problem - Steve Krull, Be Found OnlineNo Cookies No Problem - Steve Krull, Be Found Online
No Cookies No Problem - Steve Krull, Be Found Online
 
Social media, ppt. Features, characteristics
Social media, ppt. Features, characteristicsSocial media, ppt. Features, characteristics
Social media, ppt. Features, characteristics
 
The Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdfThe Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdf
 

Opticon - Scaling The Next Web’s Testing Program to Maximum Impact

  • 1. #opticon2015 Scaling The Next Web’s Testing Program to Maximum Impact Martijn Scheijbeler Lead Growth, SEO & Analytics - The Next Web @martijnsch martijn@thenextweb.com
  • 2. Yearly 5.000 attendees, 75.000 products sold.
 Monthly 6 million unique users, 12 million pageviews. The Next Web: 2006 > 2015
  • 3.
  • 4. What’s happened in the last 12 months? Scaling our testing program
  • 5. That clearly wasn’t good enough 1 test a month :(
  • 6. Currently 5-6 tests a week, how did we get there? 15-20 tests a month :)
  • 7. What did we learn along the way? • Swim lanes, KPIs, Goals. • Segments: traffic sources, etc. • Idea creation: backlog + prioritisation. • Scaling > Analysing results. • Do it live!
  • 8. Unique Users, Page Views, Emails & Revenue. Goals: UU, PV, Emails & $ Related stories, pagination, sharing, email signups. KPI: CTR, Engagement, Funnels
  • 9. New v.s. Returning - Europe v.s. USA - Facebook v.s. SEO - Mobile v.s. Desktop Type, Regions, Source, Device
  • 10. Idea creation v.s. Planning • Different templates + segments. • How much time per test? Run tests for a fixed predefined time. • The maximum of your program. • Take away: create a backlog!
  • 11. The part that we can’t scale yet
  • 13. Well, this is both the most boring and most exciting part! 1-2 main KPIs, 15 secondary KPIs
  • 14. Rails, Google Analytics API
 Built with love by the TNW Marketing team. TNW Analytics
  • 16. Adding a new idea
  • 18. Test
  • 19. Saving 20% of time per test!
  • 20. Take away: Be able to push code!
  • 22. Scaling our testing program to the next level? • Even more templates, across more projects. • 25+ tests a month with current growth of traffic. • Bigger team? We can’t keep up with coding ;-) • More automation? Push test variant to GTM. • Mix it up with personalisation/ behaviour. • Push our experiments to Slack for sharing. • Create a better style guide + Bootstrap guidelines.
  • 23. Summary & Take aways • Know your KPI framework, expectations and limitations. • Know your segments, CR% so you know how to prioritise. • Follow your planning, priorities. • Scale = 40% - 50% automation: analyse fast! • Be able to push code live!
  • 24. #opticon2015 Have fun & Thank you! Martijn Scheijbeler Lead Growth , SEO & Analytics - TNW @martijnsch martijn@thenextweb.com