SlideShare une entreprise Scribd logo
1  sur  16
Télécharger pour lire hors ligne
You have NO Idea
what your users want
         And how to learn



   Evan Solomon | Automattic
        @evansolomon
if( time() % 2 )
! $title = ‘Split testing’;
else
! $title = ‘A/B testing’;
setcookie( ‘title’, $title );
echo $title;
A/B testing

1.Try different things
2.Measure their effects
3.Profit!
      Lots of good examples at blog.performable.com
What can it do?

Determine how likely one version
   is to be better than another.

 If you have enough data you can tell how much better, but you need quite a bit
Why do it?

It makes you
  smart(er)!
 Also turns out to be good for getting clients
Start basic


1.Talk to your customers
2.Listen
3.Profit!
             Step 3 is always profit
Dashboard




Check out geckoboard.com
Running tests

We use Optimizely because:
1.Real time results (great for debugging)
2.Tests are written entirely in jQuery
3.Easy to target using WordPress logic
Version A
Version B
What happens?


   B wins and
covers the spread
     Signups increase ~25%
Handling data
  Terms you should get used to:
           “Confidence interval”
            “Significance test”


Check out isvalid.org to calculate these stats
           I made isvalid for myself so it’s a little rough, but very useful
isvalid.org
Keep in mind

•   Question assumptions
•   Unintended (or not) biases
•   Test liberally, it’s nearly free
•   Many (most) tests “fail”
Call for help


I’m working on a .org plugin

     Based on the one we made for WordPress.com
@evansolomon

Contenu connexe

Tendances

I Smell A RAT- Rapid Application Testing
I Smell A RAT- Rapid Application TestingI Smell A RAT- Rapid Application Testing
I Smell A RAT- Rapid Application Testing
Peter Presnell
 
Dr ScriptLove or How I Learned to Stop Worrying and Love JavaScript
Dr ScriptLove or How I Learned to Stop Worrying and Love JavaScriptDr ScriptLove or How I Learned to Stop Worrying and Love JavaScript
Dr ScriptLove or How I Learned to Stop Worrying and Love JavaScript
Frank La Vigne
 
Continuous Deployment
Continuous DeploymentContinuous Deployment
Continuous Deployment
Codeship
 

Tendances (20)

Test automation – the bitter truth
Test automation – the bitter truthTest automation – the bitter truth
Test automation – the bitter truth
 
SXSW 2016 - Everything you think about A/B testing is wrong
SXSW 2016 - Everything you think about A/B testing is wrongSXSW 2016 - Everything you think about A/B testing is wrong
SXSW 2016 - Everything you think about A/B testing is wrong
 
"Worst" practices of software testing
"Worst" practices of software testing"Worst" practices of software testing
"Worst" practices of software testing
 
Hindsight lessons about API testing
Hindsight lessons about API testingHindsight lessons about API testing
Hindsight lessons about API testing
 
Worst practices in software testing by the Testing troll
Worst practices in software testing by the Testing trollWorst practices in software testing by the Testing troll
Worst practices in software testing by the Testing troll
 
5 Essential Tips For Improving Your Website Mockups & Prototypes!
5 Essential Tips For Improving Your Website Mockups & Prototypes!5 Essential Tips For Improving Your Website Mockups & Prototypes!
5 Essential Tips For Improving Your Website Mockups & Prototypes!
 
Check methods params_for_validity
Check methods params_for_validityCheck methods params_for_validity
Check methods params_for_validity
 
Performance testing mistakes newbies make
Performance testing mistakes newbies makePerformance testing mistakes newbies make
Performance testing mistakes newbies make
 
Automation vs. intelligence - "follow me if you want to live"
Automation vs. intelligence - "follow me if you want to live"Automation vs. intelligence - "follow me if you want to live"
Automation vs. intelligence - "follow me if you want to live"
 
Everything you wanted to know about making an R package but were afraid to ask
Everything you wanted to know about making an R package but were afraid to askEverything you wanted to know about making an R package but were afraid to ask
Everything you wanted to know about making an R package but were afraid to ask
 
RailsGirls DC 2012 - Debugging
RailsGirls DC 2012 - DebuggingRailsGirls DC 2012 - Debugging
RailsGirls DC 2012 - Debugging
 
TestWorksConf: Experience exploratory testing
TestWorksConf: Experience exploratory testingTestWorksConf: Experience exploratory testing
TestWorksConf: Experience exploratory testing
 
Four Stages of Automated Testing by Bradley Temple
Four Stages of Automated Testing by Bradley TempleFour Stages of Automated Testing by Bradley Temple
Four Stages of Automated Testing by Bradley Temple
 
EuroSTAR Webinar - Mob testing
EuroSTAR Webinar - Mob testingEuroSTAR Webinar - Mob testing
EuroSTAR Webinar - Mob testing
 
I Smell A RAT- Rapid Application Testing
I Smell A RAT- Rapid Application TestingI Smell A RAT- Rapid Application Testing
I Smell A RAT- Rapid Application Testing
 
Keeping lab notes as a software developer
Keeping lab notes as a software developerKeeping lab notes as a software developer
Keeping lab notes as a software developer
 
The limits of unit testing by Craig Stuntz
The limits of unit testing by Craig StuntzThe limits of unit testing by Craig Stuntz
The limits of unit testing by Craig Stuntz
 
React performance
React performanceReact performance
React performance
 
Dr ScriptLove or How I Learned to Stop Worrying and Love JavaScript
Dr ScriptLove or How I Learned to Stop Worrying and Love JavaScriptDr ScriptLove or How I Learned to Stop Worrying and Love JavaScript
Dr ScriptLove or How I Learned to Stop Worrying and Love JavaScript
 
Continuous Deployment
Continuous DeploymentContinuous Deployment
Continuous Deployment
 

Similaire à You have no idea what your users want - WordCamp PDX

Similaire à You have no idea what your users want - WordCamp PDX (20)

Automated tests
Automated testsAutomated tests
Automated tests
 
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in FlexassertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
 
Creating testing tools to support development
Creating testing tools to support developmentCreating testing tools to support development
Creating testing tools to support development
 
Selenium Users Anonymous
Selenium Users AnonymousSelenium Users Anonymous
Selenium Users Anonymous
 
Unit Testing and Behavior Driven Testing with AngularJS - Jesse Liberty | Fal...
Unit Testing and Behavior Driven Testing with AngularJS - Jesse Liberty | Fal...Unit Testing and Behavior Driven Testing with AngularJS - Jesse Liberty | Fal...
Unit Testing and Behavior Driven Testing with AngularJS - Jesse Liberty | Fal...
 
Things Could Get Worse: Ideas About Regression Testing
Things Could Get Worse: Ideas About Regression TestingThings Could Get Worse: Ideas About Regression Testing
Things Could Get Worse: Ideas About Regression Testing
 
Exploratory Testing Explained
Exploratory Testing ExplainedExploratory Testing Explained
Exploratory Testing Explained
 
Agile testingandautomation
Agile testingandautomationAgile testingandautomation
Agile testingandautomation
 
Getting Started with Selenium
Getting Started with SeleniumGetting Started with Selenium
Getting Started with Selenium
 
Automated Testing with Logic Apps and Specflow
Automated Testing with Logic Apps and SpecflowAutomated Testing with Logic Apps and Specflow
Automated Testing with Logic Apps and Specflow
 
Browser Automated Testing Frameworks - Nightwatch.js
Browser Automated Testing Frameworks - Nightwatch.jsBrowser Automated Testing Frameworks - Nightwatch.js
Browser Automated Testing Frameworks - Nightwatch.js
 
Fantastic Tests - The Crimes of Bad Test Design
Fantastic Tests - The Crimes of Bad Test DesignFantastic Tests - The Crimes of Bad Test Design
Fantastic Tests - The Crimes of Bad Test Design
 
Acceptance And Story Testing Patterns - By Charles Bradley
Acceptance And Story Testing Patterns - By Charles BradleyAcceptance And Story Testing Patterns - By Charles Bradley
Acceptance And Story Testing Patterns - By Charles Bradley
 
Scryent: Plone - Hone Your Test Fu
Scryent: Plone - Hone Your Test FuScryent: Plone - Hone Your Test Fu
Scryent: Plone - Hone Your Test Fu
 
5 Steps to Usability Testing Success with Salesforce and Beyond!
5 Steps to Usability Testing Success with Salesforce and Beyond!5 Steps to Usability Testing Success with Salesforce and Beyond!
5 Steps to Usability Testing Success with Salesforce and Beyond!
 
TDD Best Practices
TDD Best PracticesTDD Best Practices
TDD Best Practices
 
Petri for kyiv.pptx
Petri for kyiv.pptxPetri for kyiv.pptx
Petri for kyiv.pptx
 
Unit testing
Unit testingUnit testing
Unit testing
 
Patrick McKenzie Opticon 2014: Advanced A/B Testing
Patrick McKenzie Opticon 2014: Advanced A/B TestingPatrick McKenzie Opticon 2014: Advanced A/B Testing
Patrick McKenzie Opticon 2014: Advanced A/B Testing
 
Test for Success: 5 Steps to Usability Testing Success
Test for Success: 5 Steps to Usability Testing SuccessTest for Success: 5 Steps to Usability Testing Success
Test for Success: 5 Steps to Usability Testing Success
 

Dernier

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Dernier (20)

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 

You have no idea what your users want - WordCamp PDX

  • 1. You have NO Idea what your users want And how to learn Evan Solomon | Automattic @evansolomon
  • 2. if( time() % 2 ) ! $title = ‘Split testing’; else ! $title = ‘A/B testing’; setcookie( ‘title’, $title ); echo $title;
  • 3. A/B testing 1.Try different things 2.Measure their effects 3.Profit! Lots of good examples at blog.performable.com
  • 4. What can it do? Determine how likely one version is to be better than another. If you have enough data you can tell how much better, but you need quite a bit
  • 5. Why do it? It makes you smart(er)! Also turns out to be good for getting clients
  • 6. Start basic 1.Talk to your customers 2.Listen 3.Profit! Step 3 is always profit
  • 8. Running tests We use Optimizely because: 1.Real time results (great for debugging) 2.Tests are written entirely in jQuery 3.Easy to target using WordPress logic
  • 11. What happens? B wins and covers the spread Signups increase ~25%
  • 12. Handling data Terms you should get used to: “Confidence interval” “Significance test” Check out isvalid.org to calculate these stats I made isvalid for myself so it’s a little rough, but very useful
  • 14. Keep in mind • Question assumptions • Unintended (or not) biases • Test liberally, it’s nearly free • Many (most) tests “fail”
  • 15. Call for help I’m working on a .org plugin Based on the one we made for WordPress.com

Notes de l'éditeur

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n