SlideShare une entreprise Scribd logo
1  sur  15
A/B Testing
8 Tests from SoftVu and the Industry
Why Test Campaigns?

Compare the effectiveness of 2 or
more campaigns in order to
discover which has better response
or conversion rate
Visitors randomly distributed
                                                 SoftVu’s
              Equal Distribution
                                                 A/B Testing Process
                                                 1. Gather Insights
                                                 2. Predict how to improve
                                                 3. Confirm theory (A/B Test)
                                                 4. Develop new theory

          A                                  B   5. Test Again




 28%                                   43%


Version B is better than version A
SoftVu’s Email Campaign “DNA”
▪ Sender          ▪ Signature Line
▪ Subject line    ▪ Call to Action
▪ Template        ▪ Distribution Rules
▪ Body Text
SoftVu’s Landing Page “DNA”
▪ Color & Images      ▪ Number of Links
▪ Video               ▪ Keywords
▪ Responsive Design   ▪ Personalization
▪ Spokesperson vs.    ▪ Masculine vs. Feminine
  Executive
Test Results
SoftVu Email Campaigns
Call to Action
Campaign: New Lead, Day 0                                                 Test Goal: Increase click-through rate
  Purpose: Company Introduction                                             Results: 23% Increase
    Sends: 13,000 – round-robin                                             Winner: Contender



     LOGO                                                                       LOGO
                 Hi [fname],                                                               Hi [fname],

     View Link   I just received your loan inquiry and noticed that you        View Info   I just received your loan inquiry and noticed that you
                 may be eligible for a VA loan. [Company] is a veteran                     may be eligible for a VA loan. [Company] is a veteran
                 owned company, so we take pride in providing top-                         owned company, so we take pride in providing top-
                 notch service to those who served our country.                            notch service to those who served our country.

                 Click the VIEW LINK button to learn the answers to                        Click the VIEW LINK button to learn the answers to
                 some of the most common questions we get from our                         some of the most common questions we get from our
                 veteran customers.                                                        veteran customers.


   Champion                                                                  Contender
Short versus Long Copy
Campaign: Aged - Day 45                                                        Test Goal: Increase click-through rate
  Purpose: Update Status                                                         Results: 88% Increase
    Sends: 5,000 – round-robin                                                   Winner: Version A


     Hi [fname],                                                                    Hi [fname],

     If you’re ready to discuss your financial goals, compare offers, or get        All else being just about equal, what do you expect from a mortgage
     started with your home loan, please click the UPDATE STATUS button.            lender?
     It only takes a minute and will help me to better serve your needs.
                                                                                    It’s the things you don’t expect that sets us apart:
     Best Regards
                                                                                       1) Our loan officers actually answer the phones live and in-person
                                                                                      95% of the time (yes, we track it).
                                                                                      2) On average, we close a week or two faster than our competition
                                                                                      3) Our loan officers have more experience and provide more
                                                                                      personalized service with fewer hassles than our competition.
                                                                                      4) We lend our own money and make our own lending decisions—
                                                                                      so we can help where other can’t.

                                                                                     Let me know if you’re ready to discuss your financial goals, compare
                                                                                     offers, or get started with your home loan by clicking the UPDATE
Subject Line
Campaign: Veterans Day Promo     Test Goal: Increase Open Rate
  Purpose: Thank Veterans          Results: 16% Increase
    Sends: 4,000 – round-robin     Winner: Version B


                                      Version A: [Client] Thanks You For Your Service
      LOGO
                                      Version B: Honoring All Who Serve

                                       The click through rate of Version
                                       B was also 83% greater, and the
                                       click-to-open rate 56% higher
                                       than Version A
Multivariate test
Campaign: Working Day 14                                         Test Goal: Increase click-through rate
  Purpose: Educational, loan points                                Results: 500% Increase
    Sends: 35,000 – round-robin                                    Winner: Version A

                                                           A                                                                B
     Tim to lock-in is Now                                         [Client Name}: A Few Points
    Sent: Wednesday, October 24, 2012 5:53PM                      Sent: Wednesday, October 24, 2012 5:55PM
     To: Jeff D                                                     To: Jeff D



          LOGO                                                          LOGO

    VIEW RATES   Jeff,                                            LEARN MORE   Jeff,

                 With today’s rate fluctuations, we are urging                 Do you know how loan points may impact your
                 our customers to lock NOW, while rates are at                 monthly payment? Or, if it makes sense to pay
                 the lowest!                                                   points on your upcoming refinance?

                 Click VIEW RATES to see current rates.                        Click on the LEARN MORE button to read more
                                                                               about origination points, the benefits, and much
                 Call me today to lock in!                                     more.

                                                                               Thanks,
Test Results
Great Examples from the Industry
Content Impact on Conversion Rate




                    108% Improvement
Increase Sign Ups




                    125%
Showing Price




 100% increase in leads
The Power of One Word




Control




                  161%
Increase in click-through rate
                                  Variation

Contenu connexe

Similaire à Soft vu a:b test report

Nsf online lecture 5 customer relationships
Nsf online lecture 5 customer relationshipsNsf online lecture 5 customer relationships
Nsf online lecture 5 customer relationships
Stanford University
 
Embrace Home Loans
Embrace Home LoansEmbrace Home Loans
Embrace Home Loans
Jenn Howard
 
Embrace Home Loans
Embrace Home LoansEmbrace Home Loans
Embrace Home Loans
kennykasnett
 
Embrace Home Loans
Embrace Home LoansEmbrace Home Loans
Embrace Home Loans
Cary Reines
 
Company Overview
Company OverviewCompany Overview
Company Overview
Jim Lavan
 
Guaranteed Rate 2015
Guaranteed Rate 2015Guaranteed Rate 2015
Guaranteed Rate 2015
Rebecca Domke
 
5 essential google analytics tools for the data driven marketer
5 essential google analytics tools for the data driven marketer5 essential google analytics tools for the data driven marketer
5 essential google analytics tools for the data driven marketer
DemandWave
 
ME - Lead nurturing tested
ME - Lead nurturing testedME - Lead nurturing tested
ME - Lead nurturing tested
Định Lê
 

Similaire à Soft vu a:b test report (20)

Investment opportunity
Investment opportunityInvestment opportunity
Investment opportunity
 
Learn How To Maximize Todays Refi Opportunity - Webinar
Learn How To Maximize Todays Refi Opportunity - WebinarLearn How To Maximize Todays Refi Opportunity - Webinar
Learn How To Maximize Todays Refi Opportunity - Webinar
 
Nsf online lecture 5 customer relationships
Nsf online lecture 5 customer relationshipsNsf online lecture 5 customer relationships
Nsf online lecture 5 customer relationships
 
How to Achieve Social Media ROI in the Hotel Industry: Turning Loyal Customer...
How to Achieve Social Media ROI in the Hotel Industry: Turning Loyal Customer...How to Achieve Social Media ROI in the Hotel Industry: Turning Loyal Customer...
How to Achieve Social Media ROI in the Hotel Industry: Turning Loyal Customer...
 
Convince & Convert Media Kit
Convince & Convert Media KitConvince & Convert Media Kit
Convince & Convert Media Kit
 
Presentation Ntp 04 Aldridge Pt 1
Presentation Ntp 04 Aldridge Pt 1Presentation Ntp 04 Aldridge Pt 1
Presentation Ntp 04 Aldridge Pt 1
 
12 meo tang ty le chuyen doi tren trang landingpage
12 meo tang ty le chuyen doi tren trang landingpage12 meo tang ty le chuyen doi tren trang landingpage
12 meo tang ty le chuyen doi tren trang landingpage
 
Beyond Search: Leveraging Ads On Facebook, Twitter and LinkedIn #SMX
Beyond Search: Leveraging Ads On Facebook, Twitter and LinkedIn #SMXBeyond Search: Leveraging Ads On Facebook, Twitter and LinkedIn #SMX
Beyond Search: Leveraging Ads On Facebook, Twitter and LinkedIn #SMX
 
Embrace Home Loans
Embrace Home LoansEmbrace Home Loans
Embrace Home Loans
 
Embrace Home Loans
Embrace Home LoansEmbrace Home Loans
Embrace Home Loans
 
Embrace Home Loans
Embrace Home LoansEmbrace Home Loans
Embrace Home Loans
 
Embrace Home Loans
Embrace Home LoansEmbrace Home Loans
Embrace Home Loans
 
Mortgage Products
Mortgage Products Mortgage Products
Mortgage Products
 
Company Overview
Company OverviewCompany Overview
Company Overview
 
Guaranteed Rate 2015
Guaranteed Rate 2015Guaranteed Rate 2015
Guaranteed Rate 2015
 
Solavei info
Solavei infoSolavei info
Solavei info
 
Online Marketing from the Fringes - DreamForce
Online Marketing from the Fringes - DreamForceOnline Marketing from the Fringes - DreamForce
Online Marketing from the Fringes - DreamForce
 
5 essential google analytics tools for the data driven marketer
5 essential google analytics tools for the data driven marketer5 essential google analytics tools for the data driven marketer
5 essential google analytics tools for the data driven marketer
 
New Era Prospects
New Era ProspectsNew Era Prospects
New Era Prospects
 
ME - Lead nurturing tested
ME - Lead nurturing testedME - Lead nurturing tested
ME - Lead nurturing tested
 

Dernier

How to Build a Simple Shopify Website
How to Build a Simple Shopify WebsiteHow to Build a Simple Shopify Website
How to Build a Simple Shopify Website
mark11275
 
怎样办理莫纳什大学毕业证(Monash毕业证书)成绩单留信认证
怎样办理莫纳什大学毕业证(Monash毕业证书)成绩单留信认证怎样办理莫纳什大学毕业证(Monash毕业证书)成绩单留信认证
怎样办理莫纳什大学毕业证(Monash毕业证书)成绩单留信认证
ehyxf
 
Top profile Call Girls In Sonipat [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In Sonipat [ 7014168258 ] Call Me For Genuine Models W...Top profile Call Girls In Sonipat [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In Sonipat [ 7014168258 ] Call Me For Genuine Models W...
nirzagarg
 
Abortion pills in Kuwait 🚚+966505195917 but home delivery available in Kuwait...
Abortion pills in Kuwait 🚚+966505195917 but home delivery available in Kuwait...Abortion pills in Kuwait 🚚+966505195917 but home delivery available in Kuwait...
Abortion pills in Kuwait 🚚+966505195917 but home delivery available in Kuwait...
drmarathore
 
Q4-Trends-Networks-Module-3.pdfqquater days sheets123456789
Q4-Trends-Networks-Module-3.pdfqquater days sheets123456789Q4-Trends-Networks-Module-3.pdfqquater days sheets123456789
Q4-Trends-Networks-Module-3.pdfqquater days sheets123456789
CristineGraceAcuyan
 
Top profile Call Girls In Meerut [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Meerut [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Meerut [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Meerut [ 7014168258 ] Call Me For Genuine Models We...
gajnagarg
 
怎样办理巴斯大学毕业证(Bath毕业证书)成绩单留信认证
怎样办理巴斯大学毕业证(Bath毕业证书)成绩单留信认证怎样办理巴斯大学毕业证(Bath毕业证书)成绩单留信认证
怎样办理巴斯大学毕业证(Bath毕业证书)成绩单留信认证
eeanqy
 
一比一定(购)滑铁卢大学毕业证(UW毕业证)成绩单学位证
一比一定(购)滑铁卢大学毕业证(UW毕业证)成绩单学位证一比一定(购)滑铁卢大学毕业证(UW毕业证)成绩单学位证
一比一定(购)滑铁卢大学毕业证(UW毕业证)成绩单学位证
wpkuukw
 
怎样办理伦敦国王学院毕业证(KCL毕业证书)成绩单留信认证
怎样办理伦敦国王学院毕业证(KCL毕业证书)成绩单留信认证怎样办理伦敦国王学院毕业证(KCL毕业证书)成绩单留信认证
怎样办理伦敦国王学院毕业证(KCL毕业证书)成绩单留信认证
eeanqy
 
Design-System - FinTech - Isadora Agency
Design-System - FinTech - Isadora AgencyDesign-System - FinTech - Isadora Agency
Design-System - FinTech - Isadora Agency
Isadora Agency
 

Dernier (20)

Hackathon evaluation template_latest_uploadpdf
Hackathon evaluation template_latest_uploadpdfHackathon evaluation template_latest_uploadpdf
Hackathon evaluation template_latest_uploadpdf
 
Furniture & Joinery Details_Designs.pptx
Furniture & Joinery Details_Designs.pptxFurniture & Joinery Details_Designs.pptx
Furniture & Joinery Details_Designs.pptx
 
How to Build a Simple Shopify Website
How to Build a Simple Shopify WebsiteHow to Build a Simple Shopify Website
How to Build a Simple Shopify Website
 
Abortion pills in Riyadh +966572737505 <> buy cytotec <> unwanted kit Saudi A...
Abortion pills in Riyadh +966572737505 <> buy cytotec <> unwanted kit Saudi A...Abortion pills in Riyadh +966572737505 <> buy cytotec <> unwanted kit Saudi A...
Abortion pills in Riyadh +966572737505 <> buy cytotec <> unwanted kit Saudi A...
 
NO1 Top Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakist...
NO1 Top Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakist...NO1 Top Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakist...
NO1 Top Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakist...
 
怎样办理莫纳什大学毕业证(Monash毕业证书)成绩单留信认证
怎样办理莫纳什大学毕业证(Monash毕业证书)成绩单留信认证怎样办理莫纳什大学毕业证(Monash毕业证书)成绩单留信认证
怎样办理莫纳什大学毕业证(Monash毕业证书)成绩单留信认证
 
Top profile Call Girls In Sonipat [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In Sonipat [ 7014168258 ] Call Me For Genuine Models W...Top profile Call Girls In Sonipat [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In Sonipat [ 7014168258 ] Call Me For Genuine Models W...
 
The hottest UI and UX Design Trends 2024
The hottest UI and UX Design Trends 2024The hottest UI and UX Design Trends 2024
The hottest UI and UX Design Trends 2024
 
Abortion pills in Kuwait 🚚+966505195917 but home delivery available in Kuwait...
Abortion pills in Kuwait 🚚+966505195917 but home delivery available in Kuwait...Abortion pills in Kuwait 🚚+966505195917 but home delivery available in Kuwait...
Abortion pills in Kuwait 🚚+966505195917 but home delivery available in Kuwait...
 
Just Call Vip call girls Fatehpur Escorts ☎️8617370543 Two shot with one girl...
Just Call Vip call girls Fatehpur Escorts ☎️8617370543 Two shot with one girl...Just Call Vip call girls Fatehpur Escorts ☎️8617370543 Two shot with one girl...
Just Call Vip call girls Fatehpur Escorts ☎️8617370543 Two shot with one girl...
 
LANDSCAPE ARCHITECTURE PORTFOLIO - MAREK MITACEK
LANDSCAPE ARCHITECTURE PORTFOLIO - MAREK MITACEKLANDSCAPE ARCHITECTURE PORTFOLIO - MAREK MITACEK
LANDSCAPE ARCHITECTURE PORTFOLIO - MAREK MITACEK
 
Call Girls Jalaun Just Call 8617370543 Top Class Call Girl Service Available
Call Girls Jalaun Just Call 8617370543 Top Class Call Girl Service AvailableCall Girls Jalaun Just Call 8617370543 Top Class Call Girl Service Available
Call Girls Jalaun Just Call 8617370543 Top Class Call Girl Service Available
 
Q4-Trends-Networks-Module-3.pdfqquater days sheets123456789
Q4-Trends-Networks-Module-3.pdfqquater days sheets123456789Q4-Trends-Networks-Module-3.pdfqquater days sheets123456789
Q4-Trends-Networks-Module-3.pdfqquater days sheets123456789
 
Top profile Call Girls In Meerut [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Meerut [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Meerut [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Meerut [ 7014168258 ] Call Me For Genuine Models We...
 
怎样办理巴斯大学毕业证(Bath毕业证书)成绩单留信认证
怎样办理巴斯大学毕业证(Bath毕业证书)成绩单留信认证怎样办理巴斯大学毕业证(Bath毕业证书)成绩单留信认证
怎样办理巴斯大学毕业证(Bath毕业证书)成绩单留信认证
 
How to Create a Productive Workspace Trends and Tips.pdf
How to Create a Productive Workspace Trends and Tips.pdfHow to Create a Productive Workspace Trends and Tips.pdf
How to Create a Productive Workspace Trends and Tips.pdf
 
一比一定(购)滑铁卢大学毕业证(UW毕业证)成绩单学位证
一比一定(购)滑铁卢大学毕业证(UW毕业证)成绩单学位证一比一定(购)滑铁卢大学毕业证(UW毕业证)成绩单学位证
一比一定(购)滑铁卢大学毕业证(UW毕业证)成绩单学位证
 
怎样办理伦敦国王学院毕业证(KCL毕业证书)成绩单留信认证
怎样办理伦敦国王学院毕业证(KCL毕业证书)成绩单留信认证怎样办理伦敦国王学院毕业证(KCL毕业证书)成绩单留信认证
怎样办理伦敦国王学院毕业证(KCL毕业证书)成绩单留信认证
 
Design-System - FinTech - Isadora Agency
Design-System - FinTech - Isadora AgencyDesign-System - FinTech - Isadora Agency
Design-System - FinTech - Isadora Agency
 
Q4-W4-SCIENCE-5 power point presentation
Q4-W4-SCIENCE-5 power point presentationQ4-W4-SCIENCE-5 power point presentation
Q4-W4-SCIENCE-5 power point presentation
 

Soft vu a:b test report

  • 1. A/B Testing 8 Tests from SoftVu and the Industry
  • 2. Why Test Campaigns? Compare the effectiveness of 2 or more campaigns in order to discover which has better response or conversion rate
  • 3. Visitors randomly distributed SoftVu’s Equal Distribution A/B Testing Process 1. Gather Insights 2. Predict how to improve 3. Confirm theory (A/B Test) 4. Develop new theory A B 5. Test Again 28% 43% Version B is better than version A
  • 4. SoftVu’s Email Campaign “DNA” ▪ Sender ▪ Signature Line ▪ Subject line ▪ Call to Action ▪ Template ▪ Distribution Rules ▪ Body Text
  • 5. SoftVu’s Landing Page “DNA” ▪ Color & Images ▪ Number of Links ▪ Video ▪ Keywords ▪ Responsive Design ▪ Personalization ▪ Spokesperson vs. ▪ Masculine vs. Feminine Executive
  • 7. Call to Action Campaign: New Lead, Day 0 Test Goal: Increase click-through rate Purpose: Company Introduction Results: 23% Increase Sends: 13,000 – round-robin Winner: Contender LOGO LOGO Hi [fname], Hi [fname], View Link I just received your loan inquiry and noticed that you View Info I just received your loan inquiry and noticed that you may be eligible for a VA loan. [Company] is a veteran may be eligible for a VA loan. [Company] is a veteran owned company, so we take pride in providing top- owned company, so we take pride in providing top- notch service to those who served our country. notch service to those who served our country. Click the VIEW LINK button to learn the answers to Click the VIEW LINK button to learn the answers to some of the most common questions we get from our some of the most common questions we get from our veteran customers. veteran customers. Champion Contender
  • 8. Short versus Long Copy Campaign: Aged - Day 45 Test Goal: Increase click-through rate Purpose: Update Status Results: 88% Increase Sends: 5,000 – round-robin Winner: Version A Hi [fname], Hi [fname], If you’re ready to discuss your financial goals, compare offers, or get All else being just about equal, what do you expect from a mortgage started with your home loan, please click the UPDATE STATUS button. lender? It only takes a minute and will help me to better serve your needs. It’s the things you don’t expect that sets us apart: Best Regards 1) Our loan officers actually answer the phones live and in-person 95% of the time (yes, we track it). 2) On average, we close a week or two faster than our competition 3) Our loan officers have more experience and provide more personalized service with fewer hassles than our competition. 4) We lend our own money and make our own lending decisions— so we can help where other can’t. Let me know if you’re ready to discuss your financial goals, compare offers, or get started with your home loan by clicking the UPDATE
  • 9. Subject Line Campaign: Veterans Day Promo Test Goal: Increase Open Rate Purpose: Thank Veterans Results: 16% Increase Sends: 4,000 – round-robin Winner: Version B Version A: [Client] Thanks You For Your Service LOGO Version B: Honoring All Who Serve The click through rate of Version B was also 83% greater, and the click-to-open rate 56% higher than Version A
  • 10. Multivariate test Campaign: Working Day 14 Test Goal: Increase click-through rate Purpose: Educational, loan points Results: 500% Increase Sends: 35,000 – round-robin Winner: Version A A B Tim to lock-in is Now [Client Name}: A Few Points Sent: Wednesday, October 24, 2012 5:53PM Sent: Wednesday, October 24, 2012 5:55PM To: Jeff D To: Jeff D LOGO LOGO VIEW RATES Jeff, LEARN MORE Jeff, With today’s rate fluctuations, we are urging Do you know how loan points may impact your our customers to lock NOW, while rates are at monthly payment? Or, if it makes sense to pay the lowest! points on your upcoming refinance? Click VIEW RATES to see current rates. Click on the LEARN MORE button to read more about origination points, the benefits, and much Call me today to lock in! more. Thanks,
  • 11. Test Results Great Examples from the Industry
  • 12. Content Impact on Conversion Rate 108% Improvement
  • 14. Showing Price 100% increase in leads
  • 15. The Power of One Word Control 161% Increase in click-through rate Variation

Notes de l'éditeur

  1. Planned to be 10 tests, but our internal split test proved that people tend to nod off after 8. So, we went with 8!