Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Using Statistical Analysis to boost your website performance

880 vues

Publié le

Statistics Analysis entails segmentation of a complex challenge to smaller sized sections then analyzing the smaller sized sections individually. Once the smaller sized segments are analyzed, it is going to once again be co-related with respect towards the total method so as to resolve the problem.

The ability to measure accurately your marketing campaign’s performance is one of the key advantages of online marketing. By tracking the number of click-throughs and responses, you can track the success of new calls-to-action on your website. You can see almost immediately the return on your expenditure on online marketing in terms of the number of clicks it generates. Wonderful statistical info may also be identified on ranking websites like Alexa. They present you clear depiction of the site statistics and also provide you with information about which of your sub-domains which are fetching you site visitors. Google analytics is a superb service supplied by Google in order to get the facts statistics analysis on your web-sites. There is a wealth of useful software which is available to make statistics analysis on websites. You can purchase such software to help you with the analysis of your website statistics. The software takes the data from your server logs and uses them to analyze your website performance and traffic. The tracking system on the visitors offered by Google analytics is truly useful

Publié dans : Business
  • Soyez le premier à commenter

Using Statistical Analysis to boost your website performance

  1. 1. WELCOME Boost Website Performance withStatistical Design of Experiment 1
  2. 2. Website Optimization Develop designs based on experience  Dated? Slow! Inputs from focus groups  Limited and expensive Optimization based on web analytics  Big step forward  Not granular enough-still making assumptions? What next?  What are elements that go on my web page  How do they interact/affect user behavior  Do these elements have the same impact on users If not which ones lead the pack  What happens if we change one or more of these? Confidential | The next-generation online customer acquisition engine
  3. 3. Example of elements Web page elements  Header  Headline  Sub-headline  Body text  Bullet list, charts, graphs, videos, pictures, callouts, or similar  Promotion  Registration Form, call-to-action  Navigation  Site search  Links on the page  Footer  User reviews Confidential | The next-generation online customer acquisition engine
  4. 4. Web pages are collection of elements: that work together! Location of search barLeft side Model selectornavigation Display pricing Add to cart Display shipping and price Image controls & guarantee video © 2010 Confiden1tial | think innovation
  5. 5. Use Heat Maps To identify elements that are getting visitors attention on web pages
  6. 6. In Page Analytics: More quantitative Identifies elements AND level of attention from visitors
  7. 7. Web page elements: What Next? We identified elements and the level of attention We still don’t know which ones have bigger impact on conversion What happens if we change one or more of them? How do we optimize these for boosting performance?Solution: Statistical Controlled Experiment where variations are present Determine the best performing combination of variations Run a validation test on the recommendation to confirm Confidential | The next-generation online customer acquisition engine
  8. 8. Statistical Controlled Experiment: Reaching back in time Controlled Experiment in 1747 in England  Testing cure for scurvy by James Lind, a surgeon Fisher started with application to agriculture – 1918-1940  Factorial Design and ANOVA Industrial Era: 1950-1970  Application in Chemical and process industries Second Industrial Era: 1970-1990  Wider application in Quality Control to most industries How do we apply this to boost website performance? Confidential | The next-generation online customer acquisition engine
  9. 9. High Level Design of Experiment (DOE) Process Select elements that are important or attract more attention  Elements should be as much independent as possible Variations are created for each element  2/3 apart from the existing (CONTROL) Develop web pages containing combination of variations  ALL possible variations or a SMALLER subset These web pages are run along with Control  Traffic volume for creating statistically significant result is directed to these pages Performance of each web page is measured  Statistical analysis yields which combination of variation is expected to yield the highest results Develop a final page with recommended combination  Run this in parallel with CONTROL to validate results Confidential | The next-generation online customer acquisition engine
  10. 10. Selecting the right Design Factors in choosing elements, variations and design  Independence, traffic volume, traffic sources, time, cost  Experiment Design: Full Factorial (all possible combination) or Partial (selected)Example:A Full Factorial Design with 3 elements with 3 levels of variations each will need 3^3 = 27 web pages to be testedTraditional A/B Testing will also require 27 pagesPartial Factorial Design (say in Taguchi Design) will need a L9 array with 9 web pages only Run duration should be enough to  Generate statistically significant results  Cover cyclical and important variations in environment Confidential | The next-generation online customer acquisition engine
  11. 11. Case Study of an E-Commerce portal Objectives  To increase the Shopping Cart creation Rate Google Analytics data revealed that the Product Details page were dropping visitors Planned DOE on the Product Details Page  Selected 7 elements with 2 variations each  Partial Factorial Design lead to L8 array (8 pages to test) Combinations were run sequentially  Each combination ran for 6 days taking 10%-15% of traffic  The page ran in parallel with CONTROL
  12. 12. Possible List of variables identified1. Add to Cart placement2. Displaying price information3. Members star ratings4. Image and video controls5. Related Products6. Left hand navigation7. Member Actions – Email to friends, Add to wish list, Price Alerts8. Product search barWe finally selected the first 7 variables with one variation eachPartial Factorial design based on Taguchi required 8 pages © 2010 Confiden1tial | think innovation
  13. 13. Example of Page Element Layout Link to Slide 6 TV1 Link to Slide 10 TV4 TV6Link to Slide 6 Link to Slide 9 TV2 13
  14. 14. Experiments Run Experiment Run ID Variable Combinations Name Experiment-1 Test Run-2 C-TV1,C-TV2,C-TV3,C+1-TV4,C+1-TV5,C+1-TV6,C+1-TV7 Experiment-2 Test Run-3 C-TV1,C+1-TV2,C+1-TV3,C-TV4,C-TV5,C+1-TV6,C+1-TV7 Experiment-3 Test Run-4 C-TV1,C+1-TV2,C+1-TV3,C+1-TV4,C+1-TV5,C-TV6,C-TV7 Experiment-4 Test Run-5 C+1-TV1,C-TV2,C+1-TV3,C-TV4,C+1-TV5,C-TV6,C+1-TV7 Experiment-5 Test Run-6 C+1-TV1,C-TV2,C+1-TV3,C+1-TV4,C-TV5,C+1-TV6,C-TV7 Experiment-6 Test Run-7 C+1-TV1,C+1-TV2,C-TV3,C-TV4,C+1-TV5,C+1-TV6,C-TV7 Experiment-7 Test Run-8 C+1-TV1,C+1-TV2,C-TV3,C+1-TV4,C-TV5,C-TV6,C+1-TV7 Taguchi New Test C+1-TV1,C-TV2,C+1-TV3,C+1-TV4,C+1-TV5,C+1-TV6,C+1-TV7Recommendation 14
  15. 15. Experiment-5: C+1-TV1,C-TV2,C+1-TV3,C+1-TV4,C-TV5,C+1-TV6,C-TV7Affected VariablesTV3: Members star ratings onthe Product details pageTV1(position change): - Add to Cart link - Price information display - Free shipping and lowestprice guaranteeTV4: View image and video linkTV6: Left side navigation links
  16. 16. Experiment-4: C+1-TV1,C-TV2,C+1-TV3,C-TV4,C+1-TV5,C-TV6,C+1-TV7Affected VariablesTV3: Members star ratings onthe Product details pageTV1(position change): - Add to Cart link - Price information display - Free shipping and lowestprice guaranteeTV5: Related products – displaylogicTV7: Member control Links
  17. 17. 1. Significance Test Analysis The Significant Test shows that Experiment 5 and Experiment 4 are ahead of the others EXPExp Number Mean. EXP St. Dev. Control Control Z- P-value # of Runs SC Cr. SC Cr. Mean SC Cr. St. Dev. score (accept H0) Rank 1 6 5.06% 0.69% 4.60% 0.80% 1.41 0.0700 4 2 6 4.83% 0.17% 4.68% 0.61% 0.60 0.5600 7 3 6 4.89% 0.42% 4.70% 0.36% 1.29 0.2000 6 4 6 5.27% 0.84% 4.67% 0.24% 6.12 0.0002 2 5 6 5.54% 0.98% 4.67% 0.38% 5.61 0.0002 1 6 6 4.90% 0.22% 4.48% 0.44% 2.34 0.0200 3 7 6 4.79% 0.65% 4.49% 0.45% 1.63 0.1000 5 A higher z-score means that the data is farther away from the population mean (CONTROL) 17
  18. 18. 2. Taguchi Design AnalysisThe Taguchi design analyzed the variables, and for each variable itcompared Control(1) vs. its Variant(2) and calculated Signal to Noiseratio for each experimentMean – The mean gives the average SC creation rateSN Ratio – Signal-to-Noise ratio is a measure of how predictable is the SC creation rate. A high SN ratiomeans that the Standard Deviation for that given EXP is proportionally smaller than the mean. A lowerSN ratio means that data for the SC creation rate is relatively unpredictable.Experiment 5 and Experiment 4 have the highest and the second highest SN ratio,meaning that their SC creation rate is higher, robust and more predictable. 18
  19. 19. 2. Taguchi Design AnalysisImpact of elements & variations: Choice 1 (Control) vs. Choice 2(Variant) for each of the 7 variablesNote that changes in variable 1 pushed the shopping cart creation rate the most whilechanges in the second variable caused the rate to drop 19
  20. 20. 2. Recommendations Based on Taguchi Design AnalysisTaguchi recommended variable combination for themostperformantShopping Cart creation rate as follows Variable Recommendation TV_1 2 TV_2 1 TV_3 2 TV_4 2 TV_5 2 TV_6 2 TV_7 2 NOTE: Taguchi however will not predict how much better than CONTROL the above hypothesized template will perform. We recommend creating a design based on this recommendation and running it for validation. 20
  21. 21. Results SC Creation Rate moved from 4.48% to 5.02% after validation Number of carts created increased from 500 to 560 per day Average Cart value moved from $155 to $169 Revenue increased by 170K a month
  22. 22. Things to keep in mind Variables to be kept as independent from each other as possible  Avoid confounding Keep environmental variables as much constant as possible  Avoid introducing new traffic sources Select more variables if you have traffic  Keep the number of variation of each limited to 2-3 Traffic should be randomly selected  Load balancer should not introduce any bias  Should not mix traffic between different variations and CONTROL
  23. 23. Regalix Online MarketingWhy partner with RegalixIndustry Best Practices & FrameworkRegalix has developed social media framework and best ―The ComplianceOnline case study offers insight into how technologypractices leveraging experience from Fortune 1,000 as well marketers can make a vision for a viable online community — thatas venture backed customers across multiple industry embraces customers, partners, and prospects alike — become real.‖verticals including Hi-tech., Retail, Manufacturing, Education, – Laura Ramos, B2B Analyst, Forrester ResearchClean Tech. and Financial Services.Full-service Execution ―Regalix has social mediaRegalix provides strategy, program management, creative, marketing experience that hastechnology, and operations services. This reduces the needfor engaging and managing multiple service providers. been leveraged on behalf of blue- ―Citibank views Regalix as a chip Fortune 100 firms. For this trusted strategic partner. They reason, I think that you’ll get far have consistently deliveredSuperior Client-service more bang for your buck out of innovative solutions ofRegalix’s client services teamconsists of industry experts and leveraging them as your turn-key exceptional quality over the lastpractitioners that have created significant and successful marketing team than you would couple of years of ourmarketing programs. out of a single FTE. engagement. Their reporting dashboards are Regalix has proven experience in also first-rate. Super stream lined supporting an enterprise of our – super time efficient.” size and diversity, while respecting the stringent quality - Alyssa Rapp, CEO Bottlenotes standards that we have set.‖ 23
  24. 24. Regalix Online MarketingAbout UsFull-Service: Digital Marketing & Technology ServicesCompany; Strategy, Creative, Campaigns, Technology,Communities Collateral & ThoughtTalent: Leadership, Advisory, 175+ Team LeadershipCustomers: Fortune 500 and Venture-Backed ContentFocus: B2B, B2C, C2CVerticals: Retail, Hi-Tech., Finance, HealthcareGlobal: HQ in Silicon Valley, 4 Offices Website, Community Portals,Background: Built on 8+ years of research & Social iPhone, iPad MediaIndustry Recognition: applications Digital Strategy Customer & Lead Prospect Generation Nurturing & Sales 24
  25. 25. Companies that trust us Retail & Financial Film & Professional Healthcare, Education Hi-Tech. & Clean-Tech.,Consumer Services Entertainment Services Bio-Tech. Software Bio-FuelProducts UCLA ICICI BANK 25
  26. 26. Thank You 26
  27. 27. Landing Page Optimization Overview Goal: Increase Lead Conversion Rate (CR)  Use Regalix’s ROITM Intelligent Design of Experiment (DOE)  Create a best page with higher CR for the same online traffic  Create a continuous optimization program for Citibank Activity 1: Plan for DOE  Break the landing pages into individual elements  Create variations on each element for testing  Create DOE test and identify the number of pages/element combinations necessary for testing  Create landing pages with the necessary combinations Activity 2: Conduct DOE  Set-up landing pages on ROI  Run DOE and collect statistically significant data Activity 3: Analyze data and build best page Activity 4: Continuous improvement Confidential | The next-generation online customer acquisition engine
  28. 28. Activity 3: Analyze data and build best page Feed data into statistical system and use Regalix’s proprietary algorithm to calculate the following:  The tested page elements that influence the conversion rate  The specific variations of the elements that worked best  The combination of elements that could provide a significant higher conversion rate Final landing page is built using the best element and their variation New Page is deployed  Data is collected to establish higher CR  Optionally to isolate any market dynamics that is likely to influence the CR: The New Page will be run simultaneously with the Older Page to get a direct measure of improvement New page is ready for wider rollout Confidential | The next-generation online customer acquisition engine