Contenu connexe



Product Testing: Methodological Issues & Design Considerations

  1. © 2014 Ipsos. All rights reserved. Contains Ipsos' Confidential and Proprietary information and may not be disclosed or reproduced without the prior written consent of Ipsos. Product Testing: Methodological Issues & Design Considerations October 30, 2014
  2. 1 Contents 2 What Business Issue to Address Unbranded vs. Branded Test Monadic vs. Sequential Monadic Design Central Location Test vs. In-Home Use Test 2 3 4
  3. What Business Issue to Address 3
  4. 4 Fact: Product Testing is critical for all Successful FMCG clients Consumers need to evaluate the physical product R&D still needs explicit guidance as to how to develop winning formulations Clients still need a trusted partner given the associated investment
  5. Maximize Sales & ROI Launch Develop & Optimize Product, Pack, Price Generate & Screen Ideas Identify & Screen Insights Develop Brand / Category Strategy Define Long-Term Brand Vision MID STAGELATE STAGE POST LAUNCH FUZZY FRONT END Define Innovation Platforms Product Testing Lifecycle Evaluate Concepts Finalize Proposition Monitor Performance Build ConceptsExploration Benchmarking Cost Savings / QI The Innovation Process 5
  6. IdentifyObjectives 6 Align research objectives not only by stage but by type of product development
  7. Research Goals:  Explore & prioritize  Identify next generation product features & benefits Exploration 1 EARLY STAGE Benefits: Capture consumer input at the earliest stages of development to uncover critical product features to fill development pipeline 7
  8. Benefits: Working with R&D / Sensory, use efficient designs and Sensory / technical data to identify drivers of liking to guide next stages of development Research Goal: Screen & optimize prototypes 2 Exploration Guidance 2 EARLY STAGE 8
  9. Benefits:  Fine tuning minor product enhancements  Disaster Check; Identify “red flags”  Prepares the product for market with realistic sense of success 9 Focuses on:  Assessing consumer feedback for a small set of products  Confirming acceptance of an optimized product (from Early Stage)  Finding the best product  Understanding overall fit and / or shifts in performance Validation 3
  10. Benefits:  Track Product Health; early warning system for production  Use variation in production as basis for quality control and product improvement  Monitor competition, identify points of difference & track internal quality processes Focuses on: assessing quality control and in- market tracking of product quality over time Benchmarking 10 4
  11. Consumer complaints have spiked & sales are declining. How can we pinpoint where / when the problem is occurring? How does my leading sku perform and / or benchmark against the leading in-market competitor? We have just launched a new sku, how can we track performance post launch to monitor & ensure year 1 success? What are clients asking at Benchmarking stage? 11 4
  12. Cost Savings / Quality Improvement (QI) 12 5 Focuses on: products that are currently in the market which may not need enhancements to improve consumer acceptance but require ingredient or process modification to extend their life in the market Benefits:  Identifying cost savings opportunities  An approach to business decision making coupling dollars at risk with consumer perceptions of preference
  13. What are clients asking at the Cost Savings / Quality Improvement stage? 13 5 Can consumers detect differences when a new ingredient is used in an existing formula? How can we increase profitability by introducing a cost reduced prototype without causing risk to the franchise? Is the new formula an equally acceptable replacement for the current product and how can we be sure that it will be successful in- market?
  14. Setting Action Standard 14 • Is consistent with Brand Performance Standard; e.g., Client’s internal protocol • Is agreed PRIOR to test and aligned with client • Include a benchmark; i.e., current product, main competitor, or norms (rarely) • Have a defined measure; e.g., mean scores, Top Box, Top 2 Boxes • Have a set of defined attributes; e.g., Overall Liking, Purchase Intent, Uniqueness, product attributes, Preference • Define Statistical Test and Confidence Level to use • Define Target group; e.g., at total sample
  15. New Product Development • Mean Score of Priced Purchase Intention of the new product is rated at least parity (at 90% confidence level) with the competitor • Mean Scores of Overall Liking, Uniqueness, and attributes “makes me feel beautiful” and “makes my skin fairer” of the new product are rated significantly higher (at 95% confidence level) vs. the competitor Benchmarking • Mean Score of Overall Liking is rated at least parity (at 90% confidence level) with the competitor Cost Savings • Overall preference of the new prototype should be at least parity (at 90% confidence level) vs. the current product Examples of Action Standards 15
  17. UNBRANDED Unbranded product evaluations are appropriate at early / Guidance stages of research when the focus is solely on product formulation acceptance and / or for screening prototypes in the absence of brand or positioning. What’s important ? Unbrandedvs.Branded BRANDED Branded product evaluations are used at later / Validation stages of research when the focus is on holistic understanding of product performance, in the context of brand equity and / or new brand / concept positioning. What’s important ? Will respondents be made aware of brand(s)? 17
  18. Unbranded Evaluations 18  The evaluation or ratings provided for an object presented to respondents when the object is not identified either through packaging or other labeling  The purpose of the blinding or brand concealment is to remove from respondent consideration the effect of branding on the object’s evaluation  This is an attempt to obtain an evaluation that focuses on object characteristics unaffected or unbiased by the influence of the brand and the image it conveys  Early stage concept or idea screening  A manufacturer is exploring entering a new category and does not want their current brand image to impact perceptions  Early stages of product development  Focus on comparing product formulations  Screening prototypes  Tuning formulas  Identifying product characteristics that drive key measures What Why
  19. Branded Evaluations 19  The evaluation or ratings provided for an object presented to respondents when the object is identified or branded  The intent is to allow the branding and its associated imagery to affect or influence perceptions of the object characteristics  The effects of branding can mask or obscure the differences between objects  Specifically, differences that are due to the influence of physical or sensory characteristics, positioning and / or attributes  Later stage evaluations when the product already has a full positioning  Understanding the impact of brand on product evaluations  Cost Reduction  When products cannot be de-branded  Cost prohibitive to blind the product  Consider evaluations among current brand users and non-users to understand impact of the brand What Why IPSOS SOS IPSOS IPSOS IPSOS IPSOS IPSOS IPSOS IPSOSIP IPSOS IPSOS IPSOS
  20. Can we test these products unbranded? 20 Lifebuoy Bar Soap Roma Kelapa Dove Bar Soap Oreo ProChiz
  21. 21 Design Considerations: Monadic vs. Sequential Monadic Design
  22. 22 Respondent tries first product and evaluates it using typical rating scales Then tries a second product and gives direct comparisons between products Testing similar products which may have close monadic ratings. Preference ratings can be used as a “tie-breaker”. Have only a small sample of qualified respondents or incidence is low Early stage Practical approach when home use context may be less important Single product evaluation represents a more realistic & natural environment Testing a novel or unfamiliar product When there are carryover effects When To Use Respondent tries one product and evaluates it and then tries a second product and evaluates it separately without comparisons Often CLT Respondent tries only one product and evaluates Often in home (IHUT) What It Is Protomonadic Sequential Monadic MonadicDesign Product Testing Designs 22
  23. 23 Same types of questions as “Monadic” design for first product only Preference questions on overall performance and attributes after second product trial Not often recommended Some clients want to use first position monadic and have preference as a tie breaker Having same respondent evaluate multiple products may increase statistical precision because there is more control over respondent variability Each product is evaluated by an independent sample then the results evaluated against each other Design Characteristics As per monadic PLUS Overall preference is sometimes added at end of second evaluation “Absolute measurements” without any comparison references Hedonic Intensity Efficacy JAR Type Of Questions Protomonadic Sequential Monadic MonadicDesign Product Testing Designs (Cont.) 23
  24. Product Testing Designs: Practical Considerations 24 Objectives Design Unbranded or Branded? Innovation Exploration Sequential monadic Usually Unbranded Guidance Sequential monadic Usually Unbranded Validation Monadic Usually Branded Renovation Benchmarking Monadic Both Unbranded and Branded Cost Savings / Quality Improvement Protomonadic Branded
  25. 25 Design Considerations: Central Location Test vs. In-Home Use Test
  26. Venue In what environment will product(s) be evaluated? 26  Used for late stage in some F&B & for early stage when the goal is to evaluate in a controlled environment What’s important ? CENTRAL LOCATION TEST (CLT) IN-HOME USE TEST (IHUT)  An IHUT is appropriate at later / Validation stages of research when goal is to confirm product acceptance in a realistic setting  Should be considered when client is seeking context around how product is prepared consumed, etc. What’s important ?
  27. Central Location Test (CLT) 27  A study conducted to evaluate the use and performance of an object in a controlled environment  Respondents are asked to come to a centrally located facility where they are exposed to the object(s) to be tested  Presentation of objects typically follows a sequential monadic design  Respondents will be pre-recruited and invited to a central location  Appropriate for early stage research  When you seek a highly controlled environment  Early stage research such as guidance research, may be more qualitative in nature, with smaller base sizes  Prototypes (limited amount of product)  Sniff Tests  Respondents sniff the fragrance of a variety of products  Products which are not normally used or prepared in-home  Restrictions on shipping What Why
  28. In-Home Use Test (IHUT) 28  A study conducted to evaluate the use and performance of an object in a setting more consistent with how the object might normally be used by consumers  Appropriate for later stage research  The data obtained from such an evaluation are considered to have good validity given the natural setting in which the object is used  Desire to allow respondents to have a real-life experience with the object  Packaging research  Cost Reduction research  Products which require usage over a more extended period of time  Measuring satiety or other characteristics over time  Understand When, Where, and How consumers use a product on different occasions What Why
  29. CLT vs. IHUT / Pro’s & Con’s 29 Central Location Test In-Home Use Test Pro Con Pro Con Use Appropriate for prototype testing where there may be limited supply of product May not reflect how consumers would actually use product Product used in natural environment; Consumers control when, how, and how much they use No control over variability in how product is used Experience Consumers may have increased awareness of product differences Product can be used repeatedly over time; Preparation / serving according to each individual Potential liability issues if problems arise when product is used (e.g., illness) Control Can be highly controlled environment; Can control product preparation and consumption Consumers may feel controlled environment is artificial; May not be as tightly controlled as expected No strict control over what happens in-home; Products can be prepared very differently Evaluations Can probe on a wide range of product characteristics; Can ensure consumers understand rating scales Product characteristics may impact evaluations differently than in an IHUT Can measure satiety and consumption rates; Appropriate when a consumer needs to have the products in their hands (packaging)
  30. CLT vs. IHUT: Practical Considerations 30 Go with CLT if ...  It’s in the early stage where a strict control is required in product preparation  It’s about screening many prototypes in the early stage in Foods & Beverages  R&D can’t provide enough samples to conduct IHUT  It’s about a fragrance sniff test However, keep in mind that ...  CLT captures the short term effect of the product  While IHUT captures the long term effect ─ Example: The infamous “New Coke” disaster (April 23, 1985) Remembering New Coke, The Ultimate Product Introduction Disaster ( - April 24, 2014
  31. To Sum Up, Touch Points to Consider When Designing a Product Test Project are ... 31 Research Objective Stage of Development Benchmarks Action Standard Unbranded vs. Branded Design Venue for Testing
  32. What else to consider in designing a product test project? 32 Sample Definition • What’s the target market? • Whom to interview? Sample Size • Depends on product tier, based on financial and strategic importance • Depends on products availability • Depends on planned subgroups analyses
  33. Begin Your Quest with InnoQuest Ipsos in Indonesia Graha Arda, 3rd Floor Jl. H.R. Rasuna Said Kav. B-6 Kuningan Jakarta 12190 © 2014 PT. Ipsos. All rights reserved. Contains Ipsos' Confidential and Proprietary information and may not be disclosed or reproduced without the prior written consent of Ipsos.

Notes de l'éditeur

  1. The FFE is at the foundation of any NPD process.