To have an appreciation for the different product testing methodologies and to understand their impact on the results obtained from the test.

1. © 2014 Ipsos. All rights reserved. Contains Ipsos' Confidential and Proprietary information and
may not be disclosed or reproduced without the prior written consent of Ipsos.
Product Testing:
Methodological Issues & Design Considerations
October 30, 2014

2. 1
Contents
2
What Business Issue to Address
Unbranded vs. Branded Test
Monadic vs. Sequential Monadic Design
Central Location Test vs. In-Home Use Test
2
3
4

3. What Business Issue
to Address
3

4. 4
Fact: Product Testing is critical for all Successful FMCG clients
Consumers need to evaluate the physical product
R&D still needs explicit guidance as to how to develop winning formulations
Clients still need a trusted partner given the associated investment

5. Maximize
Sales & ROI
Launch
Develop & Optimize
Product, Pack, Price
Generate &
Screen Ideas
Identify & Screen
Insights
Develop Brand /
Category Strategy
Define Long-Term
Brand Vision
MID STAGELATE STAGE
POST LAUNCH FUZZY FRONT END
Define Innovation
Platforms
Product
Testing
Lifecycle
Evaluate Concepts
Finalize
Proposition
Monitor
Performance
Build ConceptsExploration
Benchmarking
Cost Savings / QI
The Innovation Process
5

6. IdentifyObjectives
6
Align research objectives not only by stage but by
type of product development

7. Research Goals:
 Explore & prioritize
 Identify next generation product
features & benefits
Exploration
1
EARLY STAGE
Benefits:
Capture consumer input at the earliest stages of development to uncover
critical product features to fill development pipeline
7

8. Benefits:
Working with R&D / Sensory, use efficient designs and Sensory / technical
data to identify drivers of liking to guide next stages of development
Research Goal:
Screen & optimize prototypes
2
Exploration
Guidance
2
EARLY STAGE
8

9. Benefits:
 Fine tuning minor product enhancements
 Disaster Check; Identify “red flags”
 Prepares the product for market with realistic sense of success
9
Focuses on:
 Assessing consumer feedback
for a small set of products
 Confirming acceptance of an
optimized product (from Early
Stage)
 Finding the best product
 Understanding overall fit and /
or shifts in performance
Validation
3

10. Benefits:
 Track Product Health; early warning system for production
 Use variation in production as basis for quality control and product improvement
 Monitor competition, identify points of difference & track internal quality processes
Focuses on:
assessing quality control and in-
market tracking of product quality
over time
Benchmarking
10
4

11. Consumer complaints have spiked &
sales are declining.
How can we pinpoint where / when
the problem is occurring?
How does my leading sku perform
and / or benchmark against the
leading in-market competitor?
We have just launched a new sku,
how can we track performance post
launch to monitor & ensure year 1
success?
What are clients asking
at Benchmarking stage?
11
4

12. Cost Savings / Quality
Improvement (QI)
12
5
Focuses on:
products that are currently in the
market which may not need
enhancements to improve consumer
acceptance but require ingredient or
process modification to extend their
life in the market
Benefits:
 Identifying cost savings opportunities
 An approach to business decision making coupling dollars at risk with
consumer perceptions of preference

13. What are clients asking
at the Cost Savings /
Quality Improvement
stage?
13
5
Can consumers detect differences
when a new ingredient is used in an
existing formula?
How can we increase profitability by
introducing a cost reduced
prototype without causing risk to
the franchise?
Is the new formula an equally
acceptable replacement for the
current product and how can we be
sure that it will be successful in-
market?

14. Setting Action Standard
14
• Is consistent with Brand Performance Standard; e.g., Client’s
internal protocol
• Is agreed PRIOR to test and aligned with client
• Include a benchmark; i.e., current product, main competitor, or
norms (rarely)
• Have a defined measure; e.g., mean scores, Top Box, Top 2 Boxes
• Have a set of defined attributes; e.g., Overall Liking, Purchase
Intent, Uniqueness, product attributes, Preference
• Define Statistical Test and Confidence Level to use
• Define Target group; e.g., at total sample

15. New Product Development
• Mean Score of Priced Purchase Intention of the new product is rated at
least parity (at 90% confidence level) with the competitor
• Mean Scores of Overall Liking, Uniqueness, and attributes “makes me
feel beautiful” and “makes my skin fairer” of the new product are rated
significantly higher (at 95% confidence level) vs. the competitor
Benchmarking
• Mean Score of Overall Liking is rated at least parity (at 90% confidence
level) with the competitor
Cost Savings
• Overall preference of the new prototype should be at least parity (at
90% confidence level) vs. the current product
Examples of Action Standards
15

16. 16
Design Considerations:
Unbranded
vs.
Branded Test
IPSOS
SOS
IPSOS
IPSOS
IPSOSIPSOS
IPSOS
IPSOS
IPSOSIP
IPSOS
IPSOS
IPSOS

17. UNBRANDED
Unbranded product evaluations are
appropriate at early / Guidance stages
of research when the focus is solely
on product formulation acceptance
and / or for screening prototypes in
the absence of brand or positioning.
What’s important ?
Unbrandedvs.Branded
BRANDED
Branded product evaluations are
used at later / Validation stages of
research when the focus is on holistic
understanding of product performance,
in the context of brand equity and /
or new brand / concept positioning.
What’s important ?
Will respondents be made aware of brand(s)?
17

18. Unbranded Evaluations
18
 The evaluation or ratings provided for
an object presented to respondents
when the object is not identified
either through packaging or other
labeling
 The purpose of the blinding or brand
concealment is to remove from
respondent consideration the effect of
branding on the object’s evaluation
 This is an attempt to obtain an
evaluation that focuses on object
characteristics unaffected or unbiased
by the influence of the brand and the
image it conveys
 Early stage concept or idea screening
 A manufacturer is exploring
entering a new category and does
not want their current brand image
to impact perceptions
 Early stages of product development
 Focus on comparing product
formulations
 Screening prototypes
 Tuning formulas
 Identifying product characteristics that
drive key measures
What Why

19. Branded Evaluations
19
 The evaluation or ratings provided for an
object presented to respondents when
the object is identified or branded
 The intent is to allow the branding and
its associated imagery to affect or
influence perceptions of the object
characteristics
 The effects of branding can mask or
obscure the differences between objects
 Specifically, differences that are due to
the influence of physical or sensory
characteristics, positioning and / or
attributes
 Later stage evaluations when the product
already has a full positioning
 Understanding the impact of brand on
product evaluations
 Cost Reduction
 When products cannot be de-branded
 Cost prohibitive to blind the product
 Consider evaluations among current
brand users and non-users to understand
impact of the brand
What Why
IPSOS
SOS
IPSOS
IPSOS
IPSOS
IPSOS
IPSOS
IPSOS
IPSOSIP
IPSOS
IPSOS
IPSOS

20. Can we test these products unbranded?
20
Lifebuoy Bar Soap Roma Kelapa
Dove Bar Soap Oreo
ProChiz

21. 21
Design Considerations:
Monadic
vs.
Sequential Monadic
Design

22. 22
Respondent tries
first product and
evaluates it using
typical rating
scales
Then tries a
second product
and gives direct
comparisons
between products
Testing similar
products which
may have close
monadic ratings.
Preference ratings
can be used as a
“tie-breaker”.
Have only a small
sample of
qualified
respondents or
incidence is low
Early stage
Practical approach
when home use
context may be
less important
Single product
evaluation
represents a more
realistic & natural
environment
Testing a novel or
unfamiliar product
When there are
carryover effects
When To Use
Respondent tries
one product and
evaluates it and
then tries a second
product and
evaluates it
separately without
comparisons
Often CLT
Respondent tries
only one product
and evaluates
Often in home
(IHUT)
What It Is
Protomonadic
Sequential
Monadic
MonadicDesign
Product Testing Designs
22

23. 23
Same types of
questions as
“Monadic” design
for first product
only
Preference
questions on overall
performance and
attributes after
second product trial
Not often
recommended
Some clients want
to use first
position monadic
and have
preference as a tie
breaker
Having same
respondent
evaluate multiple
products may
increase statistical
precision because
there is more
control over
respondent
variability
Each product is
evaluated by an
independent
sample then the
results evaluated
against each other
Design
Characteristics
As per monadic
PLUS
Overall preference
is sometimes
added at end of
second evaluation
“Absolute
measurements”
without any
comparison
references
Hedonic
Intensity
Efficacy
JAR
Type Of
Questions
Protomonadic
Sequential
Monadic
MonadicDesign
Product Testing Designs (Cont.)
23

24. Product Testing Designs: Practical Considerations
24
Objectives Design
Unbranded or
Branded?
Innovation
Exploration
Sequential
monadic
Usually
Unbranded
Guidance
Sequential
monadic
Usually
Unbranded
Validation Monadic
Usually
Branded
Renovation
Benchmarking Monadic
Both
Unbranded
and Branded
Cost Savings /
Quality Improvement
Protomonadic Branded

25. 25
Design Considerations:
Central Location Test
vs.
In-Home Use Test

26. Venue In what environment will product(s) be evaluated?
26
 Used for late stage in some F&B & for
early stage when the goal is to evaluate
in a controlled environment
What’s important ?
CENTRAL LOCATION TEST (CLT) IN-HOME USE TEST (IHUT)
 An IHUT is appropriate at later /
Validation stages of research when
goal is to confirm product acceptance
in a realistic setting
 Should be considered when client is
seeking context around how product
is prepared consumed, etc.
What’s important ?

27. Central Location Test (CLT)
27
 A study conducted to evaluate the use
and performance of an object in a
controlled environment
 Respondents are asked to come to a
centrally located facility where they are
exposed to the object(s) to be tested
 Presentation of objects typically follows
a sequential monadic design
 Respondents will be pre-recruited and
invited to a central location
 Appropriate for early stage research
 When you seek a highly controlled
environment
 Early stage research such as guidance
research, may be more qualitative in
nature, with smaller base sizes
 Prototypes (limited amount of product)
 Sniff Tests
 Respondents sniff the fragrance of a
variety of products
 Products which are not normally used or
prepared in-home
 Restrictions on shipping
What Why

28. In-Home Use Test (IHUT)
28
 A study conducted to evaluate the use
and performance of an object in a setting
more consistent with how the object
might normally be used by consumers
 Appropriate for later stage research
 The data obtained from such an
evaluation are considered to have good
validity given the natural setting in which
the object is used
 Desire to allow respondents to have a
real-life experience with the object
 Packaging research
 Cost Reduction research
 Products which require usage over a
more extended period of time
 Measuring satiety or other characteristics
over time
 Understand When, Where, and How
consumers use a product on different
occasions
What Why

29. CLT vs. IHUT / Pro’s & Con’s
29
Central Location Test In-Home Use Test
Pro Con Pro Con
Use
Appropriate for
prototype testing
where there may be
limited supply of
product
May not reflect how
consumers would
actually use product
Product used in natural
environment;
Consumers control
when, how, and how
much they use
No control over
variability in how
product is used
Experience
Consumers may have
increased awareness of
product differences
Product can be used
repeatedly over time;
Preparation / serving
according to each
individual
Potential liability
issues if problems
arise when product
is used (e.g., illness)
Control
Can be highly
controlled
environment; Can
control product
preparation and
consumption
Consumers may feel
controlled environment
is artificial; May not be
as tightly controlled as
expected
No strict control
over what happens
in-home; Products
can be prepared very
differently
Evaluations
Can probe on a wide
range of product
characteristics; Can
ensure consumers
understand rating
scales
Product characteristics
may impact evaluations
differently than in an
IHUT
Can measure satiety
and consumption rates;
Appropriate when a
consumer needs to
have the products in
their hands (packaging)

30. CLT vs. IHUT: Practical Considerations
30
Go with CLT if ...
 It’s in the early stage where a strict control is required in
product preparation
 It’s about screening many prototypes in the early stage in
Foods & Beverages
 R&D can’t provide enough samples to conduct IHUT
 It’s about a fragrance sniff test
However, keep in mind that ...
 CLT captures the short term effect of the product
 While IHUT captures the long term effect
─ Example: The infamous “New Coke” disaster (April 23, 1985)
Remembering New Coke, The Ultimate Product Introduction
Disaster (http://fw.to/GbO886O) - April 24, 2014

31. To Sum Up, Touch Points to Consider When Designing a Product
Test Project are ...
31
Research Objective
Stage of Development
Benchmarks
Action Standard
Unbranded vs. Branded
Design
Venue for Testing

32. What else to consider in designing a product test project?
32
Sample Definition
• What’s the target market?
• Whom to interview?
Sample Size
• Depends on product tier, based on
financial and strategic importance
• Depends on products availability
• Depends on planned subgroups
analyses

33. Begin Your Quest
with InnoQuest
Ipsos in Indonesia
Graha Arda, 3rd Floor
Jl. H.R. Rasuna Said Kav. B-6
Kuningan
Jakarta 12190
© 2014 PT. Ipsos. All rights reserved. Contains Ipsos' Confidential and Proprietary information
and may not be disclosed or reproduced without the prior written consent of Ipsos.