1. Lecture 10
Observation: chapter 6
Test marketing: chapter 7
1

2. What is observation?
• Observation is the systematic process of
recording the behavioural patterns of people,
objects, and occurrences as they are
witnessed.
– No questioning or communicating with
people
– Example: The Australian Women’s Weekly.
2

3. When is observation scientific?
• Observation becomes a tool for scientific inquiry
when it:
– Serves a formulated research purpose
– Is planned systematically
– Is recorded systematically and related to
general propositions rather than simply
reflecting a set of interesting curiosities.
– Is subjected to checks or controls on validity
and reliability.
3

4. What can be observed?
4

5. What can be observed?
• Used to describe a wide variety of behaviour.
• Attitudes, motivations, and preferences
cannot be observed.
• Observation is also generally of short
duration.
5

6. The nature of observation studies
• Human observation versus mechanical
observation.
• Unobtrusive
• Visible observation versus hidden observation.
• Data do not have distortions, inaccuracies, or
other response biases.
6

7. Observation of human behaviour
• Toy manufacturers use observation because
children cannot express their reactions to products.
– How long does the child’s attention stay with the
product?
– How long until the child puts the toy down?
– Are the child’s peers equally interested in the
toy?
• Observation of nonverbal behaviour.
7

8. Observation of human behaviour
8

9. Direct observation
• A straightforward attempt to observe and
record what naturally occurs.
• Investigator does not create an artificial
situation.
• An observation form keeps observations
consistent.
• Response latency can be observed
– The amount of time it takes to make a
choice between two alternatives.
9

10. Errors associated with direct
observation
• Not error–free because the observer may
record events subjectively.
• Observer bias: a distortion of measurement
resulting from the cognitive behaviour or
actions of the witnessing observer.
• Accuracy may suffer if observer does not
record every detail.
• Interpretation of observation data is another
major source of error.
10

11. Scientifically contrived
observation
• Contrived observation: investigator intervenes
to create an artificial environment in order to
test a hypothesis.
• Contrived situations reduce the research time
spent waiting and observing a situation.
• Mystery shoppers.
11

12. Ethical issues in the observation of
humans
• Hidden observations and respondent’s right to
privacy
• Contrived observation and deception
– Entrapment
• Requires a balance: if researcher obtains
permission to observe, the subject may not act
in a typical manner.
12

13. Observation of physical objects
• Physical–trace evidence is a visible mark of
some past event or occurrence.
– For example, wear on library books to
determine books most read, erosion traces
on floor tiles of museums to determine
most popular exhibits, Campbell’s soup
cans in garbage.
• This method can be accurate.
13

14. Content analysis
• Content analysis is the systematic observation and
quantitative description of the manifest content of
communication.
• Content or messages of advertisements,
newspaper articles, television programs etc.,
– For example, frequency of appearance of
women or minorities in mass media, whether
advertisers use certain themes or appeals more
than others.
14

15. Mechanical observation
• Means of observation is mechanical
– For example, video cameras, traffic
counters
• OzTAM television monitoring system for
estimating national TV audiences.
– Electronic boxes hooked up to television
sets to capture program choices, length of
viewing time, and identity of viewer.
• Monitoring website traffic
• Scanner–based research.
15

16. Measuring physiological reactions
• Four major categories of devices to measure
physiological reactions:
– Eye–tracking monitors: used to observe eye
movements
– Pupilometers: used to observe and record
changes in the diameter of a subject’s pupils
– Psychogalvanometer: used to measure
galvanic skin response
– Voice pitch analysis: records abnormal
frequencies in the voice.
16

17. Chapter 7
Experimental research and test marketing
17

18. The nature of experiments
• Conditions are controlled so that one or more
independent variables can be manipulated to
test a hypothesis about a dependent variable.
• Causal relationship among variables may be
evaluated while eliminating all other
variables.
– For example, influence of brand name
identification and labels on consumers’
taste perception.
18

19. Issues using an experimental
design
• Laboratory versus field experiments
• Threats to internal and external validity that
can influence results
• How can these threats be controlled?
• Choice of experimental design.
19

20. Field and laboratory experiments
• Experiments differ in the degree of control
over the research situation.
• Experimenter either creates an artificial
situation that limits the influence of outside
factors or deliberately manipulates a real–life
situation, which allows the influence of
outside factors but makes it harder to
determine cause–and–effect relationships.
20

21. Field and laboratory experiments
• Laboratory experiment is conducted in a
setting where the researcher has almost
complete control.
– Viewing television commercial then
allowing viewer to purchase in a simulated
store environment.
– Mobile shopping van
– Tachistoscope: controls the amount of time
a subject is exposed to a visual image.
21

22. Field and laboratory experiments
• Field experiments are conducted in a natural
setting where complete control of extraneous
variables is not possible.
– Test markets
• Controlled store test is a hybrid between a
laboratory experiment and test market.
– Test products are sold in selected stores to
actual customers.
22

23. Basic issues in experimental
design
• Manipulation of the independent variable.
• Selection and measurement of the dependent
variable.
• Selection and assignment of subjects.
• Control over extraneous variables.
23

24. Manipulation of the independent
variable
• Independent variable can be manipulated,
changed or altered independently of any
other variable.
• Hypothesised to have the causal influence.
• Experimental treatments.
24

25. Manipulation of the independent
variable
• Example of variations of advertising copy and graphic
designs in marketing experiments.
25

26. Manipulation of the independent
variable
• Experimental group: group of subjects exposed to the
experimental treatment.
• Control group: group of subjects not exposed to the
experimental treatment.
– The two treatment groups are then compared to
determine any causal effects.
• There can be several experimental treatment levels.
• There can be more than one independent variable.
26

27. Selection and measurement of
the dependent variable
• The value of a dependent variable is expected
to be dependent on the experimenter’s
manipulation of the independent variable.
• Selection of dependent variable is crucial.
– Determines what type of answer is given to
the research question.
27

28. Selection and assignment
of test units
• Test units are the subjects or entities whose
responses to experimental treatments are observed
or measured.
• Sample selection error
• Random sampling error
– Repetitions of the basic experiment sometimes
favour one experimental condition and sometimes
the other on a chance basis.
• Randomisation and matching.
28

29. Control over extraneous variables
• Experimenters may strive for constancy of conditions.
• Error due to order of presentation.
• Blinding is used to control subjects’ knowledge of
whether or not they have been given an experimental
treatment.
• Constant experimental error occurs when extraneous
variables are allowed to influence the dependent
variable every time the experiment is repeated.
29

30. Issues of experimental validity
• Internal validity refers to whether the
experimental treatment was the sole cause of
observed changes in the dependent variable.
– If the observed results were influenced or
confounded by extraneous factors, then
the experiment is not internally valid.
30

31. Issues of experimental validity
• External validity is the ability of an experiment
to generalise beyond the experiment data to
other subjects or groups in the population
under study.
– If the experimental situation is artificial and
does not reflect the true setting and
conditions in which the investigated
behaviour takes place, then the experiment
is not externally valid.
31

32. Threats to internal validity
• History: history effect and cohort effect
– For example, competitors change their marketing
strategies during a test marketing experiment, two
subject groups with different histories.
• Maturation: maturation effects, guinea pig effect, and
Hawthorne effect
– For example, day–long experimental subjects may
grow hungry, tired, or bored, thus changing the
result of experiment.
– Example: subject changes behaviour in the
presence of experimenter.
32

33. Threats to internal validity
• Testing: testing effects
– For example, students exposed to the experiment
the first time, may react differently the second
time.
• Instrumentation: instrumentation effect
– Example: change in wording of questions may
cause a change in the results of experiment.
• Selection: selection effect
– Sampling bias that results from differential
selection of respondents for comparison groups.
33

34. Threats to internal validity
• Mortality: mortality effect
– Sample bias that results from the
withdrawal of some subjects from the
experiment before it is completed.
• Demand characteristics
– Experimental design procedures that
unintentionally suggest to subjects about
the experimenter’s hypothesis.
34

35. Threats to external validity
• Student surrogates: use of university students
as experimental subjects.
– Students do not provide accurate
predictions of other populations.
• Extraneous variables may have an impact on
the dependent variable, thereby distorting the
experiment.
– Not always possible to control everything in
marketing experiments.
35

36. Types of experimental designs
• Experimental for one independent variable or
outcome or factorial to consider a number of causal
factors
– Basic experimental designs with one independent
variable and one dependent variable.
– Factorial experimental designs with two or more
independent variables.
• Repeated measures or not
– Subjects exposed to all experimental treatments.
36

37. Basic experimental designs
• A single independent variable is manipulated to
measure its effect on another single dependent
variable.
– Complex or statistical experimental design for two
or more independent variables.
• Symbolism for diagramming experimental designs:
– X: Exposure of a group to experimental treatment.
– O: Observation of dependent variable.
– R: Random assignment of test units.
37

38. Three examples of quasi–
experimental designs
• One shot design: single measure is recorded
after treatment is administered.
• One–group pretest–posttest design:
experimental group is measured before and
after treatment is administered.
• Static group design: after–only design
measuring group exposed to experimental
treatment and control group without exposure
to treatment.
38

39. Three better experimental
designs
• First step of true experimental design is
randomisation of subject assignment.
– Pretest–posttest control group design: both
experimental and control groups are measured
before and after treatment administered on
experimental group.
– Posttest–only control group design: after–only
design measuring both experimental and control
groups.
– Solomon four–group design: combines both
experimental designs.
39

40. Time series designs
• Experiments are conducted over long periods
of time to distinguish temporary and
permanent changes in dependent variables.
– Example: political polls tracking candidates’
popularity.
40

41. Complex experimental designs
• Isolate the effects of confounding extraneous
variables
• Allow for manipulation of more than one
independent variable.
– Completely randomised design,
randomised block design, factorial design,
Latin square design.
41

42. Completely randomised design
• Uses a random process to assign subjects to
treatments to investigate the effects of only
one independent variable.
42

43. Randomised block design
• Identifies and blocks out effects of a single
extraneous variable that might affect the
response of the test units.
43

44. Factorial designs
• Investigates the interaction of two or more
independent variables on a single dependent
variable.
• Main effect: the influence of a single
independent variable on a dependent
variable.
• Interaction effect: the influence on a
dependent variable of combinations of two or
more independent variables.
44

45. Factorial designs
• The number of treatments and the number of
levels of each treatment identify the factorial
design.
– Example: two magazine ads tested on men
and women: 2X2
45

46. Factorial designs
46

47. Latin square design
• Balanced, two–way classification scheme that
attempts to control or block out the effect of
two or more extraneous factors by restricting
randomisation with respect to the row and
column effects.
47

48. Test marketing
• Scientific testing and controlled experimental
procedure that provides an opportunity to measure
sales or profit potential for a new product.
• Test a new marketing plan under realistic marketing
conditions.
– Offers the opportunity to estimate the outcomes of
alternative courses of action.
– Allows management to identify and correct
weaknesses in either the product or its marketing
plan before a national launch.
48

49. Test marketing
• An expensive research procedure.
– Developing local distribution, arranging
media coverage, monitoring sales results
• Many uncertainties and risks even with test
marketing.
• The firm runs the risk of exposing a new
product or its plans to competitors.
• Warranted only if it will save the company
money in the long run.
49

50. How long should a test market last?
• New product volume likely to peak out in 3 to
4 months, suggesting a number of people try
new products, but many do not repeat their
purchases.
– Test markets should be long enough for
consumers to become aware of the
product and to try it more than once.
– A test market that is too short may over–
estimate sales.
50

51. Factors to consider in test market
selection
• Population size
• Demographic composition and lifestyle
considerations
• Competitive situation
• Media coverage and efficiency
• Media isolation
• Self–contained trading area
• Over–used test markets
• Availability of scanner data.
51

52. Estimating sales volume: some
problems
• Over–attention
• Unrealistic store conditions
• Reading the competitive environment
incorrectly
• Incorrect volume forecasts
• Time lapse.
52

53. Projecting test market results
• Consumer surveys
– Measure levels of change in consumer awareness
and attitudes toward the product.
• Straight trend projections
• Ratio of test products sales to total company sales
• Market penetration X repeat–purchase rate
– Repeat–purchase rate obtained from longitudinal
research.
53

54. Standard method versus control
method of test marketing
• Standard method of test marketing has
considerable external validity.
• Control method of test marketing involves a
‘mini–market test’ using forced distribution.
– Retailers are paid for shelf space so that
the test marketer can be guaranteed
distribution.
54

55. Standard method versus control
method of test marketing
• Electronic test markets measures results based on
scanner data combined with broadcasting systems to
experiment with different ad messages.
• Simulated test markets are research laboratories in
which the traditional shopping process is
compressed.
– Virtual–reality simulated test market attempts to
reproduce actual store atmosphere with visually
appealing images on computer screen.
55