Here we discuss the challenges of convening a sensory panel in an industrial setting, options for improving sensory panel resolution and an opportunity to develop new ways of handling sensory data.

1. Yasigiworld Ltd
www.yasigiworld.com
Paul Hughes
Sensory panels
Set-up, avoiding bias and enhancing the
quality of sensory panel data

2. www.yasigiworld.com
Introduction
• Sensory data is essential to our business
– Final decisions on product release
– Evolving new product concepts
– Competitor analysis
• Output requires human intervention
• Data (good and bad!) is always forthcoming
Reliable data of required quality enhances
competitiveness and facilitates brand management

3. www.yasigiworld.com
• Challenges
– Convening and maintaining panels
– Use of scales and (in)appropriate data handling
– Prediction of sensory qualities from analytical
data
– Integrating sensory information - holistic
Introduction
Want to address scaling issues, but first, to focus
on the core of sensory analysis: the panel

4. www.yasigiworld.com
Scope
• Panel pool size paradox
• Correcting for assessors
• Towards predicting sensory performance
from analysis
– Magnitude estimation
– The Sensory Unit
• Addressing the holistic challenge

5. www.yasigiworld.com
Panel pool size paradox
• For a panel of a given size, what size of panel pool
do you need?
• Clearly depends on panellist availability
• Can model chance of convening a panel from a
pool of n panellists:
(readily handled by MS Excel!)
• Where P(A) is the probability that panellist i is
available
)1.(
)!(!
!
.)](1[.)]([)( )(
eq
ini
n
APAPpanelaConveningP
n
ri
ini





6. www.yasigiworld.com
Panel pool size paradox
0.0
0.2
0.4
0.6
0.8
1.0
15 20 25 30
Size of panel pool
P(conveningapanelof8)
P = 0.5
P = 0.6
P = 0.7
P = 0.8
P = 0.9
P = 0.95

7. www.yasigiworld.com
• So if the probability of a panellist being available is
0.5, then a panel convenor needs well over 20
trained people to choose from to have a good
chance of running a given panel
• This level of availability is not atypical for some
staff…
• So whilst training is important, so is reliable
attendance
• The performance of
Panel pool size paradox

8. www.yasigiworld.com
Panel pool size paradox
• So, assuming that a pool of 20 is maintained for
panels of 8, there are around 126,000 possible
panel compositions
• We asked 20 tasters to assess the bitterness of two
commercial lager beers
• Then, we computed the mean of each possible
panel combination for panel sizes of 6, 8 and 10….

9. www.yasigiworld.com
Panel pool size paradox
0
2
4
6
8
10
12
14
16
18
20
20 25 30 35 40 45
Panel mean
Probabilityofattainingpanelmean(%)
6 from 20
8 from 20
10 from 20A
0
5
10
15
20
25
30
35
40
45
20 25 30 35 40 45
Panel mean
Probabilityofattainingpanelmean(%)
6 from 20
8 from 20
10 from 20B

10. www.yasigiworld.com
Panel pool size paradox
0
5
10
15
20
25
30
35
40
45
0 5 10 15 20 25 30 35 40 45
Panel mean
Probabilityofachievingpanelmean(%)
6 from 20
0
5
10
15
20
25
30
35
40
45
0 5 10 15 20 25 30 35 40 45
Panel mean
Probabilityofachievingpanelmean(%)
8 from 20
0
5
10
15
20
25
30
35
40
45
0 5 10 15 20 25 30 35 40 45
Panel mean
Probabilityofachievingpanelmean(%)
10 from 20
Resolution between products improves with panel size

11. www.yasigiworld.com
Panel pool size paradox
Panel size Data set Mean (standard
deviation)
A > B/%
(Error rate/%)
6
A 30.6 (3.16)
83 (17)
B 26.7 (1.34)
8
A 30.6 (2.54)
89 (11)
B 26.7 (1.09)
10
A 30.5 (2.08)
93 (7)
B 26.7 (0.90)
Here, 10 panellists rather than six more than halves error rate

12. www.yasigiworld.com
Panel pool size paradox
• The paradox is the balancing of costs and effort
of panel maintenance with quality of resulting
information
• As tools such as profile analysis are being used
to go beyond pass/fail to distinguish between
products of similar quality, need better
resolution from sensory testing
• This has implications for the way in which we
collect and analyse profile data…

13. www.yasigiworld.com
• It is common in production companies for
assessors to evaluate a relatively small subset of
products (compared with, say, a research facility)
• Individuals tend to scale consistently within
themselves but not between themselves
– Panel means represent few if any individuals
– Adds a lot of scatter to the data
Correcting for assessors

14. www.yasigiworld.com
Correcting for assessors
0
20
40
60
Estery
Hoppy
Sulphury
Sweet
BitterSour
Astringent
Acetaldehyde
Diacetyl
Taster 1
Taster 2

15. www.yasigiworld.com
Correcting for assessors
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12
Tastescore
Taster code
Max Min Median
Six assessments of the same beer batch over a two week
period. Note internal consistency between tasters

16. www.yasigiworld.com
• Could we filter out individual variances first?
– Yes, mean-centering each individual panelist
• Step 1: Create reference beer set
Correcting for assessors
Panelist-descriptor
matrix of median
score values
The “beer samples” are presentations
of the same beer batch six times.
Result is a panelist-descriptor matrix

17. www.yasigiworld.com
• Step 2: Correct test sample using panelist correction
Correcting for assessors
Reference panelist-
descriptor matrix
Test sample panelist-
descriptor matrix
Panelist-corrected
test sample panelist-
descriptor matrix
Panelists
Descriptors
- =
Final vector contains the panelist-corrected scores for the test sample

18. www.yasigiworld.com
• Tested out using beer bitterness as an example.
Used proprietary algorithm to create panel means
for all panels of nine from a pool of 12 assessors
Correcting for assessors
Reference
Test
0
5
10
15
20
25
30
28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
Frequency
Panel scores
Reference Test
Some suggestion that
the test is less bitter
than the reference. Not
very convincing though!
Uncorrected panelist data

19. www.yasigiworld.com
• Corrected data shows that the test is
unambiguously less bitter than the reference
Correcting for assessors
0
5
10
15
20
25
30
-2.71
-2.56
-2.41
-2.26
-2.11
-1.96
-1.81
-1.66
-1.51
-1.36
-1.21
-1.06
-0.91
-0.76
-0.61
-0.46
-0.31
-0.16
-0.01
0.14
Frequency
Reference-corrected panel scores
Reference
sample
Distribution of
test sample

20. www.yasigiworld.com
• Mean-centering correction of assessors, together
with creating exhaustive combinations of panel
membership from a pool of assessors substantially
enhances panel resolution
• Particularly useful in situations where panelists
assess relatively few products, but become expert
in the assessment of those products
• Critically, no change of the sensory experiment is
required, reducing chance of bias and risking the
opportunity to track historical data
Correcting for assessors

21. www.yasigiworld.com
Towards predicting sensory
performance from analysis
• Better prediction of sensory quality from
analytical data can give more cost-effective
product monitoring and NPD
• Opportunities to get:
– The same information for less investment
– More information for the same investment
• Enhanced competitiveness

22. www.yasigiworld.com
Towards predicting sensory
performance from analysis
• Scales are often labelled arbitrarily, eg
• This has been shown to be erroneous.
Labels often better satisfy the scale below:
This is due to non-linearity of sensory responses…

23. www.yasigiworld.com
Towards predicting sensory
performance from analysis
0
5
10
15
20
25
0 10 20 30 40 50
[Bitterness] (mg/l)
NumberofJNDs
Flavour threshold

24. www.yasigiworld.com
Towards predicting sensory
performance from analysis
0
10
20
30
40
50
60
0 10 20 30 40 50 60
[Hop acids] / mg/l
JNDs
1.05 1.075 1.10 1.15

25. www.yasigiworld.com
Towards predicting sensory
performance from analysis
• Two-step procedure for relating
sensory and analytical data
1. Convert analytical data into Flavour Units
2. Apply non-linear correction to Flavour
Units to derive the Sensory Unit.
Expression of the form:
)2.(
][
)( eq
thresholdFlavour
Analyte
FUUnitsFlavour 
)3.()ln( eqbFUaSU 

26. www.yasigiworld.com
Towards predicting sensory
performance from analysis
0
1
2
3
4
5 Carbondioxide
Hopacids
Ethanol
Isoamylacetate
DMS
Ethylacetate
Methanethiol
Dimethyltrisulphide
Ethylthioacetate
MBT
Glycerol
Sulphite
Methylthioacetate
Diacetyl
Phosphate
Chloride
Hydrogensulphide
Maltose
Potassium
Acetaldehyde
Maltotriose
Sulphate
Magnesium
FlavourUnits(FU)
0
1
2
3
4
5
0 1 2 3 4 5
Expected FU's
ObservedFU's
MBT
(50 vs 20 ppt)
CO2
4.2 vs 4.5 g/l)
Hop acids
(23 vs 21 ppm)
DMS
(40 vs 55 ppb)
Isoamyl acetate
(2.2 vs 2.0 ppm)
Off-diagonal – out of “perfect” specification
0
1
2
3
4
5
0 1 2 3 4 5
Expected FU's
ObservedFU's
MBT
(50 vs 20 ppt)
CO2
4.2 vs 4.5 g/l)
Hop acids
(23 vs 21 ppm)
DMS
(40 vs 55 ppb)
Isoamyl acetate
(2.2 vs 2.0 ppm)
0
1
2
3
4
5
0 1 2 3 4 5
Expected FU's
ObservedFU's
MBT
(50 vs 20 ppt)
CO2
4.2 vs 4.5 g/l)
Hop acids
(23 vs 21 ppm)
DMS
(40 vs 55 ppb)
Isoamyl acetate
(2.2 vs 2.0 ppm)
Off-diagonal – out of “perfect” specification
Step 1

27. www.yasigiworld.com
Towards predicting sensory
performance from analysis
-70
-60
-50
-40
-30
-20
-10
0
10
20
30
-70 -60 -50 -40 -30 -20 -10 0 10 20 30
Expected SU's
Observed SU'sAbove threshold, when
spec is below threshold
Observed and expected
are below threshold
Below threshold, when
spec is above threshold
MBT
DMS
Hop acids
Ethyl thioacetate
-70
-60
-50
-40
-30
-20
-10
0
10
20
30
-70 -60 -50 -40 -30 -20 -10 0 10 20 30
Expected SU's
Observed SU'sAbove threshold, when
spec is below threshold
Observed and expected
are below threshold
Below threshold, when
spec is above threshold
MBT
DMS
Hop acids
Ethyl thioacetate

28. www.yasigiworld.com
Towards predicting sensory
performance from analysis
• Such an approach requires validation in a
commercial environment
• Certain missing data is essential, such as
the magnitudes of the JND steps, but this
can be derived by experiment
• Moves us further on, but still assumes a
1-to-1 mapping of analytes to flavours…

29. www.yasigiworld.com
Integrating sensory information
• “Holistic” is a common term
• Implies interconnectedness
• To a first approximation, can ignore minor
variables
• For more accurate information, need to
bring in more and more parameters
• Today, merely want to set the scene

30. www.yasigiworld.com
Holistic beer quality
Uncontrolled image Controlled image Intrinsic liquid properties
Flavour VisualInternet
Mouthfeel
CO2/N2
Saliva
pH
Viscosity
Polyphenols
Mass media
TV
Newspapers
Magazines
Dispense, packaging
Glasses
Fonts
Home systems
Theatre of pour
Bottle/can
Health/physiology
Taste
Sweet
Salt
Sour
Bitter
Umami
Aroma
Hop
Malt
Fermentation
Maturation
Age-related
Off-aromas
Individual
postings
Formal media
Marketing
Product
Promotion
Price
Place
ColourClarity FoamIntegrity Well-being
Chemical
Biochemical
Physical
Microbiological
GMO
Radioactivity
Nutrition
Morning-after
Psychological
Allergens
Integrating sensory information
Integration required at various levels…
Intrinsic liquid properties
Flavour Visual
Mouthfee
l
CO2/N2
Saliva
pH
Viscosity
Polyphenols
Health/physiology
Taste
Sweet
Salt
Sour
Bitter
Umami
Aroma
Hop
Malt
Fermentation
Maturation
Age-related
Off-aromas
ColourClarity FoamIntegrity Well-being
Chemical
Biochemical
Physical
Microbiological
GMO
Radioactivity
Nutrition
Morning-after
Psychological
Allergens
On-the-spot
experiences
Delayed responses

31. www.yasigiworld.com
Integrating sensory information
• Challenge is our classically reductionist view of
both sensory and chemical analysis
• How to integrate? First need to understand the
activities of specific flavours and how they interact
with the matrix…

32. www.yasigiworld.com
Integrating sensory information
Sophistication
Accuracy (correlation to sensory score)
Free diacetyl
concentration
Weighted sum
of VDK levels and
intermediates. Matrix
compensation
Sum of free diacetyl
and pentanedione
concentrations
Weighted sum of
free diacetyl and
pentanedione
concentrations

33. www.yasigiworld.com
Summary
• We ask more of our sensory analysis today
– Finer resolution between products
– NPD of food and drinks that push traditional product
envelope
• My argument is that we need to ensure that we get
the very best from our sensory panels, by
– Taking heed of already well-established scientific
observations and statistical doctrine
– Applying some simple post-processing data analysis
tricks to improve panel resolution
– Moving towards more holistic measures of sensory
attributes
• A challenge, but a competitive opportunity to those
that do it well!

34. www.yasigiworld.com
• Yasigiworld Ltd was set up with a few aims in
mind, not least to provide on-line educational
resources and cost-effective texts
• If you have any comments of queries contact me,
Paul Hughes, either at
– paul@yasigiworld.com, or
– Connect up via LinkedIn
• Coming soon to ourYoutube channel, our
100seconds on… series on alcohol-related subjects
About us