Jon Puleston's presentation to the Festival of NewMR 2010

1. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
The Eureka ExperimentsExploring ways to stimulate and evaluate creative ideas in online surveys

2. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Please note that this is a snapshot of an
ongoing research experiment.
So I don’t have all the answers!
Also recognise this is an area where people
have already done a huge amount of thinking

3. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Background
• For the last 2 years working with Engage
research exploring how to improve the quality of
feedback from online surveys by making them
more engaging
• Published 2 papers on this topic and a case
study with Sony Music UK

4. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
2 minute summary of
our discoveries...

5. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
The importance of the introduction...
=
80%
more
Ime
focusing
on
task

6. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
The role of imagery
What
foods
do
you
hate
to
eat?
Twice as many suggestions
+

7. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
The impact of projective techniques...
Evalua2on
of
a
product
= 100% more feedback
+

8. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Discovered value of humour...
Coming
up
with
crea2ve
ideas
=
30%
more
feedback
+

9. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
How showing examples can encourage
respondents to give more feedback
Task
= 100%+ more feedback
+

10. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
The amazing impact of making tasks
competitive...
Remembering
ads
=
200%+
more
feedback
+

11. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
What can be achieved
with the right stimulus...
A 6 fold increases in feedback
x6

12. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Turning our attention to the
idea generation process

13. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
The challenge of creative thinking
in online research
• Harder
– Its not just a question of engagement, it is also an
issue of how to unlock creative thinking processes
– Most existing idea generation techniques are heavily
reliant on a moderator to implement them effectively
– The difficulty of implementing these techniques often
boils down to the limitations of existing survey
technology

14. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
So how did we tackle this?
• Firstly we examined all the common idea
generation/lateral thinking techniques out there
(well as best as we could, there are lots!)
• Looked at which ones could be best adapted to
the online survey environment
• Set up a series of experiments involving a variety
of creative tasks where we tested out each
technique
• Tried to measure and calibrate the volume and
quality of ideas each generated

15. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
We are only partly through
these experiments

16. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
What we have
experimented with
so far...

17. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Stage 1: Ice breaking
• Critical part of the idea generation process is getting
people in the right frame of mind
• Techniques tested so far:
1. Writing short film summaries
2. Writing lines of limericks
3. 10 seconds to think up names game
4. Silly uses for..
• What we have measured:
– How many respondents joined in
– How much they enjoyed it
– How it helped stimulate ideas later on

18. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Example: Silly uses

19. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Impact
6 to 8 ideas
But perhaps more importantly
promotes more radical thinking....
e.g. Uses for a brick: a door stop,
hammer, diving weight
to
Uses for a brick: Play catch with it,
carry it around with you as your lucky
talisman
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
With
Stimulus
Without
Very imaginative
Quite
Not very

20. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Value of this technique
• Sends a message that this is not a normal
survey
– By showing how far we are prepared to go, underlines
that we are taking this seriously ourselves
– Breaks down the rules of conventional surveying

21. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Stage 2: Working on the briefing
process

22. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
What if...
• Instead of 10% intro and 90% questions
• 90% intro and 10% questions

23. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Case study: king of shaves
• 10 minute briefing process
• Explaining the basic
techniques of advertising
development
• Then ask respondents to
have a go

24. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)

25. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
King of shaves results
• 250 respondents generated
over 1,200 ideas between
them
• 8 of the brand proposition
statements outscored the
agencies & 3 of the ad
treatments
• 87% cross participated in 2
follow up surveys to evaluate
the ideas

26. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
1. Perfectly in tune with your face
2. A new angle on shaving
3. Flexible smooth shaving
4. A good shave hinges on design
5. A smooth clean shave
6. Lightweight. precise. flexible.
7. Flex and flow
8. Flexible close shave
9. Better because it bends
10. No slaving over shaving
Top 10 brand proposition statements
Agency suggestion

27. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Stage 3: Exploring specific Idea
stimulation techniques...
• Techniques tested so far:
1. Thinking rooms
2. Idea for an idea
3. Thinking through the minds of
4. Random idea slot machine
• What we have measured:
– The volume and quality of ideas each technique generated

28. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Thinking rooms
• Technique often used by moderators is to
guide people to think about things in different
ways
• Online Methodology:
– An adapted approach using a set of freeform
rooms that people can explore
– Each room tackles the issue from a different
perspective
– A more free form less guided approach – they
choose what room to enter each time
– More like a game experience for respondents

29. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Idea stimulation: Thinking rooms

30. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Improvements in volume of ideas
0.7 to 6.9 ice cream names

31. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Delivers richer and more creative ideas
too...
Ask people to pick favourites and 7 of
the top 10 came from this process

32. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Idea for an idea
• At the heart of any brain storming process are ideas
that feed the generation of other ideas
• Online Methodology:
– A more serious version of the silly uses technique
– You come up with an idea, we then show you another
idea from someone else and then ask you to try and
come up with more ideas, loop through stimulus until
respondents get bored
– Technical requirements are fairly basic - a random
looping process feeding the seed ideas

33. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Idea for an idea...

34. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Improvements in volume of ideas
6 to 18

35. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Thinking through the mind of...
• A projective thinking technique
• Methodology:
– Respondents are asked to think through the minds of
different people/brands
– Each respondent gets a random stimulus

36. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)

37. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Impact
1 or 2 thoughts
increased to 10 to 20

38. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
What we have learnt so far?

39. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
All these techniques work

40. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Highlights how powerful
some ideas are at
stimulating ideas v others...

41. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Great ideas can unlock other
great ideas

42. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Good ideas act as Sparkplugs

43. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Idea Generating Enzymes
Our
quest
is
to
idenIfy
the
best...

44. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Strength of these techniques in an
online environment
• Each respondent can be seeded with a unique random set
of thoughts
• A self levelling technique for each individual respondents
attention span
– If the idea does not trigger an idea they just click “next”
– If they get bored doing it they can give up
• Using iterative sample you can edit and refine the
stimulation material as new ideas get generated
• Easily adaptable to any thinking task
• A no brainer in terms of how effective they are

45. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
What it has highlighted though is...
• The seeding ideas have critical impact of
steering the process in the direction you
wish to take it
• A certain amount of creative thought is
needed to get the ball rolling

46. Jon
Puleston,
GMI
Interac2ve,
UK
Part
2:
Session
1,
Convenor
Sue
York,
Chair
Dan
Foreman,
schedule
=
9:58am
to
10:22am
(GMT/London)
Q & A
Jon Puleston and Dan Foreman