The document summarizes the findings from an evaluation of Creative Credits, an innovation voucher program in the UK that provided £5,000 grants to small businesses to partner with creative service providers. Key findings include: 1) The vouchers had a high level of additionality, with firms much more likely to undertake projects with vouchers than without. However, longer-term impacts on innovation and cooperation were weaker. 2) Outcomes varied greatly depending on the quality of match between firms and creative partners. Stronger brokerage and vetting of partners may have improved results. 3) Implementing the program during an economic recession made it difficult to isolate the impact of the vouchers from broader market conditions.

1. Creative Credits B2B Innovation
Vouchers
Creative Credits – What have we learnt from the
UK experience?
 Professor Stephen Roper
& Dr Judy Scully

2. Introducing Creative Credits
 SMEs tend to be reluctant to partner with creative service
providers for innovation
 Creative Credits are B2B innovation vouchers to encourage
SMEs to innovate in partnership with creative service
providers (e.g. designers)
 Creative Credits have a face value of £4,000. The SME is
required to add a further £1,000 to support the project.
 150 Creative Credits were distributed in Manchester between
October 2009 and November 2010 with very light-touch
brokerage using an on-line gallery of creative service firms

3. Logic model for Creative Credits

4. The Creative Credits policy experiment
 Usually policy evaluation is ex post. Here a different and novel approach.
 Evaluation designed as small scale, experimental evaluation study based on
randomised control trial methodology.
 In fact ‘RCT+’ evaluation methodology has three key features: :
 Randomised allocation of firms to treatment and control groups –
lottery based allocation of Creative Credits
 Longitudinal data collection strategy, allowing the longer term as well as
short-term impacts to be assessed,
 Mixed-methods empirical approach, combining qualitative interviews
with quantitative survey techniques.

5. Collecting evaluation data – theory

6. Collecting evaluation data – in practice
 Four surveys over two years so
attrition was a problem – used some
small financial incentives
 By survey 4. Treatment -78 per cent
response (n=117), control group
52.2 per cent (n= 157)
 Clearly bias between groups but
comparison of baseline
characteristics suggests NO
systematic bias within groups
 Longitudinal sample therefore
considered ‘representative’

7. Key results 1: Project additionality
 Evidence of a high level of project additionality - firms
receiving a Creative Credit were 78 per cent more likely to
undertake the proposed project
 This result is very similar to that obtained in the Dutch
innovation voucher pilot project
 Typical comments were:
 “…we couldn’t have afforded it … we wouldn’t have done it. It’s made a real
difference to us from that point of view.”
 “… it’s allowed us to accelerate things and it’s perhaps made me focus on it
rather than put things off. So it’s got things moving quicker than if I’d been left
to my own devices.”

8. Key results 2: Output additionality
 Strong evidence of short-term output additionality in terms of
increased innovation. But effect short lived. Weak learning
effects?
 Longer term sales gains from Creative Credits projects though
– average benefits last 2.5 years
 But … there was widespread evidence of a recession effect:
 ‘We are still experiencing a depressed market due to the recession, but
our short term future looks better and I expect to see significant up-turn
in 2012. As such our new products have not had an easy start in life, but
we are happy with their development and remain optimistic for their
future’.

9. Key results 3: Behavioural effects
 Little evidence of significant behavioural effects
 Control/treatment firms equally likely to innovate in future
three years
 Also equally likely to co-operate with creatives
 But considerable evidence (self-reported) of gains in
confidence, innovative capability etc. in treatment
group
One SME commented: ‘you can’t just sit back and just do things
the same. We’re now thinking about building a French web site
and a German web site’.
Similarly, ‘you cannot move forward if you don’t take the risk’.

10. Qualitative Insights- from 20
working relationships
 Out of 20 pairs in the qualitative interviews (stage4) 9 SMEs reported
failure
 Poor communication
 Poor coordination
 Economic climate
 Brokerage not robust enough to support selection
 One consequence of these communication and coordination difficulties – and the tensions they created in
some of the SME-creative relationships – was a lack of organisational learning and the knowledge was not
embedded
 In particular, a number of SMEs made suggestions as to how NESTA could have provided a more active
matching and brokerage function between SMEs and their creative partners. :
 ‘…we would have got loads more value if they went down a completely different route that was more
online focused. Now, I think I would have made better decisions if I had have had almost somebody from
NESTA to bounce ideas off first rather than me ... because it was quite limited and I think, perhaps, even if
there was some sort of almost a network, an open day so - that everyone that was accepted onto the
programme could actually go and meet a few of these other companies … I think that would have been a
lot better. It would have meant I would get heaps more out of the project’. ( SME)

11. Qualitative Behavioural Additionality
 Behavioural Additionality
 Despite the following loose examples of behaviour change, in Survey 4 there is limited evidence of
behavioural change:
 Mindset shift: One SME spoke of how the Creative Credits partner enabled a mindset shift in terms of
working with professional agencies and freelancers and so “we’ve actually started to work with an agency”
(W2S11).
 Thinking differently: Another SME explained how their creative partner encourages them to think about
their products and bring three websites together and “This has changed the marketing system from a “Hi
we’re here” to a “Hi we’re here and have a look’” (W1S6).
 Operational benefits: A third SME noticed that co-operation improved through a close working relationship
on innovation. This involved them being offered discounted marketing at a favourable rate as well as being
given some shared office space (W1S6).
 Sales and promotions: A fourth SME said that working with the creative credit partner sales team has led to
new products on Amazon and orders coming in daily as well as new products. They intend to work with
their creative partner again and associate the success of the new mascot design with the launch of their
new company (W2S8).


12. 11 Success Stories in the Qualitative
 In a good many cases – unsurprisingly, given the web-based nature of many of the projects
SMEs felt that their creative partners had portrayed them in a more positive and engaging
way online to their customers.
 One firm commented: ‘certainly it has improved the image and it’s improved the response
we’re getting, people are getting in touch with us which they weren’t before’ (SME).
 ‘it’s changed the way that people perceive [us] because instead of looking like a clacky old
website that I was slightly embarrassed by, it looked like I wanted it to look, because the
person that designed it originally, the two guys that were involved from the very start five
years ago, and did the update two and a half years ago, were not the ones I should have used.
I should have been using a professional organisation’ (SME)
 For other SMEs the Creative Credits projects had resulted in a more unified marketing
message:
 ‘The main impact we’ve had is the little mascot man, the little robot, because he also appears
on the hard copy material that we produce as well’ (SME).

13. Summing up …
 Creative Credits proved very popular with SMEs and creative
service providers with high application rates (6 times over
subscribed)
 Short term additionality was good but longer-term and
behavioural effects proved weaker
 Why? Timing particularly difficult but also possibly matches
could have been stronger. One of a number of firms
commented:
 ‘Unfortunately the wrong choice of creative partner was made - the
creative credits scheme was in no way at fault. The partner chosen fell
way below the standard expected and their performance bore no
relation to the picture painted at initial meetings’.

14. Implementation lessons
 Do consider brokerage carefully. Getting the matches right is crucial. This
was under-estimated and we had contrasting outcomes:
‘… Unfortunately the wrong choice of creative partner was made’.
‘Our experience with our creative partner was very positive … ‘
‘We … feel there should have been more 'vetting' of the creatives
involved’.
 And don’t implement schemes in recessions!:
‘… its difficult to know whether the Creative Credit has helped deflect
further shrinkage in trade than would have been suffered and what impact
it would have had in normal circumstances’.

15. Follow-up material
 ‘A guide to Creative Credits’ – provides a detailed overview of
the programme
Available at: http://www.nesta.org.uk/assets/features/guide_to_creative_credits
 ‘Creating innovation in SMEs’ - an overview of the interim
evaluation results
Available at:
http://www.nesta.org.uk/publications/reports/assets/features/creating_innov
ation_in_smes
 And, final report to come in a few weeks