Slides from an ARMA 2015 talk on building impact into the research process, with a focus on research management.

1. 1
Embedding impact; effecting
institutional change through the
development and rollout of an
Impact Capture System
Julie Bayley
Impact Officer
Senior Researcher in Health
Psychology
Email: j.bayley@coventry.ac.uk
Twitter @Julie_covuni
Julia Ryall
Programmes Director
Post Award
Email: j.ryall@coventry.ac.uk
•To share Coventry’s experience of embedding impactPurpose
•To offer insight into developing strategies for your own institutionAim
•Research directors, managers and officers, funders, stakeholdersAudience
•Brief, to support planning for your institutionExercises
Resources
•Blog - http://blogs.coventry.ac.uk/researchblog/category/impact/
•Session materials (slides, booklet)
•Link to paper and slides (“Strategies for the management and
adoption of impact capture processes”) from EuroCRIS 2014
•Link to “How-to guide for developing an Impact Capture
System”, LSE Impact of Social Sciences blog post

2. 2
Overview
1. Background and HE context
2. Coventry approach (with ‘ERIC’)
3. Research support context
4. Next steps
5. Q&A
Section 1
Background and
context

3. 3
Research Impact
• Research Councils UK (RCUK) definition:
– 'the demonstrable contribution that excellent
research makes to society and the economy'.
• Broad types:
– Academic
– Economic
– Societal
– Environmental
Impact Drivers
• REF 2014
– Impact added to assessment criteria (not in RAE 2008)
• Academic excellence 65%
• Impact 20%
– Retrospective tracking and capture
• Expected increased weighting for impact in REF 2020
• Increased funder requirements for impact (eg. Horizon
2020)
• ‘Return on Investment’

4. 4
Related context
• Ongoing discussions on measurement, assessment and
definitions e.g.
– Acceptable metrics
– Appropriate metrics
– Altmetrics
– Proxy measures
• Related drives eg. Open Access, international strategy
• Increasing focus on how to embed change institutionally
• How do we develop a systematised approach?
Challenges
• Experience of REF 2014
• Impact understanding /
awareness
• Impact is not static or
universally accepted
• Scepticism and suspicion
• Standardisation across
disciplines
• Attribution and ownership
• Intangible impact
• Time lag / tracking
• Control of information
• Bridging academic /
business divide
• Combining narrative with
metrics
• Non-funded projects

5. 5
Section 2
Coventry University
Approach
EXERCISE 1
Booklet page 6-7

6. 6
Impact is a good thing
Impact can be captured
by an academic self
service system

7. 7
Impact capture and
assessment are separate
activities
Different impacts can
be ranked / weighted

8. 8
Ranking / weighting impact
is motivating for
academics
Priority should be placed
on REF 2020 impact rather
than impact in broader
terms

9. 9
Impact is a ‘virtuous activity’
Academics are suspicious
about Centrally led
initiatives

10. 10
“Embedding Research Impact at
Coventry” (ERIC)
• JISC funded project (2012) to develop a pilot impact
capture system
• System to plan and capture impacts from project inception
• VC commission: redeveloped to cover whole university
• Development:
– Development of impact markers
– Review of literature and guidance
– Academic and stakeholder consultation
*http://www.publicengagement.ac.uk/about/impact-analysis
Prototype development process
1. Impact
needs analysis
Business and
Community
Engagement
2. Create
content
Research
3. Technical
process review
/ system
adaptation
Information
Management
4. Pilot test
system
Research

11. 11
Full rolloutPilot
Purchase
database Full launch
Consultation and
engagement
across faculties
Modify/ extend
database
Embedding
External
literature /
guidance
‘ERIC’ system
• Built within existing CU Research Information
Management system (BIDS)
• Add details of (multiple) planned impacts, those
which occur unexpectedly or remove those which
do not transpire.
• 4 steps to add an impact…..

12. 12
1. Plan impact
2. Plan evidence

13. 13
3. Set email
reminders
4. Add evidence / defer / update

14. 14
Benefits of ERIC
• ‘Institutional memory’
• Flexible system supports impact planning from project
inception
• Build and monitor ongoing impact portfolios
• Reduced reporting burden for REF 2020
• Support funding bids / reports to funders
• CU marketing, case studies and reputation building
Consultation feedback
Barriers Facilitators
0
2
4
6
8
10
12
14
16
18
20
Time /
workload
System
familiarity
Impact
awareness
System
difficulties
Attribution
and tracking
0
2
4
6
8
10
12
14
16
18
20
Training Raise
awareness
System
edits
Link across
university
Utilise data Access Increase
relevance

15. 15
Section 3
Research Support
Context
Supporting the Impact Agenda
• Coventry’s Story: Past, Present, Future
• Changes in the Project Life Cycle
• Tensions, concerns, worries, solutions….

16. 16
Coventry University Commitment
• Part of new “Excellence with Impact” strategic plan
• Impact Capture System (ERIC)
• Central Impact Officer (Academic)
• Faculty / research centre based impact personnel
• Broad and growing support structure for impact support
• Increasing reputation and presence across higher education
• Impact is valued

17. 17
Impact across the project lifespan
Research
project £
Academic
output
Citations Impact
From...Impact considered at the end of the project..
Academic
“project”
Academic
output
Citations and metrics
ImpactOther
dissemination
Impact monitoring / evidencing
Impact
planning
To…Impact built in from the start of a project…
CULTURAL and
SYSTEMS
integration

18. 18
CULTURAL and
SYSTEMS
integration
Academic’s
Impact
Funded
Projects
Non-
Funded
Events /
Conferences
To..
Articles
Media
Presence
Esteem
Markers
BIDS
EventsForce
Ethics
Moodle Repository
M&C
??
Publicly
Funded
Projects
£
Articles
From…
DisseminationConference
BIDS
Repository
Impact
Academics
Heterogeneous
Research
Support
Homogeneous
Strategic
leads
Themes
Everything?
Cross Cutting
Subject Specific
Indexed / Meta Tags
Immediacy
Impact ‘Cogs’
More
FUNDING
Benchmarking
Ownership
Organisational
Reputation
Own Reputation
Longevity
RE-USE
Performance
Measure?
Ranking
Standardisation
Individuals
Key Challenges
Diverse
Context
We don’t know
what the
Strategic
Leaders will
require next
New different
drivers? When?
? What is the
information for
Impact ‘Cogs’
Ownership
Organisational
Reputation
Own Reputation
Longevity
Standardisation
Individuals
Key Challenges
Diverse
Everything?
Cross Cutting
Subject Specific
Indexed
Immediacy
RE-USE
Benchmarking
Performance
Measure?
Context

19. 19
Academics
Research
Support
Strategic leads
Exercise 2: Your cogs
Booklet page 13
Impact ‘Cogs’
Ownership
Organisational
Reputation
Own Reputation
Longevity
Standardisation
Individuals
Key Challenges
Diverse
Everything?
Cross Cutting
Subject Specific
Indexed
Immediacy
RE-USE
Benchmarking
Performance
Measure?
Context
Section 4
Next steps

20. 20
.…as a Sector
• Build in impact from the start
• Translate “Impact”
• Engage Academics
– Behaviour change, embed, deal with suspicion…
• Engage Stakeholders
• Build networks (internal and external)
• Bridge academic / research support divide
….at Coventry
• Continue building and rolling out system
• Grow impact culture / infrastructure
• Embedding impact and behaviour change
• Training
• System changes
• Dissemination (papers, reports, blogs)

21. 21
Exercise 3: .…at your Organisation
1.
2.
3.
Booklet page 15
Summary
• Top-down support for bottom-up development
• Consider what works in your context
• ‘Engage not enrage’ academics
• Across-university system interaction
• Continued dialogue and revision
• Centralised and committed approach, locally translated
• Partnership is key

22. 22
Thank you
Questions?
Julie Bayley
Email: j.bayley@coventry.ac.uk
Twitter @Julie_covuni
Julia Ryall
Email: JRyall@cad.coventry.ac.uk