The Balanced Value Impact Model is intended to aid the thinking and decision making of those wishing to engage in Impact Assessment. It also acts as a guide through the process of Impact Assessment to enable the core values most appropriate to the assessment to be brought to the fore and given a balanced consideration when evaluating outcomes. It presumes that the assessment will be measuring change within an ecosystem for a digital resource. For the purposes of this Model, the definition of Impact is: The measurable outcomes arising from the existence of a digital resource that demonstrate a change in the life or life opportunities of the community. Who should use the BVI Model? The aim of this workshop is to provide key information and a strong model for the following primary communities of use: Memory institutions and cultural heritage organizations, such as libraries, museums and archives. Funding bodies who wish to promote evidence-based impact assessment of activities they support. Holders and custodians of special collections. Managers, project managers and fundraisers who are seeking to justify further investment in digital resources. Academics looking to establish digital projects and digital scholarship collaborations with collection owners. Publishing, media and business sectors which may be considering the best means to measure the impact of their digital resources and are looking to collaborate and align with collection owners, with academia or with memory institutions. Impact Assessment practitioners considering an Impact Assessment of a digital resource. What the workshop will cover: Where the value and impact can be found in digital resources, Who are the beneficiaries gaining from the impact and value, How to measure change and impact for digital resources, How to do an Impact Assessment using the Balanced Value Impact Model, and How to present a convincing evidence-based argument for digital resources? The Workshop will include case studies of how the BVI Model is being implemented at present.

2. www.kcl.ac.uk/ddh/
Digital Humanities:
the application of digital technology to humanities disciplines
reflection upon the impact of digital media upon humanity
> 50 academics & researchers
~ £2.5 million research income per annum
5+ million digital objects in 107+projects
200+ million hits over the last 5 years

3. Measuring the Impact of
Digitized Resources:
The Balanced Value Model
http://www.kdcs.kcl.ac.uk/innovation/impact.html

4. “the measurable outcomes arising from the existence of a
digital resource that demonstrate a change in the life or life
opportunities of the community”
www.kdcs.kcl.ac.uk/innovation/impact.html

9. The case for Impact
We are more effective and efficient in delivering change
and tangible benefits (Internal Impact);
Our organisation is gaining strategic advantage through
the innovation inherent in this digital activity (Innovation
Impact);
We are delivering a strong economic benefit to our
community that demonstrate the worth and value of our
endeavours in clear monetary terms (Economic Impact);
and
the community has been changed by the resource in
beneficial ways that can be clearly identified (Social
Impact)

10. www.kdcs.kcl.ac.uk/innovation/impact.html

11. www.kdcs.kcl.ac.uk/innovation/impact.html

12. Impact taskforce
Simon Tanner

13. Aggregated
• 2200+ content
providers
• 173 aggregators
• 26.9 million objects

14. Network
• 600+ individual
members, working
in taskforces and
on strategy
Breakdown
Galleries: 2
Libraries: 111
Archives: 26
Museums: 60
National Aggs: 22
Publishers: 2
Creative Ind: 5
Research: 78
Ministries: 9
Other: 174

15. Stakeholder Survey and Mapping
A person, group, community, or organization who
affects or can be affected by the ecosystem of the
digital resource to be assessed.
Questions to be answered in establishing and
identifying stakeholders include:
Have all primary stakeholders been listed?
Have all secondary stakeholders been listed?
Have all potential supporters and opponents been
identified?
Have the interests of vulnerable or minority groups been
identified?
What new primary or secondary stakeholders are likely to
emerge?

16. Stakeholders – the kinds of broad groups
Consumers
One Stop Consumers
Partners and Collaborators
Paymasters
Producers and Creators?
Commentators
Marginalised
Leavers
Non-users
Champions
Competitors

17. Stakeholder Survey and Mapping
Stakeholder mapping is a powerful tool to
enable planning
It enables a list of stakeholders to become an active
tool in three simple steps
The steps:
Name your stakeholders and put them in categories
Map where the stakeholder is now in relation to your
organisation or the initiative for which you want their
support
Now map where you need the stakeholder to be to
enable your organisation or initiative to succeed.
Use this information to plan actions/timing etc.

18. Stakeholder Mapping
Knowledge
Context
Setters
Champions &
Activists
Influencers
Bystanders
Influence / Initiative

19. Stakeholder Mapping: Example
Knowledge
Champion
Fund project
Govt. Funder
Gain IP agreement
CEO
Govt.
Publisher
Influence / Initiative

20. Exercise: Draw your own
Knowledge
Context
Setters
Champions &
Activists
Influencers
Bystanders
Influence / Initiative

23. www.kdcs.kcl.ac.uk/innovation/impact.html

25. PERSPECTIVE
+ VALUE
DRIVER
OBJECTIVES
STAKEHOLDERS
AREAS MEASURED
METHODS
INDICATORS
CURRENT
USE (SOCIAL)
Those with
an interest
in the
intellectual
content find
it useful to
their
research
Google
Users of the 1. Discovery
Code2. Engagement Analytics review
breakers
3. Usefulness
Site surveys
resource.
sampling users
on the
Codebreakers
microsite.
Tracking of
recommendatio
ns to others or
reviews.
Users
User panels.
Citations of
content on the
site.
1.
a.
b.
c.
Discovery of resource:
Web visits/visitors
Views to digitised content
Relative use to historical use figures (where
applicable)
1. Engagement with resource:
a. Average time spent on digitised content
b. Repeat visitors to Codebreakers
c. Downloads of content
d. User journeys across Codebreakers
1. Usefulness of resource:
a. Site surveying to collect reported usage and
utility of Codebreakers resource
b. Citation indices – this can only be a very long
term measure due to research and publishing
timescales
c. User panel – recruited from actual users of
the Codebreakers resource. Engage in enquiry of
the function, content and discoverability of
Codebreakers.
d. Desk research to find innovative use of the
content
e. Online media monitoring to capture people’s
mentions and recommendations of Codebreakers.

26. PERSPECTIVE +
VALUE DRIVER
OBJECTIVES
STAKEHOLDERS
AREAS MEASURED
METHODS
INDICATORS
CURRENT USE
(SOCIAL)
Community
Peer
organisation
s and
members of
our
professional
community
have
changed
their policy
or practice
concerning
digitisation
projects.
Practitioners 1.
, peer
organisation
s and
2.
members of
our
professional
community 3.
who have
been
influenced
by the
project
Awareness
of the
project
Take up of
methods/
approaches
/ standards
Impact of
take up on
partner and
peer
organisatio
ns
Survey of
partner
organisations
who worked on
the project
Survey of event
attendees / key
peer
organisations
Desk research
1.
a.
b.
c.
d.
1.
a.
b.
1.
Awareness:
Number attending digitisation open days/events
held at Wellcome Library
Contacts from peer organisations/practitioners
Online media monitoring for blogs, conference
presentations, events, workshops, open days,
social media etc.
Citations/references to the project
Take up:
Survey of partner and peer organisations, and
practitioners identified in stage 1.
Desk research to identify stakeholders influenced
by the Codebreakers project (e.g. Ronan Deazley
work on archives and copyright)
Impact:
a. Survey of partner and peer organisations, and
practitioners identified in stage 1.
b. Desk research

27. PERSPECTIVE
+ VALUE
DRIVER
OBJECTIVES
STAKEHOLDERS
AREAS MEASURED
METHODS
INDICATORS
POTENTIAL
USE
(INNOVATION)
Users
The
Codebreakers
project has
enabled new
potential
activities and
research
methods for
those
interested in
the intellectual
content.
Potential users
of the
Codebreakers
resource
1.
2.
3.
Community
The extent to
which
Codebreakers
has created
new
possibilities for
organisations
and
professional
members of
the cultural
heritage
community.
Practitioners,
peer
organisations
and members
of our
professional
community
who may be
influenced by
the project
1.
2.
3.
4.
Delivery of
the planned
functionality/
usability of
the
Codebreakers
resource.
User
understandin
g of the new
research
enabled by
Codebreakers
.
Unforeseen
potentials for
new research.
Evaluation of
functional
capabilities of the
finished site against
our initial goals.
Heuristic evaluation
of Codebreakers
resource.
Site survey.
Focus group with
potential users
1.
a.
Desk work to
determine
accessibility of
technical and
process
developments.
Awareness of Qualitative
measures such as
access to
developments questionnaires,
desk research,
.
structured
Uptake of
interviews. A
practices
initiated in the comparison of our
initial goals with the
project as
final site.
industry
standard.
Unforeseen
1.
a.
The
accessibility
of technical
developments
.
b.
1.
a.
b.
c.
d.
1.
b.
1.
a.
1.
a.
1.
a.
b.
Delivery:
Desk work to check the functionality of the site against our
initial specification.
Heuristic evaluation of usability
User understanding:
Site survey of users on Codebreakers microsite
Focus group research
Usability research with potential users
Heuristic evaluation
Unforeseen potentials:
Staff interviews to discover where unexpected benefits
occurred during the build of the site.
Focus group recruited from potential Codebreakers users to
discuss new opportunities for researchers.
Accessibility of developments:
Desk work to check accessibility of technical developments
to the peer community
Availability of documentation of process/organisational
developments
Awareness of access:
Survey of partner and peer organisations, and practitioners
identified in Current Community Awareness.
Uptake as industry standards:
As described in Current Community uptake
Unforeseen potentials:
Survey of partner and peer organisations, and practitioners
identified in Current Community Awareness.
Staff interviews to discover where unexpected benefits
occurred during the build of the site.

28. PERSPECTIVE
+ VALUE
DRIVER
OBJECTIVES
STAKEHOLDERS
AREAS MEASURED
METHODS
INDICATORS
INTERNAL
How have the
staff of the
Wellcome
Trust had their
skills, abilities,
capacity and
knowledge
Development enhanced by
developing
Codebreakers
Inheritance /
Bequest
How does
Codebreakers
represent the
inheritance of
the Wellcome
Trust Library’s
activities since
collecting
began and
how does it
prepare the
Library for the
future and
bequeath
benefits to
future
generations?
Staff of the
Wellcome
Library and
Trust.
1.
2.
Changes in
individual
knowledge or
skills.
Changes in
working
practices and
behaviours
3.
Staff and
members of
the Wellcome
Trust.
Change in
usage
enabled by
Codebreaker
s resource.
Value for
future
digitisation
activity
Benchmarkin
g against
peer
organisations
.
Comparison
to historical
strategic
direction of
Library.
2.
3.
4.
1.
a.
b.
1.
a.
b.
1.
a.
Changes in
organisation
al capacity or
ability.
1.
Survey of Trust
staff connected
with the project.
Interviews with
line-managers.
Interviews with
senior managers.
Google Analytics
Interviews with
senior managers
Review of peer
organisation
activity
Desk research
Individual knowledge:
Survey of Trust staff connected to Codebreakers activity.
Interviewing line-managers of staff involved in the
Codebreakers project.
Working practices and behaviours:
Survey of Trust staff connected to Codebreakers activity.
Interviewing line-managers of staff involved in the
Codebreakers project.
Changes to organisation:
Interviews with senior managers.
Change in usage:
a.
See Current Users usage
Value for future digitisation activity:
a.
Interviews with senior managers
b.
Data from Internal Development
Benchmarking:
a.
Review of peer organisations – desk research and
interviews to compare the Wellcome Library’s digital
status in comparison with its peers worldwide.
Comparison to historical strategic direction:
a.
Desk research
b.
Interviews with senior managers

29. PERSPECTIVE
+ VALUE
DRIVER
OBJECTIVES
STAKEHOLDERS
AREAS MEASURED
METHODS
INDICATORS
Google Analytics
review
Site survey
User panels.
This will be based on methodology developed by the
British Library in their 2013 economic evaluation. The
full British Library report is available here. It will include:
1. Comparison of Codebreakers usage with archive
usage records over the last 5 years with an
assessment of the cost of use.
2. User time spent on the Codebreakers resource
3. Users’ geographic location.
4. Equivalent cost implication for users consulting
across collections previously held in physically
separate locations.
5. Contingent valuation questions included in site
survey and user panels.
ECONOMIC
What is the
net
economic
effect of
making the
content
freely
available
online?
Users
Users of the Economic gain
Codebreaker to individual
s resource. users of the
resource.
Economic value
generated for
organisations
that are endusers of the
resource.

30. Thanks!
Thank you to all the folks at the Wellcome Library
for allowing me to share this with you.
Contacts
Alexander Green
Email: A.Green@wellcome.ac.uk
Christy Henshaw
Email: C.Henshaw@wellcome.ac.uk

31. The case for Impact
We are more effective and efficient in delivering change
and tangible benefits (Internal Impact);
Our organisation is gaining strategic advantage through
the innovation inherent in this digital activity (Innovation
Impact);
We are delivering a strong economic benefit to our
community that demonstrate the worth and value of our
endeavours in clear monetary terms (Economic Impact);
and
the community has been changed by the resource in
beneficial ways that can be clearly identified (Social
Impact)

32. www.kdcs.kcl.ac.uk/innovation/impact.html

33. With thanks to Alice Maggs for the Impact illustrations
alice.100@hotmail.com