Given at the SLA Conference in Phoenix 2017, an overview of the logic model to measure the outcomes of information services & programs from the decision-makers' perspectives.
2. My lens
•must use evidence for decision-making
• Healthy blend of evidence, emotion & risk
•key questions: what, so what, now what
•crucial to start with where you are, plan small
progressive steps that won’t kill your Achilles’ heel
•everyone’s lens looks through their mental models
9. Board Chair
Kim Kardforyour Mony • Socially connected
• Lobbying to have library
budget reallocated to other
projects
• Has never used the library
• Buys all her books from
Amazon
• Why does the library need so
many staff? And why are
they paid so much?
10. VP Research
Dr. Ural Doomed • Libraries days are numbered
• Libraries aren’t relevant
• Everyone has computers and internet
• Need information? It’s all online
• Research happens in faculties not
libraries
• With edocs, ebooks, open data and
research online, believes facilities can
be made smaller – no need for print
material
11. CFO
Roi Bottomline
• All about the bottomline
• Revenue and investments
• How do your services support
revenue generation?
• Services that don’t generate
revenue are a drain on the
system
• If it’s not efficient, outsource
it!
12. Dean of Humanities
Kee Paiper
• Influential in many circles
• Has deep seated beliefs in traditional
library services and importance of
paper
• Libraries are institutions for archives
& research
• Needs to demonstrate to college
that humanities still contributes to
reputation
13. Logic model in action for Dr. Papier
Input
(resources)
• One liaison librarian
Output
(operations)
• Faculty research support for a Grant Application
• Completed application submitted on time
Outcome
(user)
• Grant dollars received by faculty
• Faculty member is able to hire 2 grad students to conduct
research
Impact
(stakeholder)
• Increased university visibility in research area
• Enhanced university reputation
• A number of articles are submitted and accepted by academic
journals, increased citations for faculty member
14. Project Outcomes: Jones & Raven 14
Resource
Perspective
What we
put in
Operational
Perspective
What we
deliver
User
Perspective
What
comes out
of it for
customers
Stakeholder
Perspective
What
changes for
customers
input
outcome
output
impact
28. • No one magic measure
• “Impact” or value is dynamic, economic, psychological & relative to alternatives
• Identifying outcomes & especially impact takes a long time & heavy duty analysis
• Successful organizations:
• clarity of purpose
• understand their culture
• performance measurement system that fits that culture
29. Operational/Usage Measures
• We’ve long tracked “how much”
• We also need to look at differences:
• Peaks? Dips? Switches?
• What are these statistics really telling us?
Or, not?
• Who needs to see these statistics?
• Are they telling a story that you want to
tell?
Operational
30. Satisfaction Measures
• Tends to focus on existing products
and/or services
• Usually measured through surveys
• Are we also measuring importance?
• Survey burnout, plus the
expectation is to score all
“excellent” or “10 out of 10”
Satisfaction
31. Value Measures
• What do you have?
• Qualitative?
• Testimonials, attributes
• Quantitative?
• Budget
Value
32. Value
Satisfaction
Operational
Focus on the sweet spot 72 % of First-year Students engaged with
Library staff services in Fall 2017
semester
85% of these Students rated these
services as very important in
addressing their workload &
assignment completion
70% of all Students used the Library in Fall
2017
83% of all these Students rated the Library
as the 2nd most important service
related to academic success on campus,
2nd only to their faculty
35. Know the Context (& the Contexters)
How do they
measure their
progress?
Know your
Stakeholders – who
are they?
How is value or impact
defined,
measured &
communicated?
36. Know the
language of
your context
Metrics
Value
Customer
Growth
Student
Sustain
Residents
Constituent
37. We can't come to terms with new concepts until we
learn new language
Horseless Carriages
Car Phones
ATM’s or PTB
38. Start with stakeholders
Name Position How do
they
measure
their
progress?
Goals &
objectives
What do we know about
them? What type of
stakeholder are they?
What do
they know
about our
services?
Critical that you understand the context in which the measures are considered.
1. In your organization, how is value defined, measured and conveyed?
2. Who are the stakeholders who must understand the value of your services?
39. The Logic Model
…goes beyond documenting what you did and
measures what difference you made in the life
(work, reputation, etc. )of your target audience…
What has been the positive change in their
sphere… as a result of what you do?
46. Align Strategies & Objectives
Stakeholder’s Goals & Objectives Your Goals & Objectives: How they
Contribute
Accelerate product to market Catalyze product development
Improve market share in specific
industry segments
Customize segment dashboards for
business development
Increase research grants in xxx Underpin the research process and
assist with submission and completion
of grants
48. Identify Services
What services, programs or products contribute to your realizing an objective?
Stakeholder Goals & Objectives
Your Goals & Objectives
Your Services
51. Manage Measurement Data
• What will success look like?
• How will you track progress towards meeting goals?
• What indicators or measures will you use?
• Qualitative? Quantitative?
• What data needs to be collected?
• Keep it key
• Keep it simple
• Be practical
• Don’t get mired – or
overwork frontline staff
52. Manage collection
• Determine a collection schedule
• Work out agreements with other departments that
have needed data
• Are you currently collecting data or indicators that
are no longer relevant?
• How long do you need to keep data?
• Who is responsible?
54. Define outcomes & impact
• What will success look like?
• And how will you “get” there?
OUTCOME IMPACT
INPUT OUTPUT
55. Government Library Example
Input
(resources)
• $100K resource budget
• Staff to advise on development of policies
Output
(operations)
• 2500 resources utilized by 75% of department employees
• Library has contributed to the development of 6 policies in past
12 months
Outcome
(user)
• 6 policies approved and in place
• Eliminate department-specific resource purchases saving the
organization $35K
Impact
(stakeholder)
• Credible policies
• Reduce policy duplication, “blind-sides” or contradictory policies
by at least 25% (as reported during policy debriefings)
• Reduced costs to run department
56. Corporate Library Example
Input
(resources)
• $80K electronic content contract
• Tech and applications to integrate library’s products and resources into the
departments applications
• Skilled info pro to develop research portfolio for business development
department
Output
(operations)
• Customized research portfolio in response to request for background on
deal between XYZ company & ABC government agency, including specific
data on terms how ABC likes deal structured
• Library resources are integrated into business development applications
Outcome
(user)
• Client extrapolated how ABC agency likes to see deals structured and
structured deal appropriately
Impact
(stakeholder)
• Organization successfully completed a deal with ABC government agency
for $10M, the 1st in 5 years
59. Convey results
• Actually begins when you are understanding the context
• Your message: “wehave
contributed towards your goals
by……”
• If your goals & objectives are meaningful for your planning
& decision-making, your measures will be indispensable