With ever-shrinking library budgets it is more essential than ever to ensure that the library collection is targeted, relevant and well-used. Return on Investment (ROI) has become the mantra of library management and libraries need to show accountability for collection decisions. This webinar will focus on speakers who have successfully implemented assessment metrics (such as COUNTER 3, Eigenfactor and impact factors) as one determining factor of collection development decisions.
APM Welcome, APM North West Network Conference, Synergies Across Sectors
NISO Webinar: Assessment Metrics
1. NISO Webinar: Assessment Metrics
December 14, 2011
Oliver Pesch
Chief Strategist, E-Resources
EBSCO Information Services
2. • What’s new with Release 4 of the COUNTER Code of
Practice
• How the COUNTER and SUSHI XML Schemas are
affected
• Facilitating implementation with the COUNTER-SUSHI
Implementation Profile
• Ensuring Compliance through a more robust audit
• Timing
4. One Code of Practice
• Covers books, databases, journals and multimedia
collections
• Easier to manage for COUNTER
• Easier to manage for content providers –
particularly those offering multiple resource types.
5. Multimedia Report 1
• Reports on use of items in multimedia collections
• Applies to platforms like ArtStor
Consortium Report 3
• The multimedia report for consortia
Journal Report 5
• Usage of journal articles broken out by year of
publication
• No longer optional
6. DOI and Proprietary Identifier
• Improves identification of items listed in the reports
• Improves matching usage to the correct items in a
knowledge base
• Involves coordination with KBART so the same
identifier is on both the vendor’s Title List and their
COUNTER report
• Applies to most COUNTER reports
7. Institutional Identifier
• Utilizes the NISO I2 as the standard identifier for the
institution (being implemented as ISNI for
organizations)
• Improves comparative analysis– particularly for
usage processed by consortium
• Applies to all reports
8.
9. Gold Open Access
• Covers articles accessible without subscription
• Included as a separate count in Journal Report
1and Journal Report 1a
10. Result Clicks and Record Views
• Result Click counts each time a user clicked a link
from the search result to the detailed view, a link
resolver, etc.
• Record View counts the abstract/detailed records
explicitly viewed from the database
• Both indicate an explicit expression of interest in the
result
• Provides a more direct measure of database usage,
in particular for A&I databases
• Applies to Database Reports 1 and 3
11. Access Denied: content item not licensed
• In addition to Turnaway s (simultaneous user limit)
• Records a user’s attempt to access full text for which
their institution does not have a subscription
• Applies to “Access Denied” reports:
o Journal Report 2
o Database Report 2
o Book Reports 3 and 4
• Useful input for collection development
12. Sessions
• Federated Search and Discovery products have
made “Sessions” an unreliable measure of use
• Reports affected:
o Database Report 1
o Database Report 3
o Consortium Report 2
14. COUNTER Schema
• Minor changes only
• ItemIdentifier, ItemPublisher now optional elements
• PubYr attribute ( used by Journal Report 5) now
defined as a year.
• PubYrFrom and PubYrTo attributes added to allow
Journal Report 5 to report ranges of years.
15. COUNTER Enumeration Schema
• Contains the controlled vocabulary for elements like
ItemDataType, Category, MetricType, etc.
• Added new values as required by the new and
updated reports
18. Why
• SUSHI and COUNTER tend to be abstract
• Abstraction leaves the opportunity for concepts to
be interpreted differently
• Different interpretations leads to inconsistencies
• Examples:
o Different authentication methods that require custom development
o Wrong capitalization on controlled data values
o Including “Totals” in the XML
o Including invalid ISSNs (e.g. “N/A”)
19. Authentication for SUSHI Client
• IP and use of requester and customer IDs allowed
• Use of SOAP extensions not allowed
20. Report Names
• Must exactly match the value in the “Name”
column on the SUSHI Reports Registry
Date Ranges for SUSHI Request
• Format must by yyyy-mm-dd
• Begin date must be first of the month
• End date must be last day of the month
• End date must be later than begin date
21. SUSHI Exceptions
• Standard error numbers must be used for standard
error conditions (see Table 17 in the SUSHI standard)
SUSHI Server Registry
• SUSHI Server must be registered on SUSHI Server
Registry
22. Data to Return in the COUNTER Report
• Only the months of usage asked for
• Include exception if some or all usage not available
• Only include items appropriate to the report (e.g.
don’t return book usage in a Journal Report)
• Do not include totals for the report or an item
23. Data Elements Values
• Use registered values with proper casing for
enumerated values (Item Data Type, Category,
Metric Type, etc.)
• Use values appropriate for the report
24. Designed for use by implementers
• Prescriptive in nature
• Lots of tables and charts
25. Designed for use by implementers
• Prescriptive in nature
• Lots of tables and charts
27. • SUSHI and COUNTER XML will receive more focus in
the audit
• COUNTER-SUSHI Implementation Profile should server
as a guide for auditors
• Auditors will be expected to use automation to
reduce the cost of the audit
29. January 20, 2012
• Last day for comments on the draft of release 4 of
COUNTER Code of Practice and the COUNTER-SUSHI
Implementation Profile
March/April 2012
• Final publication of Release 4 of COUNTER CoP and
the COUNTER-SUSHI Implementation Profile
December 31, 2013
• Deadline for compliance with Release 4 of the
COUNTER CoP
30. COUNTER CoP R4 http://www.projectcounter.org/code_practice.html
COUNTER-SUSHI http://www.niso.org/apps/group_public/document.php?
Implementation Profile document_id=7639&wg_abbrev=sushimaintenance
COUNTER Schema Date http://www.niso.org/workrooms/sushi/values_R4
Element Values
KBART http://www.uksg.org/kbart
NISO SUSHI website http://www.niso.org/workrooms/sushi
SUSHI Reports Registry http://www.niso.org/workrooms/sushi/reports_R4/
SUSHI Server Registry https://sites.google.com/site/sushiserverregistry/
Thank you