Presentation of the 2nd Content Providers Community Call, targeting the following topics: 1) OpenAIRE infrastructure updates; 2) Main topic: OpenAIRE Usage Statistics service: technical details and upcoming developments; 3) Community questions & comments.

1. @openaire_eu
2nd Community Call
OpenAIRE content providers managers
04/12/2019

2. 1) OpenAIRE provide updates
2) Usage Statistics Service
- service overview and specifications
- workflows and data collection
3) Questions & comments (please share
your use cases, issues)
AGENDA:
www.openaire.eu/provide-community-calls
Notes & Agenda  https://bit.ly/2rTgJwy

3. Provide – recent news
Newbrokerevents
available inprovide
dashboard (orcid…)
https://provide.openaire.eu
Newusagestatsgeneric
trackerscript(asapython
scriptforothercases)
https://github.com/openaire/Gen
eric-Matomo-Tracker
Guidelines v4compatibility
inDspace 5&6(CanadianUniv.
librariesaresupportingthedevelopmentof
anextensiontoDSpace5&6thatwillsupport
compliancewiththeOpenAIREGuidelinesv4.
www.coar-repositories.org/news-
updates/collaborating-to-support-greater-visibility-and-
discoverability-of-open-scholarship/
Provide Public Roadmap
weneedandwantyourfeedback
https://trello.com/b/JHbHKLZ4/openaire-
provide-roadmap

4. New broker events – ORCID, Software
4
Missing ORCID
Missing Software
Link Publications to Software

5. UsageStatisticsService
-serviceoverviewandspecifications
-workflowsanddatacollection
Andreas Czerniak, Bielefeld University
Dimitris Pierrakos, Athena Research and Innovation Center
Jochen Schirrwagen, Bielefeld University
Pedro Príncipe, University of Minho

6. OpenAIRE’s e-infrastructure Commons & Usage Stats sketch

7. Usage Statistics Service Features
Tracking of views and
downloads (PUSH)/
collecting COUNTER
reports (Pull)
Anonymisation of IP-
addresses.
Metadata de-
duplication enables
accumulation of views
and downloads for
same documents
COUNTER Code of
Practice compatibility.
standards based usage
statistics.
enables comparability with
statistics from other data
sources.
Thisiswhere youtypeintheevent

8. Metadata-Index
UsageStatistics-DB
● Repository
● CRIS
● eJournal
● National
Statistics Node
● Publisher
PULL
COUNTER Report
PUSH
tracked event
IP-
Anonym.
processing script
processing script
Architecture and Workflows for Usage Statistics

9. Push Usage Statistics Tracking Workflow
▪ An institutional repository registers in OpenAIRE’s Usage
Statistics Service via the Content Provider Dashboard.
▪ Server side real-time tracking: Plugins (DSpace), patches
(Eprints) using Matomo’s HTTP API.
▪ Usage Activity is tracked and logged at OpenAIRE’s
Analytics platform in real time.
▪ Generic Log file parser: A python script that parses
repository log files and sends the usage events to
OpenAIRE’s Analytics platform (not in real time).

10. Push Usage Statistics Tracking Workflow
▪ An offline workflow, based on Matomo API, transfers
and stores usage events, to OpenAIRE’s DBs for
statistical analysis.
▪ Statistics are deployed for human (OpenAIRE’s Portal)
and machine (Sushi-Lite API) consumption.

11. Enable Metrics for selected Datasource
11

12. Configure Metrics for selected Datasource
12
1223456

13. DSpace oapiwik.cfg
#------------------------#
# OpenAIRE Piwik Tracker #
#------------------------#
#Base Tracker URL
oapiwik.piwik.trackerURL = https://analytics.openaire.eu/piwik.php
#Piwik siteID
oapiwik.piwik.siteID = 1
#Piwik Authentication Token
oapiwik.piwik.tokenAuth=123456789
#IP Address Anonymization Bytes. Values in {1,2,3}
#oapiwik.piwik.ipanonymizationbytes=2
# Flag to enable/disable tracking. Default is true
oapiwik.piwik.enabled = true
Plugin/Patch Configuration
13
Eprints OAPiwik.pl
##################
# CONFIG START #
################
# Modify the following URL to the Piwik tracker location
$c->{OAPiwik}->{tracker} = "https://analytics.openaire.eu/piwik.php";
# Enter the OpenAIRE Piwik Site ID
$c->{OAPiwik}->{siteID} = "1";
# Enter the piwik token_auth
$c->{OAPiwik}->{token_auth} = "32846584f571be9b57488bf4088f30ea";
# Specify the number of bytes, 1,2 or 3, for IP Anonymization (empty for no IP
Anonymization)
$c->{OAPiwik}->{noOfBytes} = "";
# Other Config Parameters
$c->{OAPiwik}->{ua} = LWP::UserAgent->new(conn_cache => LWP::ConnCache->new,);
$c->{plugins}->{"Event::OAPiwik"}->{params}->{disable} = 0;
123456

14. 1. Install virtualenv $ pip install virtualenv
2. Create a virtual environment for the project $ virtualenv -p /usr/bin/python2.7
matomo_tracker
3. Activate the virtual environment: $ source matomo_tracker/bin/activate
4. Download COUNTER Robots List: wget
https://raw.githubusercontent.com/atmire/COUNTER-
Robots/master/COUNTER_Robots_list.json
5. Download matomo_import_logs.py and matomo.yaml to activated project folder
6. Configure matomo parameters in matomo.yaml
7. Install the following packages:
○ pip install pyyaml
8. Run the importer /usr/bin/python2.7 {logs_folder}
Generic Tracker Script Configuration
14

15. matomo_config.yaml
Matomo_Parameters:
#Matomo tracker Endpoint
matomo_url: "https://analytics.openaire.eu/"
#Repository Base Url
repository_base_url: "https://repository.com"
#Matomo Repository Site ID
idSite: 47
#Matomo authenticastion token
#token_auth: "12345"
token_auth: "12345"
#matomo parameters (default values)
delay_after_failure: 10
default_max_attempts: 3
default_socket_timeout: 300
ip_anonymization: false
Generic Tracker Script Configuration
15
# Number of recorders. It should be set to the number of CPU cores in your server.
# You can also experiment with higher values which may increase performance until a certain
point
recorders: 1
#OAI-PMH Configuration
oaipmh_preamble: "oai:repository.com:"
oaipmh_regex: "/[0-9]*"
#Metadata Location (regexp supported)
tracking_metadata:
#- "/[0-9]*"
#- "^((?!.*pdf.*).)*$"
- "/handle/[0-9]*"
#- "/record"
#- "/publication/"
#Item Location (regexp supported)
tracking_download:
#- "pdf"
#- "/[0-9]*"
- "/bitstream/handle/[0-9]*"
#- "/download"

16. ▪ Gathering of consolidated statistics reports from
aggregation services, such as IRUS-UK, using protocols
such as SUSHI-Lite.
▪ Statistics are stored to OpenAIRE’s DB for statistical
analysis.
▪ Statistics are deployed via OpenAIRE’s Portal or Sushi-
Lite API.
Collecting (Pull) Consolidated
Usage Statistics Reports

17. • 61 repositories tracked with Matomo ID
• 78 repositories via IRUS-UK
• 10 journals via SARC-OJS
Number of Platforms using Metrics Service actively
17

18. Metrics Service Evolution
18

19. Metrics Service Statistics
19

20. User-Interface in the Content Provider Dashboard
20

21. Aggregated and Combined Statistics of a Document from
different Repositories
21

22. Supported Report Types (COUNTER R4)
22

23. Retrieve Usage Statistics via API
23
based on
http://services.openaire.eu/usagestats/sushilite/
Supports COUNTER R4 compatible reports:
○ Article Reports (AR) and Book Reports (BR)
using identifiers like openaire, doi, oai-
record-id
○ Item Reports (IR)
○ Repository Reports (RR) using identifiers
issued by OpenAIRE or OpenDOAR
○ Journal Reports (JR) using identifiers like
ISSN
COUNTER USAGE REPORTS
RELEVANT FOR OPENAIRE
Repository Platform Report – RR1
Journal Report – JR1
Item Report – IR1
Article Report – AR1
Book Report – BR1
Book Chapter Report – BR2
Research Dataset Report tbc.
Research Software Usage Report tbc.

24. 24
SUSHI response example (JSON)
Repository Report Item Report

25. • Enhance the visualization tools in the Dashboard.
• Offer “snippets” to embed the Metrics statistics in the
repositories.
• Support CoP for Research Data.
• Update to CoP Release 5.
• Setting up a Usage Statistics Hub (OpenAIRE, LA
Referencia, IRUS-UK, ...)
Next Steps
25

26. In 2020: Usage Statistics in Extended Contexts
26
Open Research Analytics Service
Usage
Statistics
Transactio
nal Data
Analytics
Interface
● Combining different types of data to provide insights of OA-Transformation
● a cooperation between OpenAIRE, OpenAPC and Knowledge Unlatched

27. Thank you!
www.openaire.eu/provide-community-calls
Andreas Czerniak, Bielefeld University
Dimitris Pierrakos, Athena Research and Innovation Center
Jochen Schirrwagen, Bielefeld University
Pedro Príncipe, University of Minho