4. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
RESEARCH DATA
ALLIANCE
BIBLIOMETRICS FOR
DATA
SURVEY RESULTS
• How do researchers currently measure the impact
of their data?
• Organised by the RDA Publishing Data
Bibliometrics group, 2014
• Slides adapted from Sarah Callaghan
5. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
4) What do you currently use to evaluate the impact of data?
0 10 20 30 40 50 60 70
Nothing
Data citation counts
Downloads
Social media (likes/shares/tweets)
Mentions in peer-reviewed papers
Hits in search engines
Mentions in blogs
Bookmarks in Zotero and/or Mendeley
Other (please specify)
n=115
6. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
5) If you don't use anything to evaluate the impact of data, why?
• 33 responses:
– Majority opinion: current metrics not good enough, no
standards, don’t know what to do
– Other opinions:
• Impact metrics not important for respondee
• Interest in quantifying impact, but repository/policies
still under development
• Metrics are too easily gamed, or too complicated
7. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
6) Are the methods you use to evaluate impact adequate for
your needs?
31.5%
68.5%
Are the methods you use to evaluate impact adequate for your needs?
Yes
No
n=111
8. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
7) Why do you want to evaluate the impact of data?
0 10 20 30 40 50 60 70 80 90
Funder requirements
Promotion/tenure/job
evaluation
Understanding the
quality of the data
Curiosity
Interested in the reuse
of data
Other (please specify)
n=111
9. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
8) In the future, what would you like to use to evaluate the
impact of data?
• CITATIONS! (and better tools to track them)
• Downloads
• Altmetrics / “anything and everything”
• Peer review / community feedback
• Use outside scholarly literature (e.g., in patents)
• Reuse / “actual use”
10. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
9) What is currently missing and/or needs to be created for
bibliometrics for data to become widely used?
• STANDARDS!
• Data Citation
• Consistent use of PIDs / DOIs
• Culture Change / “A belief that they are valid.”
11. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
To return control of publishing to researchers, providing them
with the infrastructure and support to advance publishing in
ways that legacy publishers are not willing to do.
About Ubiquity Press
Background
Mission
Spun out of University College London in 2012
Researcher-led
Extensive publishing background as well
(BioMed Central, PLoS, Elsevier, IOP)
Based in London
Comprehensive approach: journals,
books, data, software, hardware, wetware….
12. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
The Social Contract
of Science
• Validation
• Dissemination
• Further development
Scientific Malpractice
• Data
• Results
• Software
• Hardware, wetware…
#@%$#@
% #@%$#
Source: http://www.smbc-comics.com/index.php?db=comics&id=2015
15. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
The basics of the UP data journal model
Data papers are short
1) Low barrier data publication
Peer review is quick and objective
2) Online authoring
Low APC: £100 / €130
Lower cost (straight to XML)
Encourages shorter form
3) Open access only (CC-BY)
4) The publisher is not the repository
No-questions-asked waivers
17. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
1. The paper contents
a. The methods section of the paper must provide
sufficient detail that a reader can understand how
the resource was created.
b. The resource must be correctly described.
c. The reuse section must provide concrete and useful
suggestions for reuse of the reuse.
2. The deposited resource
a. The repository must be suitable for resource
and have a sustainability model.
b. Open license permits unrestricted access (e.g. CC0),
or access guaranteed if criteria met (must qualify)
c. A version in an open, non-proprietary format.
d. Labeled in such a way that a 3rd party can make
sense of it.
e. Must be actionable.
Peer review
27. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
• No single way to cite data, but good guidelines
available (e.g. Force 11)
Data citation: better publisher guidelines
Alexander NS, Wint W (2013) Data from: Projected population
proximity indices (30km) for 2005, 2030 & 2050. Dryad
Digital Repository. http://dx.doi.org/10.5061/dryad.12734
• Journal must have clear guidelines about how to
cite data, e.g.:
• Creators, date of publication, host repository,
version, persistent identifier
• Must be included in reference list
28. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
• Publishers need to provide better guidelines for
copyeditors:
Data citation: better copyediting
• Make sure journal guidelines for data
citation are being followed
• Go back to authors if no citation included
• Fix incorrect citations (e.g. simple hyperlinks
in text)
29. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
• Many data reuse scenarios involve locating,
querying and recombining data from a large
number of sources
Machine Readable Citations
• This can be made significantly easier by
making data citations machine readable
• Enables locating of data via text mining of
relevant literature
• Two possible methods – XML and RDF
30. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
• Journal Article Tag
Suite (JATS)
maintained by
NISO used by most
publishers
XML
• JATS currently
recommends
tagging data
references as web
publications
<ref>
<element-citation publication-type="database"
publication-format="web">
<source>Database of Human Disease Causing
Gene Homologues in Dictyostelium Discoideum
[Internet] </source>
<publisher-loc>San Diego (CA)</publisher-loc>
<publisher-name>San Diego Supercomputer
Center</publisher-name>
<year>2003</year>
<date-in-citation>cited 2007 Feb 2
</date-in-citation>
<comment>Available from: <uri>http://
dictyworkbench.sdsc.edu/HDGDD/</uri>.
</comment>
</element-citation>
</ref>
• Not ideal, but
available now
31. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
• Several proposals for improvements with more
suitable terms:
XML
<name> -> <collab collab-type="curators">
<source> -> <data-title>
<edition> -> <version>
<license>
• NISO-JATS Data Citation Implementation Workshop
held at the British Library in June 2014
• Force11 Data Citation Implementation Group
• <JATS4R> publisher group
• E.g.:
32. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
• Can make data not only discoverable through citation,
but also the relationship of it to the research.
RDF
• JATS to RDF provides a start for this, but publishers
still need to make data citations more specific
<rdf:Description rdf:about="reference-item-4"><co:index>4</co:index></rdf:Description><rdf:Description rdf:about="reference-4"><dcterms:bibliographicCitation>
Oldenburg, H (1665). </dcterms:bibliographicCitation></rdf:Description><rdf:Description rdf:about="reference-4"><rdf:type
rdf:resource="http://purl.org/spar/biro/BibliographicReference"/></rdf:Description><rdf:Description rdf:about="reference-
4"><dcterms:identifier>b4</dcterms:identifier></rdf:Description><rdf:Description rdf:about="reference-4"><biro:references rdf:resource="reference-4-textual-
entity"/></rdf:Description><rdf:Description rdf:about="textual-entity"><cito:cites rdf:resource="reference-4-textual-entity"/></rdf:Description><rdf:Description
rdf:about="reference-4-textual-entity"><rdf:type rdf:resource="http://purl.org/spar/fabio/Expression"/><frbr:realizationOf rdf:resource="reference-4-conceptual-
work"/></rdf:Description><rdf:Description rdf:about="reference-4-textual-entity"><rdf:type
rdf:resource="http://purl.org/spar/fabio/JournalArticle"/></rdf:Description><rdf:Description rdf:about="reference-4-conceptual-work"><dcterms:creator
rdf:resource="reference-4-agent-1"/></rdf:Description><rdf:Description rdf:about="reference-4-agent-1"><rdf:type
rdf:resource="http://xmlns.com/foaf/0.1/Person"/></rdf:Description><rdf:Description rdf:about="reference-4-agent-
1"><foaf:familyName>Oldenburg</foaf:familyName></rdf:Description><rdf:Description rdf:about="reference-4-agent-
1"><foaf:givenName>H</foaf:givenName></rdf:Description><rdf:Description rdf:about="reference-4-textual-
entity"><fabio:hasPublicationYear>1665</fabio:hasPublicationYear></rdf:Description><rdf:Description rdf:about="reference-4-textual-entity"><dcterms:title>Epistle
Dedicatory</dcterms:title></rdf:Description><rdf:Description rdf:about="reference-4-textual-entity-source"><dcterms:title>Philosophical Transactions of the Royal Society
of London</dcterms:title></rdf:Description><rdf:Description rdf:about="reference-4-textual-entity"><frbr:partOf rdf:resource="reference-4-textual-entity-
source"/></rdf:Description><rdf:Description rdf:about="reference-4-textual-entity"><frbr:partOf rdf:resource="periodical-volume-reference-4-textual-
entity"/></rdf:Description><rdf:Description rdf:about="periodical-volume-reference-4-textual-entity"><rdf:type
rdf:resource="http://purl.org/spar/fabio/PeriodicalVolume"/><prism:volume>1</prism:volume><frbr:partOf><rdf:Description><rdf:type
rdf:resource="http://purl.org/spar/fabio/Periodical"/></rdf:Description></frbr:partOf></rdf:Description><rdf:Description rdf:about="reference-4-textual-
entity"><frbr:embodiment rdf:resource="digital-embodiment-d1e2589"/></rdf:Description><rdf:Description rdf:about="digital-embodiment-
d1e2589"><prism:startingPage rdf:resource="0"/></rdf:Description><rdf:Description rdf:about="reference-4-textual-
entity"><prism:doi>10.1098/rstl.1665.0001</prism:doi></rdf:Description>
4. Oldenburg H (1665). "Epistle Dedicatory". Philosophical Transactions of the Royal Society
of London 1: 0–0. doi:10.1098/rstl.1665.0001.
33. brian.hole@ubiquitypress.com www.ubiquitypress.com / @ubiquitypress
RDF
<a about="http://dx.doi.org/10.5334 /jophd.ab"
rel="cito:Uses_Data_From"
href="http://dx.doi.org/10.5061/dryad.12734”>http://
dx.doi.org/10.5061/dryad.12734</a>
• The Citation Typing Ontology (CiTO) is available
now, and makes the relationship explicit: