Science progresses by building upon previous research. Progress can be most rapid, efficient, and focused when raw datasets from previous studies are available for reuse. To facilitate this practice, funders and journals have begun to request and require that investigators share their primary datasets with other researchers. Unfortunately, it is difficult to evaluate the effectiveness of these policies. This study aims to develop foundations for evaluating data sharing and reuse decisions in the biomedical literature by developing tools to answer the following research questions, within the context of biomedical gene expression datasets: What is the prevalence of biomedical research data sharing? Biomedical research data reuse? What features are most associated with an investigator’s decision to share or reuse a biomedical research dataset? Does sharing or reusing data contribute to the impact of a research article, independently of other factors? What do the results suggest for developing efficient, effective policies, tools, and initiatives for promoting data sharing and reuse? I suggest a novel approach to identifying publications that share and reuse datasets, through the application of natural language processing techniques to the full text of primary research articles. Using these classifications and extracted covariates, univariate and multivariate analysis will assess which features are most important to data sharing and reuse prevalence, and also estimate the contribution that sharing data and reusing data make to a publication’s research impact. I hope the results will inform the development of effective policies and tools to facilitate this important aspect of scientific research and information exchange.

1. Prevalence and Patterns of
Biomedical Research Data
Sharing and Reuse
Heather Piwowar
Department of Biomedical Informatics
University of Pittsburgh
JCDL Doctoral Consortium
June 2008
except clipart

7. $$

10. ?

11. Is it working? Is it worth it?
Are scientists sharing their data?
Are other scientists reusing the data?
Who, if anyone, is benefiting from the
policies, tools, and initiatives?

12. We cannot manage
what we do not measure

13. Dissertation Objective:
Evaluate the patterns and prevalence
of biomedical research data sharing
and reuse

14. Prior work in this area
• Surveys
• Manual audits
• Automated classification of citation
contexts
See http://www.citeulike.org/user/hpiwowar for bibliography

15. Missing:
a study of data sharing and reuse
behavior and impact
based on a broad spectrum of instances

16. Dissertation Research Questions
1. prevalence of sharing and reuse
2. patterns of sharing and reuse
3. affect of sharing and reuse on impact
4. implications of findings

18. Data type
• Gene expression microarrays
http://en.wikipedia.org/wiki/DNA_microarray
http://en.wikipedia.org/wiki/Image:Heatmap.png

19. Sharing type
• Openly online, mentioned in publication
– PubMed
– filtered with MeSH terms for gene-expression
– English, machine-readable full text
– 2000-2007

20. Dissertation dataset
Article ID Link to full
text
234 http://…
456 http://…
657 http://…
897 http://…

21. 1. What is the prevalence
of biomedical research data sharing?
of biomedical research data reuse?

22. Endpoints
• Three endpoints:
– Does this study produce raw data?
– If so, does this study share the raw data?
– Does this study reuse others’ raw data?

23. Example text cues
• Sharing
– “our data has been deposited in the GEO
database”
– “the microarray expression values from this
study are available at the following website”
• Reuse
– “using the data of Smith et al, we… ”
– “we downloaded four datasets from…”

24. Identification of endpoints
• Train and evaluate a Natural Language
Processing system to recognize endpoint
cues within full text
• Performance to be summarized as
precision and recall with confidence
intervals
Pilot data for NLP identification of sharing: Piwowar and Chapman, submitted to AMIA 2008.

25. Research Question 1: Prevalence
| Endpoints |
Article ID Link to full Produces Shares data? Reuses
text data? data?
234 http://… TRUE TRUE FALSE
456 http://… TRUE TRUE TRUE
657 http://… TRUE FALSE FALSE
897 http://… FALSE n/a TRUE
Calculate Percentages

26. 2. What features are most associated
with an investigator’s decision
to share or reuse a biomedical
research dataset?

27. • Features to include:
– Journal Impact Factor
– Number of Authors
– Are the samples from humans
– Subdiscipline
– Strictness of Journal policy on data sharing
– Institution
– Year of publication
– …
Pilot data for journal policies: Piwowar and Chapman, ELPUB 2008.

28. | Endpoints | | Covariates |
Article Link to Produces Shares Reuses Journal # Human
ID full data? data? data? Impact Authors Data
text Factor
234 http://… TRUE TRUE FALSE 1.5 2 TRUE
456 http://… TRUE TRUE TRUE 23.5 1 FALSE
657 http://… TRUE FALSE FALSE 2.4 6 FALSE
897 http://… FALSE n/a TRUE 0.6 2 TRUE
Journal Human
Impact Factor # Authors Data …
Shares data? Reuses data?
Multivariate logistic regressions

29. 3. Does sharing or reusing data
contribute to
the impact of a research article,
independently of other factors?

30. Assumption:
citation count is a proxy for research impact

31. | Endpoints | | Covariates |
Article … Produces Shares Reuses … … … Number of
ID data? data? data? Citations
234 TRUE TRUE FALSE 1.5 2 TRUE 0
456 TRUE TRUE TRUE 23.5 1 FALSE 4
657 TRUE FALSE FALSE 2.4 6 FALSE 4
897 FALSE n/a TRUE 0.6 2 TRUE 5
Journal Human
Impact Factor # Authors Data …
Shares data? Reuses data?
Number of Citations
Multivariate linear regressions
Pilot data on citation impact for sharing: Piwowar, Day and Fridsma, PLoS ONE 2007.

32. 4. What do the results suggest for
developing efficient, effective policies,
tools, and initiatives for promoting
data sharing and reuse?

33. We might discover, for example:
• Lots of sharing for non-human data
• All reuse within the first 5 years
• Journal and funder requests are ineffective

34. Significance
• Dataset
– social network analysis, simulation, …
• NLP Classifiers
• Best-practice patterns, communities
• Novel research connections
• Inspire further work in this area
– policy evaluation, reusability metrics
– citations for data (Data Reuse Registry)
Piwowar, Chapman. Envisioning a Data Reuse Registry. Poster submitted to AMIA 2008.

35. Limitations
• Causation?
• Other data types?
• Other sharing mechanisms?

36. “Does anyone want your data?
That’s hard to predict[…]
After all, no one ever knocked on your
door asking to buy those figurines
collecting dust in your cabinet before you
listed them on eBay.
Your data, too, may simply be awaiting an
effective matchmaker.”
Got data? Nature Neuroscience 10, 931 (2007)

37. My data is here
www.dbmi.pitt.edu/piwowar
I urge you to share yours, too.

38. Thank you
Funding: NLM informatics training grant
Advisor: Dr. Wendy Chapman
Committee: Dr. Ellen Detlefsen
Dr. Madhavi Ganapathiraju
+ JCDL Funding and Reviewers!
Questions, Comments, or
Suggestions?
except clipart