Thinking about Open Science practices, data sharing and lifetime, and communication from Climate Scientists. Slides based on a presentation given at the Lunchtime talk sessions from the MetOS Section, Department of Geosciences, University of Oslo, November 12th 2015.

1. From byte to mind
Thinking about Open practices, data sharing and
lifetime, and communication for Climate Scientists
@benlaken www.benlaken.com

2. What do you think when you hear ‘Open Science’ ?
Community/Network inc. many NPOs to spread technical skills
Responsible data management/archival for
long-term data preservation
Data is archived and accessible (not just on request)
Free-to-access journals (no paywall)
Code is open, archived, version controlled, and peer reviewed
Optimised for reproducible work and rapid improvement
Lowers/removes cost barrier of software tools
Accountable peer-review process, and even post-pub review

3. J.M. Wicherts, M. Bakker, & D. Molenaar: "Willingness to Share Research Data Is Related to the Strength of the
Evidence and the Quality of Reporting of Statistical Results". PLoS ONE, 6(11): e26828, 2011, doi:10.1371/
journal.pone.0026828.
Widespread reluctance to share published research data is often hypothesized to be due to the authors' fear that reanalysis
may expose errors in their work or may produce conclusions that contradict their own. However, these hypotheses have not
previously been studied systematically.
We related the reluctance to share research data for reanalysis to 1,148 statistically significant results reported in 49
papers published in two major psychology journals. We found the reluctance to share data to be associated with weaker
evidence (against the null hypothesis of no effect) and a higher prevalence of apparent errors in the reporting of
statistical results. The unwillingness to share data was particularly clear when reporting errors had a bearing on
statistical significance.
Our findings on the basis of psychological papers suggest that statistical results are particularly hard to verify when reanalysis
is more likely to lead to contrasting conclusions. This highlights the importance of establishing mandatory data archiving
policies.
“…the unwillingness to share data was most pronounced when the errors concerned
statistical significance.”
Immediate value of Open Science: transparency

4. “…most did not follow proper archiving standards (e.g., by keeping
code books and writing meta-data [37]), but simply stored data on
their own (current) computer (32%), on CDs/DVDs (18%), or on the
shelf (20%). Haphazard data management is documented in a
number of scientific fields [37], [38], [39], may result in errors in
analyzing and reporting of results, and obviously impedes the
sharing of data after results are published.”
Openness tied to the fate of data: Archiving
Archiving strategies are for more than just reliability…
Default academic archival strategy

5. “Digital information lasts forever—or five years, whichever comes first.”
Jeff Rothenberg, 1999, Ensuring the longevity of digital information
Digital information has a life cycle
A baby called Data
…5 yrs…
If your work/data has not ‘reproduced’ itself within a short(ening) timeframe, it will be lost.
Due to both ‘lifecycle’ of scientific work
(e.g. see “Attention decay in science” http://arxiv.org/pdf/1503.01881.pdf)
and the technological challenges of ensuring reproducibility over the long-term…

6. A computer program circa 1970
http://celebrating200years.noaa.gov/magazine/tsunami_database/NOAApunchcard.html

7. 1955
Program (and programmer):
62,500 punched cards
for SAGE military computer network
(a 5mb program)
http://www.computerhistory.org/revolution/memory-storage/8/326/924
…a larger program…
And what about the data
these programs were
using?…

8. UK Met Office, 1960’s, climate data storage on magnetic tape reels
http://www.sciencephoto.com/media/435868/view
Sizes
of ~5 to 140mb
from ~1950s to 1970’s

9. Causes of death for data: Trapped on obsolete storage media and operating
systems. Closed source/propriety file types dependent on unsupported systems.
Or even just long-term degradation on storage media (‘data rot’).
1988
1983
Open Science encourages data preservation/archiving:

10. “The primary barrier to computational reproducibility in many domains has
nothing to do with the technological approaches …, but stems rather from a
reluctance to publish the code used in generating the results in the first place.”
Barnes, N. (2010). Publish your computer code: it is good enough. Nature. 467, 7317, 753-753
“Despite extensive evidence to the contrary, many researchers and journals
continue to assume that summary descriptions or pseudo-code provide a
sufficient description of computational methods used in data gathering,
processing, simulation, visualisation, or analysis.”
Boettiger, C (2014) An introduction to Docker for reproducible research. ArXiV: 1410.0846v1
Like archiving data, coding practices are crucial
NPO to encourage use of reproducible computer skills
and Open Science practices among Scientists.
Modern scientists are now essentially software developers

11. Being asked to review a paper with no code or data attached
Requests for data/code of a paper from corresponding authors
partially addressed or just ignored
Failure to reproduce results from a paper due to poor/partial
documentation of methods
Published results you don't find credible
Problems reproducing your own past work after some time, due to lost
data/no documentation, or obsolescence of hardware software!
Code/models/data are lost with people on short-term contracts moving?
Common symptoms of ‘closed’/traditional science
Fortunately, all these nasty symptoms have a simple cure:
adopting Open Science practices!

12. Wider research communication: philosophy and methods
Related to OS, the spreading of your ideas to wider audience. This needs
some philosophical consideration in Climate Science, as it can get messy…

13. Pew research centre (2009)
http://www.people-press.org/2009/07/09/section-4-scientists-politics-and-religion/
Surveys of both gen. public and Scientists show high approval for Scientists to
advocate on ‘political’ issues
Do you think you should be involved in a public discussion?

14. The public has high levels of ‘trust’ in Scientists
Source IPSOS MORI (2014)
https://www.ipsos-mori.com/researchpublications/researcharchive/3504/Politicians-trusted-less-than-estate-agents-bankers-and-journalists.aspx
Consistent question since 1983, asked to UK public, “Would you
generally trust them to tell the truth or not?”

15. You have your own bias, what you consider important.
These value judgements influence what you value in life,
pursue scientifically, and how you interpret objective facts.
Supporting this unique mix of values is advocacy.
The support of your normative values.
Advocacy
What this means:
Connection between Science, Open Science,
communication, and advocacy
(Normative value —> a claim about how things ought to be)
Adapted from Gavin Schmidts 2013 Stephen Schneider lecture at AGU

16. There is an idea that
Advocates and Scientists
are/ (and should be kept)
distinct.
http://smbc-comics.com/index.php?db=comics&id=2088#comic

17. See if you agree with the following statements and would publicly support them:
Anthropogenic emissions should be limited by international regulations.
Geoengineering should be seriously considered.
Nuclear and solar power should become the primary source of
power generation globally.
Vegetarianism and Entomophagy should be encouraged to
ease climate impacts of animal agriculture.
Funding for Scientific Research should be a higher priority.
Developing nations should not be allowed to invest in coal technology.
Scientists should communicate more about what they do and find.
Adapted from Gavin Schmidts 2013 Stephen Schneider lecture at AGU
Public understanding of Science should be encouraged.
A personal case study from your feelings on hot-button issues
(n.b. Entomophagy = practice of eating insects)

18. Judith Curry (2012), http://judithcurry.com/2012/09/10/too-much-advocacy/
Some typical criticisms
“But by advocating policy positions- overtly or by stealth - scientists may be forfeiting
their privileged positions as scientists and becoming just ordinary guys with opinions,
and in the process, undercutting the credibility of their scientific work.”
Problems with advocacy
Seen as a threat to scientific objectivity
Politicization may damage reputation of scientists/scientific bodies
Potentially seen as abuse of a position of trust
Jeopardy to employment/job security
Dichotomy between science telling us what is, and the scientist telling us what to value?
Science doesn’t tell us the answer to our problems. Neither should scientists.
Tasmin Edwards, (2013) The Gaurdian, http://blogs.plos.org/models/climate-scientists-must-not-advocate-particular-policies/
Summary of perceived issues:
The previous test should have shown you that not all types of
advocacy are viewed equally: most people are happy to
support advocating more funding, but some are not happy to
support climate scientists advocating policies.
(Yet, generally, scientists see no problem advocating for:
Science, Funding, public understanding, and even themselves…)

19. Everyone has value positions (consciously and unconsciously), the
keys to being a responsible advocate are:
a. Making your value positions conscious
b. Making them explicit
c. Not letting your value positions distort your subjective priors (we
need a community to do this effectively)
d. Defending your value positions separately from arguments of
probabilities and consequences
Some Scientists that that it is impossible to do these things, and
instead say they will be neutral, but no one is.
S. Schnider’s rules for responsible advocacy
Steven Schnider (1996)

20. Gavin Schmidt, (2015), What should
climate scientists advocate for?
Bulletin of the Atomic Scientists, doi:
10.1177/0096340214563677
Stephen Schneider (2009) Is the
Scientist-Advocate an Oxymoron?.
Diss. Stanford University
“What's the use in having developed science well enough to make predictions
if all we are willing to do is sit around and wait for them to come true? ”
More on this topic:
In the Climate Science community many peoples attitudes
to these questions are likely summed up by this quote:
Prof. Sherwood Rowland, on his discovery CFCs damage the ozone layer said:

21. Steps to disseminate your scientific work and values
1. Get a unique researcher ID (many journals now use this)
2. Create profiles on researcher-based social networks
3. Generate some original content from code,
presentations, posters, papers and reviews.
4. Spread the word/join conversations, and see many benefits: e.g.
feedback, research awareness, citations, wider networks, more
impact, contribute to cultural conversation…