No research is done in a void: science is constantly expanding previous hypotheses, building upon past knowledge. We live in a digital age where information is ubiquitous, yet we struggle to preserve accurate machine readable and quantitative descriptions of our research compromising our capacity to use them in our inferences. In the following talk I will show how and why we incorporate assumptions in our studies based on three experiments we have conducted: (i) dissociating metacognitive subdomains in medial and lateral anterior prefrontal cortex, (ii) relating reading comprehension to individual differences in the default mode network, and (iii) exploring neural correlates of the content and form of self-generated thoughts. This will be followed by introducing a new inference method - probabilistic Regions of Interest (pROI) - which allows the use of prior knowledge in the form of a probabilistic map. This approach provides the middle ground between ROI and full brain analysis, by giving researchers more flexibility in formalizing priors. The quality of prior probability maps based on the literature can be improved by using unthresholded statistical maps instead of peak coordinates. To facilitate this we have created NeuroVault.org - a community - wide effort to collect unthresholded statistical maps. Taking the initiative a step further I will describe the concept of data papers - publications purely dedicated to datasets. Together those three mechanisms (pROI, NeuroVault.org and data papers) are a small but significant steps towards better, more reusable and reproducible science.

1. Reusable Science:
How not to slip from
the shoulders of giants
Chris Gorgolewski
Max Planck Research Group: Neuroanatomy &
Connectivity

2. Anatomy of a giant
I. Example studies
II. Probabilistic ROIs
III.Sharing statistical maps
IV.Data papers

3. Anatomy of a giant
I. Example studies
II. Probabilistic ROIs
III.Sharing statistical maps
IV.Data papers

4. Study I
Medial and Lateral Networks in Anterior
Prefrontal Cortex Support Metacognitive
Ability for Memory and Perception
Benjamin Baird, Jonathan Smallwood, Krzysztof J.
Gorgolewski, and Daniel S. Margulies
Journal of Neuroscience (in press)

5. Meta-cognition
• Are we equally good in judging our
performance of memory or perception
tasks?
• Is metacognition related to medial or
lateral prefrontal cortex? Does it depend
on modality?

6. Measuring metacognition

7. Metacognition of memory and
perception are distinct systems

8. Sources of seed points
Gilbert et al. 2006, Functional specialization within rostral prefrontal cortex
(area 10): a meta-analysis. Journal of cognitive neuroscience

9. Sources of seed points
Fleming, S. M., Weil, R. S., Nagy, Z., Dolan, R. J., & Rees, G. (2010).
Relating introspective accuracy to individual differences in brain
structure. Science (New York, N.Y.), 329(5998), 1541–3.

10. Meta-cognition of Perception
vs.
Meta-cognition of Memory
Baird, Smallwood et al. (in press) JON

11. Double dissociation of
metacognitive abilities

12. Study II
The Default Modes of Reading: Modulation
of posterior cingulate and medial prefrontal
cortex connectivity associated with
subjective and objective differences in
reading experience
Jonathan Smallwood, Krzysztof J. Gorgolewski, Johannes
Golchert, Florence J.M. Ruby, Haakon G. Engen, Benjamin
Baird, Melaina Vinski, Jonathan Schooler, Daniel S. Margulies
Frontiers in Neuroscience (in press)

13. Reading comprehension
• What is the relation between task focus
and reading comprehension?
• What role does Default Mode Network
play in reading comprehension and task
focus?

14. Task focus is inversely correlated
with reading comprehension

15. Reading by Default
Seed locations
Andrews-hanna, J. R., Reidler, J. S., Sepulcre, J., Poulin, R., Buckner, R. L., &
Temp, P. (2010). Functional-anatomic fractionation of the brain’s default
network. Neuron, 65(4), 550–62.
Smallwood, et al., Frontiers in Human Neuroscience

16. Reading by Default
Smallwood, et al., Frontiers in Human Neuroscience

17. Reading by Default
Smallwood, et al., Frontiers in Human Neuroscience

18. Why mind-wandering may disrupt
reading

19. Study III
A correspondence between the brain's
intrinsic functional architecture and the
content and form of self-generated
thoughts
Krzysztof J. Gorgolewski, Dan Lurie, Sebastian Urchs,
Judy A. Kipping, R. Cameron Craddock, Michael P. Milham,
Daniel S. Margulies, and Jonathan Smallwood
PLoS One (submitted)

20. Mind wandering
• What is the content and form of thoughts
in mind wandering?
• How does it relate to various aspects of
intrinsic BOLD activity?

21. Questions about the content

22. Questions about the form

23. Future vs. Past = Words vs.
Images

24. Resting state measures
Group average
Fractional Amplitude of
Low Frequency
Fluctuations
Regional
Homogeneity
Degree Centrality

25. Not only Default Mode Network

26. Not only Default Mode Network

27. What these studies have in
common?

28. Anatomy of a giant
I. Example studies
II. Probabilistic ROIs
III.Sharing statistical maps
IV.Data papers

29. Signal to Noise ratio

30. Looking in the wrong places

31. Lower SNR = we miss more
stuff

32. Lower SNR = higher FDR threshold

34. How to improve power?
• stronger effects?
• fewer null/noise samples -> ROI

35. What is wrong with ROI analysis?

36. What is wrong with ROI analysis?

37. Binary nature of masks

38. Fifty
Shades
of Grey,
Matter
Fifty shades of grey
A probabilistic view on the ROI
analysis
A probabilistic approach to ROI analysis
Gorgolewski et al. PRNI 2013

48. Extensions and disclaimers
• Kernel density estimation
• Markov Random Field reguralization
• Posterior maps cannot be used in meta
analysis – circularity!
• Prior maps are integral part of the analysis
and need to be included in publications

52. Anatomy of a giant
I. Example studies
II. Probabilistic ROIs
III.Sharing statistical maps
IV.Data papers

53. Just coordinates?
• Databases such as Neurosynth or
BrainMap rely on peak coordinates
reported in papers (only strong effects)

54. Are we throwing money away?

56. Data sharing?

58. Data sharing?
• Ok, ok so we should share data.
• We all know it’s good.
• But almost no one does it.
– You have to prepare data
– You risk that your mistakes will be found!

59. “I swear I’ve heard it before”
• In the past there were many attempts to
propagate data sharing
– For example fMRI DC:
• Failed because of technical issues
• …and the amount of time it took to prepare data
for submission (a week, a very frustrating week)
• fMRI DC was however too ambitious for its
time:
– They wanted to collect raw data and all
metadata required to reproduce the analysis
Van Horn & Gazzaniga (2013). Why share data? Lessons learned from the fMRIDC. NeuroImage

60. Baby steps
• Everything is a question of cost and
benefit
– If we keep the cost low even small benefit (or
just conviction that data sharing is GOOD) will
suffice

61. NeuroVault.org
simple data sharing
• Minimize the cost!
• We just want your statistical maps with
minimum description (DOI)
– If you want you can put more metadata, but
you don’t have to
• We streamline login process (external
services such as Google, Facebook)

62. Benefits?
• In return authors get interactive web
based visualization of their statistical maps
– Something they can embed on their lab
website
• We are keeping both cost and benefit
low…
– …but we also plan to work with journal editors
to popularize the idea

65. Share your stat maps!
?
Make science more reproducibl

66. NeuroVault.org

67. Anatomy of a giant
I. Example studies
II. Probabilistic ROIs
III.Sharing statistical maps
IV.Data papers

68. Motivation
• Share your stat maps!
vs
.
institutions
scientists

69. Quality control
• Share your stat maps!
Complex datasets require
elaborate descriptions

70. Solution – data papers
• Authors get recognizable credit for their
work.
– Even smaller contributors such as RAs can be
included.
• Acquisition methods are described in
detail.
• Quality of metadata is being controlled by
peer review.

72. Where to publish data papers?
• Neuroinformatics (Springer)
• Frontiers in Human Brain Methods
(Nature Publishing
(Frontiers Media) Group)
• GigaScience (BGI, BioMed Central)
• Scientific Data (Nature Publising Group,
coming soon)

75. Read more
• Probabilistic ROIS
Gorgolewski et al. PRNI, 2013
• NeuroVault.org
Gorgolewski et al. OHBM, 2013
• Data papers
Gorgolewski et al. Frontiers in Brain Imaging
Methods, 2012

76. Acknowledgements
(my personal giants)
Pierre-Louie Bazin
Haakon Engen
Satrajit Ghosh
Russell A. Poldrack
Jean-Baptiste Poline
Yannick Schwarz
Tal Yarkoni
Michael Milham
Daniel Margulies
Benjamin Baird
Jonathan Smallwood
Johannes Golchert
Florence J.M. Ruby
Melaina Vinski
Jonathan Schooler
Dan Lurie
Sebastian Urchs
Judy A. Kipping
R. Cameron Craddock
MPI CBS Resting state group

77. THANK YOU!

78. Details

79. Details

80. Details