Brief slide presentation from Roundtable at European Health Psychology Conference, Bordeaux, France, July 17, 2013

1. Provocations Roundtable,
EHPS, Bordeaux, 2013
Most Findings in Health
Psychology
Are Not Believable
James C. Coyne
University of Groningen, University Medical Center
Groningen (UMCG), The Netherlands
@CoyneoftheRealm

2. Psychology and psychiatry articles
5 times more likely to report
positive findings than articles in
physics and other hard sciences.

3. Can we trust what we read in
journals?
Ioannidis, J. P. (2005). Why most published
research findings are false. PLoS medicine,
2(8), e124.
Shun-Shin, M. J., & Francis, D. P. (2013). Why
Even More Clinical Research Studies May Be
False: Effect of Asymmetrical Handling of
Clinically Unexpected Values. PLOS ONE,
8(6), e65323.

4. Copyright ©2005 American Heart Association
Modified from Loscalzo, J. Circulation 2005;112:3026-3029
Incentives and Pressures ForIncentives and Pressures For
Confirmatory BiasConfirmatory Bias

5. Ioannidis, J. (2012). Scientific inbreeding
and same-team replication: Type D
personality as an example. Journal of
Psychosomatic Research.
 Obedient replication: Investigators feel that
the prevailing views are so dominant that
finding consistent results is a sign of being a
good scientist and there is no room for
dissenting results and objections.

6. Ioannidis, J. (2012). Scientific inbreeding
and same-team replication: Type D
personality as an example. Journal of
Psychosomatic Research.
 Obliged replication: Proponents of dominant
view are so strong in controlling the
publication venues that they can largely
select and mold the results, wording, and
interpretation of studies eventually published.

7. Telling It Like It Ain’t: How toTelling It Like It Ain’t: How to
Succeed in Health PsychologySucceed in Health Psychology

Have Lots of Endpoints and Ignore
Negative Results in Main Analyses of
Primary Endpoints.

Favor Secondary Analyses, Subgroup
Analyses, and Endpoints Developed Post
Hoc over Negative Findings for Primary
Analyses.

Ignore Methodological Shortcomings that
Would Make Trial or Meta Analyses
Invalid.

8. Telling It Like It Ain’t: How toTelling It Like It Ain’t: How to
Succeed in Health PsychologySucceed in Health Psychology
• Present Negative Findings as if Positive in
Subsequent Publications, Exaggerate
Positive Findings.

Assess Multiple Endpoints, Treat any
Significant Finding as a Replication of Past
Findings.

Create a False Consensus and Seeming
Unanimity in the Literature by
Cherrypicking Findings that can be
Construed as Positive, Ignoring the Rest.

9. Do we modify Sackett’s definition of evidence-
based medicine as "the conscientious, explicit
and judicious use of current best evidence in
making decisions about the care of individual
patients”
To
Evidence-based health psychology is ‘the careful
selection, analysis, and interpretation of results to
put theories and interventions in the best
possible light, upholding the credibility of the
field.’

10. Change what is viewed as “for
the good of the field”
 Need to stop viewing producing evidence
upholding dominant theories and efficacy of
interventions as overarching goal of
publication.
 Need to recognize null findings,
nonreplication, and criticism of dominant
views as accomplishments in need of
protecting and publishing.