The multiplicity of analysis strategies jeopardizes replicability: lessons learned across disciplines.
interdisciplinary perspective
metaresearch
open science
replicability crisis
uncertainty
Journal
Royal Society open science
ISSN: 2054-5703
Titre abrégé: R Soc Open Sci
Pays: England
ID NLM: 101647528
Informations de publication
Date de publication:
21 Apr 2021
21 Apr 2021
Historique:
entrez:
17
5
2021
pubmed:
18
5
2021
medline:
18
5
2021
Statut:
epublish
Résumé
For a given research question, there are usually a large variety of possible analysis strategies acceptable according to the scientific standards of the field, and there are concerns that this multiplicity of analysis strategies plays an important role in the non-replicability of research findings. Here, we define a general framework on common sources of uncertainty arising in computational analyses that lead to this multiplicity, and apply this framework within an overview of approaches proposed across disciplines to address the issue. Armed with this framework, and a set of recommendations derived therefrom, researchers will be able to recognize strategies applicable to their field and use them to generate findings more likely to be replicated in future studies, ultimately improving the credibility of the scientific process.
Identifiants
pubmed: 33996122
doi: 10.1098/rsos.201925
pii: rsos201925
pmc: PMC8059606
doi:
Types de publication
Journal Article
Langues
eng
Pagination
201925Informations de copyright
© 2021 The Authors.
Références
Semin Hematol. 2008 Jul;45(3):135-40
pubmed: 18582619
Epidemiology. 2008 Sep;19(5):640-8
pubmed: 18633328
BMC Nutr. 2017 Jul 10;3:54
pubmed: 32153834
JAMA. 2013 Apr 3;309(13):1351-2
pubmed: 23549579
PLoS One. 2011;6(6):e20456
pubmed: 21674008
Nature. 2010 Jul 1;466(7302):24-6
pubmed: 20595989
Psychol Sci. 2011 Nov;22(11):1359-66
pubmed: 22006061
Glob Chall. 2017 Jan 23;1(2):1600008
pubmed: 31565263
Stat Med. 1992 May;11(7):875-9
pubmed: 1604067
Nature. 2015 Feb 5;518(7537):27-9
pubmed: 25652980
Nat Genet. 2009 Feb;41(2):149-55
pubmed: 19174838
PLoS Genet. 2014 Oct 30;10(10):e1004722
pubmed: 25357204
Science. 2015 Aug 28;349(6251):aac4716
pubmed: 26315443
Perspect Psychol Sci. 2016 Sep;11(5):702-712
pubmed: 27694465
Nature. 2019 Mar;567(7748):305-307
pubmed: 30894741
BMC Med Res Methodol. 2017 Sep 19;17(1):146
pubmed: 28927376
Nature. 2020 Jun;582(7810):84-88
pubmed: 32483374
Nature. 2012 Mar 28;483(7391):531-3
pubmed: 22460880
Front Comput Neurosci. 2012 Mar 05;6:8
pubmed: 22403538
BMC Med Res Methodol. 2017 Sep 9;17(1):138
pubmed: 28888225
Perspect Psychol Sci. 2020 Sep;15(5):1158-1177
pubmed: 32598854
Psychol Assess. 2014 Jun;26(2):419-32
pubmed: 24447279
Soc Sci Res. 2016 Sep;59:1-12
pubmed: 27480367
Nat Rev Neurosci. 2013 May;14(5):365-76
pubmed: 23571845
Science. 2018 Feb 16;359(6377):725-726
pubmed: 29449469
Nat Methods. 2018 Apr;15(4):233-234
pubmed: 30100822
Trends Cogn Sci. 2014 May;18(5):235-41
pubmed: 24656991
PLoS Med. 2005 Aug;2(8):e124
pubmed: 16060722
Eur J Epidemiol. 2016 Apr;31(4):337-50
pubmed: 27209009
Philos Trans R Soc Lond B Biol Sci. 2014 Apr 14;369(1643):20130190
pubmed: 24733945
Perspect Psychol Sci. 2017 Nov;12(6):1100-1122
pubmed: 28841086
Int J Cancer. 2016 Jan 15;138(2):293-302
pubmed: 25583132
Biom J. 2020 May;62(3):670-687
pubmed: 31099917
Nature. 2015 Oct 8;526(7572):182-5
pubmed: 26450039
Science. 2017 Feb 10;355(6325):584-585
pubmed: 28183939
Stat Med. 2014 Dec 30;33(30):5413-32
pubmed: 25074480
Int J Epidemiol. 2016 Dec 1;45(6):1866-1886
pubmed: 28108528
Perspect Psychol Sci. 2017 Nov;12(6):1123-1128
pubmed: 28853993
Sci Transl Med. 2016 Jun 1;8(341):341ps12
pubmed: 27252173
Perspect Psychol Sci. 2012 Nov;7(6):632-8
pubmed: 26168122
Nature. 2011 Sep 28;477(7366):511
pubmed: 21956292
Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2600-2606
pubmed: 29531091
Trends Pharmacol Sci. 2016 Apr;37(4):290-302
pubmed: 26776451
Nature. 2016 May 25;533(7604):452-4
pubmed: 27225100
Stat Appl Genet Mol Biol. 2007;6:Article25
pubmed: 17910531
Nature. 2015 Oct 8;526(7572):187-9
pubmed: 26450040
Pharm Stat. 2014 Jan-Feb;13(1):41-54
pubmed: 23913901
Psychol Methods. 2004 Jun;9(2):147-63
pubmed: 15137886
Nature. 2018 Jan 25;553(7689):399-401
pubmed: 29368721
Am J Epidemiol. 2016 Mar 1;183(5):415-22
pubmed: 26893297
Nature. 2014 Feb 13;506(7487):150-2
pubmed: 24522584
Nat Rev Drug Discov. 2011 Aug 31;10(9):712
pubmed: 21892149
Science. 2017 Feb 3;355(6324):486-488
pubmed: 28154051
Wiley Interdiscip Rev Clim Change. 2016 Jan;7(1):23-41
pubmed: 26877771
Am J Epidemiol. 1993 Sep 15;138(6):430-42
pubmed: 8213748
PLoS Med. 2014 Oct 21;11(10):e1001747
pubmed: 25334033
Science. 2014 Nov 28;346(6213):1054-5
pubmed: 25430753
Science. 2015 Jul 17;349(6245):255-60
pubmed: 26185243
PLoS Biol. 2015 Jun 09;13(6):e1002165
pubmed: 26057340
Nat Hum Behav. 2020 Apr;4(4):329-331
pubmed: 31915371
Psychol Sci. 2014 Jan;25(1):7-29
pubmed: 24220629
Science. 2015 Jun 26;348(6242):1422-5
pubmed: 26113702
Nat Hum Behav. 2018 Jan;2(1):6-10
pubmed: 30980045
Cancer Causes Control. 2013 Apr;24(4):611-27
pubmed: 23380943
Psychon Bull Rev. 2007 Oct;14(5):779-804
pubmed: 18087943
Cortex. 2013 Mar;49(3):609-10
pubmed: 23347556