Challenges for assessing replicability in preclinical cancer biology.
Reproducibility Project: Cancer Biology
cancer biology
human
mouse
open data
open science
preregistration
replication
reproducibility
Journal
eLife
ISSN: 2050-084X
Titre abrégé: Elife
Pays: England
ID NLM: 101579614
Informations de publication
Date de publication:
07 12 2021
07 12 2021
Historique:
received:
02
03
2021
accepted:
20
07
2021
entrez:
7
12
2021
pubmed:
8
12
2021
medline:
16
12
2021
Statut:
epublish
Résumé
We conducted the Reproducibility Project: Cancer Biology to investigate the replicability of preclinical research in cancer biology. The initial aim of the project was to repeat 193 experiments from 53 high-impact papers, using an approach in which the experimental protocols and plans for data analysis had to be peer reviewed and accepted for publication before experimental work could begin. However, the various barriers and challenges we encountered while designing and conducting the experiments meant that we were only able to repeat 50 experiments from 23 papers. Here we report these barriers and challenges. First, many original papers failed to report key descriptive and inferential statistics: the data needed to compute effect sizes and conduct power analyses was publicly accessible for just 4 of 193 experiments. Moreover, despite contacting the authors of the original papers, we were unable to obtain these data for 68% of the experiments. Second, none of the 193 experiments were described in sufficient detail in the original paper to enable us to design protocols to repeat the experiments, so we had to seek clarifications from the original authors. While authors were
Identifiants
pubmed: 34874008
doi: 10.7554/eLife.67995
pii: 67995
pmc: PMC8651289
doi:
pii:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Commentaires et corrections
Type : CommentIn
Informations de copyright
© 2021, Errington et al.
Déclaration de conflit d'intérêts
TE, AD, BN Employed by the nonprofit Center for Open Science that has a mission to increase openness, integrity, and reproducibility of research, NP, EI Employed by and hold shares in Science Exchange Inc
Références
Nature. 2011 Oct 02;478(7370):529-33
pubmed: 21964340
Cell. 2011 Oct 14;147(2):344-57
pubmed: 22000013
Sci Rep. 2013 Nov 06;3:3146
pubmed: 24192909
JAMA. 1990 Mar 9;263(10):1420-3
pubmed: 2406475
PLoS One. 2017 Sep 13;12(9):e0183591
pubmed: 28902887
Nature. 2012 Aug 23;488(7412):522-6
pubmed: 22854781
Nat Med. 2011 Feb;17(2):211-5
pubmed: 21240262
Cell Metab. 2016 Sep 13;24(3):348-360
pubmed: 27626191
Science. 2015 Aug 28;349(6251):aac4716
pubmed: 26315443
Cell. 2011 Sep 16;146(6):904-17
pubmed: 21889194
Science. 2010 May 21;328(5981):1031-5
pubmed: 20378772
Nature. 2012 Mar 28;483(7391):531-3
pubmed: 22460880
Elife. 2016 Jul 07;5:
pubmed: 27387362
Perspect Psychol Sci. 2012 Nov;7(6):562-71
pubmed: 26168113
Circ Res. 2015 Jan 2;116(1):116-26
pubmed: 25552691
Elife. 2015 Sep 03;4:
pubmed: 26335297
Elife. 2017 Jun 27;6:
pubmed: 28653617
Lancet. 2014 Jan 18;383(9913):267-76
pubmed: 24411647
Elife. 2017 Jan 19;6:
pubmed: 28100397
R Soc Open Sci. 2016 Sep 21;3(9):160384
pubmed: 27703703
Scientometrics. 2016;108:315-328
pubmed: 27390387
Nature. 2012 Jul 26;487(7408):500-4
pubmed: 22763439
PLoS Biol. 2011 Dec;9(12):e1001195
pubmed: 22162946
Elife. 2015 Jul 31;4:e06434
pubmed: 26231042
Elife. 2019 Jun 19;8:
pubmed: 31215867
Science. 2016 Mar 4;351(6277):1024-6
pubmed: 26941302
Nature. 2011 Nov 20;481(7381):380-4
pubmed: 22101433
Nature. 2010 Jun 24;465(7301):1033-8
pubmed: 20577206
PLoS Med. 2005 Aug;2(8):e124
pubmed: 16060722
Elife. 2016 Mar 11;5:e10860
pubmed: 26971564
J Empir Res Hum Res Ethics. 2007 Dec;2(4):3-14
pubmed: 19385804
Elife. 2014 Dec 10;3:
pubmed: 25490933
Cell. 2010 Apr 2;141(1):69-80
pubmed: 20371346
Cell. 2011 Jul 8;146(1):148-63
pubmed: 21729786
Nature. 2011 Nov 09;479(7374):547-51
pubmed: 22080947
Elife. 2021 Dec 07;10:
pubmed: 34874009
PLoS Biol. 2021 Mar 1;19(3):e3001107
pubmed: 33647013
PLoS One. 2009 Nov 30;4(11):e7824
pubmed: 19956596
Genome Res. 2012 Feb;22(2):299-306
pubmed: 22009989
Lancet. 2014 Jan 11;383(9912):156-65
pubmed: 24411644
Elife. 2015 Jan 26;4:
pubmed: 25621566
Nature. 2010 Jan 21;463(7279):318-25
pubmed: 20032975
Elife. 2020 Oct 19;9:
pubmed: 33073769
Cancer Cell. 2010 Mar 16;17(3):225-34
pubmed: 20171147
Nature. 2011 Oct 05;478(7368):197-203
pubmed: 21976023
Elife. 2017 Jan 19;6:
pubmed: 28100400
Nature. 2010 Aug 12;466(7308):869-73
pubmed: 20668451
Nature. 2012 Oct 11;490(7419):187-91
pubmed: 23060188
Elife. 2016 Feb 16;5:
pubmed: 26882073
Nature. 2012 Jan 26;483(7387):100-3
pubmed: 22281684
Elife. 2018 Jan 09;7:
pubmed: 29313488
Nature. 2010 Dec 16;468(7326):973-7
pubmed: 21107323
Prev Sci. 2011 Jun;12(2):103-17
pubmed: 21541692
Nat Hum Behav. 2021 Aug;5(8):990-997
pubmed: 34168323
Nature. 2012 May 09;485(7399):502-6
pubmed: 22622578
Sci Data. 2016 Mar 15;3:160018
pubmed: 26978244
PLoS One. 2013 Jun 21;8(6):e67111
pubmed: 23805293
Nat Hum Behav. 2018 Sep;2(9):637-644
pubmed: 31346273
Nature. 2014 Mar 27;507(7493):423-5
pubmed: 24678540
Perspect Psychol Sci. 2012 Nov;7(6):632-8
pubmed: 26168122
Nature. 2011 Jul 06;475(7354):106-9
pubmed: 21734707
PLoS Biol. 2019 May 1;17(5):e3000246
pubmed: 31042704
Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2600-2606
pubmed: 29531091
PLoS Negl Trop Dis. 2015 Nov 20;9(11):e0004194
pubmed: 26587586
Psychol Sci. 2012 May 1;23(5):524-32
pubmed: 22508865
Nat Cell Biol. 2010 May;12(5):468-76
pubmed: 20418870
Elife. 2019 Dec 17;8:
pubmed: 31845647
Elife. 2016 Feb 26;5:
pubmed: 26943899
Nature. 2010 Dec 9;468(7325):839-43
pubmed: 21057493
Nature. 2011 Jul 13;475(7355):231-4
pubmed: 21753854
Nature. 2010 Apr 15;464(7291):1071-6
pubmed: 20393566
Nature. 2011 Sep 07;477(7363):179-84
pubmed: 21901007
Circ Res. 2017 Jun 9;120(12):1916-1926
pubmed: 28373349
Perspect Psychol Sci. 2012 Nov;7(6):537-42
pubmed: 26168110
Lancet. 2014 Jan 11;383(9912):166-75
pubmed: 24411645
Cell. 2012 Sep 28;151(1):56-67
pubmed: 23021215
Cell. 2012 May 11;149(4):780-94
pubmed: 22579283
Nature. 2010 Dec 9;468(7325):824-8
pubmed: 21102434
Elife. 2016 Feb 19;5:
pubmed: 26894955
Nature. 2011 Aug 03;478(7370):524-8
pubmed: 21814200
Cancer Cell. 2010 Dec 14;18(6):553-67
pubmed: 21130701
Elife. 2015 Feb 25;4:
pubmed: 25714925
Elife. 2018 Jan 09;7:
pubmed: 29313490
Elife. 2015 Jun 18;4:e06938
pubmed: 26086719
Nature. 2020 Jul;583(7817):518-520
pubmed: 32694846
Cell. 2016 Aug 25;166(5):1059-1060
pubmed: 27565332
Infect Immun. 2012 Mar;80(3):891-6
pubmed: 22184414
Proc Natl Acad Sci U S A. 2011 May 10;108(19):7950-5
pubmed: 21498687
Elife. 2018 Oct 08;7:
pubmed: 30295289
BMJ Open Sci. 2020 Aug 27;4(1):e100051
pubmed: 35047690
Elife. 2015 Jan 26;4:
pubmed: 25621564
Elife. 2016 Jan 29;5:e07383
pubmed: 26826285
Nature. 2011 Dec 14;481(7380):190-4
pubmed: 22170610
Elife. 2016 Feb 17;5:
pubmed: 26885666
Elife. 2020 Apr 21;9:
pubmed: 32314732
Psychol Bull. 2009 Jul;135(4):555-88
pubmed: 19586162
Psychol Sci. 2011 Nov;22(11):1359-66
pubmed: 22006061
Pers Soc Psychol Rev. 1998;2(3):196-217
pubmed: 15647155
Nature. 2010 Mar 18;464(7287):427-30
pubmed: 20179705
Nature. 2010 Dec 16;468(7326):968-72
pubmed: 21107320
Science. 2012 Aug 10;337(6095):730-5
pubmed: 22855427
Sci Eng Ethics. 2015 Feb;21(1):127-37
pubmed: 24668038
Elife. 2016 Mar 01;5:
pubmed: 26943900
Bull Med Libr Assoc. 1999 Oct;87(4):437-43
pubmed: 10550028
Cell. 2011 Oct 14;147(2):370-81
pubmed: 22000015
Elife. 2016 Feb 23;5:
pubmed: 26905833
PLoS One. 2010 Apr 07;5(4):e10068
pubmed: 20383332
Elife. 2017 Jun 27;6:
pubmed: 28653623
Cell. 2010 Jan 22;140(2):209-21
pubmed: 20141835
Elife. 2017 Jan 19;6:
pubmed: 28100392
Nature. 2011 Jun 08;475(7355):222-5
pubmed: 21654748
Cancer Cell. 2011 Jan 18;19(1):17-30
pubmed: 21251613
Trends Cogn Sci. 2019 Oct;23(10):815-818
pubmed: 31421987
Nat Rev Drug Discov. 2011 Aug 31;10(9):712
pubmed: 21892149
PLoS Biol. 2014 Jan;12(1):e1001756
pubmed: 24409096
Elife. 2021 Dec 07;10:
pubmed: 34874005
Elife. 2017 Jan 19;6:
pubmed: 28100395
Elife. 2015 Sep 01;4:
pubmed: 26327698
Nature. 2015 Jun 11;522(7555):151-3
pubmed: 26062496
Elife. 2015 May 05;4:e06847
pubmed: 25939392
Nature. 2010 Mar 18;464(7287):431-5
pubmed: 20130576
Elife. 2018 Mar 13;7:
pubmed: 29533182
Nature. 2012 Aug 23;488(7412):527-30
pubmed: 22854777
Elife. 2014 Dec 10;3:
pubmed: 25490934
Elife. 2016 Mar 21;5:
pubmed: 26999821
Elife. 2015 Aug 19;4:
pubmed: 26287525
Elife. 2015 Jul 16;4:e04796
pubmed: 26179155
Nat Med. 2012 Jun;18(6):883-91
pubmed: 22635005
Elife. 2017 Jan 19;6:
pubmed: 28100394
Elife. 2019 Mar 12;8:
pubmed: 30860027
Lancet. 2014 Jan 11;383(9912):101-4
pubmed: 24411643
Sci Transl Med. 2011 Aug 17;3(96):96ra77
pubmed: 21849665
Nature. 2012 Jul 26;487(7408):505-9
pubmed: 22763448
Elife. 2018 Dec 11;7:
pubmed: 30526855
Nature. 2011 Dec 07;481(7379):85-9
pubmed: 22158103
Nature. 2012 Mar 28;483(7391):570-5
pubmed: 22460902
Nature. 2020 Mar;579(7798):190-192
pubmed: 32157231
Elife. 2015 Jul 31;4:e07420
pubmed: 26231040
Nature. 2012 Jan 18;482(7383):53-8
pubmed: 22258507
Elife. 2014 Dec 10;3:
pubmed: 25490932
Nature. 2011 Aug 18;476(7360):346-50
pubmed: 21760589
PLoS One. 2015 Aug 05;10(8):e0132382
pubmed: 26244868
Elife. 2015 Jun 25;4:e07072
pubmed: 26111384
Perspect Psychol Sci. 2012 Nov;7(6):615-31
pubmed: 26168121
Proc Natl Acad Sci U S A. 2012 Apr 24;109(17):6662-7
pubmed: 22451913
Elife. 2016 Feb 11;5:
pubmed: 26882501
Nat Genet. 2010 Feb;42(2):181-5
pubmed: 20081860
Nature. 2019 Sep;573(7773):187-189
pubmed: 31506624
Elife. 2014 Dec 10;3:
pubmed: 25490935
Elife. 2015 Jan 26;4:
pubmed: 25621565
Elife. 2015 May 22;4:
pubmed: 27879198
Science. 2012 Oct 5;338(6103):120-3
pubmed: 22903521
Elife. 2015 Mar 17;4:
pubmed: 25781343
Science. 2015 Jun 26;348(6242):1422-5
pubmed: 26113702
Proc Natl Acad Sci U S A. 2021 Apr 27;118(17):
pubmed: 33893240
PLoS Biol. 2020 Mar 27;18(3):e3000691
pubmed: 32218571
Nature. 2012 Feb 15;483(7390):474-8
pubmed: 22343901
Psychol Bull. 1990 Nov;108(3):480-98
pubmed: 2270237
Exp Neurol. 2012 Feb;233(2):597-605
pubmed: 22078756
BMC Med. 2008 Jun 16;6:13
pubmed: 18558004
Elife. 2016 Jun 23;5:
pubmed: 27336789
Science. 2016 Mar 25;351(6280):1433-6
pubmed: 26940865