Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
11 12 2020
11 12 2020
Historique:
received:
29
01
2020
accepted:
13
11
2020
entrez:
14
12
2020
pubmed:
15
12
2020
medline:
5
1
2021
Statut:
epublish
Résumé
Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.
Identifiants
pubmed: 33311448
doi: 10.1038/s41467-020-20142-y
pii: 10.1038/s41467-020-20142-y
pmc: PMC7733498
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Research Support, U.S. Gov't, Non-P.H.S.
Langues
eng
Sous-ensembles de citation
IM
Pagination
6377Références
PLoS Med. 2020 Jun 30;17(6):e1003266
pubmed: 32603323
Stat Med. 2017 Apr 15;36(8):1210-1226
pubmed: 28083901
Res Synth Methods. 2015 Jun;6(2):124-33
pubmed: 26099480
N Engl J Med. 2000 Jun 22;342(25):1878-86
pubmed: 10861324
BMJ. 2016 Oct 12;355:i4919
pubmed: 27733354
PLoS Biol. 2018 Mar 13;16(3):e2005468
pubmed: 29534060
Trends Ecol Evol. 2009 Mar;24(3):127-35
pubmed: 19185386
J Clin Epidemiol. 2020 Jan;117:36-45
pubmed: 31541691
Trends Ecol Evol. 2004 Jun;19(6):305-8
pubmed: 16701275
Nature. 2018 Jul;559(7715):517-526
pubmed: 30046075
Science. 2015 Aug 28;349(6251):aac4716
pubmed: 26315443
Am Stat. 2019 Mar 20;73(1):56-68
pubmed: 31057338
JAMA. 2014 Dec 10;312(22):2401-2
pubmed: 25490331
J Clin Epidemiol. 2013 Feb;66(2):151-7
pubmed: 22542023
PLoS Med. 2005 Aug;2(8):e124
pubmed: 16060722
J R Stat Soc Ser A Stat Soc. 2009 Jan;172(1):21-47
pubmed: 19381328
J Clin Epidemiol. 1995 Jan;48(1):9-18
pubmed: 7853053
PLoS One. 2009 Nov 30;4(11):e7824
pubmed: 19956596
Res Synth Methods. 2020 Mar;11(2):181-217
pubmed: 31614060
Eval Rev. 2019 Jun-Aug;43(3-4):152-188
pubmed: 31390890
Nature. 2018 Jun;558(7710):361-364
pubmed: 29925978
J Clin Epidemiol. 2015 Feb;68(2):122-33
pubmed: 25579639
J Occup Med. 1994 Jul;36(7):763-75
pubmed: 7931743
JAMA. 2001 Aug 15;286(7):821-30
pubmed: 11497536
Conserv Biol. 2020 Jun 24;:
pubmed: 32583521
Psychol Sci. 2012 May 1;23(5):524-32
pubmed: 22508865
BMJ. 2017 Sep 21;358:j4008
pubmed: 28935701
Ecol Evol. 2015 Oct 26;5(22):5295-5304
pubmed: 30151132
Bioscience. 2020 Aug 26;70(9):794-803
pubmed: 32973409
Environ Manage. 2020 Sep;66(3):435-449
pubmed: 32594203
Res Synth Methods. 2020 Sep;11(5):698-713
pubmed: 32618107
Pers Soc Psychol Rev. 1998;2(3):196-217
pubmed: 15647155
Nature. 2015 Dec 10;528(7581):185-7
pubmed: 26659166
J R Stat Soc Ser A Stat Soc. 2020 Jan;183(1):193-209
pubmed: 31857745
Orthop Nurs. 2008 Mar-Apr;27(2):116-22; quiz 123-4
pubmed: 18385595