Publication and related biases in health services research: a systematic review of empirical evidence.
Dissemination bias
Funnel plots
Grey literature
Health services research
Outcome reporting bias
Publication bias
Research methodology
Research publication
Research registration
Systematic review
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
01 06 2020
01 06 2020
Historique:
received:
28
01
2019
accepted:
07
05
2020
entrez:
4
6
2020
pubmed:
4
6
2020
medline:
25
6
2021
Statut:
epublish
Résumé
Publication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR. Databases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively. After screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases. Methodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper. PROSPERO 2016 CRD42016052333.
Sections du résumé
BACKGROUND
Publication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR.
METHODS
Databases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively.
RESULTS
After screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases.
CONCLUSIONS
Methodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper.
SYSTEMATIC REVIEW REGISTRATION
PROSPERO 2016 CRD42016052333.
Identifiants
pubmed: 32487022
doi: 10.1186/s12874-020-01010-1
pii: 10.1186/s12874-020-01010-1
pmc: PMC7268600
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
137Subventions
Organisme : Department of Health
ID : 15/71/06
Pays : United Kingdom
Organisme : National Institute for Health Research
ID : Applied Research Collaboration West Midlands
Pays : International
Organisme : Health Services and Delivery Research Programme
ID : HS&DR 15/71/06
Pays : International
Références
J Am Med Inform Assoc. 2007 May-Jun;14(3):368-71
pubmed: 17329724
BMJ. 2000 Dec 9;321(7274):1428
pubmed: 11110723
Int J Psychophysiol. 2019 Aug;142:33-49
pubmed: 31195065
PLoS One. 2014 Oct 15;9(10):e110418
pubmed: 25335091
PLoS Biol. 2015 Mar 13;13(3):e1002106
pubmed: 25768323
PLoS One. 2016 Aug 03;11(8):e0159290
pubmed: 27487090
J Exp Psychol Gen. 2015 Dec;144(6):1146-52
pubmed: 26595842
PLoS One. 2016 Feb 17;11(2):e0149144
pubmed: 26886098
BMC Med. 2018 Jun 20;16(1):94
pubmed: 29921277
PLoS One. 2013 Jul 05;8(7):e66844
pubmed: 23861749
BMJ. 2010 Aug 27;341:c4413
pubmed: 20802000
Health Policy. 2015 Apr;119(4):503-10
pubmed: 25260911
Cochrane Database Syst Rev. 2014 Nov 20;(11):CD000011
pubmed: 25412402
Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000011
pubmed: 17443632
Stat Med. 2015 Sep 10;34(20):2781-93
pubmed: 25988604
BMC Med. 2018 Jun 20;16(1):95
pubmed: 29921272
Health Psychol Rev. 2019 Sep;13(3):318-343
pubmed: 31122177
Lancet. 2005 May 28-Jun 3;365(9474):1829-31
pubmed: 15924966
Vaccine. 2005 Feb 18;23(13):1624-35
pubmed: 15694515
CMAJ. 2004 Nov 9;171(10):1179-85
pubmed: 15534310
Ann Epidemiol. 2016 Aug;26(8):597-599.e4
pubmed: 27545753
Bull World Health Organ. 2004 Sep;82(9):689-96
pubmed: 15628207
BMJ. 2009 Jul 21;339:b2700
pubmed: 19622552
Health Policy. 2013 Jan;109(1):78-87
pubmed: 23182564
J Exp Psychol Gen. 2014 Apr;143(2):534-47
pubmed: 23855496
Psychol Methods. 2018 Sep;23(3):546-560
pubmed: 28425729
BMJ. 2012 May 16;344:e3480
pubmed: 22592904
Account Res. 2016;23(5):309-17
pubmed: 27028494
Perspect Psychol Sci. 2014 Nov;9(6):666-81
pubmed: 26186117
PeerJ. 2016 Feb 18;4:e1715
pubmed: 26925335
PLoS One. 2013;8(2):e55574
pubmed: 23451028
BMJ Qual Saf. 2017 Apr;26(4):296-303
pubmed: 27099359
BMJ. 2006 Sep 16;333(7568):597-600
pubmed: 16974018
J Biomed Inform. 2013 Feb;46(1):139-41
pubmed: 22975314
Stud Health Technol Inform. 2006;124:957-62
pubmed: 17108634
BMC Health Serv Res. 2012 Jul 24;12:216
pubmed: 22827968
J Health Serv Res Policy. 2005 Jul;10 Suppl 1:6-20
pubmed: 16053580
Am J Prev Med. 2002 Jul;23(1):43-6
pubmed: 12093422
BMJ. 2011 Jul 22;343:d4002
pubmed: 21784880
Health Technol Assess. 2010 Feb;14(8):iii, ix-xi, 1-193
pubmed: 20181324