Comparison of Reporting and Transparency in Published Protocols and Publications in Umbrella Reviews: Scoping Review.
inconsistency
protocol
publication
transparency
umbrella reviews
Journal
Journal of medical Internet research
ISSN: 1438-8871
Titre abrégé: J Med Internet Res
Pays: Canada
ID NLM: 100959882
Informations de publication
Date de publication:
02 08 2023
02 08 2023
Historique:
received:
07
10
2022
accepted:
05
05
2023
revised:
15
02
2023
medline:
3
8
2023
pubmed:
2
8
2023
entrez:
2
8
2023
Statut:
epublish
Résumé
Inconsistencies between a protocol and its umbrella review (UR) may mislead readers about the importance of findings or lead to false-positive results. Furthermore, not documenting and explaining inconsistencies in the UR could reduce its transparency. To our knowledge, no study has examined the methodological consistency of the protocols with their URs and assessed the transparency of the URs when generating evidence. This study aimed to investigate the inconsistency of protocols with their URs in the methodology and assess the transparency of the URs. We searched medical-related electronic databases from their inception to January 1, 2022. We investigated inconsistencies between protocols and their publications and transparencies in the search strategy, inclusion criteria, methods of screening and data extraction, quality assessment, and statistical analysis. We included 31 protocols and 35 publications. For the search strategy, 39 inconsistencies between the protocols and their publications were found in 26 of the 35 (74%) URs, and 16 of these inconsistencies were indicated and explained. There were 84 inconsistencies between the protocols and their URs regarding the inclusion criteria in 31 of the 35 (89%) URs, and 29 of the inconsistencies were indicated and explained. Deviations from their protocols were found in 12 of the 32 (38%) URs reporting the methods of screening, 14 of the 30 (47%) URs reporting the methods of data extraction, and 11 of the 32 (34%) URs reporting the methods for quality assessment. Of the 35 URs, 6 (17%) were inconsistent with their protocols in terms of the tools for quality assessment; one-half (3/6, 50%) of them indicated and explained the deviations. As for the statistical analysis, 31 of the 35 (89%) URs generated 61 inconsistencies between the publications and their protocols, and 16 inconsistencies were indicated and explained. There was a high prevalence of inconsistencies between protocols and publications of URs, and more than one-half of the inconsistencies were not indicated and explained in the publications. Therefore, how to promote the transparency of URs will be a major part of future work.
Sections du résumé
BACKGROUND
Inconsistencies between a protocol and its umbrella review (UR) may mislead readers about the importance of findings or lead to false-positive results. Furthermore, not documenting and explaining inconsistencies in the UR could reduce its transparency. To our knowledge, no study has examined the methodological consistency of the protocols with their URs and assessed the transparency of the URs when generating evidence.
OBJECTIVE
This study aimed to investigate the inconsistency of protocols with their URs in the methodology and assess the transparency of the URs.
METHODS
We searched medical-related electronic databases from their inception to January 1, 2022. We investigated inconsistencies between protocols and their publications and transparencies in the search strategy, inclusion criteria, methods of screening and data extraction, quality assessment, and statistical analysis.
RESULTS
We included 31 protocols and 35 publications. For the search strategy, 39 inconsistencies between the protocols and their publications were found in 26 of the 35 (74%) URs, and 16 of these inconsistencies were indicated and explained. There were 84 inconsistencies between the protocols and their URs regarding the inclusion criteria in 31 of the 35 (89%) URs, and 29 of the inconsistencies were indicated and explained. Deviations from their protocols were found in 12 of the 32 (38%) URs reporting the methods of screening, 14 of the 30 (47%) URs reporting the methods of data extraction, and 11 of the 32 (34%) URs reporting the methods for quality assessment. Of the 35 URs, 6 (17%) were inconsistent with their protocols in terms of the tools for quality assessment; one-half (3/6, 50%) of them indicated and explained the deviations. As for the statistical analysis, 31 of the 35 (89%) URs generated 61 inconsistencies between the publications and their protocols, and 16 inconsistencies were indicated and explained.
CONCLUSIONS
There was a high prevalence of inconsistencies between protocols and publications of URs, and more than one-half of the inconsistencies were not indicated and explained in the publications. Therefore, how to promote the transparency of URs will be a major part of future work.
Identifiants
pubmed: 37531172
pii: v25i1e43299
doi: 10.2196/43299
pmc: PMC10433027
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
e43299Informations de copyright
©Liang Zhao, Caiyi Shen, Ming Liu, Jiaoyan Zhang, Luying Cheng, Yuanyuan Li, Lanbin Yuan, Junhua Zhang, Jinhui Tian. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 02.08.2023.
Références
BMJ. 2021 Mar 29;372:n71
pubmed: 33782057
BMJ. 2015 Jan 02;350:g7647
pubmed: 25555855
J Clin Epidemiol. 2021 Mar;131:59-69
pubmed: 33227446
BMJ. 2017 Sep 21;358:j4008
pubmed: 28935701
Updates Surg. 2021 Oct;73(5):1673-1689
pubmed: 34031848
Res Synth Methods. 2017 Mar;8(1):92-108
pubmed: 28074553
BMC Neurol. 2021 Sep 29;21(1):378
pubmed: 34587933
PLoS Biol. 2016 Jan 04;14(1):e1002333
pubmed: 26726926
CMAJ. 2009 Oct 13;181(8):488-93
pubmed: 19654195
Science. 2014 Jan 3;343(6166):30-1
pubmed: 24385620
BMC Med Res Methodol. 2020 Jun 1;20(1):138
pubmed: 32487023
J Clin Epidemiol. 2021 Apr;132:146-147
pubmed: 33383129
J Clin Epidemiol. 2016 Nov;79:46-54
pubmed: 27079845
BMC Pregnancy Childbirth. 2021 Mar 26;21(1):254
pubmed: 33771100
PLoS Med. 2005 Aug;2(8):e124
pubmed: 16060722
Am J Nurs. 2014 May;114(5):49-56
pubmed: 24759479
JBI Database System Rev Implement Rep. 2017 Apr;15(4):1154-1208
pubmed: 28398987
BMC Public Health. 2019 Dec 17;19(1):1678
pubmed: 31842835
STAR Protoc. 2021 May 14;2(2):100468
pubmed: 34036281
Syst Rev. 2020 Nov 4;9(1):254
pubmed: 33148319
JAMA. 2002 Jun 5;287(21):2831-4
pubmed: 12038926
Evid Based Ment Health. 2018 Aug;21(3):95-100
pubmed: 30006442
Res Integr Peer Rev. 2021 Jun 2;6(1):9
pubmed: 34078479
Syst Rev. 2016 Feb 09;5:27
pubmed: 26862061
Cochrane Database Syst Rev. 2014 Oct 01;(10):MR000035
pubmed: 25271098
J Med Internet Res. 2020 Nov 11;22(11):e23851
pubmed: 33174851
J Neurotrauma. 2020 Dec 15;37(24):2605-2615
pubmed: 32791886
J Evid Based Med. 2012 Aug;5(3):166-73
pubmed: 23672223
JBI Database System Rev Implement Rep. 2018 Mar;16(3):752-775
pubmed: 29521871
J Clin Epidemiol. 2018 Jun;98:144-145
pubmed: 29222061
Nat Hum Behav. 2017 Jan 10;1:0021
pubmed: 33954258
Cochrane Database Syst Rev. 2011 Jan 19;(1):MR000031
pubmed: 21249714
J Clin Epidemiol. 2014 Apr;67(4):368-75
pubmed: 24581293
Ann Intern Med. 2016 Sep 20;165(6):421-30
pubmed: 27294570
Stat Med. 2005 May 30;24(10):1547-61
pubmed: 15580591
Res Synth Methods. 2021 May;12(3):394-405
pubmed: 33522101
Syst Rev. 2018 Aug 23;7(1):129
pubmed: 30139391
Eur J Epidemiol. 2019 Jun;34(6):543-546
pubmed: 30852716
BMJ Open. 2022 Jan 12;12(1):e053312
pubmed: 35022174
J Clin Epidemiol. 2009 Oct;62(10):1006-12
pubmed: 19631508
J Clin Epidemiol. 2011 Apr;64(4):383-94
pubmed: 21195583