Do reporting guidelines have an impact? Empirical assessment of changes in reporting before and after the PRISMA extension statement for network meta-analysis.
Multiple treatment meta-analysis
PRISMA-NMA
Reporting
Systematic review
Journal
Systematic reviews
ISSN: 2046-4053
Titre abrégé: Syst Rev
Pays: England
ID NLM: 101580575
Informations de publication
Date de publication:
10 09 2021
10 09 2021
Historique:
received:
27
01
2021
accepted:
28
07
2021
entrez:
11
9
2021
pubmed:
12
9
2021
medline:
27
10
2021
Statut:
epublish
Résumé
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement for network meta-analysis (NMA) published in 2015 promotes comprehensive reporting in published systematic reviews with NMA. PRISMA-NMA includes 32 items: 27 core items as indicated in the 2009 PRISMA Statement and five items specific to the reporting of NMAs. Although NMA reporting is improving, it is unclear whether PRISMA-NMA has accelerated this improvement. We aimed to investigate the impact of PRISMA-NMA and highlight key items that require attention and improvement. We updated our previous collection of NMAs with articles published between April 2015 and July 2018. We assessed the completeness of reporting for each NMA, including main manuscript and online supplements, using the PRISMA-NMA checklist. The PRISMA-NMA checklist originally includes 32 total items (i.e. a 32-point scale original PRISMA-NMA score). We also prepared a modified version of the PRISMA-NMA checklist with 49 items to evaluate separately at a more granular level all multiple-content items (i.e. a 49-point scale modified PRISMA-NMA score). We compared average reporting scores of articles published until and after 2015. In the 1144 included NMAs the mean modified PRISMA-NMA score was 32.1 (95% CI 31.8-32.4) of a possible 49-excellence-score. For 1-year increase, the mean modified score increased by 0.96 (95% CI 0.32 to 1.59) for 389 NMAs published until 2015 and by 0.53 (95% CI 0.02 to 1.04) for 755 NMAs published after 2015. The mean modified PRISMA-NMA score for NMAs published after 2015 was higher by 0.81 (95% CI 0.23 to 1.39) compared to before 2015 when adjusting for journal impact factor, type of review, funding, and treatment category. Description of summary effect sizes to be used, presentation of individual study data, sources of funding for the systematic review, and role of funders dropped in frequency after 2015 by 6-16%. NMAs published after 2015 more frequently reported the five items associated with NMA compared to those published until 2015. However, improvement in reporting after 2015 is compatible with that observed on a yearly basis until 2015, and hence, it could not be attributed solely to the publication of the PRISMA-NMA.
Sections du résumé
BACKGROUND
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement for network meta-analysis (NMA) published in 2015 promotes comprehensive reporting in published systematic reviews with NMA. PRISMA-NMA includes 32 items: 27 core items as indicated in the 2009 PRISMA Statement and five items specific to the reporting of NMAs. Although NMA reporting is improving, it is unclear whether PRISMA-NMA has accelerated this improvement. We aimed to investigate the impact of PRISMA-NMA and highlight key items that require attention and improvement.
METHODS
We updated our previous collection of NMAs with articles published between April 2015 and July 2018. We assessed the completeness of reporting for each NMA, including main manuscript and online supplements, using the PRISMA-NMA checklist. The PRISMA-NMA checklist originally includes 32 total items (i.e. a 32-point scale original PRISMA-NMA score). We also prepared a modified version of the PRISMA-NMA checklist with 49 items to evaluate separately at a more granular level all multiple-content items (i.e. a 49-point scale modified PRISMA-NMA score). We compared average reporting scores of articles published until and after 2015.
RESULTS
In the 1144 included NMAs the mean modified PRISMA-NMA score was 32.1 (95% CI 31.8-32.4) of a possible 49-excellence-score. For 1-year increase, the mean modified score increased by 0.96 (95% CI 0.32 to 1.59) for 389 NMAs published until 2015 and by 0.53 (95% CI 0.02 to 1.04) for 755 NMAs published after 2015. The mean modified PRISMA-NMA score for NMAs published after 2015 was higher by 0.81 (95% CI 0.23 to 1.39) compared to before 2015 when adjusting for journal impact factor, type of review, funding, and treatment category. Description of summary effect sizes to be used, presentation of individual study data, sources of funding for the systematic review, and role of funders dropped in frequency after 2015 by 6-16%.
CONCLUSIONS
NMAs published after 2015 more frequently reported the five items associated with NMA compared to those published until 2015. However, improvement in reporting after 2015 is compatible with that observed on a yearly basis until 2015, and hence, it could not be attributed solely to the publication of the PRISMA-NMA.
Identifiants
pubmed: 34507621
doi: 10.1186/s13643-021-01780-9
pii: 10.1186/s13643-021-01780-9
pmc: PMC8434710
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
246Commentaires et corrections
Type : ErratumIn
Informations de copyright
© 2021. The Author(s).
Références
Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLOS Med. 2009;6(7):e1000097.
doi: 10.1371/journal.pmed.1000097
Page MJ, Moher D. Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review. Syst Rev. 2017;6(1):263.
doi: 10.1186/s13643-017-0663-8
Page M J, McKenzie J E, Bossuyt P M, Boutron I, Hoffmann T C, Mulrow C D et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.1136/bmj.n71 .
Page MJ, Altman DG, Shamseer L, McKenzie JE, Ahmadzai N, Wolfe D, et al. Reproducible research practices are underused in systematic reviews of biomedical interventions. J Clin Epidemiol. 2018;94:8–18.
doi: 10.1016/j.jclinepi.2017.10.017
Carroll K, Hemmings R. On the need for increased rigour and care in the conduct and interpretation of network meta-analyses in drug development. Pharm Stat. 2016;15(2):135–42.
doi: 10.1002/pst.1733
Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162(11):777–84.
doi: 10.7326/M14-2385
Zarin W, Veroniki AA, Nincic V, Vafaei A, Reynen E, Motiwala SS, et al. Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review. BMC Med. 2017;15(1):3.
doi: 10.1186/s12916-016-0764-6
Petropoulou M, Nikolakopoulou A, Veroniki AA, Rios P, Vafaei A, Zarin W, et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol. 2017;82:20–8.
doi: 10.1016/j.jclinepi.2016.11.002
Lee DW, Shin IS. Critical quality evaluation of network meta-analyses in dental care. J Dent. 2018;75:7–11.
doi: 10.1016/j.jdent.2018.05.010
Tonin FS, Rotta I, Mendes AM, Pontarolo R. Network meta-analysis: a technique to gather evidence from direct and indirect comparisons. Pharmacy Practice. 2017;15(1):943.
doi: 10.18549/PharmPract.2017.01.943
Pratt M, Wieland S, Ahmadzai N, Butler C, Wolfe D, Pussagoda K, et al. A scoping review of network meta-analyses assessing the efficacy and safety of complementary and alternative medicine interventions. Syst Rev. 2020;9(1):97.
doi: 10.1186/s13643-020-01328-3
Nikolakopoulou A, Chaimani A, Veroniki AA, Vasiliadis HS, Schmid CH, Salanti G. Characteristics of networks of interventions: a description of a database of 186 published networks. PLoS ONE. 2014;9(1):e86754.
doi: 10.1371/journal.pone.0086754
Papakonstantinou T. nmadata: R package for accessing redcap database of network meta-analyses hosted by ISPM University of Bern. Available at https://github.com/esm-ispm-unibe-ch/nmadata .
R Development Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2008. ISBN 3–900051–07–0 https://www.R-project.org.
R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2019.
Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, et al. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health J Int Soc Pharmacoecon Outcomes Res. 2011;14(4):429–37.
doi: 10.1016/j.jval.2011.01.011
Jansen JP, Fleurence R, Devine B, Itzler R, Barrett A, Hawkins N, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health J Int Soc Pharmacoecon Outcomes Res. 2011;14(4):417–28.
doi: 10.1016/j.jval.2011.04.002
Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. 2011; last updated April 2014; Available from http://www.nicedsu.org.uk .
Sun X, Zhou X, Yu Y, Liu H. Exploring reporting quality of systematic reviews and Meta-analyses on nursing interventions in patients with Alzheimer’s disease before and after PRISMA introduction. BMC Med Res Methodol. 2018;18(1):154.
doi: 10.1186/s12874-018-0622-7