Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study.
adequacy
cochrane risk of bias tool 2.0
improvement
inconsistency
non-cochrane systematic reviews
Journal
Research synthesis methods
ISSN: 1759-2887
Titre abrégé: Res Synth Methods
Pays: England
ID NLM: 101543738
Informations de publication
Date de publication:
23 Jan 2024
23 Jan 2024
Historique:
revised:
04
12
2023
received:
15
03
2023
accepted:
06
12
2023
medline:
24
1
2024
pubmed:
24
1
2024
entrez:
23
1
2024
Statut:
aheadofprint
Résumé
Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the RoB 2 assessments in non-Cochrane reviews published in 2020. This meta-research study included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that used the RoB 2 tool, we analyzed the reporting of the RoB 2 assessment. Among 3880 included reviews, the Cochrane RoB 1 tool was the most frequently used (N = 2228; 57.4%), followed by the Cochrane RoB 2 tool (N = 267; 6.9%). From 267 reviews that reported using the RoB 2 tool, 213 (79.8%) actually used it. In 26 (12.2%) reviews, erroneous statements were used to indicate the RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment with a detailed table of answers to all signaling questions. The judgment of risk of bias by the RoB 2 tool was not justified by a comment in 158 (74.2%) reviews. Only in 33 (14.5%) of reviews the judgment in all domains was justified in the accompanying comment. In most reviews (81.7%), the RoB was inadequately assessed at the study level. In conclusion, the majority of non-Cochrane reviews published in 2020 still used the Cochrane RoB 1 tool. Many reviews used the RoB 2 tool inadequately. Further studies about the uptake and the use of the RoB 2 tool are needed.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024 John Wiley & Sons Ltd.
Références
Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
Sterne JAC, Savovic J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.
Babic A, Pijuk A, Brazdilova L, et al. The judgement of biases included in the category “other bias” in Cochrane systematic reviews of interventions: a systematic survey. BMC Med Res Methodol. 2019;19(1):77.
Babic A, Tokalic R, Amilcar Silva Cunha J, et al. Assessments of attrition bias in Cochrane systematic reviews are highly inconsistent and thus hindering trial comparability. BMC Med Res Methodol. 2019;19(1):76.
Babic A, Vuka I, Saric F, et al. Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent. J Clin Epidemiol. 2020;119:57-64.
Propadalo I, Tranfic M, Vuka I, Barcot O, Pericic TP, Puljak L. In Cochrane reviews, risk of bias assessments for allocation concealment were frequently not in line with Cochrane's Handbook guidance. J Clin Epidemiol. 2019;106:10-17.
Barcot O, Boric M, Dosenovic S, Puljak L. Assessing the risk of performance and detection bias in Cochrane reviews as a joint domain is less accurate compared to two separate domains. BMC Med Res Methodol. 2021;21(1):149.
Barcot O, Boric M, Dosenovic S, Poklepovic Pericic T, Cavar M, Puljak L. Risk of bias assessments for blinding of participants and personnel in Cochrane reviews were frequently inadequate. J Clin Epidemiol. 2019;113:104-113.
Barcot O, Boric M, Poklepovic Pericic T, et al. Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook. BMC Med Res Methodol. 2019;19(1):170.
Barcot O, Dosenovic S, Boric M, et al. Assessing risk of bias judgments for blinding of outcome assessors in Cochrane reviews. J Comp Eff Res. 2020;9(8):585-593.
Saric F, Barcot O, Puljak L. Risk of bias assessments for selective reporting were inadequate in the majority of Cochrane reviews. J Clin Epidemiol. 2019;112:53-58.
Puljak L, Ramic I, Arriola Naharro C, et al. Cochrane risk of bias tool was used inadequately in the majority of non-Cochrane systematic reviews. J Clin Epidemiol. 2020;123:114-119.
Minozzi S, Gonzalez-Lorenzo M, Cinquini M, et al. Adherence of systematic reviews to Cochrane RoB2 guidance was frequently poor: a meta epidemiological study. J Clin Epidemiol. 2022;152:47-55.
Runjic E, Behmen D, Pieper D, et al. Following Cochrane review protocols to completion 10 years later: a retrospective cohort study and author survey. J Clin Epidemiol. 2019;111:41-48.
Runjic E, Rombey T, Pieper D, Puljak L. Half of systematic reviews about pain registered in PROSPERO were not published and the majority had inaccurate status. J Clin Epidemiol. 2019;116:114-121.
Carev M, Civljak M, Puljak L, Dosenovic S. Characteristics, completion and publication of PROSPERO records in regional anesthesia for acute perioperative pain. J Comp Eff Res. 2023;12:e220129.
Bojcic R, Todoric M, Puljak L. Adopting AMSTAR 2 critical appraisal tool for systematic reviews: speed of the tool uptake and barriers for its adoption. BMC Med Res Methodol. 2022;22(1):104.
Status and expectations of implementation of Risk of Bias 2 in Cochrane intervention reviews. November 19, 2020 Available at: https://community.cochrane.org/news/status-and-expectations-implementation-risk-bias-2-cochrane-intervention-reviews
Minozzi S, Dwan K, Borrelli F, Filippini G. Reliability of the revised Cochrane risk-of-bias tool for randomised trials (RoB2) improved with the use of implementation instruction. J Clin Epidemiol. 2022;141:99-105.
Minozzi S, Cinquini M, Gianola S, Gonzalez-Lorenzo M, Banzi R. The revised Cochrane risk of bias tool for randomized trials (RoB 2) showed low interrater reliability and challenges in its application. J Clin Epidemiol. 2020;126:37-44.
Crocker TF, Lam N, Jordao M, et al. Risk-of-bias assessment using Cochrane's revised tool for randomized trials (RoB 2) was useful but challenging and resource-intensive: observations from a systematic review. J Clin Epidemiol. 2023;161:39-45.
Barcot O, Ivanda M, Buljan I, Pieper D, Puljak L. Enhanced access to recommendations from the Cochrane Handbook for improving authors' judgments about risk of bias: a randomized controlled trial. Res Synth Methods. 2021;12(5):618-629.
Nussbaumer-Streit B, Sommer I, Hamel C, et al. Rapid reviews methods series: guidance on team considerations, study selection, data extraction and risk of bias assessment. BMJ Evid Based Med. 2023;28:418-423.
Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71