Meta-analyses of diagnostic test accuracy could not be reproduced.
Diagnostic test accuracy
Meta-analyses
Reproducibility
Sensitivity and specificity
Systematic reviews
Transparency
Journal
Journal of clinical epidemiology
ISSN: 1878-5921
Titre abrégé: J Clin Epidemiol
Pays: United States
ID NLM: 8801383
Informations de publication
Date de publication:
Nov 2020
Nov 2020
Historique:
received:
11
11
2019
revised:
11
06
2020
accepted:
27
06
2020
pubmed:
18
7
2020
medline:
6
3
2021
entrez:
18
7
2020
Statut:
ppublish
Résumé
The aim of our study was to investigate the reproducibility of diagnostic accuracy meta-analyses, as reported in published systematic reviews. We selected all systematic reviews of diagnostic test accuracy containing a meta-analysis, published in January 2018 and retrieved in Medline through Ovid. All reviews reported a summary estimate of sensitivity and specificity. We requested the protocol from their authors and used the protocol and the information in the published review to reproduce the reported meta-analysis. Successful reproduction was defined as a result differing <1% point from the reported point estimates; or reported primary study results that were in line with those of the actual primary study results; or if the data from the primary studies could be extracted without checking the data in the review first. Of the 51 included reviews, 16 had a protocol registered in PROSPERO and five of those responded to our request for a protocol. Nineteen reviews (37%) provided the 2×2 tables that were included in the meta-analysis. In 14 of those, the outcome of the meta-analysis could be reproduced. Considering the correctness of the numbers from the primary articles and the complete reporting of the search strategy, only one meta-analysis was fully replicable. Published meta-analyses of diagnostic test accuracy were poorly replicable. This was partly because of lack of information about the methods and data used, and partly because of mistakes in the data extraction or data reporting.
Sections du résumé
BACKGROUND AND OBJECTIVES
OBJECTIVE
The aim of our study was to investigate the reproducibility of diagnostic accuracy meta-analyses, as reported in published systematic reviews.
STUDY DESIGN AND SETTING
METHODS
We selected all systematic reviews of diagnostic test accuracy containing a meta-analysis, published in January 2018 and retrieved in Medline through Ovid. All reviews reported a summary estimate of sensitivity and specificity. We requested the protocol from their authors and used the protocol and the information in the published review to reproduce the reported meta-analysis. Successful reproduction was defined as a result differing <1% point from the reported point estimates; or reported primary study results that were in line with those of the actual primary study results; or if the data from the primary studies could be extracted without checking the data in the review first.
RESULTS
RESULTS
Of the 51 included reviews, 16 had a protocol registered in PROSPERO and five of those responded to our request for a protocol. Nineteen reviews (37%) provided the 2×2 tables that were included in the meta-analysis. In 14 of those, the outcome of the meta-analysis could be reproduced. Considering the correctness of the numbers from the primary articles and the complete reporting of the search strategy, only one meta-analysis was fully replicable.
CONCLUSION
CONCLUSIONS
Published meta-analyses of diagnostic test accuracy were poorly replicable. This was partly because of lack of information about the methods and data used, and partly because of mistakes in the data extraction or data reporting.
Identifiants
pubmed: 32679314
pii: S0895-4356(19)31036-4
doi: 10.1016/j.jclinepi.2020.06.033
pii:
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
161-166Informations de copyright
Copyright © 2020. Published by Elsevier Inc.