Comparative analysis of Cochrane and non-Cochrane reviews over three decades.
Bibliometrics
Cochrane
Evidence-based medicine
Information science
Systematic review
Journal
Systematic reviews
ISSN: 2046-4053
Titre abrégé: Syst Rev
Pays: England
ID NLM: 101580575
Informations de publication
Date de publication:
02 May 2024
02 May 2024
Historique:
received:
20
10
2023
accepted:
13
04
2024
medline:
3
5
2024
pubmed:
3
5
2024
entrez:
2
5
2024
Statut:
epublish
Résumé
Systematic reviews are viewed as the best study design to guide clinical decision-making as they are the least biased publications assuming they are well-conducted and include well-designed studies. Cochrane was initiated in 1993 with an aim of conducting high-quality systematic reviews. We aimed to examine the publication rates of non-Cochrane systematic reviews (henceforth referred to simply as "systematic reviews") and Cochrane reviews produced throughout Cochrane's existence and characterize changes throughout the period. This observational study collected data on systematic reviews published between 1993 and 2022 in PubMed. Identified Cochrane reviews were linked to data from the Cochrane Database of Systematic Reviews via their Digital Object Identifier. Systematic reviews and Cochrane reviews were analyzed separately. Two authors screened a random sample of records to validate the overall sample, providing a precision of 98%. We identified 231,602 (94%) systematic reviews and 15,038 (6%) Cochrane reviews. Publication of systematic reviews has continuously increased with a median yearly increase rate of 26%, while publication of Cochrane reviews has decreased since 2015. From 1993 to 2002, Cochrane reviews constituted 35% of all systematic reviews in PubMed compared with 3.5% in 2013-2022. Systematic reviews consistently had fewer authors than Cochrane reviews, but the number of authors increased over time for both. Chinese first authors conducted 15% and 4% of systematic reviews published from 2013-2022 and 2003-2012, respectively. Most Cochrane reviews had first authors from the UK (36%). The native English-speaking countries the USA, the UK, Canada, and Australia produced a large share of systematic reviews (42%) and Cochrane reviews (62%). The largest publishers of systematic reviews in the last 10 years were gold open access journals. Publication of systematic reviews is increasing rapidly, while fewer Cochrane reviews have been published through the last decade. Native English-speaking countries produced a large proportion of both types of systematic reviews. Gold open access journals and Chinese first authors dominated the publication of systematic reviews for the past 10 years. More research is warranted examining why fewer Cochrane reviews are being published. Additionally, examining these systematic reviews for research waste metrics may provide a clearer picture of their utility.
Sections du résumé
BACKGROUND
BACKGROUND
Systematic reviews are viewed as the best study design to guide clinical decision-making as they are the least biased publications assuming they are well-conducted and include well-designed studies. Cochrane was initiated in 1993 with an aim of conducting high-quality systematic reviews. We aimed to examine the publication rates of non-Cochrane systematic reviews (henceforth referred to simply as "systematic reviews") and Cochrane reviews produced throughout Cochrane's existence and characterize changes throughout the period.
METHODS
METHODS
This observational study collected data on systematic reviews published between 1993 and 2022 in PubMed. Identified Cochrane reviews were linked to data from the Cochrane Database of Systematic Reviews via their Digital Object Identifier. Systematic reviews and Cochrane reviews were analyzed separately. Two authors screened a random sample of records to validate the overall sample, providing a precision of 98%.
RESULTS
RESULTS
We identified 231,602 (94%) systematic reviews and 15,038 (6%) Cochrane reviews. Publication of systematic reviews has continuously increased with a median yearly increase rate of 26%, while publication of Cochrane reviews has decreased since 2015. From 1993 to 2002, Cochrane reviews constituted 35% of all systematic reviews in PubMed compared with 3.5% in 2013-2022. Systematic reviews consistently had fewer authors than Cochrane reviews, but the number of authors increased over time for both. Chinese first authors conducted 15% and 4% of systematic reviews published from 2013-2022 and 2003-2012, respectively. Most Cochrane reviews had first authors from the UK (36%). The native English-speaking countries the USA, the UK, Canada, and Australia produced a large share of systematic reviews (42%) and Cochrane reviews (62%). The largest publishers of systematic reviews in the last 10 years were gold open access journals.
CONCLUSIONS
CONCLUSIONS
Publication of systematic reviews is increasing rapidly, while fewer Cochrane reviews have been published through the last decade. Native English-speaking countries produced a large proportion of both types of systematic reviews. Gold open access journals and Chinese first authors dominated the publication of systematic reviews for the past 10 years. More research is warranted examining why fewer Cochrane reviews are being published. Additionally, examining these systematic reviews for research waste metrics may provide a clearer picture of their utility.
Identifiants
pubmed: 38698429
doi: 10.1186/s13643-024-02531-2
pii: 10.1186/s13643-024-02531-2
doi:
Types de publication
Journal Article
Comparative Study
Research Support, Non-U.S. Gov't
Observational Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
120Informations de copyright
© 2024. The Author(s).
Références
Bornmann L, Haunschild R, Mutz R. Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanit Soc Sci Commun. 2021;8:224. https://doi.org/10.1057/s41599-021-00903-w .
doi: 10.1057/s41599-021-00903-w
Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71. https://doi.org/10.1136/bmj.312.7023.71 .
doi: 10.1136/bmj.312.7023.71
pubmed: 8555924
pmcid: 2349778
Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. BMJ Evid Based Med. 2016;21:125–7. https://doi.org/10.1136/ebmed-2016-110401 .
doi: 10.1136/ebmed-2016-110401
Murad MH. Clinical practice guidelines: a primer on development and dissemination. Mayo Clin Proc. 2017;92:423–33. https://doi.org/10.1016/j.mayocp.2017.01.001 .
doi: 10.1016/j.mayocp.2017.01.001
pubmed: 28259229
Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156:525–31. https://doi.org/10.7326/0003-4819-156-7-201204030-00009 .
doi: 10.7326/0003-4819-156-7-201204030-00009
pubmed: 22473437
Ault A. Climbing a medical everest. Science. 2003;300:2024–5. https://doi.org/10.1126/science.300.5628.2024 .
doi: 10.1126/science.300.5628.2024
pubmed: 12829761
Clarke M, Chalmers I. Reflections on the history of systematic reviews. BMJ Evidence-Based Med. 2018;23:121–2. https://doi.org/10.1136/BMJEBM-2018-110968 .
doi: 10.1136/BMJEBM-2018-110968
Lind J. A Treatise of the scurvy in three parts. Edinburgh: Sands, Murray and Cochran; 1753. https://www.jameslindlibrary.org/lind-j-1753/ .
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook of Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane 2022. https://training.cochrane.org/handbook (accessed 19 Apr 2023).
Higgins JPT, Lasserson T, Thomas J, Flemyng E, Churchill R. Methodological Expectations of Cochrane Intervention Reviews. London: Cochrane; 2023. https://community.cochrane.org/mecir-manual/key-points-and-introduction/how-cite-mecir-standards .
Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283:2008–12. https://doi.org/10.1001/jama.283.15.2008 .
doi: 10.1001/jama.283.15.2008
pubmed: 10789670
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372: n71. https://doi.org/10.1136/bmj.n71 .
doi: 10.1136/bmj.n71
pubmed: 33782057
pmcid: 8005924
Home Page. JBI 2023. https://jbi.global/ . Accessed 13 Sept 2023.
Benchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I, et al. The Reporting of studies conducted using observational routinely-collected health data (RECORD) statement. PLOS Med. 2015;12:e1001885. https://doi.org/10.1371/journal.pmed.1001885 .
doi: 10.1371/journal.pmed.1001885
pubmed: 26440803
pmcid: 4595218
Buciek JH, Andersen MZ, Fonnes S, Andresen K, Rosenberg J. The historical group of withdrawn Cochrane reviews should be distinguished from retracted papers. J Evid Based Med. 2021;14:269–71. https://doi.org/10.1111/JEBM.12449 .
doi: 10.1111/JEBM.12449
pubmed: 34632706
Bramer WM, Giustini D, Kramer BM, Anderson PF. The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Syst Rev. 2013;2:115. https://doi.org/10.1186/2046-4053-2-115 .
doi: 10.1186/2046-4053-2-115
pubmed: 24360284
pmcid: 3882110
PubMed. NLM 2023. https://pubmed.ncbi.nlm.nih.gov/ . Accessed 13 Sept 2023.
Cochrane Database of Systematic Reviews. Cochrane 2023. https://www.cochranelibrary.com/cdsr/reviews . Accessed 13 Sept 2023.
Halladay CW, Trikalinos TA, Schmid IT, Schmid CH, Dahabreh IJ. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol. 2015;68:1076–84. https://doi.org/10.1016/J.JCLINEPI.2014.12.017 .
doi: 10.1016/J.JCLINEPI.2014.12.017
pubmed: 26279401
United Nations. Eleventh United Nations conference on the standardization of geographical names. United Nations; 2018. https://doi.org/10.18356/823a24b2-en .
Andersen MZ, Gülen S, Fonnes S, Andresen K, Rosenberg J. Half of Cochrane reviews were published more than 2 years after the protocol. J Clin Epidemiol. 2020;124:85–93. https://doi.org/10.1016/j.jclinepi.2020.05.011 .
doi: 10.1016/j.jclinepi.2020.05.011
pubmed: 32413390
Tricco AC, Brehaut J, Chen MH, Moher D. Following 411 Cochrane protocols to completion: a retrospective cohort study. PLoS ONE. 2008;3:e3684. https://doi.org/10.1371/journal.pone.0003684 .
doi: 10.1371/journal.pone.0003684
pubmed: 18997866
pmcid: 2577297
Runjic E, Behmen D, Pieper D, Mathes T, Tricco AC, Moher D, et al. Following Cochrane review protocols to completion 10 years later: a retrospective cohort study and author survey. J Clin Epidemiol. 2019;111:41–8. https://doi.org/10.1016/j.jclinepi.2019.03.006 .
doi: 10.1016/j.jclinepi.2019.03.006
pubmed: 30928507
Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7:e012545. https://doi.org/10.1136/bmjopen-2016-012545 .
doi: 10.1136/bmjopen-2016-012545
pubmed: 28242767
pmcid: 5337708
Tsujimoto H, Tsujimoto Y, Kataoka Y. Unpublished systematic reviews and financial support: a meta-epidemiological study. BMC Res Notes. 2017;10:703. https://doi.org/10.1186/s13104-017-3043-5 .
doi: 10.1186/s13104-017-3043-5
pubmed: 29208054
pmcid: 5717810
Andersen MZ, Fonnes S, Andresen K, Rosenberg J. Most published meta-analyses were made available within two years of protocol registration. Eur J Integr Med. 2021;44:101342. https://doi.org/10.1016/j.eujim.2021.101342 .
doi: 10.1016/j.eujim.2021.101342
Collier A, Heilig L, Schilling L, Williams H, Dellavalle RP. Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology. Br J Dermatol. 2006;155:1230–5. https://doi.org/10.1111/j.1365-2133.2006.07496.x .
doi: 10.1111/j.1365-2133.2006.07496.x
pubmed: 17107394
Windsor B, Popovich I, Jordan V, Showell M, Shea B, Farquhar C. Methodological quality of systematic reviews in subfertility: a comparison of Cochrane and non-Cochrane systematic reviews in assisted reproductive technologies. Hum Reprod. 2012;27:3460–6. https://doi.org/10.1093/humrep/des342 .
doi: 10.1093/humrep/des342
pubmed: 23034152
Fleming PS, Seehra J, Polychronopoulou A, Fedorowicz Z, Pandis N. Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm? Eur J Orthod. 2013;35:244–8. https://doi.org/10.1093/ejo/cjs016 .
doi: 10.1093/ejo/cjs016
pubmed: 22510325
Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78. https://doi.org/10.1371/journal.pmed.0040078 .
doi: 10.1371/journal.pmed.0040078
pubmed: 17388659
pmcid: 1831728
Wen J, Ren Y, Wang L, Li Y, Liu Y, Zhou M, et al. The reporting quality of meta-analyses improves: a random sampling study. J Clin Epidemiol. 2008;61:770–5. https://doi.org/10.1016/j.jclinepi.2007.10.008 .
doi: 10.1016/j.jclinepi.2007.10.008
pubmed: 18411041
Delaney A, Bagshaw SM, Ferland A, Laupland K, Manns B, Doig C. The quality of reports of critical care meta-analyses in the Cochrane Database of Systematic Reviews: an independent appraisal. Crit Care Med. 2007;35:589–94. https://doi.org/10.1097/01.CCM.0000253394.15628.FD .
doi: 10.1097/01.CCM.0000253394.15628.FD
pubmed: 17205029
Bunn F, Trivedi D, Alderson P, Hamilton L, Martin A, Iliffe S. The impact of Cochrane Systematic Reviews: a mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research. Syst Rev. 2014;3:125. https://doi.org/10.1186/2046-4053-3-125 .
doi: 10.1186/2046-4053-3-125
pubmed: 25348511
pmcid: 4238314
Champion C. Cochrane Author Satisfaction Survey 2019. London, UK: 2020. https://community.cochrane.org/sites/default/files/uploads/inline-files/Cochrane%20Author%20Satisfaction%20Survey%202019%20Report.pdf .
Cochrane’s new production model. Cochrane 2023. https://futurecochrane.org/new-production-model . Accessed 13 Sept 2023.
Key projects and milestones. Cochrane 2023. https://futurecochrane.org/key-projects-and-milestones-nav . Accessed 13 Sept 2023.
Tian J, Zhang J, Ge L, Yang K, Song F. The methodological and reporting quality of systematic reviews from China and the USA are similar. J Clin Epidemiol. 2017;85:50–8. https://doi.org/10.1016/J.JCLINEPI.2016.12.004 .
doi: 10.1016/J.JCLINEPI.2016.12.004
pubmed: 28063911
Tennant JP, Waldner F, Jacques DC, Masuzzo P, Collister LB, Hartgerink CHJ. The academic, economic and societal impacts of Open Access: an evidence-based review [version 3; peer review: 4 approved, 1 approved with reservations]. F1000Research. 2016;5:632. https://doi.org/10.12688/f1000research.8460.3 .
doi: 10.12688/f1000research.8460.3
pubmed: 27158456
pmcid: 4837983
Khoo SY-S. Article processing charge hyperinflation and price insensitivity: an open access sequel to the serials crisis. Lib Q. 2019;29:1–18. https://doi.org/10.18352/lq.10280 .
doi: 10.18352/lq.10280
Fuchs C, Sandoval M. The diamond model of open access publishing: why policy makers, scholars, universities, libraries, labour unions and the publishing world need to take non-commercial, non-profit open access serious. TripleC: Communication, capitalism & critique. 2013;11:428–43. https://doi.org/10.31269/triplec.v11i2.502 .
doi: 10.31269/triplec.v11i2.502
Glasziou P, Chalmers I. Research waste is still a scandal—an essay by Paul Glasziou and Iain Chalmers. BMJ. 2018;363:k4645. https://doi.org/10.1136/BMJ.K4645 .
doi: 10.1136/BMJ.K4645
Altman DG. The scandal of poor medical research. BMJ. 1994;308:283–4. https://doi.org/10.1136/BMJ.308.6924.283 .
doi: 10.1136/BMJ.308.6924.283
pubmed: 8124111
pmcid: 2539276
Puljak L, Lund H. Definition, harms, and prevention of redundant systematic reviews. Syst Rev. 2023;12:63. https://doi.org/10.1186/S13643-023-02191-8 .
doi: 10.1186/S13643-023-02191-8
pubmed: 37016459
pmcid: 10071231
Grainger MJ, Bolam FC, Stewart GB, Nilsen EB. Evidence synthesis for tackling research waste. Nat Ecol Evol. 2020;4:495–7. https://doi.org/10.1038/s41559-020-1141-6 .
doi: 10.1038/s41559-020-1141-6
pubmed: 32203478
Gülen S, Fonnes S, Andresen K, Rosenberg J. Increasing number of authors in Cochrane reviews. J Evid Based Med. 2020;13:34–41. https://doi.org/10.1111/jebm.12371 .
doi: 10.1111/jebm.12371
pubmed: 32086993
Tilak G, Prasad V, Jena AB. Authorship inflation in medical publications. Inquiry. 2015;52:0046958015598311. https://doi.org/10.1177/0046958015598311 .
doi: 10.1177/0046958015598311
pubmed: 26228035
pmcid: 4943864
Vale RD. Accelerating scientific publication in biology. Proc Natl Acad Sci U S A. 2015;112:13439–46. https://doi.org/10.1073/pnas.1511912112 .
doi: 10.1073/pnas.1511912112
pubmed: 26508643
pmcid: 4640799
Papatheodorou SI, Trikalinos TA, Ioannidis JPA. Inflated numbers of authors over time have not been just due to increasing research complexity. J Clin Epidemiol. 2008;61:546–51. https://doi.org/10.1016/J.JCLINEPI.2007.07.017 .
doi: 10.1016/J.JCLINEPI.2007.07.017
pubmed: 18471658
Bachmann LM, Estermann P, Kronenberg C, Ter Riet G. Identifying diagnostic accuracy studies in EMBASE. J Med Libr Assoc. 2003;91:341–6.
pubmed: 12883560
pmcid: 164397
Hoffmeyer BD, Andersen MZ, Fonnes S, Rosenberg J. Most Cochrane reviews have not been updated for more than 5 years. J Evid Based Med. 2021;14:181–4. https://doi.org/10.1111/JEBM.12447 .
doi: 10.1111/JEBM.12447
pubmed: 34427395