Feasibility study and evaluation of expert opinion on the semi-automated meta-analysis and the conventional meta-analysis.
Artificial intelligence
Automation
Machine learning
Meta-analysis
Systematic review
Journal
European journal of clinical pharmacology
ISSN: 1432-1041
Titre abrégé: Eur J Clin Pharmacol
Pays: Germany
ID NLM: 1256165
Informations de publication
Date de publication:
Jul 2022
Jul 2022
Historique:
received:
13
01
2022
accepted:
24
04
2022
pubmed:
3
5
2022
medline:
14
6
2022
entrez:
2
5
2022
Statut:
ppublish
Résumé
To assess the feasibility and acceptance of the semi-automated meta-analysis (SAMA). The objectives are twofold, namely (1) to compare expert opinion on the quality of protocols, methods, and results of one conventional meta-analysis (CMA) and one SAMA and (2) to compare the time to execute the CMA and the SAMA. Experts evaluated the protocols and manuscripts/reports of the CMA and SAMA conducted independently on the safety of metronidazole in pregnancy. Expert opinion was collected using AMSTAR 2 checklist. Time spent was recorded using case report forms. The overall scores of the opinion of all experts for protocols, methods, and results for SAMA (6.75) and CMA (6.87) were not statistically different (p = 0.88). The experts' confidence in the results of each MA was 7.89 ± 1.17 and 8.11 ± 0.92, respectively. The time to completion was 14 working days for SAMA and 24.7 for CMA. MA tasks such as calculation of effect estimates, subgroup/sensitivity analysis, and publication bias investigation required no investment in time for SAMA. In conclusion, our study demonstrated the feasibility of SAMA and suggests acceptance for risk assessment by an expert committee. Our results suggest that SAMA reduces the time required for a MA without altering expert confidence in the methodological and scientific rigor. As our study was limited to one example, the generalization of our results requires confirmation by other studies.
Identifiants
pubmed: 35501476
doi: 10.1007/s00228-022-03329-8
pii: 10.1007/s00228-022-03329-8
doi:
Types de publication
Journal Article
Meta-Analysis
Langues
eng
Sous-ensembles de citation
IM
Pagination
1177-1184Informations de copyright
© 2022. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.
Références
Haidich AB (2010) Meta-analysis in medical research. Hippokratia 14(Suppl 1):29–37
pubmed: 21487488
pmcid: 3049418
Greco T, Zangrillo A, Biondi-Zoccai G, Landoni G (2013) Meta-analysis: pitfalls and hints. Heart Lung Vessels 5(4):219–225
pubmed: 24364016
pmcid: 3868184
Bastian H, Glasziou P, Chalmers I (2010) Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 7(9):e1000326
doi: 10.1371/journal.pmed.1000326
Allen IE, Olkin I (1999) Estimating time to conduct a meta-analysis from number of citations retrieved. JAMA 282(7):634–635
doi: 10.1001/jama.282.7.634
Borah R, Brown AW, Capers PL, Kaiser KA (2017) Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2):e012545
doi: 10.1136/bmjopen-2016-012545
Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S et al (2017) Living systematic review: 1. Introduction—the why, what, when, and how. J Clin Epidemiol 91:23–30
doi: 10.1016/j.jclinepi.2017.08.010
Thomas J, Noel-Storr A, Marshall I, Wallace B, McDonald S, Mavergames C et al (2017) Living systematic reviews: 2. Combining human and machine effort. J Clin Epidemiol 91:31–37
doi: 10.1016/j.jclinepi.2017.08.011
Safi S, Thiessen T, Schmailzl KJ (2018) Acceptance and resistance of new digital technologies in medicine: qualitative study. JMIR Res Protoc 7(12):e11072
doi: 10.2196/11072
O’Connor AM, Tsafnat G, Thomas J, Glasziou P, Gilbert SB, Hutton B (2019) A question of trust: can we build an evidence base to gain trust in systematic review automation technologies? Syst Rev 8:143
doi: 10.1186/s13643-019-1062-0
Clark J, Glasziou P, Del Mar C, Bannach-Brown A, Stehlik P, Scott AM (2020) A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol 121:81–90
doi: 10.1016/j.jclinepi.2020.01.008
O’Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS (2018) Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Rev 7(1):3
doi: 10.1186/s13643-017-0667-4
Marshall IJ, Wallace BC (2019) Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev 8(1):163
doi: 10.1186/s13643-019-1074-9
Gates A, Gates M, DaRosa D, Elliott SA, Pillay J, Rahman S et al (2020) Decoding semi-automated title-abstract screening: findings from a convenience sample of reviews. Syst Rev 9(1):272
doi: 10.1186/s13643-020-01528-x
Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J et al (2017) AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017;358:j4008. [Accessed 13 févr 2020];358. https://www.bmj.com/content/358/bmj.j4008
Pussegoda K, Turner L, Garritty C, Mayhew A, Skidmore B, Stevens A et al (2017) Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study. Syst Rev 6(1):117
doi: 10.1186/s13643-017-0507-6
Hemming V, Burgman MA, Hanea AM, McBride MF, Wintle BC (2018) A practical guide to structured expert elicitation using the IDEA protocol. Methods Ecol Evol 9(1):169–180
doi: 10.1111/2041-210X.12857
Speirs-Bridge A, Fidler F, McBride M, Flander L, Cumming G, Burgman M (2010) Reducing overconfidence in the interval judgments of experts. Risk Anal Off Publ Soc Risk Anal 30(3):512–523
R Core Team (2020) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria [Internet]. 2020 [Accessed 30 mars 2020] https://www.r-project.org/
Viechtbauer W (2010) Conducting meta-analyses in R with the metafor package. J Stat Softw 36(1):1–48
Balduzzi S, Rücker G, Schwarzer G (2019) How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health 22(4):153–160
doi: 10.1136/ebmental-2019-300117
Collignon A, Cuxac P (2017) ISTEX: des enrichissements au web de données. I2D - Inf Donnees Doc 54(4):8‑15
Higgins JP, Green S (2008) Cochrane Handbook for Systematic Reviews of Interventions: Chichester, UK: John Wiley & Sons, Ltd. http://doi.wiley.com/10.1002/9780470712184
van Altena AJ, Spijker R, Olabarriaga SD (2019) Usage of automation tools in systematic reviews. Res Synth Methods 10(1):72–82
doi: 10.1002/jrsm.1335
Chai KEK, Lines RLJ, Gucciardi DF et al (2021) Research screener: a machine learning tool to semi-automate abstract screening for systematic reviews | SpringerLink [Internet]. [Accessed 28 avr 2021]. https://link.springer.com/article/10.1186/s13643-021-01635-3
Low J, Ross JS, Ritchie JD, Gross CP, Lehman R, Lin H et al (2017) Comparison of two independent systematic reviews of trials of recombinant human bone morphogenetic protein-2 (rhBMP-2): the Yale Open Data Access Medtronic Project. Syst Rev 6(1):28
doi: 10.1186/s13643-017-0422-x
Wiseman MJ (2008) Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer. Public Health Nutrition, 11 (10). 1006 - 1014 . ISSN 1368-9800 https://doi.org/10.1017/S1368980007001334 [Accessed 24 févr 2021] https://core.ac.uk/reader/13505414?utm_source=linkout