Semi-automating abstract screening with a natural language model pretrained on biomedical literature.
Abstract
Classification
Semi-automation
Journal
Systematic reviews
ISSN: 2046-4053
Titre abrégé: Syst Rev
Pays: England
ID NLM: 101580575
Informations de publication
Date de publication:
23 09 2023
23 09 2023
Historique:
received:
11
08
2023
accepted:
13
09
2023
medline:
25
9
2023
pubmed:
23
9
2023
entrez:
22
9
2023
Statut:
epublish
Résumé
We demonstrate the performance and workload impact of incorporating a natural language model, pretrained on citations of biomedical literature, on a workflow of abstract screening for studies on prognostic factors in end-stage lung disease. The model was optimized on one-third of the abstracts, and model performance on the remaining abstracts was reported. Performance of the model, in terms of sensitivity, precision, F1 and inter-rater agreement, was moderate in comparison with other published models. However, incorporating it into the screening workflow, with the second reviewer screening only abstracts with conflicting decisions, translated into a 65% reduction in the number of abstracts screened by the second reviewer. Subsequent work will look at incorporating the pre-trained BERT model into screening workflows for other studies prospectively, as well as improving model performance.
Identifiants
pubmed: 37740227
doi: 10.1186/s13643-023-02353-8
pii: 10.1186/s13643-023-02353-8
pmc: PMC10517490
doi:
Types de publication
Letter
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
172Informations de copyright
© 2023. BioMed Central Ltd., part of Springer Nature.
Références
Syst Rev. 2019 Jul 11;8(1):163
pubmed: 31296265
Syst Rev. 2020 Dec 13;9(1):293
pubmed: 33308292
J Clin Epidemiol. 2021 May;133:121-129
pubmed: 33485929
J Am Med Inform Assoc. 2022 Jul 12;29(8):1425-1432
pubmed: 35641139
J Palliat Med. 2023 Sep 13;:
pubmed: 37702606
Res Synth Methods. 2022 May;13(3):353-362
pubmed: 35174972
Syst Rev. 2021 Oct 30;10(1):285
pubmed: 34717768
Syst Rev. 2019 Nov 15;8(1):278
pubmed: 31727150
Syst Rev. 2020 Nov 27;9(1):272
pubmed: 33243276