Optimizing Signal Management in a Vaccine Adverse Event Reporting System: A Proof-of-Concept with COVID-19 Vaccines Using Signs, Symptoms, and Natural Language Processing.
Journal
Drug safety
ISSN: 1179-1942
Titre abrégé: Drug Saf
Pays: New Zealand
ID NLM: 9002928
Informations de publication
Date de publication:
07 Dec 2023
07 Dec 2023
Historique:
accepted:
14
11
2023
medline:
8
12
2023
pubmed:
8
12
2023
entrez:
7
12
2023
Statut:
aheadofprint
Résumé
The Vaccine Adverse Event Reporting System (VAERS) has already been challenged by an extreme increase in the number of individual case safety reports (ICSRs) after the market introduction of coronavirus disease 2019 (COVID-19) vaccines. Evidence from scientific literature suggests that when there is an extreme increase in the number of ICSRs recorded in spontaneous reporting databases (such as the VAERS), an accompanying increase in the number of disproportionality signals (sometimes referred to as 'statistical alerts') generated is expected. The objective of this study was to develop a natural language processing (NLP)-based approach to optimize signal management by excluding disproportionality signals related to listed adverse events following immunization (AEFIs). COVID-19 vaccines were used as a proof-of-concept. The VAERS was used as a data source, and the Finding Associated Concepts with Text Analysis (FACTA+) was used to extract signs and symptoms of listed AEFIs from MEDLINE for COVID-19 vaccines. Disproportionality analyses were conducted according to guidelines and recommendations provided by the US Centers for Disease Control and Prevention. By using signs and symptoms of listed AEFIs, we computed the proportion of disproportionality signals dismissed for COVID-19 vaccines using this approach. Nine NLP techniques, including Generative Pre-Trained Transformer 3.5 (GPT-3.5), were used to automatically retrieve Medical Dictionary for Regulatory Activities Preferred Terms (MedDRA PTs) from signs and symptoms extracted from FACTA+. Overall, 17% of disproportionality signals for COVID-19 vaccines were dismissed as they reported signs and symptoms of listed AEFIs. Eight of nine NLP techniques used to automatically retrieve MedDRA PTs from signs and symptoms extracted from FACTA+ showed suboptimal performance. GPT-3.5 achieved an accuracy of 78% in correctly assigning MedDRA PTs. Our approach reduced the need for manual exclusion of disproportionality signals related to listed AEFIs and may lead to better optimization of time and resources in signal management.
Identifiants
pubmed: 38062261
doi: 10.1007/s40264-023-01381-6
pii: 10.1007/s40264-023-01381-6
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2023. The Author(s).
Références
Arepally GM, Ortel TL. Vaccine-induced immune thrombotic thrombocytopenia: what we know and do not know. Blood. 2021;138(4):293–8.
doi: 10.1182/blood.2021012152
pubmed: 34323940
pmcid: 8172307
Prasad N, Derado G, Nanduri SA, Reses HE, Dubendris H, Wong E, et al. Effectiveness of a COVID-19 additional primary or booster vaccine dose in preventing SARS-CoV-2 infection among nursing home residents during widespread circulation of the omicron variant—United States, February 14–March 27, 2022. MMWR Morb Mortal Wkly Rep. 2022;71(18):633–7.
doi: 10.15585/mmwr.mm7118a4
pubmed: 35511708
pmcid: 9098239
Ceacareanu A, Wintrob ZP. Summary of COVID-19 vaccine-related reports in the vaccine adverse event reporting system. J Res Pharm Pract. 2021;10(3):107.
doi: 10.4103/jrpp.jrpp_49_21
pubmed: 35198503
pmcid: 8809454
Spiteri J. Media bias exposure and the incidence of COVID-19 in the USA. BMJ Glob Health. 2021;6(9): e006798.
doi: 10.1136/bmjgh-2021-006798
pubmed: 34518207
Vaccine Adverse Event Reporting System (VAERS) standard operating procedures for COVID-19 Centers for Diease Control and Prevention. 2022 [cited 10 Oct 2022]. Available at: https://www.cdc.gov/vaccinesafety/pdf/VAERS-COVID19-SOP-02-02-2022-508.pdf .
Grundmark B, Holmberg L, Garmo H, Zethelius B. Reducing the noise in signal detection of adverse drug reactions by standardizing the background: a pilot study on analyses of proportional reporting ratios-by-therapeutic area. Eur J Clin Pharmacol. 2014;70(5):627–35.
doi: 10.1007/s00228-014-1658-1
pubmed: 24599513
pmcid: 3978377
Chen RT, Rastogi SC, Mullen JR, Hayes SW, Cochi SL, Donlon JA, et al. The Vaccine Adverse Event Reporting System (VAERS). Vaccine. 1994;12(6):542–50.
doi: 10.1016/0264-410X(94)90315-8
pubmed: 8036829
Faillie JL. Case-non-case studies: principle, methods, bias and interpretation. Therapie. 2019;74(2):225–32.
doi: 10.1016/j.therap.2019.01.006
pubmed: 30773344
Tsuruoka Y, Miwa M, Hamamoto K, Tsujii J, Ananiadou S. Discovering and visualizing indirect associations between biomedical concepts. Bioinformatics. 2011;27(13):i111–9.
doi: 10.1093/bioinformatics/btr214
pubmed: 21685059
pmcid: 3117364
Lu J, Lin C, Wang W, Li C, Wang H (eds). String similarity measures and joins with synonyms. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data. New York: ACM; 2013.
Gu Y, Tinn R, Cheng H, Lucas M, Usuyama N, Liu X, et al. Domain-specific language model pretraining for biomedical natural language processing. arXivorg. 2021.
Reimers N, Gurevych I. Sentence-BERT: sentence embeddings using Siamese BERT-Networks. arXivorg. 2019.
Bate A, Brown EG, Goldman SA, Hauben M. Terminological challenges in safety surveillance. Drug Saf. 2012;35(1):79–84.
doi: 10.2165/11598700-000000000-00000
pubmed: 22136184
Inclusion/exclusion criteria for the “Important Medical Events” list European Medicines Agency 2021 [cited 8 May 2023]. Available at: https://www.ema.europa.eu/en/documents/other/inclusion-exclusion-criteria-important-medical-events-list-meddra_en.pdf .
Tsuruoka Y, Tsujii J, Ananiadou S. FACTA: a text search engine for finding associated biomedical concepts. Bioinformatics. 2008;24(21):2559–60.
doi: 10.1093/bioinformatics/btn469
pubmed: 18772154
pmcid: 2572701
Definition and application of terms for vaccine pharmacovigilance Council for International Organizations of Medical Sciences (CIOMS) 2012 [cited 24 Apr 2023]. Available at: https://cioms.ch/wp-content/uploads/2017/01/report_working_group_on_vaccine_LR.pdf .
Sessa M, Mascolo A, Callreus T, Capuano A, Rossi F, Andersen M. Direct-acting oral anticoagulants (DOACs) in pregnancy: new insight from VigiBase
doi: 10.1038/s41598-019-43715-4
pubmed: 31076635
pmcid: 6510783
Sessa M, Rafaniello C, Sportiello L, Mascolo A, Scavone C, Maccariello A, et al. Campania Region (Italy) spontaneous reporting system and preventability assessment through a case-by-case approach: a pilot study on psychotropic drugs. Expert Opin Drug Saf. 2016;15(Suppl 2):9–15.
doi: 10.1080/14740338.2016.1221397
pubmed: 27875917
Sessa M, Sportiello L, Mascolo A, Scavone C, Gallipoli S, di Mauro G, et al. Campania Preventability Assessment Committee (Italy): a focus on the preventability of non-steroidal anti-inflammatory drugs’ adverse drug reactions. Front Pharmacol. 2017;8:305.
doi: 10.3389/fphar.2017.00305
pubmed: 28603499
pmcid: 5445158
Sessa M, Rossi C, Mascolo A, Grassi E, Fiorentino S, Scavone C, et al. Suspected adverse reactions to contrast media in Campania Region (Italy): results from 14 years of post-marketing surveillance. Expert Opin Drug Saf. 2015;14(9):1341–51.
doi: 10.1517/14740338.2015.1067301
pubmed: 26156557
Sessa M, Rossi C, Rafaniello C, Mascolo A, Cimmaruta D, Scavone C, et al. Campania preventability assessment committee: a focus on the preventability of the contrast media adverse drug reactions. Expert Opin Drug Saf. 2016;15(Suppl 2):51–9.
doi: 10.1080/14740338.2016.1226280
pubmed: 27855534
Sessa M, di Mauro G, Mascolo A, Rafaniello C, Sportiello L, Scavone C, et al. Pillars and pitfalls of the new pharmacovigilance legislation: consequences for the identification of adverse drug reactions deriving from abuse, misuse, overdose, occupational exposure, and medication errors. Front Pharmacol. 2018;9:611.
doi: 10.3389/fphar.2018.00611
pubmed: 29946258
pmcid: 6006791
Avillach P, Coloma PM, Gini R, Schuemie M, Mougin F, Dufour JC, et al. Harmonization process for the identification of medical events in eight European healthcare databases: the experience from the EU-ADR project. J Am Med Inform Assoc. 2013;20(1):184–92.
doi: 10.1136/amiajnl-2012-000933
pubmed: 22955495
Voss EA, Boyce RD, Ryan PB, van der Lei J, Rijnbeek PR, Schuemie MJ. Accuracy of an automated knowledge base for identifying drug adverse reactions. J Biomed Inform. 2017;66:72–81.
doi: 10.1016/j.jbi.2016.12.005
pubmed: 27993747
Navarro G. A guided tour to approximate string matching. ACM Comput Surv. 2001;33(1):31–88.
doi: 10.1145/375360.375365
Belousov M, Dixon WG, Nenadic G (eds). MedNorm: a corpus and embeddings for cross-terminology medical concept normalisation. In: Proceedings of the Fourth Social Media Mining for Health Applications (#SMM4H) Workshop & Shared Task. Stroudsburg: Association for Computational Linguistics; 2019.
Cho H, Choi W, Lee H. A method for named entity normalization in biomedical articles: application to diseases and plants. BMC Bioinform. 2017;18(1):451.
doi: 10.1186/s12859-017-1857-8
Wang P, Wang S, Huang R, Huang Z. Quantifying bounds of model gap for synchronous generators. arXiv pre-print server. 2021.
Harpaz R, Callahan A, Tamang S, Low Y, Odgers D, Finlayson S, et al. Text mining for adverse drug events: the promise, challenges, and state of the art. Drug Saf. 2014;37(10):777–90.
doi: 10.1007/s40264-014-0218-z
pubmed: 25151493
pmcid: 4217510
Shivade C, Raghavan P, Fosler-Lussier E, Embi PJ, Elhadad N, Johnson SB, et al. A review of approaches to identifying patient phenotype cohorts using electronic health records. J Am Med Inform Assoc. 2014;21(2):221–30.
doi: 10.1136/amiajnl-2013-001935
pubmed: 24201027