Black Swan Events and Intelligent Automation for Routine Safety Surveillance.


Journal

Drug safety
ISSN: 1179-1942
Titre abrégé: Drug Saf
Pays: New Zealand
ID NLM: 9002928

Informations de publication

Date de publication:
05 2022
Historique:
accepted: 27 02 2022
entrez: 17 5 2022
pubmed: 18 5 2022
medline: 20 5 2022
Statut: ppublish

Résumé

Effective identification of previously implausible safety signals is a core component of successful pharmacovigilance. Timely, reliable, and efficient data ingestion and related processing are critical to this. The term 'black swan events' was coined by Taleb to describe events with three attributes: unpredictability, severe and widespread consequences, and retrospective bias. These rare events are not well understood at their emergence but are often rationalized in retrospect as predictable. Pharmacovigilance strives to rapidly respond to potential black swan events associated with medicine or vaccine use. Machine learning (ML) is increasingly being explored in data ingestion tasks. In contrast to rule-based automation approaches, ML can use historical data (i.e., 'training data') to effectively predict emerging data patterns and support effective data intake, processing, and organisation. At first sight, this reliance on previous data might be considered a limitation when building ML models for effective data ingestion in systems that look to focus on the identification of potential black swan events. We argue that, first, some apparent black swan events-although unexpected medically-will exhibit data attributes similar to those of other safety data and not prove algorithmically unpredictable, and, second, standard and emerging ML approaches can still be robust to such data outliers with proper awareness and consideration in ML system design and with the incorporation of specific mitigatory and support strategies. We argue that effective approaches to managing data on potential black swan events are essential for trust and outline several strategies to address data on potential black swan events during data ingestion.

Identifiants

pubmed: 35579807
doi: 10.1007/s40264-022-01169-0
pii: 10.1007/s40264-022-01169-0
pmc: PMC9112242
doi:

Types de publication

Journal Article Research Support, Non-U.S. Gov't

Langues

eng

Sous-ensembles de citation

IM

Pagination

419-427

Informations de copyright

© 2022. TransCelerate BioPharma, Inc.

Références

Lindquist M. Data quality management in pharmacovigilance. Drug Saf. 2004;27(12):857–70.
pubmed: 15366974 doi: 10.2165/00002018-200427120-00003
Stergiopoulos S, Fehrle M, Caubel P, Tan L, Jebson L. Adverse drug reaction case safety practices in large biopharmaceutical organizations from 2007 to 2017: an industry survey. Pharmaceut Med. 2019;33(6):499–510.
pubmed: 31933240
Bate A, Stegmann JU. Safety of medicines and vaccines—building next generation capability. Trends Pharmacol Sci. 2021;42(12):1051–63.
pubmed: 34635346 doi: 10.1016/j.tips.2021.09.007
Ghosh R, Kempf D, Pufko A, Barrios Martinez LF, Davis CM, Sethi S. Automation opportunities in pharmacovigilance: an industry survey. Pharmaceut Med. 2020;34(1):7–18.
pubmed: 32036574
IBM. What is intelligent automation. 2021. https://www.ibm.com/cloud/learn/intelligent-automation
Bate A, Hobbiger SF. Artificial intelligence, real-world automation and the safety of medicines. Drug Saf. 2021;44(2):125–32.
pubmed: 33026641 doi: 10.1007/s40264-020-01001-7
Lewis DJ, McCallum JF. Utilizing advanced technologies to augment pharmacovigilance systems: challenges and opportunities. Ther Innov Regul Sci. 2020;54(4):888–99.
pubmed: 32557311 doi: 10.1007/s43441-019-00023-3
Kassekert R, Easwar M, Glaser M, Ventham R, Bate A. Automation in routine use for data collection and processing for scalable faster RWE generation. Value Health. 2020 (in Press).
Glaser M, Cranfield C, Dsouza D, Duma A, Hastie K, Kassekert R, et al. Automating individual case safety report identification within scientific literature using natural language processing. Pharmacoepidemiol Drug Saf. 2021;30:118–881.
Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. 2019;380(14):1347–58.
pubmed: 30943338 doi: 10.1056/NEJMra1814259
Kumah-Crystal YA, Pirtle CJ, Whyte H, Goode ES, Anders SH, Lehmann CU. Electronic health record interactions through voice: a review. Appl Clin Inform. 2018;9(03):541–52.
pubmed: 30040113 pmcid: 6051768 doi: 10.1055/s-0038-1666844
Huysentruyt K, Kjoersvik O, Dobracki P, Savage E, Mishalov E, Cherry M, et al. Validating intelligent automation systems in pharmacovigilance: insights from good manufacturing practices. Drug Saf. 2021;44(3):261–72.
pubmed: 33523400 pmcid: 7892696 doi: 10.1007/s40264-020-01030-2
Sessa M, Khan AR, Liang D, Andersen M, Kulahci M. Artificial intelligence in pharmacoepidemiology: a systematic review. Part 1—overview of knowledge discovery techniques in artificial intelligence. Front Pharmacol. 2020;11:1028.
pubmed: 32765261 pmcid: 7378532 doi: 10.3389/fphar.2020.01028
Brown EG, Wood L, Wood S. The medical dictionary for regulatory activities (MedDRA). Drug Saf. 1999;20(2):109–17.
pubmed: 10082069 doi: 10.2165/00002018-199920020-00002
Létinier L, Jouganous J, Benkebil M, Bel-Létoile A, Goehrs C, Singier A, et al. Artificial intelligence for unstructured healthcare data: application to coding of patient reporting of adverse drug reactions. Clin Pharmacol Therap. 2021;130:392.
doi: 10.1002/cpt.2266
Bate A, Evans SJ. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiol Drug Saf. 2009;18(6):427–36.
pubmed: 19358225 doi: 10.1002/pds.1742
Taleb NN. Black swans and the domains of statistics. Am Stat. 2007;61(3):198–200.
doi: 10.1198/000313007X219996
Spiegelhalter D. Risk and uncertainty communication. Annu Rev Stat Appl. 2017;4:31–60.
doi: 10.1146/annurev-statistics-010814-020148
Sandman PM, Miller PM, Johnson BB, Weinstein ND. Agency communication, community outrage, and perception of risk: three simulation experiments. Risk Anal. 1993;13(6):585–98.
doi: 10.1111/j.1539-6924.1993.tb01321.x
Kasperson RE, Renn O, Slovic P, Brown HS, Emel J, Goble R, et al. The social amplification of risk: a conceptual framework. Risk Anal. 1988;8(2):177–87.
doi: 10.1111/j.1539-6924.1988.tb01168.x
Bekiros S, Boubaker S, Nguyen DK, Uddin GS. Black swan events and safe havens: the role of gold in globally integrated emerging markets. J Int Money Financ. 2017;73:317–34.
doi: 10.1016/j.jimonfin.2017.02.010
Osterholm MT, Moore KA, Gostin LO. Public health in the age of Ebola in West Africa. JAMA Intern Med. 2015;175(1):7–8.
pubmed: 25317856 doi: 10.1001/jamainternmed.2014.6235
Gray GL, Alles MG. Measuring a business’s grit and survivability when faced with “black swan” events like the coronavirus pandemic. J Emerg Technol Acc. 2021;18(1):195–204.
doi: 10.2308/JETA-2020-060
Yarovaya L, Matkovskyy R, Jalan A. The effects of a “black swan” event (COVID-19) on herding behavior in cryptocurrency markets. J Int Financ Markets Inst Money. 2021;75:101321.
doi: 10.1016/j.intfin.2021.101321
Edwards IR. Causality assessment in pharmacovigilance: still a challenge. Drug Saf. 2017;40(5):365.
doi: 10.1007/s40264-017-0509-2
Fan BE, Shen JY, Lim XR, Tu TM, Chang CCR, Khin HSW, et al. Cerebral venous thrombosis post BNT162b2 mRNA SARS-CoV-2 vaccination: a black swan event. Am J Hematol. 2021;96(9):E357–61.
pubmed: 34133027 pmcid: 8420211 doi: 10.1002/ajh.26272
Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv preprint arXiv:161201474. 2016.
Farina F, Phillips L, Richmond NJ. Intrinsic uncertainties and where to find them. arXiv preprint arXiv:210702526. 2021.
Ovadia Y, Fertig E, Ren J, Nado Z, Sculley D, Nowozin S, et al. Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. arXiv preprint arXiv:190602530. 2019.
Hendrycks D, Mazeika M, Dietterich T. Deep anomaly detection with outlier exposure. arXiv preprint arXiv:181204606. 2018.
Finelli LA, Narasimhan V. Leading a digital transformation in the pharmaceutical industry: reimagining the way we work in global drug development. Clin Pharmacol Ther. 2020;108(4):756–61.
pubmed: 32294230 pmcid: 7540662 doi: 10.1002/cpt.1850
Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D. Weight uncertainty in neural network. International Conference on Machine Learning; 2015: PMLR; 2015. p. 1613–22.
Shafaei A, Schmidt M, Little JJ. A less biased evaluation of out-of-distribution sample detectors. arXiv preprint arXiv:180904729. 2018.
Meinke A, Bitterwolf J, Hein M. Provably Robust Detection of Out-of-distribution Data (almost) for free. arXiv preprint arXiv:210604260. 2021.
Ditzler G, Roveri M, Alippi C, Polikar R. Learning in nonstationary environments: a survey. IEEE Comput Intell Mag. 2015;10(4):12–25.
doi: 10.1109/MCI.2015.2471196
Finlayson SG, Subbaswamy A, Singh K, Bowers J, Kupke A, Zittrain J, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med. 2021;385(3):283–6.
pubmed: 34260843 pmcid: 8665481 doi: 10.1056/NEJMc2104626
Chandra SR. Scalable and secure learning with limited supervision over data streams. https://utd-ir.tdl.org/bitstream/handle/10735.1/6196/ETD-5608-011-CHANDRA-8457.95.pdf?sequence=6&isAllowed=y : Texas; 2018.
Ackerman S, Farchi E, Raz O, Zalmanovici M, Dube P. Detection of data drift and outliers affecting machine learning model performance over time. arXiv preprint arXiv:201209258. 2020.
Lund R. Revenge of the white swan. Am Stat. 2007;61(3):189–92.
doi: 10.1198/000313007X219374
Dietterich TG. Ensemble methods in machine learning. International workshop on multiple classifier systems; 2000: Springer; 2000. p. 1–15.
Finlayson SG, Bowers JD, Ito J, Zittrain JL, Beam AL, Kohane IS. Adversarial attacks on medical machine learning. Science. 2019;363(6433):1287–9.
pubmed: 30898923 pmcid: 7657648 doi: 10.1126/science.aaw4399
Gennatas ED, Friedman JH, Ungar LH, Pirracchio R, Eaton E, Reichmann LG, et al. Expert-augmented machine learning. Proc Natl Acad Sci. 2020;117(9):4571–7.
pubmed: 32071251 pmcid: 7060733 doi: 10.1073/pnas.1906831117
Madras D, Pitassi T, Zemel R. Predict responsibly: improving fairness and accuracy by learning to defer. arXiv preprint arXiv:171106664. 2017.
Mozannar H, Sontag D. Consistent estimators for learning to defer to an expert. In: International Conference on Machine Learning; 2020: PMLR; 2020. p. 7076–87.
Wabartha M, Durand A, Francois-Lavet V, Pineau J. Handling black swan events in deep learning with diversely extrapolated neural networks. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, International Joint Conferences on Artificial Intelligence Organization; 2020; 2020. p. 2140–7.
Kompa B, Snoek J, Beam AL. Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digit Med. 2021;4(1):1–6.
doi: 10.1038/s41746-020-00367-3
McDermott MBA, Wang S, Marinsek N, Ranganath R, Foschini L, Ghassemi M. Reproducibility in machine learning for health research: still a ways to go. Sci Transl Med. 2021;13(586).
Molnar C. Interpretable machine learning: Lulu. com; 2020.
Danysz K, Cicirello S, Mingle E, Assuncao B, Tetarenko N, Mockute R, et al. Artificial intelligence and the future of the drug safety professional. Drug Saf. 2019;42(4):491–7.
pubmed: 30343417 doi: 10.1007/s40264-018-0746-z
Peters J, Janzing D, Schölkopf B. Elements of causal inference: foundations and learning algorithms. The MIT Press; 2017.
Markatou M, Ball R. A pattern discovery framework for adverse event evaluation and inference in spontaneous reporting systems. Stat Anal Data Mining ASA Data Sci J. 2014;7(5):352–67.
doi: 10.1002/sam.11233
Olsson S, Edwards IR. Tachycardia during cisapride treatment. BMJ. 1992;305(6856):748–9.
pubmed: 1384896 pmcid: 1883438 doi: 10.1136/bmj.305.6856.748-a
Inman W, Kubota K. Tachycardia during cisapride treatment. BMJ. 1992;305(6860):1019.
pubmed: 1458129 pmcid: 1884002 doi: 10.1136/bmj.305.6860.1019-a
Layton D, Key C, Shakir SA. Prolongation of the QT interval and cardiac arrhythmias associated with cisapride: limitations of the pharmacoepidemiological studies conducted and proposals for the future. Pharmacoepidemiol Drug Saf. 2003;12(1):31–40.
pubmed: 12616845 doi: 10.1002/pds.781
Bate A, Lindquist M, Orre R, Edwards IR, Meyboom RH. Data-mining analyses of pharmacovigilance signals in relation to relevant comparison drugs. Eur J Clin Pharmacol. 2002;58(7):483–90.
pubmed: 12389072 doi: 10.1007/s00228-002-0484-z
Mann RD. An instructive example of a long-latency adverse drug reaction–sclerosing peritonitis due to practolol. Pharmacoepidemiol Drug Saf. 2007;16(11):1211–6.
pubmed: 17853493 doi: 10.1002/pds.1466
Brewer T, Colditz GA. Postmarketing surveillance and adverse drug reactions: current perspectives and future needs. JAMA. 1999;281(9):824–9.
pubmed: 10071004 doi: 10.1001/jama.281.9.824
Kessler DA. Introducing MEDWatch. A new approach to reporting medication and device adverse effects and product problems. JAMA. 1993;269(21):2765–8.
pubmed: 8492403 doi: 10.1001/jama.1993.03500210065033
Bate A, Reynolds RF, Caubel P. The hope, hype and reality of Big Data for pharmacovigilance. Ther Adv Drug Saf. 2018;9(1):5–11.
pubmed: 29318002 doi: 10.1177/2042098617736422
LePendu P, Iyer SV, Bauer-Mehren A, Harpaz R, Mortensen JM, Podchiyska T, et al. Pharmacovigilance using clinical notes. Clin Pharmacol Ther. 2013;93(6):547–55.
pubmed: 23571773 doi: 10.1038/clpt.2013.47
Bhattacharya M, Snyder S, Malin M, Truffa MM, Marinic S, Engelmann R, et al. Using social media data in routine pharmacovigilance: a pilot study to identify safety signals and patient perspectives. Pharmaceut Med. 2017;31(3):167–74.
Norén GN, Orre R, Bate A, Edwards IR. Duplicate detection in adverse drug reaction surveillance. Data Min Knowl Disc. 2007;14(3):305–28.
doi: 10.1007/s10618-006-0052-8
Star K, Caster O, Bate A, Edwards IR. Dose variations associated with formulations of NSAID prescriptions for children: a descriptive analysis of electronic health records in the UK. Drug Saf. 2011;34(4):307–17.
pubmed: 21417503 doi: 10.2165/11586610-000000000-00000
Nath J. Chatbot, machine learning and artificial intelligence in pharmacovigilance: maintaining privacy, optimizing efficiency. 2018 [cited 2021 25th November]; https://chatbotsmagazine.com/chatbot-machine-learning-and-artificial-intelligence-in-pharmacovigilance-maintaining-privacy-877283e4b4b7 . Accessed 11 Mar 2022.

Auteurs

Oeystein Kjoersvik (O)

R&D IT, MSD, Prague, Czech Republic.

Andrew Bate (A)

Global Safety, GSK, 980 Great West Road, Brentford, TW8 9GS, Middlesex, UK. andrew.x.bate@gsk.com.
Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, UK. andrew.x.bate@gsk.com.
Department of Medicine at NYU Grossman School of Medicine, New York, USA. andrew.x.bate@gsk.com.

Articles similaires

[Redispensing of expensive oral anticancer medicines: a practical application].

Lisanne N van Merendonk, Kübra Akgöl, Bastiaan Nuijen
1.00
Humans Antineoplastic Agents Administration, Oral Drug Costs Counterfeit Drugs

Smoking Cessation and Incident Cardiovascular Disease.

Jun Hwan Cho, Seung Yong Shin, Hoseob Kim et al.
1.00
Humans Male Smoking Cessation Cardiovascular Diseases Female
Humans United States Aged Cross-Sectional Studies Medicare Part C
1.00
Humans Yoga Low Back Pain Female Male

Classifications MeSH