Budget constrained machine learning for early prediction of adverse outcomes for COVID-19 patients.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
01 10 2021
01 10 2021
Historique:
received:
05
06
2021
accepted:
25
08
2021
entrez:
2
10
2021
pubmed:
3
10
2021
medline:
14
10
2021
Statut:
epublish
Résumé
The combination of machine learning (ML) and electronic health records (EHR) data may be able to improve outcomes of hospitalized COVID-19 patients through improved risk stratification and patient outcome prediction. However, in resource constrained environments the clinical utility of such data-driven predictive tools may be limited by the cost or unavailability of certain laboratory tests. We leveraged EHR data to develop an ML-based tool for predicting adverse outcomes that optimizes clinical utility under a given cost structure. We further gained insights into the decision-making process of the ML models through an explainable AI tool. This cohort study was performed using deidentified EHR data from COVID-19 patients from ProMedica Health System in northwest Ohio and southeastern Michigan. We tested the performance of various ML approaches for predicting either increasing ventilatory support or mortality. We performed post hoc analysis to obtain optimal feature sets under various budget constraints. We demonstrate that it is possible to achieve a significant reduction in cost at the expense of a small reduction in predictive performance. For example, when predicting ventilation, it is possible to achieve a 43% reduction in cost with only a 3% reduction in performance. Similarly, when predicting mortality, it is possible to achieve a 50% reduction in cost with only a 1% reduction in performance. This study presents a quick, accurate, and cost-effective method to evaluate risk of deterioration for patients with SARS-CoV-2 infection at the time of clinical evaluation.
Identifiants
pubmed: 34599200
doi: 10.1038/s41598-021-98071-z
pii: 10.1038/s41598-021-98071-z
pmc: PMC8486861
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
19543Subventions
Organisme : Laboratory Directed Research and Development
ID : 19-ERD-009
Informations de copyright
© 2021. The Author(s).
Références
Holshue, M. L. et al. First case of 2019 novel coronavirus in the United States. N. Engl. J. Med. 382, 929–936 (2020).
pubmed: 32004427
pmcid: 7092802
doi: 10.1056/NEJMoa2001191
Sen-Crowe, B., Sutherland, M., McKenney, M. & Elkbuli, A. A closer look into global hospital beds capacity and resource shortages during the COVID-19 pandemic. J. Surg. Res. 260, 56–63 (2021).
pubmed: 33321393
doi: 10.1016/j.jss.2020.11.062
Yan, L. et al. An interpretable mortality prediction model for COVID-19 patients. Nat. Mach. Intell. 2, 283–288 (2020).
doi: 10.1038/s42256-020-0180-7
Yadaw, A. S. et al. Clinical features of COVID-19 mortality: Development and validation of a clinical prediction model. Lancet Digit. Health 2, e516–e525 (2020).
pubmed: 32984797
pmcid: 7508513
doi: 10.1016/S2589-7500(20)30217-X
Gao, Y. et al. Machine learning based early warning system enables accurate mortality risk prediction for COVID-19. Nat. Commun. 11, 5033 (2020).
pubmed: 33024092
pmcid: 7538910
doi: 10.1038/s41467-020-18684-2
Wollenstein-Betech, S., Cassandras, C. G. & Paschalidis, I. C. Personalized predictive models for symptomatic COVID-19 patients using basic preconditions: Hospitalizations, mortality, and the need for an ICU or ventilator. Int. J. Med. Inform. 142, 104258 (2020).
pubmed: 32927229
pmcid: 7442577
doi: 10.1016/j.ijmedinf.2020.104258
Li, W. et al. Early predictors for mechanical ventilation in COVID-19 patients. Ther. Adv. Respir. Dis. 14, 1753466620963017 (2020).
pubmed: 33054630
pmcid: 7570781
doi: 10.1177/1753466620963017
Stiglic, G. et al. Interpretability of machine learning-based prediction models in healthcare. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 10, e1379 (2020).
doi: 10.1002/widm.1379
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R. & Yu, B. Definitions, methods, and applications in interpretable machine learning. Proc. Natl. Acad. Sci. 116, 22071–22080 (2019).
pubmed: 31619572
pmcid: 6825274
doi: 10.1073/pnas.1900654116
Qayyum, A., Qadir, J., Bilal, M. & Al-Fuqaha, A. Secure and robust machine learning for healthcare: A survey. IEEE Rev. Biomed. Eng. 14, 156–180 (2020).
doi: 10.1109/RBME.2020.3013489
Lundberg, S. M. & Lee, S.-In. A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems 4768–4777.
Bertsimas, D. et al. COVID-19 mortality risk assessment: An international multi-center study. PLoS ONE 15, e0243262 (2020).
pubmed: 33296405
pmcid: 7725386
doi: 10.1371/journal.pone.0243262
Abdulaal, A. et al. Prognostic modeling of COVID-19 using artificial intelligence in the United Kingdom: Model development and validation. J. Med. Internet Res. 22, e20259 (2020).
pubmed: 32735549
pmcid: 7451108
doi: 10.2196/20259
Jagdhuber, R., Lang, M. & Rahnenführer, J. Feature Selection Methods for Cost-Constrained Classification in Random Forests. arXiv preprint arXiv: 2008.06298 (2020).
Erion, G. et al. CoAI: Cost-aware artificial intelligence for health care. medRxiv 2, 2 (2021).
Min, F., Hu, Q. & Zhu, W. Feature selection with test cost constraint. Int. J. Approx. Reason. 55, 167–179 (2014).
doi: 10.1016/j.ijar.2013.04.003
Yoon, J., Jordon, J. & Schaar, M. ASAC: Active sensing using actor-critic models. 451–473.
Jagdhuber, R., Lang, M., Stenzl, A., Neuhaus, J. & Rahnenführer, J. Cost-constrained feature selection in binary classification: Adaptations for greedy forward selection and genetic algorithms. BMC Bioinform. 21, 1–21 (2020).
doi: 10.1186/s12859-020-3361-9
Verma, A., Hanawal, M. K. & Hemachandra, N. Unsupervised online feature selection for cost-sensitive medical diagnosis. 1–6.
Wells, B. J., Chagin, K. M., Nowacki, A. S. & Kattan, M. W. Strategies for handling missing data in electronic health record derived data. Egems 1, 2 (2013).
doi: 10.13063/2327-9214.1035
Chen, T. & Guestrin, C. Xgboost: A scalable tree boosting system. 785–794.
Rasmussen, C. E. Gaussian processes in machine learning. 63–71.
Shipe, M. E., Deppen, S. A., Farjah, F. & Grogan, E. L. Developing prediction models for clinical use using logistic regression: an overview. J. Thorac. Dis. 11, S574 (2019).
pubmed: 31032076
pmcid: 6465431
doi: 10.21037/jtd.2019.01.25
Nick, T. G. & Campbell, K. M. Logistic regression. Top. Biostat. 2, 273–301 (2007).
doi: 10.1007/978-1-59745-530-5_14
Schober, P. & Vetter, T. R. Logistic regression in medical research. Anesth. Analg. 132, 365 (2021).
pubmed: 33449558
pmcid: 7785709
doi: 10.1213/ANE.0000000000005247
Moor, M., Horn, M., Rieck, B., Roqueiro, D. & Borgwardt, K. Early recognition of sepsis with Gaussian process temporal convolutional networks and dynamic time warping. 2–26.
Rinta-Koski, O.-P., Särkkä, S., Hollmén, J., Leskinen, M. & Andersson, S. Gaussian process classification for prediction of in-hospital mortality among preterm infants. Neurocomputing 298, 134–141 (2018).
doi: 10.1016/j.neucom.2017.12.064
Clifton, L., Clifton, D. A., Pimentel, M. A. F., Watkinson, P. J. & Tarassenko, L. Gaussian processes for personalized e-health monitoring with wearable sensors. IEEE Trans. Biomed. Eng. 60, 193–197 (2012).
doi: 10.1109/TBME.2012.2208459
Sharma, A. & Verbeke, W. J. M. I. Improving diagnosis of depression with XGBOOST machine learning model and a large biomarkers dutch dataset (n= 11,081). Front. Big Data 3, 15 (2020).
pubmed: 33693389
pmcid: 7931945
doi: 10.3389/fdata.2020.00015
Chang, W. et al. A machine-learning-based prediction method for hypertension outcomes based on medical data. Diagnostics 9, 178 (2019).
pmcid: 6963807
doi: 10.3390/diagnostics9040178
Akiba, T., Sano, S., Yanase, T., Ohta, T. & Koyama, M. Optuna: A next-generation hyperparameter optimization framework. 2623–2631.
Buitinck, L. et al. API design for machine learning software: experiences from the scikit-learn project. arXiv preprint arXiv: 1309.0238 (2013).
Hu, R., Han, C., Pei, S., Yin, M. & Chen, X. Procalcitonin levels in COVID-19 patients. Int. J. Antimicrob. Agents 56, 106051 (2020).
pubmed: 32534186
pmcid: 7286278
doi: 10.1016/j.ijantimicag.2020.106051
Lippi, G. & Plebani, M. Procalcitonin in patients with severe coronavirus disease 2019 (COVID-19): A meta-analysis. Clin. Chim. Acta 505, 190 (2020).
pubmed: 32145275
pmcid: 7094472
doi: 10.1016/j.cca.2020.03.004
Krause, M. et al. Association between procalcitonin levels and duration of mechanical ventilation in COVID-19 patients. PLoS ONE 15, e0239174 (2020).
pubmed: 32946466
pmcid: 7500634
doi: 10.1371/journal.pone.0239174
Kang, S.-J. & Jung, S. I. Age-related morbidity and mortality among patients with COVID-19. Infect. Chemotherapy 52, 154 (2020).
doi: 10.3947/ic.2020.52.2.154
Trabulus, S. et al. Kidney function on admission predicts in-hospital mortality in COVID-19. PLoS ONE 15, e0238680 (2020).
pubmed: 32881976
pmcid: 7470363
doi: 10.1371/journal.pone.0238680
Cheng, A. et al. Diagnostic performance of initial blood urea nitrogen combined with D-dimer levels for predicting in-hospital mortality in COVID-19 patients. Int. J. Antimicrob. Agents 56, 106110 (2020).
pubmed: 32712332
pmcid: 7377803
doi: 10.1016/j.ijantimicag.2020.106110
Altschul, D. J. et al. A novel severity score to predict inpatient mortality in COVID-19 patients. Sci. Rep. 10, 1–8 (2020).
doi: 10.1038/s41598-020-73962-9
Askari, H. et al. Kidney diseases and COVID-19 infection: causes and effect, supportive therapeutics and nutritional perspectives. Heliyon 2, e06008 (2021).
doi: 10.1016/j.heliyon.2021.e06008
Palevsky, P. M. COVID-19 and AKI: Where do we stand?. J. Am. Soc. Nephrol. 32, 1029–1032 (2021).
pubmed: 33637516
doi: 10.1681/ASN.2020121768
pmcid: 8259685
Wald, R. & Bagshaw, S. M. COVID-19–associated acute kidney injury: Learning from the first wave. (2021).
Zheng, S. et al. Immunodeficiency promotes adaptive alterations of host gut microbiome: An observational metagenomic study in mice. Front. Microbiol. 10, 2415 (2019).
pubmed: 31781050
pmcid: 6853035
doi: 10.3389/fmicb.2019.02415
Formica, V. et al. Complete blood count might help to identify subjects with high probability of testing positive to SARS-CoV-2. Clin. Med. 20, e114 (2020).
doi: 10.7861/clinmed.2020-0373
Wang, C. et al. Preliminary study to identify severe from moderate cases of COVID-19 using combined hematology parameters. Ann. Transl. Med. 8, 2 (2020).
pubmed: 32055593
pmcid: 6995746
Kompaniyets, L. et al. Body mass index and risk for COVID-19–related hospitalization, intensive care unit admission, invasive mechanical ventilation, and death—united states, march–december 2020. Morb. Mortal. Wkly Rep. 70, 355 (2021).
doi: 10.15585/mmwr.mm7010e4
Bravata, D. M. et al. Association of intensive care unit patient load and demand with mortality rates in US Department of Veterans Affairs Hospitals during the COVID-19 pandemic. JAMA Netw. Open 4, e2034266–e2034266 (2021).
pubmed: 33464319
pmcid: 7816100
doi: 10.1001/jamanetworkopen.2020.34266
Chen, J. et al. Genetic regulatory subnetworks and key regulating genes in rat hippocampus perturbed by prenatal malnutrition: Implications for major brain disorders. Aging (Albany NY) 12, 8434–8458 (2020).
doi: 10.18632/aging.103150
Li, H. et al. Co-expression network analysis identified hub genes critical to triglyceride and free fatty acid metabolism as key regulators of age-related vascular dysfunction in mice. Aging (Albany NY) 11, 7620–7638 (2019).
doi: 10.18632/aging.102275
Jiang, L. et al. Sex-specific association of circulating ferritin level and risk of type 2 diabetes: A dose-response meta-analysis of prospective studies. J. Clin. Endocrinol. Metab. 104, 4539–4551 (2019).
pubmed: 31074789
doi: 10.1210/jc.2019-00495
Wu, Y. et al. Multi-trait analysis for genome-wide association study of five psychiatric disorders. Transl. Psychiatry 10, 1–11 (2020).
Zhang, F. et al. Causal influences of neuroticism on mental health and cardiovascular disease. Hum. Genet. 140, 1267–1281 (2021).
pubmed: 33973063
doi: 10.1007/s00439-021-02288-x
Zhang, F. et al. Genetic evidence suggests posttraumatic stress disorder as a subtype of major depressive disorder. J. Clin. Invest. https://doi.org/10.1172/JCI145942 (2021).
doi: 10.1172/JCI145942
pubmed: 34907911
pmcid: 8670847
Wang, X. et al. Genetic support of a causal relationship between iron status and type 2 diabetes: A Mendelian randomization study. J. Clin. Endocrinol. Metab. https://doi.org/10.1210/clinem/dgab454 (2021).
doi: 10.1210/clinem/dgab454
pubmed: 34314489
pmcid: 8947318