Machine Learning for Prediction of Patients on Hemodialysis with an Undetected SARS-CoV-2 Infection.
COVID-19
SARS-CoV-2
artificial intelligence
coronavirus
dialysis
end stage kidney disease
machine learning
prediction
Journal
Kidney360
ISSN: 2641-7650
Titre abrégé: Kidney360
Pays: United States
ID NLM: 101766381
Informations de publication
Date de publication:
25 03 2021
25 03 2021
Historique:
received:
22
06
2020
accepted:
12
01
2021
entrez:
4
4
2022
pubmed:
13
1
2021
medline:
8
4
2022
Statut:
epublish
Résumé
We developed a machine learning (ML) model that predicts the risk of a patient on hemodialysis (HD) having an undetected SARS-CoV-2 infection that is identified after the following ≥3 days. As part of a healthcare operations effort, we used patient data from a national network of dialysis clinics (February-September 2020) to develop an ML model (XGBoost) that uses 81 variables to predict the likelihood of an adult patient on HD having an undetected SARS-CoV-2 infection that is identified in the subsequent ≥3 days. We used a 60%:20%:20% randomized split of COVID-19-positive samples for the training, validation, and testing datasets. We used a select cohort of 40,490 patients on HD to build the ML model (11,166 patients who were COVID-19 positive and 29,324 patients who were unaffected controls). The prevalence of COVID-19 in the cohort (28% COVID-19 positive) was by design higher than the HD population. The prevalence of COVID-19 was set to 10% in the testing dataset to estimate the prevalence observed in the national HD population. The threshold for classifying observations as positive or negative was set at 0.80 to minimize false positives. Precision for the model was 0.52, the recall was 0.07, and the lift was 5.3 in the testing dataset. Area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC) for the model was 0.68 and 0.24 in the testing dataset, respectively. Top predictors of a patient on HD having a SARS-CoV-2 infection were the change in interdialytic weight gain from the previous month, mean pre-HD body temperature in the prior week, and the change in post-HD heart rate from the previous month. The developed ML model appears suitable for predicting patients on HD at risk of having COVID-19 at least 3 days before there would be a clinical suspicion of the disease.
Sections du résumé
Background
We developed a machine learning (ML) model that predicts the risk of a patient on hemodialysis (HD) having an undetected SARS-CoV-2 infection that is identified after the following ≥3 days.
Methods
As part of a healthcare operations effort, we used patient data from a national network of dialysis clinics (February-September 2020) to develop an ML model (XGBoost) that uses 81 variables to predict the likelihood of an adult patient on HD having an undetected SARS-CoV-2 infection that is identified in the subsequent ≥3 days. We used a 60%:20%:20% randomized split of COVID-19-positive samples for the training, validation, and testing datasets.
Results
We used a select cohort of 40,490 patients on HD to build the ML model (11,166 patients who were COVID-19 positive and 29,324 patients who were unaffected controls). The prevalence of COVID-19 in the cohort (28% COVID-19 positive) was by design higher than the HD population. The prevalence of COVID-19 was set to 10% in the testing dataset to estimate the prevalence observed in the national HD population. The threshold for classifying observations as positive or negative was set at 0.80 to minimize false positives. Precision for the model was 0.52, the recall was 0.07, and the lift was 5.3 in the testing dataset. Area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC) for the model was 0.68 and 0.24 in the testing dataset, respectively. Top predictors of a patient on HD having a SARS-CoV-2 infection were the change in interdialytic weight gain from the previous month, mean pre-HD body temperature in the prior week, and the change in post-HD heart rate from the previous month.
Conclusions
The developed ML model appears suitable for predicting patients on HD at risk of having COVID-19 at least 3 days before there would be a clinical suspicion of the disease.
Identifiants
pubmed: 35369017
doi: 10.34067/KID.0003802020
pii: 02200512-202103000-00009
pmc: PMC8786002
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
456-468Informations de copyright
Copyright © 2021 by the American Society of Nephrology.
Déclaration de conflit d'intérêts
C. Monaghan, F. Maddux, H. Han, J. Larkin, L. Usvyat, S. Chaudhuri, and Y. Jiao are employees of Fresenius Medical Care in the Global Medical Office. E. Weinhandl, I. Dahne-Steuber, J. Hymes, K. Belmonte, K. Bermudez, and R. Kossmann are employees of Fresenius Medical Care North America. F. Maddux has directorships in the Fresenius Medical Care Management Board, Goldfinch Bio, and Vifor Fresenius Medical Care Renal Pharma. F. Maddux, I. Dahne-Steuber, J. Hymes, K. Belmonte, L. Usvyat, P. Kotanko, and R. Kossmann have share options/ownership in Fresenius Medical Care. L. Neri is an employee of Fresenius Medical Care Deutschland GmbH in the Europe, the Middle East, and Africa Medical Office. P. Kotanko is an employee of Renal Research Institute, a wholly owned subsidiary of Fresenius Medical Care; reports receiving honorarium from Up-To-Date; and is on the Editorial Board of Blood Purification and Kidney and Blood Pressure Research. All remaining authors have nothing to disclose.
Références
Kliger AS, Silberzweig J: Mitigating risk of COVID-19 in dialysis facilities. Clin J Am Soc Nephrol 15: 707–709, 2020 https://doi.org/10.2215/CJN.03340320
doi: 10.2215/CJN.03340320
Ikizler TA: COVID-19 and dialysis units: What do we know now and what should we do? Am J Kidney Dis 76: 1–3, 2020 https://doi.org/10.1053/j.ajkd.2020.03.008
doi: 10.1053/j.ajkd.2020.03.008
Basile C, Combe C, Pizzarelli F, Covic A, Davenport A, Kanbay M, Kirmizis D, Schneditz D, van der Sande F, Mitra S: Recommendations for the prevention, mitigation and containment of the emerging SARS-CoV-2 (COVID-19) pandemic in haemodialysis centres. Nephrol Dial Transplant 35: 737–741, 2020 https://doi.org/10.1093/ndt/gfaa069
doi: 10.1093/ndt/gfaa069
Mokrzycki MH, Coco M: Management of hemodialysis patients with suspected or confirmed COVID-19 infection: Perspective of two nephrologists in the United States. Kidney360 1: 273–278, 2020 https://doi.org/10.34067/KID.0001452020
doi: 10.34067/KID.0001452020
Gallieni M, Sabiu G, Scorza D: Delivering safe and effective hemodialysis in patients with suspected or confirmed COVID-19 infection: a single-center perspective from Italy. Kidney360 1: 403–409, 2020 https://doi.org/10.34067/KID.0001782020
doi: 10.34067/KID.0001782020
Roncon L, Zuin M, Rigatelli G, Zuliani G: Diabetic patients with COVID-19 infection are at higher risk of ICU admission and poor short-term outcome. J Clin Virol 127: 104354, 2020 https://doi.org/10.1016/j.jcv.2020.104354
doi: 10.1016/j.jcv.2020.104354
Guo T, Fan Y, Chen M, Wu X, Zhang L, He T, Wang H, Wan J, Wang X, Lu Z: Cardiovascular implications of fatal outcomes of patients with Coronavirus Disease 2019 (COVID-19). JAMA Cardiol 5: 811–818, 2020 https://doi.org/10.1001/jamacardio.2020.1017
doi: 10.1001/jamacardio.2020.1017
Li X, Xu S, Yu M, Wang K, Tao Y, Zhou Y, Shi J, Zhou M, Wu B, Yang Z, Zhang C, Yue J, Zhang Z, Renz H, Liu X, Xie J, Xie M, Zhao J: Risk factors for severity and mortality in adult COVID-19 inpatients in Wuhan. J Allergy Clin Immunol 146: 110–118, 2020 https://doi.org/10.1016/j.jaci.2020.04.006
doi: 10.1016/j.jaci.2020.04.006
Cheng Y, Luo R, Wang K, Zhang M, Wang Z, Dong L, Li J, Yao Y, Ge S, Xu G: Kidney disease is associated with in-hospital death of patients with COVID-19. Kidney Int 97: 829–838, 2020 https://doi.org/10.1016/j.kint.2020.03.005
doi: 10.1016/j.kint.2020.03.005
Du RH, Liang LR, Yang CQ, Wang W, Cao TZ, Li M, Guo GY, Du J, Zheng CL, Zhu Q, Hu M, Li XY, Peng P, Shi HZ: Predictors of mortality for patients with COVID-19 pneumonia caused by SARS-CoV-2: A prospective cohort study. Eur Respir J 55: 2000524, 2020 https://doi.org/10.1183/13993003.00524-2020
doi: 10.1183/13993003.00524-2020
Adams ML, Katz DL, Grandpre J: Population-based estimates of chronic conditions affecting risk for complications from coronavirus disease, United States. Emerg Infect Dis 26: 1831–1833, 2020 https://doi.org/10.3201/eid2608.200679
doi: 10.3201/eid2608.200679
United States Renal Data System: 2019 USRDS Annual Data Report: Epidemiology of Kidney Disease in the United States, Bethesda, MD, National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases, 2019. Available at: https://www.usrds.org/annual-data-report/previous-adrs/ . Accessed June 10, 2020
Neumann ME: Latest data show 305 dialysis patient deaths due to COVID-19 in the US. Nephrology News & Issues . Available at: https://www.healio.com/nephrology/infection-control/news/online/%7B3a263aa9-ad59-4c3f-aab7-07b8395508e5%7D/latest-data-show-305-dialysis-patient-deaths-due-to-covid-19-in-the-us . Accessed April 22, 2020
CDC COVID-19 Response Team: Geographic differences in COVID-19 cases, deaths, and incidence - United States, february 12-april 7, 2020. MMWR Morb Mortal Wkly Rep 69: 465–471, 2020 https://doi.org/10.15585/mmwr.mm6915e4
doi: 10.15585/mmwr.mm6915e4
Johns Hopkins Coronavirus Resource Center Johns Hopkins University School of Medicine. 2020. Available at: https://coronavirus.jhu.edu/data/mortality . Accessed October 15, 2020
ERA-EDTA: ERACODA - The ERA-EDTA COVID-19 database for patients on kidney replacement therapy, 2020. Available at: https://www.era-edta.org/en/wp-content/uploads/2020/04/ERACODA-Study-Report-2020-04-29.pdf . Accessed October 15, 2020
Wang H: Maintenance hemodialysis and coronavirus disease 2019 (COVID-19): Saving lives with caution, care, and courage. Kidney Med 2: 365–366, 2020
Jager KJ, Kramer A, Chesnaye NC, Couchoud C, Sánchez-Álvarez JE, Garneata L, Collart F, Hemmelder MH, Ambühl P, Kerschbaum J, Legeai C, Del Pino Y Pino MD, Mircescu G, Mazzoleni L, Hoekstra T, Winzeler R, Mayer G, Stel VS, Wanner C, Zoccali C, Massy ZA: Results from the ERA-EDTA Registry indicate a high mortality due to COVID-19 in dialysis patients and kidney transplant recipients across Europe. Kidney Int 98: 1540–1548, 2020 https://doi.org/10.1016/j.kint.2020.09.006
doi: 10.1016/j.kint.2020.09.006
Kikuchi K, Nangaku M, Ryuzaki M, Yamakawa T, Hanafusa N, Sakai K, Kanno Y, Ando R, Shinoda T, Nakamoto H, Akizawa T; COVID-19 Task Force Committee of the Japanese Association of Dialysis Physicians; Japanese Society for Dialysis Therapy; Japanese Society of Nephrology: COVID-19 of dialysis patients in Japan: Current status and guidance on preventive measures. Ther Apher Dial 24: 361–365, 2020
Siordia JA Jr: Epidemiology and clinical features of COVID-19: A review of current literature. J Clin Virol 127: 104357, 2020 https://doi.org/10.1016/j.jcv.2020.104357
doi: 10.1016/j.jcv.2020.104357
Xiong F, Tang H, Liu L, Tu C, Tian JB, Lei CT, Liu J, Dong JW, Chen WL, Wang XH, Luo D, Shi M, Miao XP, Zhang C: Clinical characteristics of and medical interventions for COVID-19 in hemodialysis patients in wuhan, China. J Am Soc Nephrol 31: 1387–1397, 2020 https://doi.org/10.1681/ASN.2020030354
doi: 10.1681/ASN.2020030354
Niiler E: An AI epidemiologist sent the first warnings of the Wuhan virus, 2020. Wired . Available at: https://www.wired.com/story/ai-epidemiologist-wuhan-public-health-warnings/ . Accessed April 22, 2020
Bogoch II, Watts A, Thomas-Bachli A, Huber C, Kraemer MUG, Khan K: Pneumonia of unknown aetiology in Wuhan, China: Potential for international spread via commercial air travel. J Travel Med 27: taaa008, 2020 https://doi.org/10.1093/jtm/taaa008
doi: 10.1093/jtm/taaa008
McCall B: COVID-19 and artificial intelligence: Protecting health-care workers and curbing the spread. Lancet Digit Health 2: e166–e167, 2020 https://doi.org/10.1016/S2589-7500(20)30054-6
doi: 10.1016/S2589-7500(20)30054-6
Alimadadi A, Aryal S, Manandhar I, Munroe PB, Joe B, Cheng X: Artificial intelligence and machine learning to fight COVID-19. Physiol Genomics 52: 200–202, 2020 https://doi.org/10.1152/physiolgenomics.00029.2020
doi: 10.1152/physiolgenomics.00029.2020
Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, Bonten MMJ, Damen JAA, Debray TPA, De Vos M, Dhiman P, Haller MC, Harhay MO, Henckaerts L, Kreuzberger N, Lohman A, Luijken K, Ma J, Andaur CL, Reitsma JB, Sergeant JC, Shi C, Skoetz N, Smits LJM, Snell KIE, Sperrin M, Spijker R, Steyerberg EW, Takada T, van Kuijk SMJ, van Royen FS, Wallisch C, Hooft L, Moons KGM, van Smeden M: Prediction models for diagnosis and prognosis of covid-19 infection: Systematic review and critical appraisal [published correction appears in BMJ 369: m2204, 2020 10.1136/bmj.m2204]. BMJ 369: m1328, 2020 https://doi.org/10.1136/bmj.m1328
doi: 10.1136/bmj.m1328
Fresenius Medical Care North America: COVID-19 resource and education center. Available at: https://fmcna.com/company/covid-19-resource-center/ . Accessed August 31, 2020
Chen T, Guestrin C: XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco, California, Association for Computing Machinery, 2016 pp 785–794 https://doi.org/10.1145/2939672.2939785
Centers for Medicare & Medicaid Services: Preliminary medicare COVID-19 data snapshot, 2020. Available at: https://www.cms.gov/research-statistics-data-systems/preliminary-medicare-covid-19-data-snapshot . Accessed October 14, 2020
Anand S, Montez-Rath M, Han J, Bozeman J, Kerschmann R, Beyer P, Parsonnet J, Chertow GM: Prevalence of SARS-CoV-2 antibodies in a large nationwide sample of patients on dialysis in the USA: A cross-sectional study. Lancet 396: 1335–1344, 2020 https://doi.org/10.1016/S0140-6736(20)32009-2
doi: 10.1016/S0140-6736(20)32009-2
Shapley LS: A value for n-person games. In: Contributions to the Theory of Games II. Annals of Mathematics Studies, edited by Kuhn HW, Tucker AW, Vol. 28, Princeton, Princeton University Press, 1953, pp 307–317
Štrumbelj E, Kononenko I: Explaining prediction models and individual predictions with feature contributions. Knowl Inf Syst 41: 647–665, 2013 https://doi.org/10.1007/s10115-013-0679-x
doi: 10.1007/s10115-013-0679-x
Lundberg SM, Lee SI: A unified approach to interpreting model predictions. Proceedings from the Advances in Neural Information Processing Systems, Vol. 30, 2017. Available at: https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html . Accessed June 10, 2020
Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, Katz R, Himmelfarb J, Bansal N, Lee S-I: From local explanations to global understanding with explainable AI for trees. Nat Mach Intell 2: 56–67, 2020 https://doi.org/10.1038/s42256-019-0138-9
doi: 10.1038/s42256-019-0138-9
Saito T, Rehmsmeier M: The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One 10: e0118432, 2015 https://doi.org/10.1371/journal.pone.0118432
doi: 10.1371/journal.pone.0118432
Usvyat LA, Kotanko P, van der Sande FM, Kooman JP, Carter M, Leunissen KM, Levin NW: Circadian variations in body temperature during dialysis. Nephrol Dial Transplant 27: 1139–1144, 2012 https://doi.org/10.1093/ndt/gfr395
doi: 10.1093/ndt/gfr395
Gedney N: Long-Term hemodialysis during the COVID-19 pandemic. Clin J Am Soc Nephrol 15: 1073–1074, 2020 https://doi.org/10.2215/CJN.09100620
doi: 10.2215/CJN.09100620
Gagliardi I, Patella G, Michael A, Serra R, Provenzano M, Andreucci M: COVID-19 and the kidney: From epidemiology to clinical practice. J Clin Med 9: 2506, 2020 https://doi.org/10.3390/jcm9082506
doi: 10.3390/jcm9082506
Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, Xiang J, Wang Y, Song B, Gu X, Guan L, Wei Y, Li H, Wu X, Xu J, Tu S, Zhang Y, Chen H, Cao B: Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet 395: 1054–1062, 2020 https://doi.org/10.1016/S0140-6736(20)30566-3
doi: 10.1016/S0140-6736(20)30566-3
Fontana F, Giaroni F, Frisina M, Alfano G, Mori G, Lucchi L, Magistroni R, Cappelli G: SARS-CoV-2 infection in dialysis patients in northern Italy: A single-centre experience. Clin Kidney J 13: 334–339, 2020 https://doi.org/10.1093/ckj/sfaa148
doi: 10.1093/ckj/sfaa148
Shaikh Aisha, Zeldis Etti, Campbell Kirk N, Chan Lili: Prolonged SARS-CoV-2 Viral RNA Shedding and IgG Antibody Response to SARS-CoV-2 in Patients on Hemodialysis. Clin J Am Soc Nephrol, 2020 10.2215/CJN.11120720 33055191
doi: 10.2215/CJN.11120720
Vaishya R, Javaid M, Khan IH, Haleem A: Artificial Intelligence (AI) applications for COVID-19 pandemic. Diabetes Metab Syndr 14: 337–339, 2020 https://doi.org/10.1016/j.dsx.2020.04.012
doi: 10.1016/j.dsx.2020.04.012
Feng C, Wang L, Chen X, Zhai Y, Zhu F, Chen H, Wang Y, Su X, Huang S, Tian L, Zhu W, Sun W, Zhang L, Han Q, Zhang J, Pan F, Chen L, Zhu Z, Xiao H, Liu Y, Liu G, Chen W, Li T: A novel artificial intelligence-assisted triage tool to aid in the diagnosis of suspected COVID-19 pneumonia cases in fever clinics. Annals of Translational Medicine, 2021. Available at: https://atm.amegroups.com/article/view/6078 . Accessed January 29, 2021
Leung K, Wu JT, Liu D, Leung GM: First-wave COVID-19 transmissibility and severity in China outside Hubei after control measures, and second-wave scenario planning: A modelling impact assessment. Lancet 395: 1382–1393, 2020 https://doi.org/10.1016/S0140-6736(20)30746-7
doi: 10.1016/S0140-6736(20)30746-7
Xu S, Li Y: Beware of the second wave of COVID-19. Lancet 395: 1321–1322, 2020 https://doi.org/10.1016/S0140-6736(20)30845-X
doi: 10.1016/S0140-6736(20)30845-X