Development and External Validation of a Machine Learning Model for Prediction of Potential Transfer to the PICU.
Journal
Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies
ISSN: 1529-7535
Titre abrégé: Pediatr Crit Care Med
Pays: United States
ID NLM: 100954653
Informations de publication
Date de publication:
01 07 2022
01 07 2022
Historique:
pubmed:
22
4
2022
medline:
12
7
2022
entrez:
21
4
2022
Statut:
ppublish
Résumé
Unrecognized clinical deterioration during illness requiring hospitalization is associated with high risk of mortality and long-term morbidity among children. Our objective was to develop and externally validate machine learning algorithms using electronic health records for identifying ICU transfer within 12 hours indicative of a child's condition. Observational cohort study. Two urban, tertiary-care, academic hospitals (sites 1 and 2). Pediatric inpatients (age <18 yr). None. Our primary outcome was direct ward to ICU transfer. Using age, vital signs, and laboratory results, we derived logistic regression with regularization, restricted cubic spline regression, random forest, and gradient boosted machine learning models. Among 50,830 admissions at site 1 and 88,970 admissions at site 2, 1,993 (3.92%) and 2,317 (2.60%) experienced the primary outcome, respectively. Site 1 data were split longitudinally into derivation (2009-2017) and validation (2018-2019), whereas site 2 constituted the external test cohort. Across both sites, the gradient boosted machine was the most accurate model and outperformed a modified version of the Bedside Pediatric Early Warning Score that only used physiologic variables in terms of discrimination ( C -statistic site 1: 0.84 vs 0.71, p < 0.001; site 2: 0.80 vs 0.74, p < 0.001), sensitivity, specificity, and number needed to alert. We developed and externally validated a novel machine learning model that identifies ICU transfers in hospitalized children more accurately than current tools. Our model enables early detection of children at risk for deterioration, thereby creating opportunities for intervention and improvement in outcomes.
Identifiants
pubmed: 35446816
doi: 10.1097/PCC.0000000000002965
pii: 00130478-202207000-00005
pmc: PMC9262766
mid: NIHMS1790668
doi:
Types de publication
Journal Article
Observational Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
514-523Subventions
Organisme : NIDDK NIH HHS
ID : R01 DK126933
Pays : United States
Organisme : NIA NIH HHS
ID : R21 AG068720
Pays : United States
Organisme : NIGMS NIH HHS
ID : R01 GM123193
Pays : United States
Organisme : NHLBI NIH HHS
ID : K01 HL148390
Pays : United States
Organisme : NIGMS NIH HHS
ID : R01 GM137083
Pays : United States
Organisme : NIDA NIH HHS
ID : R01 DA051464
Pays : United States
Organisme : NHLBI NIH HHS
ID : R01 HL157262
Pays : United States
Informations de copyright
Copyright © 2022 by the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies.
Déclaration de conflit d'intérêts
Dr. Mayampurath received funding from the National Heart, Lung, and Blood Institutes (NHLBI; K01HL148390); he received support for article research from the National Institutes of Health (NIH). Dr. Mayampurath has performed consulting services for Litmus Health, Austin, TX, outside of submitted work. Ms. Hegermiller received funding from FabFitFun, Indus Consulting Inc., University of Chicago, and Northwell Health, outside of submitted work. Dr. Edelson received funding from the NIH/University of Wisconsin-Madison (R01 GM123193), the NIH/Idaho State University (R01 GM137083), the U.S. Health and Human Services/AgileMD (ASPR-BARDA 21-00592) (Principal Investigator), the Department of Defense/University of Wisconsin-Madison, and EarlySense; she disclosed that she owns equity in Agile MD stock and serves as the President and Co-Founder of AgileMD, and that she owns Intellectual Property Rights (ARCD.P0535US.P2); she also disclosed that AgileMD acquired the exclusive licensing rights to electronic Cardiac Arrest Risk and Triage when it acquired QuantHC. She received research support and honoraria from Philips Healthcare (Andover, MA). Dr. Churpek’s institution received funding from the National Institute of General Medical Sciences (R01 GM123193-05), the Department of Defense (PRMRP-W81XWH-21-1-0009), the NHLBI (R01- HL157262), the National Institute of Diabetes and Digestive and Kidney Diseases (R01-DK126933-A1), the National Institute on Aging (R21 AG068720-01), the National Institute on Drug Abuse (R01 DA051464-01), and an EarlySense Research Grant; he disclosed that he has a patent-pending (ARCD. P0535US.P2) and that he is a consultant with AgileMD. The remaining authors have disclosed that they do not have any potential conflicts of interest.
Références
Nadkarni VM, Larkin GL, Peberdy MA, et al.; National Registry of Cardiopulmonary Resuscitation Investigators: First documented rhythm and clinical outcome from in-hospital cardiac arrest among children and adults. JAMA. 2006; 295:50–57
Tress EE, Kochanek PM, Saladino RA, et al.: Cardiac arrest in children. J Emerg Trauma Shock. 2010; 3:267–272
Lambert V, Matthews A, MacDonell R, et al.: Paediatric early warning systems for detecting and responding to clinical deterioration in children: A systematic review. BMJ Open. 2017; 7:e014497
Hussain FS, Sosa T, Ambroggio L, et al.: Emergency transfers: An important predictor of adverse outcomes in hospitalized children. J Hosp Med. 2019; 14:482–485
Bonafide CP, Localio AR, Roberts KE, et al.: Impact of rapid response system implementation on critical deterioration events in children. JAMA Pediatr. 2014; 168:25–33
Parshuram CS, Duncan HP, Joffe AR, et al.: Multicentre validation of the bedside paediatric early warning system score: A severity of illness score to detect evolving critical illness in hospitalised children. Crit Care. 2011; 15:R184
Monaghan A: Detecting and managing deterioration in children. Paediatr Nurs. 2005; 17:32–35
Duncan H, Hutchison J, Parshuram CS: The Pediatric Early Warning System score: A severity of illness score to predict urgent medical need in hospitalized children. J Crit Care. 2006; 21:271–278
Parshuram CS, Hutchison J, Middaugh K: Development and initial validation of the Bedside Paediatric Early Warning System score. Crit Care. 2009; 13:R135
Chapman SM, Maconochie IK: Early warning scores in paediatrics: An overview. Arch Dis Child. 2019; 104:395–399
Chapman SM, Wray J, Oulton K, et al.: ‘The Score Matters’: Wide variations in predictive performance of 18 paediatric track and trigger systems. Arch Dis Child. 2017; 102:487–495
Trubey R, Huang C, Lugg-Widger FV, et al.: Validity and effectiveness of paediatric early warning systems and track and trigger tools for identifying and reducing clinical deterioration in hospitalised children: A systematic review. BMJ Open. 2019; 9:e022105
Parshuram CS, Dryden-Palmer K, Farrell C, et al.; Canadian Critical Care Trials Group and the EPOCH Investigators: Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: The EPOCH Randomized Clinical Trial. JAMA. 2018; 319:1002–1012
Mayampurath A, Jani P, Dai Y, et al.: A vital sign-based model to predict clinical deterioration in hospitalized children. Pediatr Crit Care Med. 2020; 21:820–826
Churpek MM, Yuen TC, Winslow C, et al.: Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med. 2016; 44:368–374
Kipnis P, Turk BJ, Wulf DA, et al.: Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU. J Biomed Inform. 2016; 64:10–19
Escobar GJ, Liu VX, Schuler A, et al.: Automated identification of adults at risk for in-hospital clinical deterioration. N Engl J Med. 2020; 383:1951–1960
Teheux L, Verlaat CW, Lemson J, et al.: Risk stratification to improve Pediatric Early Warning Systems: It is all about the context. Eur J Pediatr. 2019; 178:1589–1596
Singer JD, Willett JB: It’s about time: Using discrete-time survival analysis to study duration and the timing of events. J Educ Stat. 1993; 18:155–195
Moons KG, Altman DG, Reitsma JB, et al.: Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration. Ann Intern Med. 2015; 162:W1–73
Cohen JF, Korevaar DA, Altman DG, et al.: STARD 2015 guidelines for reporting diagnostic accuracy studies: Explanation and elaboration. BMJ Open. 2016; 6:e012799
Biecek P: DALEX: Explainers for complex predictive models in R. J Mach Learn Res. 2018; 19:1–5
Shapley LS: A value for N-person games. In: Contribution to the Theory of Fames. Princeton, Princeton University Press, 1953, pp 307–317
Štrumbelj E, Kononenko I: Explaining prediction models and individual predictions with feature contributions. Knowl Inf Syst. 2014; 41:647–665
DeLong ER, DeLong DM, Clarke-Pearson DL: Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics. 1988; 44:837–845
Bode R, Yang J, Rust S, et al.: Chasing the holy grail - predicting pediatric inpatient deterioration using a vitals risk index. Pediatrics. 2018; 142(1 MeetingAbstract):570–570
Lonsdale H, Jalali A, Ahumada L, et al.: Machine learning and artificial intelligence in pediatric research: Current state, future prospects, and examples in perioperative and critical care. J Pediatr. 2020; 221S:S3–S10
Wellner B, Grand J, Canzone E, et al.: Predicting unplanned transfers to the intensive care unit: A machine learning approach leveraging diverse clinical elements. JMIR Med Inform. 2017; 5:e45
Zhai H, Brady P, Li Q, et al.: Developing and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children. Resuscitation. 2014; 85:1065–1071
Rubin J, Potes C, Xu-Wilson M, et al.: An ensemble boosting model for predicting transfer to the pediatric intensive care unit. Int J Med Inform. 2018; 112:15–20
Gorham T, Rust S, Bambach S, et al.: The deterioration risk index: Predicting pediatric inpatient deterioration with machine learning and the electronic health record. Pediatrics. 2020; 146(1 MeetingAbstract):229–230
Lin ED, Hefner JL, Zeng X, et al.: A deep learning model for pediatric patient risk stratification. Am J Manag Care. 2019; 25:e310–e315
Park SJ, Cho KJ, Kwon O, et al.: Development and validation of a deep-learning-based pediatric early warning system: A single-center study. Biomedical Journal. 2021 Jan 18. [online ahead of print]
Poole S, Shah N: Addressing vital sign alarm fatigue using personalized alarm thresholds. Pac Symp Biocomput. 2018; 23:472–483
Bonafide CP, Brady PW, Keren R, et al.: Development of heart and respiratory rate percentile curves for hospitalized children. Pediatrics. 2013; 131:e1150–e1157
Walsh BK, Smallwood CD: Pediatric oxygen therapy: A review and update. Respir Care. 2017; 62:645–661
Britos M, Smoot E, Liu KD, et al.; National Institutes of Health Acute Respiratory Distress Syndrome Network Investigators: The value of positive end-expiratory pressure and Fio 2 criteria in the definition of the acute respiratory distress syndrome. Crit Care Med. 2011; 39:2025–2030
Lockwood J, Reese J, Wathen B, et al.: The association between fever and subsequent deterioration among hospitalized children with elevated PEWS. Hosp Pediatr. 2019; 9:170–178
Tamelytė E, Vaičekauskienė G, Dagys A, Lapinskas T, Jankauskaitė L: Early blood biomarkers to improve sepsis/bacteremia diagnostics in pediatric emergency settings. Medicina (Kaunas). 2019; 55:99
Biban P, Gaffuri M, Spaggiari S, et al.: Early recognition and management of septic shock in children. Pediatr Rep. 2012; 4:e13
Agniel D, Kohane IS, Weber GM: Biases in electronic health record data due to processes within the healthcare system: Retrospective observational study. BMJ. 2018; 361:k1479
Churpek MM, Adhikari R, Edelson DP: The value of vital sign trends for detecting clinical deterioration on the wards. Resuscitation. 2016; 102:1–5
Sosa TK, Dewan M, Tegtmeyer K: Back to the basics or back to the future? The art and science of predicting clinical deterioration in hospitalized children. Pediatr Crit Care Med. 2020; 21:839–841