Prospective, multicenter validation of the deep learning-based cardiac arrest risk management system for predicting in-hospital cardiac arrest or unplanned intensive care unit transfer in patients admitted to general wards.
Artificial intelligence
Cardiac arrest
Deep learning
Early warning score
Intensive care unit
Rapid response system
Journal
Critical care (London, England)
ISSN: 1466-609X
Titre abrégé: Crit Care
Pays: England
ID NLM: 9801902
Informations de publication
Date de publication:
05 09 2023
05 09 2023
Historique:
received:
22
05
2023
accepted:
10
08
2023
medline:
7
9
2023
pubmed:
6
9
2023
entrez:
5
9
2023
Statut:
epublish
Résumé
Retrospective studies have demonstrated that the deep learning-based cardiac arrest risk management system (DeepCARS™) is superior to the conventional methods in predicting in-hospital cardiac arrest (IHCA). This prospective study aimed to investigate the predictive accuracy of the DeepCARS™ for IHCA or unplanned intensive care unit transfer (UIT) among general ward patients, compared with that of conventional methods in real-world practice. This prospective, multicenter cohort study was conducted at four teaching hospitals in South Korea. All adult patients admitted to general wards during the 3-month study period were included. The primary outcome was predictive accuracy for the occurrence of IHCA or UIT within 24 h of the alarm being triggered. Area under the receiver operating characteristic curve (AUROC) values were used to compare the DeepCARS™ with the modified early warning score (MEWS), national early warning Score (NEWS), and single-parameter track-and-trigger systems. Among 55,083 patients, the incidence rates of IHCA and UIT were 0.90 and 6.44 per 1,000 admissions, respectively. In terms of the composite outcome, the AUROC for the DeepCARS™ was superior to those for the MEWS and NEWS (0.869 vs. 0.756/0.767). At the same sensitivity level of the cutoff values, the mean alarm counts per day per 1,000 beds were significantly reduced for the DeepCARS™, and the rate of appropriate alarms was higher when using the DeepCARS™ than when using conventional systems. The DeepCARS™ predicts IHCA and UIT more accurately and efficiently than conventional methods. Thus, the DeepCARS™ may be an effective screening tool for detecting clinical deterioration in real-world clinical practice. Trial registration This study was registered at ClinicalTrials.gov ( NCT04951973 ) on June 30, 2021.
Sections du résumé
BACKGROUND
Retrospective studies have demonstrated that the deep learning-based cardiac arrest risk management system (DeepCARS™) is superior to the conventional methods in predicting in-hospital cardiac arrest (IHCA). This prospective study aimed to investigate the predictive accuracy of the DeepCARS™ for IHCA or unplanned intensive care unit transfer (UIT) among general ward patients, compared with that of conventional methods in real-world practice.
METHODS
This prospective, multicenter cohort study was conducted at four teaching hospitals in South Korea. All adult patients admitted to general wards during the 3-month study period were included. The primary outcome was predictive accuracy for the occurrence of IHCA or UIT within 24 h of the alarm being triggered. Area under the receiver operating characteristic curve (AUROC) values were used to compare the DeepCARS™ with the modified early warning score (MEWS), national early warning Score (NEWS), and single-parameter track-and-trigger systems.
RESULTS
Among 55,083 patients, the incidence rates of IHCA and UIT were 0.90 and 6.44 per 1,000 admissions, respectively. In terms of the composite outcome, the AUROC for the DeepCARS™ was superior to those for the MEWS and NEWS (0.869 vs. 0.756/0.767). At the same sensitivity level of the cutoff values, the mean alarm counts per day per 1,000 beds were significantly reduced for the DeepCARS™, and the rate of appropriate alarms was higher when using the DeepCARS™ than when using conventional systems.
CONCLUSION
The DeepCARS™ predicts IHCA and UIT more accurately and efficiently than conventional methods. Thus, the DeepCARS™ may be an effective screening tool for detecting clinical deterioration in real-world clinical practice. Trial registration This study was registered at ClinicalTrials.gov ( NCT04951973 ) on June 30, 2021.
Identifiants
pubmed: 37670324
doi: 10.1186/s13054-023-04609-0
pii: 10.1186/s13054-023-04609-0
pmc: PMC10481524
doi:
Banques de données
ClinicalTrials.gov
['NCT04951973']
Types de publication
Multicenter Study
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
346Subventions
Organisme : Korea Medical Device Development Fund
ID : 202015X02
Organisme : Seoul National University Bundang Hospital, South Korea
ID : 14-2017-0021
Informations de copyright
© 2023. BioMed Central Ltd., part of Springer Nature.
Références
Jones DA, DeVita MA, Bellomo R. Rapid-response teams. N Engl J Med. 2011;365:139–46.
doi: 10.1056/NEJMra0910926
pubmed: 21751906
Devita MA, Bellomo R, Hillman K, Kellum J, Rotondi A, Teres D, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med. 2006;34:2463–78.
doi: 10.1097/01.CCM.0000235743.38172.6E
pubmed: 16878033
DeVita MA, Smith GB, Adam SK, Adams-Pizarro I, Buist M, Bellomo R, et al. ‘Identifying the hospitalised patient in crisis’–A consensus conference on the afferent limb of rapid response systems. Resuscitation. 2010;81:375–82.
doi: 10.1016/j.resuscitation.2009.12.008
pubmed: 20149516
Smith GB, Prytherch DR, Schmidt PE, Featherstone PI, Higgins B. A review, and performance evaluation, of single-parameter “track and trigger” systems. Resuscitation. 2008;79:11–21.
doi: 10.1016/j.resuscitation.2008.05.004
pubmed: 18620794
Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted “track and trigger” systems. Resuscitation. 2008;77:170–9.
doi: 10.1016/j.resuscitation.2007.12.004
pubmed: 18249483
Liu VX, Lu Y, Carey KA, Gilbert ER, Afshar M, Akel M, et al. Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the Intensive Care Unit. JAMA Netw Open. 2020;3: e205191.
doi: 10.1001/jamanetworkopen.2020.5191
pubmed: 32427324
pmcid: 7237982
Lee YJ, Cho KJ, Kwon O, Park H, Lee Y, Kwon JM, et al. A multicentre validation study of the deep learning-based early warning score for predicting in-hospital cardiac arrest in patients admitted to general wards. Resuscitation. 2021;163:78–85.
doi: 10.1016/j.resuscitation.2021.04.013
pubmed: 33895236
Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc. 2018;7.
Andersen LW, Holmberg MJ, Berg KM, Donnino MW, Granfeldt A. In-hospital cardiac arrest: a review. JAMA. 2019;321:1200–10.
doi: 10.1001/jama.2019.1696
pubmed: 30912843
pmcid: 6482460
Miles AH, Spaeder MC, Stockwell DC. Unplanned ICU transfers from inpatient units: examining the prevalence and preventability of adverse events associated with ICU transfer in pediatrics. J Pediatr Intensive Care. 2016;5:21–7.
pubmed: 31110878
Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: Causes and relationship to preventable errors in care. J Hosp Med. 2011;6:68–72.
doi: 10.1002/jhm.812
pubmed: 21290577
Weng CG, Poon J. A new evaluation measure for imbalanced datasets. In: Proceedings of the 7th Australasian Data Mining Conference; 2008, vol 87, p. 27–32.
Leening MJ, Vedder MM, Witteman JC, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician’s guide. Ann Intern Med. 2014;160:122–31.
doi: 10.7326/M13-1522
pubmed: 24592497
Wax R. Key elements of an RRS. In: Sebat F, editor. Designing, implementing and enhancing a rapid response system. Chicago: Society of Critical Care Medicine; 2009. p. 31–42.
Lee YJ, Park JJ, Yoon YE, Kim JW, Park JS, Kim T, et al. Successful implementation of a rapid response system in the department of internal medicine. KJCCM. 2014;29:77–82.
Calzavacca P, Licari E, Tee A, Egi M, Downey A, Quach J, et al. The impact of rapid response system on delayed emergency team activation patient characteristics and outcomes—a follow-up study. Resuscitation. 2010;81:31–5.
doi: 10.1016/j.resuscitation.2009.09.026
pubmed: 19854557
Bellomo R, Goldsmith D, Uchino S, Buckmaster J, Hart GK, Opdam H, et al. A prospective before-and-after trial of a medical emergency team. Med J Aust. 2003;179:283–7.
doi: 10.5694/j.1326-5377.2003.tb05548.x
pubmed: 12964909
Kim JS, Lee MJ, Park MH, Park JY, Kim AJ. Role of the rapid response system in end-of-life care decisions. Am J Hosp Palliat Care. 2020;37:943–9.
doi: 10.1177/1049909120927372
pubmed: 32452209
Jones DA, McIntyre T, Baldwin I, Mercer I, Kattula A, Bellomo R. The medical emergency team and end-of-life care: a pilot study. Crit Care Resusc. 2007;9:151–6.
pubmed: 17536983
Churpek MM, Yuen TC, Winslow C, Meltzer DO, Kattan MW, Edelson DP. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med. 2016;44:368–74.
doi: 10.1097/CCM.0000000000001571
pubmed: 26771782
pmcid: 4736499
Su CF, Chiu SI, Jang JR, Lai F. Improved inpatient deterioration detection in general wards by using time-series vital signs. Sci Rep. 2022;12:11901.
doi: 10.1038/s41598-022-16195-2
pubmed: 35831415
pmcid: 9279370
Cummings BC, Ansari S, Motyka JR, Wang G, Medlin RP Jr, Kronick SL, et al. Predicting intensive care transfers and other unforeseen events: Analytic model validation study and comparison to existing methods. JMIR Med Inform. 2021;9: e25066.
doi: 10.2196/25066
pubmed: 33818393
pmcid: 8061893
Romero-Brufau S, Whitford D, Johnson MG, Hickman J, Morlan BW, Therneau T, et al. Using machine learning to improve the accuracy of patient deterioration predictions: Mayo Clinic Early Warning Score (MC-EWS). J Am Med Inform Assoc. 2021;28:1207–15.
doi: 10.1093/jamia/ocaa347
pubmed: 33638343
pmcid: 8661441
Soffer S, Klang E, Barash Y, Grossman E, Zimlichman E. Predicting in-hospital mortality at admission to the medical ward: a big-data machine learning model. Am J Med. 2021;134:227-34.e4.
doi: 10.1016/j.amjmed.2020.07.014
pubmed: 32810465
Green M, Lander H, Snyder A, Hudson P, Churpek M, Edelson D. Comparison of the between the FLAGS calling criteria to the MEWS, NEWS and the electronic Cardiac Arrest Risk Triage (eCART) score for the identification of deteriorating ward patients. Resuscitation. 2018;123:86–91.
doi: 10.1016/j.resuscitation.2017.10.028
pubmed: 29169912
Subbe CP, Bannard-Smith J, Bunch J, Champunot R, DeVita MA, Durham L, et al. Quality metrics for the evaluation of rapid response systems: proceedings from the third international consensus conference on rapid response systems. Resuscitation. 2019;141:1–12.
doi: 10.1016/j.resuscitation.2019.05.012
pubmed: 31129229
Mardini L, Lipes J, Jayaraman D. Adverse outcomes associated with delayed intensive care consultation in medical and surgical inpatients. J Crit Care. 2012;27:688–93.
doi: 10.1016/j.jcrc.2012.04.011
pubmed: 22699035
Chen J, Bellomo R, Flabouris A, Hillman K, Assareh H, Ou L. Delayed emergency team calls and associated hospital mortality: a multicenter study. Crit Care Med. 2015;43:2059–65.
doi: 10.1097/CCM.0000000000001192
pubmed: 26181217