Developing and validating a machine learning ensemble model to predict postoperative delirium in a cohort of high-risk surgical patients: A secondary cohort analysis.
Journal
European journal of anaesthesiology
ISSN: 1365-2346
Titre abrégé: Eur J Anaesthesiol
Pays: England
ID NLM: 8411711
Informations de publication
Date de publication:
01 05 2023
01 05 2023
Historique:
medline:
6
4
2023
pubmed:
3
3
2023
entrez:
2
3
2023
Statut:
ppublish
Résumé
Postoperative delirium (POD) has a negative impact on prognosis, length of stay and the burden of care. Although its prediction and identification may improve postoperative care, this need is largely unmet in the Brazilian public health system. To develop and validate a machine-learning prediction model and estimate the incidence of delirium. We hypothesised that an ensemble machine-learning prediction model that incorporates predisposing and precipitating features could accurately predict POD. A secondary analysis nested in a cohort of high-risk surgical patients. An 800-bed, quaternary university-affiliated teaching hospital in Southern Brazil. We included patients operated on from September 2015 to February 2020. We recruited 1453 inpatients with an all-cause postoperative 30-day mortality risk greater than 5% assessed preoperatively by the ExCare Model. The incidence of POD classified by the Confusion Assessment Method, up to 7 days postoperatively. Predictive model performance with different feature scenarios were compared with the area under the receiver operating characteristic curve. The cumulative incidence of delirium was 117, giving an absolute risk of 8.05/100 patients. We developed multiple machine-learning nested cross-validated ensemble models. We selected features through partial dependence plot analysis and theoretical framework. We treated the class imbalance with undersampling. Different feature scenarios included: 52 preoperative, 60 postoperative and only three features (age, preoperative length of stay and the number of postoperative complications). The mean areas (95% confidence interval) under the curve ranged from 0.61 (0.59 to 0.63) to 0.74 (0.73 to 0.75). A predictive model composed of three indicative readily available features performed better than those with numerous perioperative features, pointing to its feasibility as a prognostic tool for POD. Further research is required to test the generalisability of this model. Institutional Review Board Registration number 04448018.8.0000.5327 (Brazilian CEP/CONEP System, available in https://plataformabrasil.saude.gov.br/ ).
Sections du résumé
BACKGROUND
Postoperative delirium (POD) has a negative impact on prognosis, length of stay and the burden of care. Although its prediction and identification may improve postoperative care, this need is largely unmet in the Brazilian public health system.
OBJECTIVE
To develop and validate a machine-learning prediction model and estimate the incidence of delirium. We hypothesised that an ensemble machine-learning prediction model that incorporates predisposing and precipitating features could accurately predict POD.
DESIGN
A secondary analysis nested in a cohort of high-risk surgical patients.
SETTING
An 800-bed, quaternary university-affiliated teaching hospital in Southern Brazil. We included patients operated on from September 2015 to February 2020.
PATIENTS
We recruited 1453 inpatients with an all-cause postoperative 30-day mortality risk greater than 5% assessed preoperatively by the ExCare Model.
MAIN OUTCOME MEASURE
The incidence of POD classified by the Confusion Assessment Method, up to 7 days postoperatively. Predictive model performance with different feature scenarios were compared with the area under the receiver operating characteristic curve.
RESULTS
The cumulative incidence of delirium was 117, giving an absolute risk of 8.05/100 patients. We developed multiple machine-learning nested cross-validated ensemble models. We selected features through partial dependence plot analysis and theoretical framework. We treated the class imbalance with undersampling. Different feature scenarios included: 52 preoperative, 60 postoperative and only three features (age, preoperative length of stay and the number of postoperative complications). The mean areas (95% confidence interval) under the curve ranged from 0.61 (0.59 to 0.63) to 0.74 (0.73 to 0.75).
CONCLUSION
A predictive model composed of three indicative readily available features performed better than those with numerous perioperative features, pointing to its feasibility as a prognostic tool for POD. Further research is required to test the generalisability of this model.
TRIAL REGISTRATION
Institutional Review Board Registration number 04448018.8.0000.5327 (Brazilian CEP/CONEP System, available in https://plataformabrasil.saude.gov.br/ ).
Identifiants
pubmed: 36860180
doi: 10.1097/EJA.0000000000001811
pii: 00003643-202305000-00007
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
356-364Informations de copyright
Copyright © 2023 European Society of Anaesthesiology and Intensive Care. Unauthorized reproduction of this article is prohibited.
Références
Pearse RM, Harrison DA, James P, et al. Identification and characterisation of the high-risk surgical population in the United Kingdom. Crit Care 2006; 10:R81.
Evered L, Silbert B, Knopman DS, et al. Recommendations for the nomenclature of cognitive change associated with anaesthesia and surgery—2018. Br J Anaesth 2018; 121:1005–1012.
Chaiwat O, Chanidnuan M, Pancharoen W, et al. Postoperative delirium in critically ill surgical patients: incidence, risk factors, and predictive scores. BMC Anesthesiol 2019; 19:39.
Furlaneto ME, Garcez-Leme LE. Delirium in elderly individuals with hip fracture: causes, incidence, prevalence, and risk factors. Clinics 2006; 61:35–40.
Fong TG, Jones RN, Marcantonio ER, et al. Adverse outcomes after hospitalization and delirium in persons with alzheimer disease. Ann Intern Med 2012; 156:848.
Krähenbühl ES, Immer FF, Stalder M, et al. Temporary neurological dysfunction after surgery of the thoracic aorta: a predictor of poor outcome and impaired quality of life. Eur J Cardiothorac Surg 2008; 33:1025–1029.
González M, Martínez G, Calderón J, et al. Impact of delirium on short-term mortality in elderly inpatients: a prospective cohort study. Psychosomatics 2009; 50:234–238.
Franco K, Litaker D, Locala J, et al. The cost of delirium in the surgical patient. Psychosomatics 2001; 42:68–73.
Lingehall HC, Smulter NS, Lindahl E, et al. Preoperative cognitive performance and postoperative delirium are independently associated with future dementia in older people who have undergone cardiac surgery: a longitudinal cohort study. Crit Care Med 2017; 45:1295–1303.
Sprung J, Roberts RO, Weingarten TN, et al. Postoperative delirium in elderly patients is associated with subsequent cognitive impairment. Br J Anaesth 2017; 119:316–323.
Portuondo JI, Shah SR, Singh H, et al. Failure to rescue as a surgical quality indicator. Anesthesiology 2019; 131:426–437.
Castro MC, Massuda A, Almeida G, et al. Brazil's unified health system: the first 30 years and prospects for the future. Lancet 2019; 394:345–356.
Jin Z, Hu J, Ma D. Postoperative delirium: perioperative assessment, risk reduction, and management. Br J Anaesth 2020; 125:492–504.
Cascella M, Muzio MR, Bimonte S, et al. Postoperative delirium and postoperative cognitive dysfunction: updates in pathophysiology, potential translational approaches to clinical practice and further research perspectives. Minerva Anestesiol 2018; 84:246–260.
Rengel KF, Pandharipande PP, Hughes CG. Special considerations for the aging brain and perioperative neurocognitive dysfunction. Anesthesiol Clin 2019; 37:521–536.
Wang P, Velagapudi R, Kong C, et al. Neurovascular and immune mechanisms that regulate postoperative delirium superimposed on dementia. Alzheimers Dement 2020; 16:734–749.
Whitlock EL, Vannucci A, Avidan MS. Postoperative delirium. Minerva Anestesiol 2011; 77:448–456.
Wang CG, Qin YF, Wan X, et al. Incidence and risk factors of postoperative delirium in the elderly patients with hip fracture. J Orthop Surg 2018; 13:186.
Ansaloni L, Catena F, Chattat R, et al. Risk factors and incidence of postoperative delirium in elderly patients after elective and emergency surgery. Br J Surg 2010; 97:273–280.
Inouye SK, Westendorp RG, Saczynski JS. Delirium in elderly people. Lancet 2014; 383:911–922.
Berian JR, Zhou L, Russell MM, et al. Postoperative delirium as a target for surgical quality improvement. Ann Surg 2018; 268:93–99.
Hughes CG, Boncyk CS, Culley DJ, et al. American Society for Enhanced Recovery and Perioperative Quality Initiative Joint Consensus Statement on Postoperative Delirium Prevention. Anesth Analg 2020; 130:1572–1590.
Freter SH. Predicting postoperative delirium in elective orthopaedic patients: the Delirium Elderly At-Risk (DEAR) instrument. Age Ageing 2005; 34:169–171.
Kim MY, Park UJ, Kim HT, et al. DELirium Prediction Based on Hospital Information (Delphi) in general surgery patients. Medicine (Baltimore) 2016; 95:e3072.
Whitlock EL, Braehler MR, Kaplan JA, et al. Derivation, validation, sustained performance, and clinical impact of an electronic medical record-based perioperative delirium risk stratification tool. Anesth Analg 2020; 131:1901–1910.
Wong A, Young AT, Liang AS, et al. Development and validation of an electronic health record–based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment. JAMA Netw Open 2018; 1:e181018.
Xue B, Li D, Lu C, et al. Use of machine learning to develop and evaluate models using preoperative and intraoperative data to identify risks of postoperative complications. JAMA Netw Open 2021; 4:e212240–e1212240.
Racine AM, Tommet D, D’Aquila ML, et al. The RISE Study Group. Machine learning to develop and internally validate a predictive model for postoperative delirium in a prospective, observational clinical cohort study of older surgical patients. J Gen Intern Med 2021; 36:265–273.
Zhao H, You J, Peng Y, et al. Machine learning algorithm using electronic chart-derived data to predict delirium after elderly hip fracture surgeries: a retrospective case–control study. Front Surg 2021; 8:248.
Davoudi A, Ebadi A, Rashidi P, et al. Delirium prediction using machine learning models on preoperative electronic health records data. Proc IEEE Int Symp Bioinforma Bioeng 2017; 2017:568–573.
Mikalsen KØ, Soguero-Ruiz C, Jensen K, et al. Using anchors from free text in electronic health records to diagnose postoperative delirium. Comput Methods Programs Biomed 2017; 152:105–114.
Bihorac A, Ozrazgat-Baslanti T, Ebadi A, et al. MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery. Ann Surg 2019; 269:652–662.
Wang Y, Lei L, Ji M, et al. Predicting postoperative delirium after microvascular decompression surgery with machine learning. J Clin Anesth 2020; 66:109896.
Stahlschmidt A, Passos SC, Cardoso GR, et al. Enhanced peri-operative care to improve outcomes for high-risk surgical patients in Brazil: a single-centre before-and-after cohort study. Anaesthesia 2022; 77:416–427.
Luo W, Phung D, Tran T, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res 2016; 18:e323.
Moons KGM, Altman DG, Reitsma JB, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015; 162:W1–73.
Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015 an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015; 351:h5527.
Stefani LC, Gutierrez CDS, Castro SM, et al. Derivation and validation of a preoperative risk model for postoperative mortality (SAMPE model): an approach to care stratification. PLoS One 2017; 12:e0187122.
Gutierrez CS, Passos SC, Castro SMJ, et al. Few and feasible preoperative variables can identify high-risk surgical patients: derivation and validation of the Ex-Care risk model. Br J Anaesth 2021; 126:525–532.
Fabbri RMA, Moreira MA, Garrido R, et al. Validity and reliability of the Portuguese version of the Confusion Assessment Method (CAM) for the detection of delirium in the elderly. Arq Neuropsiquiatr 2001; 59 (2A):175–179.
Grocott MPW, Browne JP, Van der Meulen J, et al. The postoperative morbidity survey was validated and used to describe morbidity after major surgery. J Clin Epidemiol 2007; 60:919–928.
Elshawi R, Al-Mallah MH, Sakr S. On the interpretability of machine learning-based model for predicting hypertension. BMC Med Inform Decis Mak 2019; 19:146.
Vabalas A, Gowen E, Poliakoff E, et al. Machine learning algorithm validation with a limited sample size. PLoS One 2019; 14:e0224365.
Cawley GC, Talbot NLC. On over-fitting in model selection and subsequent selection bias in performance evaluation. J Mach Learn Res 2010; 11:2079–2107.
Brnabic A, Hess LM. Systematic literature review of machine learning methods used in the analysis of real-world data for patient-provider decision making. BMC Med Inform Decis Mak 2021; 21:54.
Leão BF, Bernardes MM, Levin J, et al. The Brazilian National Health Informatics Strategy. Stud Health Technol Inform 2001; 84 (Pt 1):38–42.
Bishara A, Chiu C, Whitlock EL, et al. Postoperative delirium prediction using machine learning models and preoperative electronic health record data. BMC Anesthesiol 2022; 22:8.