Implementation Experience with a 30-Day Hospital Readmission Risk Score in a Large, Integrated Health System: A Retrospective Study.
decision support model
electronic medical record
hospital readmission
Journal
Journal of general internal medicine
ISSN: 1525-1497
Titre abrégé: J Gen Intern Med
Pays: United States
ID NLM: 8605834
Informations de publication
Date de publication:
09 2022
09 2022
Historique:
received:
08
06
2021
accepted:
10
11
2021
pubmed:
9
2
2022
medline:
23
9
2022
entrez:
8
2
2022
Statut:
ppublish
Résumé
Driven by quality outcomes and economic incentives, predicting 30-day hospital readmissions remains important for healthcare systems. The Cleveland Clinic Health System (CCHS) implemented an internally validated readmission risk score in the electronic medical record (EMR). We evaluated the predictive accuracy of the readmission risk score across CCHS hospitals, across primary discharge diagnosis categories, between surgical/medical specialties, and by race and ethnicity. Retrospective cohort study. Adult patients discharged from a CCHS hospital April 2017-September 2020. Data was obtained from the CCHS EMR and billing databases. All patients discharged from a CCHS hospital were included except those from Oncology and Labor/Delivery, patients with hospice orders, or patients who died during admission. Discharges were categorized as surgical if from a surgical department or surgery was performed. Primary discharge diagnoses were classified per Agency for Healthcare Research and Quality Clinical Classifications Software Level 1 categories. Discrimination performance predicting 30-day readmission is reported using the c-statistic. The final cohort included 600,872 discharges from 11 Northeast Ohio and Florida CCHS hospitals. The readmission risk score for the cohort had a c-statistic of 0.6875 with consistent yearly performance. The c-statistic for hospital sites ranged from 0.6762, CI [0.6634, 0.6876], to 0.7023, CI [0.6903, 0.7132]. Medical and surgical discharges showed consistent performance with c-statistics of 0.6923, CI [0.6807, 0.7045], and 0.6802, CI [0.6681, 0.6925], respectively. Primary discharge diagnosis showed variation, with lower performance for congenital anomalies and neoplasms. COVID-19 had a c-statistic of 0.6387. Subgroup analyses showed c-statistics of > 0.65 across race and ethnicity categories. The CCHS readmission risk score showed good performance across diverse hospitals, across diagnosis categories, between surgical/medical specialties, and by patient race and ethnicity categories for 3 years after implementation, including during COVID-19. Evaluating clinical decision-making tools post-implementation is crucial to determine their continued relevance, identify opportunities to improve performance, and guide their appropriate use.
Sections du résumé
BACKGROUND
Driven by quality outcomes and economic incentives, predicting 30-day hospital readmissions remains important for healthcare systems. The Cleveland Clinic Health System (CCHS) implemented an internally validated readmission risk score in the electronic medical record (EMR).
OBJECTIVE
We evaluated the predictive accuracy of the readmission risk score across CCHS hospitals, across primary discharge diagnosis categories, between surgical/medical specialties, and by race and ethnicity.
DESIGN
Retrospective cohort study.
PARTICIPANTS
Adult patients discharged from a CCHS hospital April 2017-September 2020.
MAIN MEASURES
Data was obtained from the CCHS EMR and billing databases. All patients discharged from a CCHS hospital were included except those from Oncology and Labor/Delivery, patients with hospice orders, or patients who died during admission. Discharges were categorized as surgical if from a surgical department or surgery was performed. Primary discharge diagnoses were classified per Agency for Healthcare Research and Quality Clinical Classifications Software Level 1 categories. Discrimination performance predicting 30-day readmission is reported using the c-statistic.
RESULTS
The final cohort included 600,872 discharges from 11 Northeast Ohio and Florida CCHS hospitals. The readmission risk score for the cohort had a c-statistic of 0.6875 with consistent yearly performance. The c-statistic for hospital sites ranged from 0.6762, CI [0.6634, 0.6876], to 0.7023, CI [0.6903, 0.7132]. Medical and surgical discharges showed consistent performance with c-statistics of 0.6923, CI [0.6807, 0.7045], and 0.6802, CI [0.6681, 0.6925], respectively. Primary discharge diagnosis showed variation, with lower performance for congenital anomalies and neoplasms. COVID-19 had a c-statistic of 0.6387. Subgroup analyses showed c-statistics of > 0.65 across race and ethnicity categories.
CONCLUSIONS
The CCHS readmission risk score showed good performance across diverse hospitals, across diagnosis categories, between surgical/medical specialties, and by patient race and ethnicity categories for 3 years after implementation, including during COVID-19. Evaluating clinical decision-making tools post-implementation is crucial to determine their continued relevance, identify opportunities to improve performance, and guide their appropriate use.
Identifiants
pubmed: 35132549
doi: 10.1007/s11606-021-07277-4
pii: 10.1007/s11606-021-07277-4
pmc: PMC8821785
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
3054-3061Informations de copyright
© 2022. The Author(s) under exclusive licence to Society of General Internal Medicine.
Références
Kansagara D, Englander H, Salanitro A, et al. Risk Prediction Models for Hospital Readmission: A Systematic Review. JAMA J Am Med Assoc. 2011;306(15):1688-1698. https://doi.org/10.1001/jama.2011.1515
doi: 10.1001/jama.2011.1515
Donzé J, Aujesky D, Williams D, Schnipper JL. Potentially Avoidable 30-Day Hospital Readmissions in Medical Patients: Derivation and Validation of a Prediction Model. JAMA Intern Med. 2013;173(8):632-638. https://doi.org/10.1001/jamainternmed.2013.3023
doi: 10.1001/jamainternmed.2013.3023
pubmed: 23529115
Tsui E, Au SY, Wong CP, Cheung A, Lam P. Development of an automated model to predict the risk of elderly emergency medical admissions within a month following an index hospital visit: a Hong Kong experience. Health Informatics J. 2015;21(1):46-56. https://doi.org/10.1177/1460458213501095
doi: 10.1177/1460458213501095
pubmed: 24352596
Gildersleeve R, Cooper P. Development of an automated, real time surveillance tool for predicting readmissions at a community hospital. Appl Clin Inform. 2013;4(2):153-169. https://doi.org/10.4338/ACI-2012-12-RA-0058
doi: 10.4338/ACI-2012-12-RA-0058
pubmed: 23874355
pmcid: 3716420
Uhlmann M, Lécureux E, Griesser A-C, Duong HD, Lamy O. Prediction of potentially avoidable readmission risk in a division of general internal medicine. Swiss Med Wkly. 2017;147:w14470. https://doi.org/10.4414/smw.2017.14470
doi: 10.4414/smw.2017.14470
pubmed: 28750417
Zapatero A, Barba R, Marco J, et al. Predictive model of readmission to internal medicine wards. Eur J Intern Med. 2012;23(5):451-456. https://doi.org/10.1016/j.ejim.2012.01.005
doi: 10.1016/j.ejim.2012.01.005
pubmed: 22726375
Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community | CMAJ. Accessed April 14, 2021. https://www.cmaj.ca/content/182/6/551
Hospital Readmissions Reduction Program (HRRP) | CMS. Accessed April 14, 2021. https://www.cms.gov/medicare/medicare-fee-for-service-payment/acuteinpatientpps/readmissions-reduction-program
Tsai TC, Joynt KE, Orav EJ, Gawande AA, Jha AK. Variation in Surgical-Readmission Rates and Quality of Hospital Care. N Engl J Med. 2013;369(12):1134-1142. https://doi.org/10.1056/NEJMsa1303118
doi: 10.1056/NEJMsa1303118
pubmed: 24047062
pmcid: 4107655
Haneuse S, Dominici F, Normand S-L, Schrag D. Assessment of Between-Hospital Variation in Readmission and Mortality After Cancer Surgical Procedures. JAMA Netw Open. 2018;1(6):e183038-e183038. https://doi.org/10.1001/jamanetworkopen.2018.3038
doi: 10.1001/jamanetworkopen.2018.3038
pubmed: 30646221
pmcid: 6324436
Singh S, Lin Y-L, Kuo Y-F, Nattinger AB, Goodwin JS. Variation in the risk of readmission among hospitals: the relative contribution of patient, hospital and inpatient provider characteristics. J Gen Intern Med. 2014;29(4):572-578. https://doi.org/10.1007/s11606-013-2723-7
doi: 10.1007/s11606-013-2723-7
pubmed: 24307260
Yu S, Farooq F, van Esbroeck A, Fung G, Anand V, Krishnapuram B. Predicting readmission risk with institution-specific prediction models. Artif Intell Med. 2015;65(2):89-96. https://doi.org/10.1016/j.artmed.2015.08.005
doi: 10.1016/j.artmed.2015.08.005
pubmed: 26363683
Yeh RW, Rosenfield K, Zelevinsky K, et al. Sources of hospital variation in short-term readmission rates after percutaneous coronary intervention. Circ Cardiovasc Interv. 2012;5(2):227-236. https://doi.org/10.1161/CIRCINTERVENTIONS.111.967638
doi: 10.1161/CIRCINTERVENTIONS.111.967638
pubmed: 22438431
Hekkert K, Kool RB, Rake E, et al. To what degree can variations in readmission rates be explained on the level of the hospital? a multilevel study using a large Dutch database. BMC Health Serv Res. 2018;18(1):999. https://doi.org/10.1186/s12913-018-3761-y
doi: 10.1186/s12913-018-3761-y
pubmed: 30591058
pmcid: 6307249
Gallagher D, Zhao C, Brucker A, et al. Implementation and Continuous Monitoring of an Electronic Health Record Embedded Readmissions Clinical Decision Support Tool. J Pers Med. 2020;10(3):E103. https://doi.org/10.3390/jpm10030103
doi: 10.3390/jpm10030103
pubmed: 32858890
Epic Systems, Inc., Verona, WI. Accessed May 26, 2021. https://www.epic.com/
Clinical Classifications Software (CCS) for ICD-10-PCS (beta version). Accessed April 14, 2021. https://www.hcup-us.ahrq.gov/toolssoftware/ccs10/ccs10.jsp
Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453. https://doi.org/10.1126/science.aax2342
doi: 10.1126/science.aax2342
pubmed: 31649194
Coley RY, Johnson E, Simon GE, Cruz M, Shortreed SM. Racial/Ethnic Disparities in the Performance of Prediction Models for Death by Suicide After Mental Health Visits. JAMA Psychiatry. 2021;78(7):726-734. https://doi.org/10.1001/jamapsychiatry.2021.0493
doi: 10.1001/jamapsychiatry.2021.0493
pubmed: 33909019
Matheny ME, Whicher D, Thadaney Israni S. Artificial Intelligence in Health Care: A Report From the National Academy of Medicine. JAMA. 2020;323(6):509-510. https://doi.org/10.1001/jama.2019.21579
doi: 10.1001/jama.2019.21579
pubmed: 31845963
Hilton CB, Milinovich A, Felix C, et al. Personalized predictions of patient outcomes during and after hospitalization using artificial intelligence. Npj Digit Med. 2020;3(1):1-8. https://doi.org/10.1038/s41746-020-0249-z
doi: 10.1038/s41746-020-0249-z
Struja T, Baechli C, Koch D, et al. What Are They Worth? Six 30-Day Readmission Risk Scores for Medical Inpatients Externally Validated in a Swiss Cohort. J Gen Intern Med. 2020;35(7):2017-2024. https://doi.org/10.1007/s11606-020-05638-z
doi: 10.1007/s11606-020-05638-z
pubmed: 31965531
pmcid: 7351934
Leppin AL, Gionfriddo MR, Kessler M, et al. Preventing 30-Day Hospital Readmissions: A Systematic Review and Meta-analysis of Randomized Trials. JAMA Intern Med. 2014;174(7):1095. https://doi.org/10.1001/jamainternmed.2014.1608
doi: 10.1001/jamainternmed.2014.1608
pubmed: 24820131
pmcid: 4249925
Misra-Hebert AD, Rothberg MB, Fox J, et al. Healthcare utilization and patient and provider experience with a home visit program for patients discharged from the hospital at high risk for readmission. Healthc Amst Neth. 2021;9(1):100518. https://doi.org/10.1016/j.hjdsi.2020.100518
doi: 10.1016/j.hjdsi.2020.100518
Nguyen OK, Washington C, Clark CR, et al. Man vs. Machine: Comparing Physician vs. Electronic Health Record–Based Model Predictions for 30-Day Hospital Readmissions. J Gen Intern Med. 2021;36(9):2555-2562. https://doi.org/10.1007/s11606-020-06355-3
doi: 10.1007/s11606-020-06355-3
pubmed: 33443694
pmcid: 8390613
Flaks-Manov N, Srulovici E, Yahalom R, Perry-Mezre H, Balicer R, Shadmi E. Preventing Hospital Readmissions: Healthcare Providers’ Perspectives on “Impactibility” Beyond EHR 30-Day Readmission Risk Prediction. J Gen Intern Med. 2020;35(5):1484-1489. https://doi.org/10.1007/s11606-020-05739-9
doi: 10.1007/s11606-020-05739-9
pubmed: 32141041
pmcid: 7210355
Marcotte LM, Reddy A, Zhou L, Miller SC, Hudelson C, Liao JM. Trends in Utilization of Transitional Care Management in the United States. JAMA Netw Open. 2020;3(1):e1919571-e1919571. https://doi.org/10.1001/jamanetworkopen.2019.19571
doi: 10.1001/jamanetworkopen.2019.19571
pubmed: 31968111
pmcid: 6991271
Hoyer EH, Golden B, Dougherty G, et al. The Paradox of Readmission Prevention Interventions: Missing Those Most in Need. Am J Med. Published online May 7, 2021. https://doi.org/10.1016/j.amjmed.2021.04.006
Fakha A, Groenvynck L, de Boer B, van Achterberg T, Hamers J, Verbeek H. A myriad of factors influencing the implementation of transitional care innovations: a scoping review. Implement Sci. 2021;16(1):21. https://doi.org/10.1186/s13012-021-01087-2
doi: 10.1186/s13012-021-01087-2
pubmed: 33637097
pmcid: 7912549
Bensken WP, Alberti PM, Koroukian SM. Health-Related Social Needs and Increased Readmission Rates: Findings from the Nationwide Readmissions Database. J Gen Intern Med. 2021;36(5):1173-1180. https://doi.org/10.1007/s11606-021-06646-3
doi: 10.1007/s11606-021-06646-3
pubmed: 33634384
pmcid: 8131460
Anderson TS, O’Donoghue AL, Dechen T, Herzig SJ, Stevens JP. Trends in telehealth and in-person transitional care management visits during the COVID-19 pandemic. J Am Geriatr Soc. n/a(n/a). https://doi.org/10.1111/jgs.17329