Performance of SAPS II according to ICU length of stay: A Danish nationwide cohort study.
Aged
Calibration
Cohort Studies
Critical Care
/ statistics & numerical data
Databases, Factual
Denmark
/ epidemiology
Female
Hospital Mortality
Humans
Intensive Care Units
/ statistics & numerical data
Length of Stay
/ statistics & numerical data
Male
Middle Aged
Predictive Value of Tests
Prognosis
Reproducibility of Results
Sensitivity and Specificity
Simplified Acute Physiology Score
Treatment Outcome
SAPS II
intensive care
length of stay
mortality
prediction
prediction models
Journal
Acta anaesthesiologica Scandinavica
ISSN: 1399-6576
Titre abrégé: Acta Anaesthesiol Scand
Pays: England
ID NLM: 0370270
Informations de publication
Date de publication:
10 2019
10 2019
Historique:
received:
06
11
2018
revised:
26
04
2019
accepted:
02
05
2019
pubmed:
15
6
2019
medline:
29
9
2020
entrez:
15
6
2019
Statut:
ppublish
Résumé
Intensive care unit (ICU) severity scores use data available at admission or shortly thereafter. There are limited contemporary data on how the prognostic performance of these scores is affected by ICU length of stay (LOS). We conducted a nationwide cohort study using routinely collected health data from the Danish Intensive Care Database. We included adults with ICU admissions ≥24 hours between 1 January 2012 and 30 June 2016, who survived to ICU discharge and had valid ICU LOS and vital status data registered. We assessed discrimination of the Simplified Acute Physiology Score (SAPS) II for predicting mortality 90 days after ICU discharge, followed by recalibration of the model and assessment of standardized mortality ratios (SMRs) and calibration. Performance was assessed in the entire cohort and stratified by ICU LOS quartiles. We included 44 523 patients. Increasing SAPS II was associated with increasing ICU LOS. Overall discrimination (area under the receiver-operating characteristics curve) of SAPS II was 0.70 (95% CI: 0.70-0.71), with decreasing discrimination from the first (0.75, 95% CI: 0.73-0.76) to the last (0.64, 95% CI: 0.63-0.65) ICU LOS quartile. SMRs were lower (less deaths) than expected in the first ICU LOS quartile and higher (more deaths) than expected in the last two ICU LOS quartiles. Calibration decreased with increasing ICU LOS. We observed that discrimination and calibration of SAPS II decreased with increasing ICU LOS, and that this affected SMRs. These findings should be acknowledged when using SAPS II for clinical, research and administrative purposes.
Sections du résumé
BACKGROUND
Intensive care unit (ICU) severity scores use data available at admission or shortly thereafter. There are limited contemporary data on how the prognostic performance of these scores is affected by ICU length of stay (LOS).
METHODS
We conducted a nationwide cohort study using routinely collected health data from the Danish Intensive Care Database. We included adults with ICU admissions ≥24 hours between 1 January 2012 and 30 June 2016, who survived to ICU discharge and had valid ICU LOS and vital status data registered. We assessed discrimination of the Simplified Acute Physiology Score (SAPS) II for predicting mortality 90 days after ICU discharge, followed by recalibration of the model and assessment of standardized mortality ratios (SMRs) and calibration. Performance was assessed in the entire cohort and stratified by ICU LOS quartiles.
RESULTS
We included 44 523 patients. Increasing SAPS II was associated with increasing ICU LOS. Overall discrimination (area under the receiver-operating characteristics curve) of SAPS II was 0.70 (95% CI: 0.70-0.71), with decreasing discrimination from the first (0.75, 95% CI: 0.73-0.76) to the last (0.64, 95% CI: 0.63-0.65) ICU LOS quartile. SMRs were lower (less deaths) than expected in the first ICU LOS quartile and higher (more deaths) than expected in the last two ICU LOS quartiles. Calibration decreased with increasing ICU LOS.
CONCLUSIONS
We observed that discrimination and calibration of SAPS II decreased with increasing ICU LOS, and that this affected SMRs. These findings should be acknowledged when using SAPS II for clinical, research and administrative purposes.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Validation Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
1200-1209Informations de copyright
© 2019 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.
Références
Vincent J-L, Moreno R. Clinical review: scoring systems in the critically ill. Crit Care. 2010;14:207. https://doi.org/10.1186/cc8204.
Le Gall J-R, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957-2963. [Erratum, JAMA. 1994;271:1321].
Sicignano A, Carozzi C, Giudici D, Merli G, Arlati S, Pulici M. The Influence of length of stay in the ICU on power of discrimination of a multipurpose severity score (SAPS). Intensive Care Med. 1996;22:1048-1051. https://doi.org/10.1007/s001340050211.
Suistomaa M, Niskanen M, Kari A, Hynynen M, Takala J. Customised prediction models based on APACHE II and SAPS II scores in patients with prolonged length of stay in the ICU. Intensive Care Med. 2002;28:479-485. https://doi.org/10.1007/s00134-002-1214-9.
Iwashyna TJ, Hodgson CL, Pilcher D, et al. Timing of onset and burden of persistent critical illness in Australia and New Zealand: a retrospective, population-based, observational study. Lancet Respir Med. 2016;4:566-573. https://doi.org/10.1016/S2213-2600(16)30098-4.
Jammer I, Wickboldt N, Sander M, et al. Standards for definitions and use of outcome measures for clinical effectiveness research in perioperative medicine: European Perioperative Clinical Outcome (EPCO) definitions: a statement from the ESA-ESICM joint taskforce on perioperative outcome measures. Eur J Anaesthesiol. 2015;88-105. https://doi.org/10.1097/EJA.0000000000000118.
Brinkman S, Abu-Hanna A, de Jonge E, de Keizer NF. Prediction of long-term mortality in ICU patients: model validation and assessing the effect of using in-hospital versus long-term mortality on benchmarking. Intensive Care Med. 2013;39:1925-1931. https://doi.org/10.1007/s00134-013-3042-5.
Lévesque LE, Hanley JA, Kezouh A, Suissa S. Problem of immortal time bias in cohort studies: example using statins for preventing progression of diabetes. BMJ. 2010;340:907-911. https://doi.org/10.1136/bmj.b5087.
Granholm A, Christiansen CF, Christensen S, Perner A, Møller MH. Performance of SAPS II according to ICU length of stay: Protocol for an observational study. Acta Anaesthesiol Scand. 2019;63(1):122-127. https://doi.org/10.1111/aas.13233.
Benchimol EI, Smeeth L, Guttmann A, et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement. PLoS Medicine. 2015;12:1-22. https://doi.org/10.1371/journal.pmed.1001885.
Lynge E, Sandegaard JL, Rebolj M. The Danish National Patient Register. Scand J Public Health. 2011;39:30-33. https://doi.org/10.1177/1403494811401482.
Pedersen CB. The Danish Civil Registration System. Scand J Public Health. 2011;39:22-25. https://doi.org/10.1177/1403494810387965.
Christiansen CF, Møller MH, Nielsen H, Christensen S. The Danish Intensive Care Database. Clin Epidemiol. 2016;8:525-530.
Wickham H, et al. Tidyverse. 2018. https://www.tidyverse.org/. Accessed November 2, 2018.
van Buuren S, Groothuis-Oudshoorn K. MICE: multivariate imputation by chained equations in R. J Stat Softw. 2011;45. https://doi.org/10.18637/jss.v045.i03.
Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373-383. https://doi.org/10.1016/0021-9681(87)90171-8.
Thygesen SK, Christiansen CF, Christensen S, Lash TL, Sørensen HT. The predictive value of ICD-10 diagnostic coding used to assess Charlson comorbidity index conditions in the population-based Danish National Registry of Patients. BMC Med Res Methodol. 2011;11:83. https://doi.org/10.1186/1471-2288-11-83.
Vesin A, Azoulay E, Ruckly S, et al. Reporting and handling missing values in clinical studies in intensive care units. Intensive Care Med. 2013;39:1396-1404. https://doi.org/10.1007/s00134-013-2949-1.
Marshall A, Altman DG, Holder RL, Royston P. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol. 2009;9:57. https://doi.org/10.1186/1471-2288-9-57.
Labarère J, Bertrand R, Fine MJ. How to derive and validate clinical prediction models for use in intensive care medicine. Intensive Care Med. 2014;40:513-527. https://doi.org/10.1007/s00134-014-3227-6.
Gönen M. Analyzing Receiver Operating Characteristic Curves with SAS®. 1st ed. Cary, NC: SAS Institute Inc; 2007.
Bland JM, Altman DG. Statistics notes: bootstrap resampling methods. BMJ. 2015;350:h2622-h2622. https://doi.org/10.1136/bmj.h2622.
Schomaker M, Heumann C. Bootstrap inference when using multiple imputation. Stat Med. 2018;2252-2266. https://doi.org/10.1002/sim.7654.
Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol. 2005;58:475-483. https://doi.org/10.1016/j.jclinepi.2004.06.017.
Le Gall J-R, Loirat P, Alperovitch A, et al. A simplified acute physiology score for ICU patients. Crit Care Med. 1984;12:975-977.
Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13:818-829. https://doi.org/10.1097/00003465-198603000-00013.
Vincent J-L, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med. 1996;22:707-710. https://doi.org/10.1007/BF01709751.
Nfor TK, Walsh TS, Prescott RJ. The impact of organ failures and their relationship with outcome in intensive care: analysis of a prospective multicentre database of adult admissions. Anaesthesia. 2006;61:731-738. https://doi.org/10.1111/j.1365-2044.2006.04707.x.
Bagshaw SM, Stelfox HT, Iwashyna TJ, Bellomo R, Zuege D, Wang X. Timing of onset of persistent critical illness: a multi-centre retrospective cohort study. Intensive Care Med. 2018;44:2134-2144. https://doi.org/10.1007/s00134-018-5440-1.
Ho AM-H, Dion PW, Ng CSH, Karmakar MK. Understanding immortal time bias in observational cohort studies. Anaesthesia. 2013;68:126-130. https://doi.org/10.1111/anae.12120.
Lee H, Lim CW, Hong HP, et al. Efficacy of the APACHE II score at ICU discharge in predicting post-ICU mortality and ICU readmission in critically ill surgical patients. Anaesth Intensive Care. 2015;43:175-186.
Peat G, Riley RD, Croft P, et al. Improving the transparency of prognosis research: the role of reporting, data sharing, registration, and protocols. PLoS Medicine. 2014;11(7):e1001671. https://doi.org/10.1371/journal.pmed.1001671.
Dal-Ré R, Ioannidis JP, Bracken MB, et al. Making prospective registration of observational research a reality. Sci Transl Med. 2014; 6(224):224cm1. https://doi.org/10.1126/scitranslmed.3007513.
Thomas L, Peterson ED. The value of statistical analysis plans in observational research. J Am Med Assoc. 2012;308:773-774.
The PLOS Medicine Editors. Observational studies: getting clear about transparency. PLoS Medicine. 2014;11:e1001711. https://doi.org/10.1371/journal.pmed.1001711.
Perner A, Haase N, Guttormsen AB, et al. Hydroxyethyl Starch 130/0.42 versus Ringer's Acetate in Severe Sepsis. N Engl J Med. 2012;367:124-134. https://doi.org/10.1056/NEJMoa1204242.
Holst LB, Haase N, Wetterslev J, et al. Lower versus higher hemoglobin threshold for transfusion in septic shock. N Engl J Med. 2014;371:1381-1391. https://doi.org/10.1056/NEJMoa1406617.
Krag M, Perner A, Wetterslev J, et al. Prevalence and outcome of gastrointestinal bleeding and use of acid suppressants in acutely ill adult intensive care patients. Intensive Care Med. 2015;41:833-845. https://doi.org/10.1007/s00134-015-3725-1.
Granholm A, Perner A, Krag M, et al. Simplified Mortality Score for the Intensive Care Unit (SMS-ICU): protocol for the development and validation of a bedside clinical prediction rule. BMJ Open. 2017;7:e015339. https://doi.org/10.1136/bmjopen-2016-015339.
Granholm A, Perner A, Krag M, et al. Development and internal validation of the Simplified Mortality Score for the Intensive Care Unit (SMS-ICU). Acta Anaesthesiol Scand. 2018;62:336-346. https://doi.org/10.1111/aas.13048.
Haniffa R, Mukaka M, Munasinghe SB, et al. Simplified prognostic model for critically ill patients in resource limited settings in South Asia. Crit Care. 2017;21:250. https://doi.org/10.1186/s13054-017-1843-6.
Dólera-Moreno C, Palazón-Bru A, Colomina-Climent F, Gil-Guillén VF. Construction and internal validation of a new mortality risk score for patients admitted to the intensive care unit. Int J Clin Pract. 2016;70:916-922. https://doi.org/10.1111/ijcp.12851.
Minne L, Eslami S, de Keizer N, de Jonge E, de Rooij SE, Abu-Hanna A. Effect of changes over time in the performance of a customized SAPS-II model on the quality of care assessment. Intensive Care Med. 2012;38:40-46. https://doi.org/10.1007/s00134-011-2390-2.
Harrison DA, Brady AR, Parry GJ, Carpenter JR, Rowan K. Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom. Crit Care Med. 2006;34:1378-1388. https://doi.org/10.1097/01.CCM.0000216702.94014.75.
Sprung CL, Cohen SL, Sjokvist P, et al. End-of-life practices in European intensive care units: the Ethicus study. JAMA. 2003;290:790-797. https://doi.org/10.1001/jama.290.6.790.