External validation of a 2-year all-cause mortality prediction tool developed using machine learning in patients with stage 4-5 chronic kidney disease.
Chronic kidney disease
Machine learning
Mortality
Validation prediction model
Journal
Journal of nephrology
ISSN: 1724-6059
Titre abrégé: J Nephrol
Pays: Italy
ID NLM: 9012268
Informations de publication
Date de publication:
04 Jul 2024
04 Jul 2024
Historique:
received:
10
01
2024
accepted:
13
06
2024
medline:
5
7
2024
pubmed:
5
7
2024
entrez:
4
7
2024
Statut:
aheadofprint
Résumé
Chronic kidney disease (CKD) is associated with increased mortality. Individual mortality prediction could be of interest to improve individual clinical outcomes. Using an independent regional dataset, the aim of the present study was to externally validate the recently published 2-year all-cause mortality prediction tool developed using machine learning. A validation dataset of stage 4 or 5 CKD outpatients was used. External validation performance of the prediction tool at the optimal cutoff-point was assessed by the area under the receiver operating characteristic curve (AUC-ROC), accuracy, sensitivity, and specificity. A survival analysis was then performed using the Kaplan-Meier method. Data of 527 outpatients with stage 4 or 5 CKD were analyzed. During the 2 years of follow-up, 91 patients died and 436 survived. Compared to the learning dataset, patients in the validation dataset were significantly younger, and the ratio of deceased patients in the validation dataset was significantly lower. The performance of the prediction tool at the optimal cutoff-point was: AUC-ROC = 0.72, accuracy = 63.6%, sensitivity = 72.5%, and specificity = 61.7%. The survival curves of the predicted survived and the predicted deceased groups were significantly different (p < 0.001). The 2-year all-cause mortality prediction tool for patients with stage 4 or 5 CKD showed satisfactory discriminatory capacity with emphasis on sensitivity. The proposed prediction tool appears to be of clinical interest for further development.
Sections du résumé
BACKGROUND
BACKGROUND
Chronic kidney disease (CKD) is associated with increased mortality. Individual mortality prediction could be of interest to improve individual clinical outcomes. Using an independent regional dataset, the aim of the present study was to externally validate the recently published 2-year all-cause mortality prediction tool developed using machine learning.
METHODS
METHODS
A validation dataset of stage 4 or 5 CKD outpatients was used. External validation performance of the prediction tool at the optimal cutoff-point was assessed by the area under the receiver operating characteristic curve (AUC-ROC), accuracy, sensitivity, and specificity. A survival analysis was then performed using the Kaplan-Meier method.
RESULTS
RESULTS
Data of 527 outpatients with stage 4 or 5 CKD were analyzed. During the 2 years of follow-up, 91 patients died and 436 survived. Compared to the learning dataset, patients in the validation dataset were significantly younger, and the ratio of deceased patients in the validation dataset was significantly lower. The performance of the prediction tool at the optimal cutoff-point was: AUC-ROC = 0.72, accuracy = 63.6%, sensitivity = 72.5%, and specificity = 61.7%. The survival curves of the predicted survived and the predicted deceased groups were significantly different (p < 0.001).
CONCLUSION
CONCLUSIONS
The 2-year all-cause mortality prediction tool for patients with stage 4 or 5 CKD showed satisfactory discriminatory capacity with emphasis on sensitivity. The proposed prediction tool appears to be of clinical interest for further development.
Identifiants
pubmed: 38965199
doi: 10.1007/s40620-024-02011-9
pii: 10.1007/s40620-024-02011-9
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024. The Author(s) under exclusive licence to Italian Society of Nephrology.
Références
WHO (2020) The top 10 causes of death. WHO. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death . Accessed 31 Aug 2023
Junaid Nazar CM, Kindratt TB, Ahmad SM, Ahmed M, Anderson J (2014) Barriers to the successful practice of chronic kidney diseases at the primary health care level; a systematic review. J Renal Inj Prev 3:61–67. https://doi.org/10.12861/jrip.2014.20
doi: 10.12861/jrip.2014.20
pubmed: 25340171
pmcid: 4206053
Couser WG, Remuzzi G, Mendis S, Tonelli M (2011) The contribution of chronic kidney disease to the global burden of major noncommunicable diseases. Kidney Int 80:1258–1270. https://doi.org/10.1038/ki.2011.368
doi: 10.1038/ki.2011.368
pubmed: 21993585
SCORE2 working group and ESC Cardiovascular risk collaboration (2021) SCORE2 risk prediction algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe. Eur Heart J 42:2439–2454. https://doi.org/10.1093/eurheartj/ehab309
doi: 10.1093/eurheartj/ehab309
Hippisley-Cox J, Coupland C, Brindle P (2017) Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ 357:j2099. https://doi.org/10.1136/bmj.j2099
doi: 10.1136/bmj.j2099
pubmed: 28536104
pmcid: 5441081
Hansrivijit P, Chen YJ, Lnu K, Trongtorsak A, Puthenpura MM, Thongprayoon C, Bathini T, Mao MA, Cheungpasitporn W (2021) Prediction of mortality among patients with chronic kidney disease: a systematic review. World J Nephrol 10:59–75. https://doi.org/10.5527/wjn.v10.i4.59
doi: 10.5527/wjn.v10.i4.59
pubmed: 34430385
pmcid: 8353601
Bansal N, Katz R, De Boer IH, Peralta CA, Fried LF, Siscovick DS, Rifkin DE, Hirsch C, Cummings SR, Harris TB, Kritchevsky SB, Sarnak MJ, Shlipak MG, Ix JH (2015) Development and validation of a model to predict 5-year risk of death without ESRD among older adults with CKD. Clin J Am Soc Nephrol 10:363–371. https://doi.org/10.2215/cjn.04650514
doi: 10.2215/cjn.04650514
pubmed: 25710804
pmcid: 4348680
Danial M, Hassali MA, Meng OL, Kin YC, Khan AH (2019) Development of a mortality score to assess risk of adverse drug reactions among hospitalized patients with moderate to severe chronic kidney disease. BMC Pharmacol Toxicol 20:41. https://doi.org/10.1186/s40360-019-0318-6
doi: 10.1186/s40360-019-0318-6
pubmed: 31287030
pmcid: 6615098
Gan GCH, Kadappu KK, Bhat A, Fernandez F, Gu KH, Cai L, Byth K, Eshoo S, Thomas L (2021) Left atrial strain is the best predictor of adverse cardiovascular outcomes in patients with chronic kidney disease. J Am Soc Echocardiog 34:166–175. https://doi.org/10.1016/j.echo.2020.09.015
doi: 10.1016/j.echo.2020.09.015
Jagadeswaran D, Indhumathi E, Hemamalini AJ, Sivakumar V, Soundararajan P, Jayakumar M (2018) Inflammation and nutritional status assessment by malnutrition inflammation score and its outcome in pre-dialysis chronic kidney disease patients. Clin Nutr 38:341–347. https://doi.org/10.1016/j.clnu.2018.01.001
doi: 10.1016/j.clnu.2018.01.001
pubmed: 29398341
Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M (2021) External validation of prognostic models: what, why, how, when and where? Clin Kidney J 14:49–58. https://doi.org/10.1093/ckj/sfaa188
doi: 10.1093/ckj/sfaa188
pubmed: 33564405
Steyerberg EW (2019) Clinical prediction models: a practical approach to development, validation, and updating. Springer Link, New York
doi: 10.1007/978-3-030-16399-0
Tran NTD, Balezeaux M, Granal M, Fouque D, Ducher M, Fauvel JP (2022) Prediction of all-cause mortality for chronic kidney disease patients using four models of machine learnings. Nephrol Dial Transplant 38:1691–1699. https://doi.org/10.1093/ndt/gfac316
doi: 10.1093/ndt/gfac316
Fouque D, Roth H, Darne B, Bouchet JL, Daugas E, Drüeke TB, Hannedouche T, Jean G, London GM (2018) Achievement of 2009 and 2017 kidney disease: improving global outcomes mineral and bone targets and survival in a French cohort of chronic kidney disease Stages 4 and 5 non-dialysis patients. Clin Kidney J 11:710–719. https://doi.org/10.1093/ckj/sfy015
doi: 10.1093/ckj/sfy015
pubmed: 30288267
pmcid: 6165763
Schisterman EF, Faraggi D, Reiser B, Hu J (2008) Youden Index and the optimal threshold for markers with mass at zero. Stat Med 27:297–315.
RapidMiner (2023). https://rapidminer.com/ . Accessed 31 Aug 2023
Riley RD, Debray TPA, Collins GS, Archer L, Ensor J, van Smeden M, Snell KIE (2021) Minimum sample size for external validation of a clinical prediction model with a binary outcome. Stat Med 40:4230–4425. https://doi.org/10.1002/sim.9025
doi: 10.1002/sim.9025
pubmed: 34031906
Bland JM, Altman DG (2004) The logrank test. BMJ 328:1073. https://doi.org/10.1136/bmj.328.7447.1073
doi: 10.1136/bmj.328.7447.1073
pubmed: 15117797
pmcid: 403858
Ramspek CL, de Jong Y, Dekker FW, van Diepen M (2020) Towards the best kidney failure prediction tool: a systematic review and selection aid. Nephrol Dial Transplant 35:1527–1538. https://doi.org/10.1093/ndt/gfz018
doi: 10.1093/ndt/gfz018
pubmed: 30830157
Sanmarchi F, Fanconi C, Golinelli D, Gori D, Hernandez-Boussard T, Capodici A (2023) Predict, diagnose, and treat chronic kidney disease with machine learning: a systematic literature review. J Nephrol 36:1101–1117. https://doi.org/10.1007/s40620-023-01573-4
doi: 10.1007/s40620-023-01573-4
pubmed: 36786976
pmcid: 10227138
Tangri N, Kitsios GD, Inker LA, Griffith J, Naimark DM, Walker S, Rigatto C, Uhlig K, Kent DM, Levey AS (2013) Risk prediction models for patients with chronic kidney disease: a systematic review. Ann Intern Med 158:596–603. https://doi.org/10.7326/0003-4819-158-8-201304160-00004
doi: 10.7326/0003-4819-158-8-201304160-00004
pubmed: 23588748
Lei N, Zhang X, Wei M, Lao B, Xu X, Zhang M, Chen H, Xu Y, Xia B, Zhang D, Dong C, Fu L, Tang F, Wu Y (2022) Machine learning algorithms’ accuracy in predicting kidney disease progression: a systematic review and meta-analysis. BMC Med Inform Decis Mak 22:205. https://doi.org/10.1186/s12911-022-01951-1
doi: 10.1186/s12911-022-01951-1
pubmed: 35915457
pmcid: 9341041
Tuena C, Semonella M, Fernández-Álvarez J, Colombo D, Cipresso P (2020) Predictive precision medicine: towards the computational challenge. Springer, Cham
Vogenberg FR (2009) Predictive and prognostic models: implications for healthcare decision-making in a modern recession. Am Health Drug Benefits 2:218–222
pubmed: 25126292
pmcid: 4106488