Temporal shift and predictive performance of machine learning for heart transplant outcomes.
artificial intelligence
donor selection
heart transplantation
machine learning
Journal
The Journal of heart and lung transplantation : the official publication of the International Society for Heart Transplantation
ISSN: 1557-3117
Titre abrégé: J Heart Lung Transplant
Pays: United States
ID NLM: 9102703
Informations de publication
Date de publication:
07 2022
07 2022
Historique:
received:
18
07
2021
revised:
25
02
2022
accepted:
23
03
2022
pubmed:
15
5
2022
medline:
30
6
2022
entrez:
14
5
2022
Statut:
ppublish
Résumé
Outcome prediction following heart transplant is critical to explaining risks and benefits to patients and decision-making when considering potential organ offers. Given the large number of potential variables to be considered, this task may be most efficiently performed using machine learning (ML). We trained and tested ML and statistical algorithms to predict outcomes following cardiac transplant using the United Network of Organ Sharing (UNOS) database. We included 59,590 adult and 8,349 pediatric patients enrolled in the UNOS database between January 1994 and December 2016 who underwent cardiac transplantation. We evaluated 3 classification and 3 survival methods. Algorithms were evaluated using shuffled 10-fold cross-validation (CV) and rolling CV. Predictive performance for 1 year and 90 days all-cause mortality was characterized using the area under the receiver-operating characteristic curve (AUC) with 95% confidence interval. In total, 8,394 (12.4%) patients died within 1 year of transplant. For predicting 1-year survival, using the shuffled 10-fold CV, Random Forest achieved the highest AUC (0.893; 0.889-0.897) followed by XGBoost and logistic regression. In the rolling CV, prediction performance was more modest and comparable among the models with XGBoost and Logistic regression achieving the highest AUC 0.657 (0.647-0.667) and 0.641(0.631-0.651), respectively. There was a trend toward higher prediction performance in pediatric patients. Our study suggests that ML and statistical models can be used to predict mortality post-transplant, but based on the results from rolling CV, the overall prediction performance will be limited by temporal shifts inpatient and donor selection.
Sections du résumé
BACKGROUND
Outcome prediction following heart transplant is critical to explaining risks and benefits to patients and decision-making when considering potential organ offers. Given the large number of potential variables to be considered, this task may be most efficiently performed using machine learning (ML). We trained and tested ML and statistical algorithms to predict outcomes following cardiac transplant using the United Network of Organ Sharing (UNOS) database.
METHODS
We included 59,590 adult and 8,349 pediatric patients enrolled in the UNOS database between January 1994 and December 2016 who underwent cardiac transplantation. We evaluated 3 classification and 3 survival methods. Algorithms were evaluated using shuffled 10-fold cross-validation (CV) and rolling CV. Predictive performance for 1 year and 90 days all-cause mortality was characterized using the area under the receiver-operating characteristic curve (AUC) with 95% confidence interval.
RESULTS
In total, 8,394 (12.4%) patients died within 1 year of transplant. For predicting 1-year survival, using the shuffled 10-fold CV, Random Forest achieved the highest AUC (0.893; 0.889-0.897) followed by XGBoost and logistic regression. In the rolling CV, prediction performance was more modest and comparable among the models with XGBoost and Logistic regression achieving the highest AUC 0.657 (0.647-0.667) and 0.641(0.631-0.651), respectively. There was a trend toward higher prediction performance in pediatric patients.
CONCLUSIONS
Our study suggests that ML and statistical models can be used to predict mortality post-transplant, but based on the results from rolling CV, the overall prediction performance will be limited by temporal shifts inpatient and donor selection.
Identifiants
pubmed: 35568604
pii: S1053-2498(22)01882-4
doi: 10.1016/j.healun.2022.03.019
pii:
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Research Support, U.S. Gov't, P.H.S.
Langues
eng
Sous-ensembles de citation
IM
Pagination
928-936Informations de copyright
Copyright © 2022 International Society for Heart and Lung Transplantation. Published by Elsevier Inc. All rights reserved.
Déclaration de conflit d'intérêts
Disclosure statement The authors have no relevant conflicts of interest to disclose.