Predicting haemoglobin deferral using machine learning models: Can we use the same prediction model across countries?
donor health
haemoglobin deferral
haemoglobin measurement
prediction
Journal
Vox sanguinis
ISSN: 1423-0410
Titre abrégé: Vox Sang
Pays: England
ID NLM: 0413606
Informations de publication
Date de publication:
18 Apr 2024
18 Apr 2024
Historique:
revised:
10
04
2024
received:
15
12
2023
accepted:
11
04
2024
medline:
19
4
2024
pubmed:
19
4
2024
entrez:
18
4
2024
Statut:
aheadofprint
Résumé
Personalized donation strategies based on haemoglobin (Hb) prediction models may reduce Hb deferrals and hence costs of donation, meanwhile improving commitment of donors. We previously found that prediction models perform better in validation data with a high Hb deferral rate. We therefore investigate how Hb deferral prediction models perform when exchanged with other blood establishments. Donation data from the past 5 years from random samples of 10,000 donors from Australia, Belgium, Finland, the Netherlands and South Africa were used to fit random forest models for Hb deferral prediction. Trained models were exchanged between blood establishments. Model performance was evaluated using the area under the precision-recall curve (AUPR). Variable importance was assessed using SHapley Additive exPlanations (SHAP) values. Across the validation datasets and exchanged models, the AUPR ranged from 0.05 to 0.43. Exchanged models performed similarly within validation datasets, irrespective of the origin of the training data. Apart from subtle differences, the importance of most predictor variables was similar in all trained models. Our results suggest that Hb deferral prediction models trained in different blood establishments perform similarly within different validation datasets, regardless of the deferral rate of their training data. Models learn similar associations in different blood establishments.
Sections du résumé
BACKGROUND AND OBJECTIVES
OBJECTIVE
Personalized donation strategies based on haemoglobin (Hb) prediction models may reduce Hb deferrals and hence costs of donation, meanwhile improving commitment of donors. We previously found that prediction models perform better in validation data with a high Hb deferral rate. We therefore investigate how Hb deferral prediction models perform when exchanged with other blood establishments.
MATERIALS AND METHODS
METHODS
Donation data from the past 5 years from random samples of 10,000 donors from Australia, Belgium, Finland, the Netherlands and South Africa were used to fit random forest models for Hb deferral prediction. Trained models were exchanged between blood establishments. Model performance was evaluated using the area under the precision-recall curve (AUPR). Variable importance was assessed using SHapley Additive exPlanations (SHAP) values.
RESULTS
RESULTS
Across the validation datasets and exchanged models, the AUPR ranged from 0.05 to 0.43. Exchanged models performed similarly within validation datasets, irrespective of the origin of the training data. Apart from subtle differences, the importance of most predictor variables was similar in all trained models.
CONCLUSION
CONCLUSIONS
Our results suggest that Hb deferral prediction models trained in different blood establishments perform similarly within different validation datasets, regardless of the deferral rate of their training data. Models learn similar associations in different blood establishments.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : Stichting Sanquin Bloedvoorziening
ID : 21-10/L2590
Organisme : Punainen Risti Veripalvelu (FRCBS)
Organisme : Australian Government
Informations de copyright
© 2024 The Authors. Vox Sanguinis published by John Wiley & Sons Ltd on behalf of International Society of Blood Transfusion.
Références
Spekman MLC, Tilburg TG, Merz E. Do deferred donors continue their donations? A large‐scale register study on whole blood donor return in the Netherlands. Transfusion. 2019;59:3657–3665.
Vinkenoog M, Toivonen J, Brits T, de Clippel D, Compernolle V, Karki S, et al. An international comparison of haemoglobin deferral prediction models for blood banking. Vox Sang. 2023;118:430–439.
Baart AM, Atsma F, McSweeney EN, Moons KGM, Vergouwe Y, De Kort WLAM. External validation and updating of a Dutch prediction model for low hemoglobin deferral in Irish whole blood donors: validity of prediction of Hb deferral. Transfusion. 2014;54:762–769.
Baart AM, Fontana S, Tschaggelar A, Heymans MW, De Kort WLAM. Generalizability of Dutch prediction models for low hemoglobin deferral: a study on external validation and updating in swiss whole blood donors. Transfus Med Hemother. 2016;43:407–414.
Saito T, Rehmsmeier M. The precision‐recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One. 2015;10:e0118432.
Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2:56–67.
Toivonen J, Koski Y, Turkulainen E, Prinsze F, della Briotta Parolo P, Heinonen M, et al. Prediction and impact of personalized donation intervals. Vox Sang. 2022;117:504–512.