An explainable machine learning model to solid adnexal masses diagnosis based on clinical data and qualitative ultrasound indicators.
gynecological ultrasound
machine learning
ovarian cancer
precision medicine
solid adnexal masses
Journal
Cancer medicine
ISSN: 2045-7634
Titre abrégé: Cancer Med
Pays: United States
ID NLM: 101595310
Informations de publication
Date de publication:
Jun 2024
Jun 2024
Historique:
revised:
10
06
2024
received:
09
10
2023
accepted:
11
06
2024
medline:
26
6
2024
pubmed:
26
6
2024
entrez:
26
6
2024
Statut:
ppublish
Résumé
Accurate characterization of newly diagnosed a solid adnexal lesion is a key step in defining the most appropriate therapeutic approach. Despite guidance from the International Ovarian Tumor Analyzes Panel, the evaluation of these lesions can be challenging. Recent studies have demonstrated how machine learning techniques can be applied to clinical data to solve this diagnostic problem. However, ML models can often consider as black-boxes due to the difficulty of understanding the decision-making process used by the algorithm to obtain a specific result. For this purpose, we propose an Explainable Artificial Intelligence model trained on clinical characteristics and qualitative ultrasound indicators to predict solid adnexal masses diagnosis. Since the diagnostic task was a three-class problem (benign tumor, invasive cancer, or ovarian metastasis), we proposed a waterfall classification model: a first model was trained and validated to discriminate benign versus malignant, a second model was trained to distinguish nonmetastatic versus metastatic malignant lesion which occurs when a patient is predicted to be malignant by the first model. Firstly, a stepwise feature selection procedure was implemented. The classification performances were validated on Leave One Out scheme. The accuracy of the three-class model reaches an overall accuracy of 86.36%, and the precision per-class of the benign, nonmetastatic malignant, and metastatic malignant classes were 86.96%, 87.27%, and 77.78%, respectively. SHapley Additive exPlanations were performed to visually show how the machine learning model made a specific decision. For each patient, the SHAP values expressed how each characteristic contributed to the classification result. Such information represents an added value for the clinical usability of a diagnostic system. This is the first work that attempts to design an explainable machine-learning tool for the histological diagnosis of solid masses of the ovary.
Sections du résumé
BACKGROUND
BACKGROUND
Accurate characterization of newly diagnosed a solid adnexal lesion is a key step in defining the most appropriate therapeutic approach. Despite guidance from the International Ovarian Tumor Analyzes Panel, the evaluation of these lesions can be challenging. Recent studies have demonstrated how machine learning techniques can be applied to clinical data to solve this diagnostic problem. However, ML models can often consider as black-boxes due to the difficulty of understanding the decision-making process used by the algorithm to obtain a specific result.
AIMS
OBJECTIVE
For this purpose, we propose an Explainable Artificial Intelligence model trained on clinical characteristics and qualitative ultrasound indicators to predict solid adnexal masses diagnosis.
MATERIALS & METHODS
METHODS
Since the diagnostic task was a three-class problem (benign tumor, invasive cancer, or ovarian metastasis), we proposed a waterfall classification model: a first model was trained and validated to discriminate benign versus malignant, a second model was trained to distinguish nonmetastatic versus metastatic malignant lesion which occurs when a patient is predicted to be malignant by the first model. Firstly, a stepwise feature selection procedure was implemented. The classification performances were validated on Leave One Out scheme.
RESULTS
RESULTS
The accuracy of the three-class model reaches an overall accuracy of 86.36%, and the precision per-class of the benign, nonmetastatic malignant, and metastatic malignant classes were 86.96%, 87.27%, and 77.78%, respectively.
DISCUSSION
CONCLUSIONS
SHapley Additive exPlanations were performed to visually show how the machine learning model made a specific decision. For each patient, the SHAP values expressed how each characteristic contributed to the classification result. Such information represents an added value for the clinical usability of a diagnostic system.
CONCLUSIONS
CONCLUSIONS
This is the first work that attempts to design an explainable machine-learning tool for the histological diagnosis of solid masses of the ovary.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
e7425Subventions
Organisme : Ministero della Salute
ID : 5 per 1000
Organisme : Ministero della Salute
ID : 5 per 1000
Informations de copyright
© 2024 The Author(s). Cancer Medicine published by John Wiley & Sons Ltd.
Références
Kaijser J, Vandecaveye V, Deroose CM, et al. Imaging techniques for the pre‐surgical diagnosis of adnexal tumours. Best Pract Res Clin Obstet Gynaecol. 2014;28(5):683‐695.
Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the international ovarian tumor analysis (IOTA) group. Ultrasound Obstet Gynecol. 2000;16(5):500‐505.
Querleu D, Planchamp F, Chiva L, et al. European Society of Gynaecological Oncology (ESGO) guidelines for ovarian cancer surgery. Int J Gynecol Cancer. 2017;27(7):1534‐1542.
Froyman W, Landolfo C, De Cock B, et al. Risk of complications in patients with conservatively managed ovarian tumours (IOTA5): a 2‐year interim analysis of a multicentre, prospective, cohort study. Lancet Oncol. 2019;20(3):448‐458.
Arezzo F, Loizzi V, La Forgia D, et al. The role of ultrasound guided sampling procedures in the diagnosis of pelvic masses: a narrative review of the literature. Diagnostics. 2021;11(12):2204.
Bhavsar KA, Singla J, Al‐Otaibi YD, Song OY, Zikria YB, Bashir AK. Medical diagnosis using machine learning: a statistical review. Computers, Materials and Continua. 2021;67(1):107‐125.
Gunning D, Aha D. DARPA's explainable artificial intelligence (XAI) program. AI Mag. 2019;40(2):44‐58.
Tjoa E, Guan C. A survey on explainable artificial intelligence (xai): toward medical xai. IEEE Transactions on Neural Networks and Learning Systems. 2020;32(11):4793‐4813.
Watson DS, Krutzinna J, Bruce IN, et al. Clinical applications of machine learning algorithms: beyond the black box. BMJ. 2019;364:l886.
Massafra R, Fanizzi A, Amoroso N, et al. Analyzing breast cancer invasive disease event classification through explainable artificial intelligence. Front Med. 2023;10:1116354.
Lauritsen SM, Kristensen M, Olsen MV, et al. Explainable artificial intelligence model to predict acute critical illness from electronic health records. Nat Commun. 2020;11(1):3852.
Miller T. Explanation in artificial intelligence: insights from the social sciences. Artif Intell. 2019;267:1‐38.
Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Adv Neural Inf Proces Syst. 2017;30.
Timmerman D, Ameye L, Fischerova D, et al. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group. BMJ. 2010;341:c6839.
Alcázar JL, Pascual MA, Olartecoechea B, et al. IOTA simple rules for discriminating between benign and malignant adnexal masses: prospective external validation. Ultrasound Obstet Gynecol. 2013;42(4):467‐471.
Van Calster B, Van Hoorde K, Froyman W, et al. Practical guidance for applying the ADNEX model from the IOTA group to discriminate between different subtypes of adnexal tumors. Facts Views Vis Obgyn. 2015;7(1):32.
Fratello M, Tagliaferri R. Decision trees and random forests. Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics. 2018;1(S3):374.
Breiman L. Random forests. Mach Learn. 2001;45:5‐32. doi:10.1023/A:1010933404324
Devi D, Biswas SK, Purkayastha B. A review on solution to class imbalance problem: Undersampling approaches. In 2020 International Conference on Computational Performance Evaluation (ComPE). IEEE; 2020:626‐631.
Kwak SK, Kim JH. Statistical data preparation: management of missing values and outliers. Korean J Anesthesiol. 2017;70(4):407‐411.
Shan G. Improved confidence intervals for the Youden index. PLoS One. 2015;10(7):e0127272.
Slack D, Hilgard S, Jia E, Singh S, Lakkaraju H. Fooling lime and shap: adversarial attacks on post hoc explanation methods. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. ACM; 2020:180‐186.
Strumbelj E, Kononenko I. An efficient explanation of individual classifications using game theory. The Journal of Machine Learning Research. 2010;11:1‐18.
Valentin L. Use of morphology to characterize and manage common adnexal masses. Best Pract Res Clin Obstet Gynaecol. 2004;18(1):71‐89.
Paladini D, Testa A, Van Holsbeke C, Mancari R, Timmerman D, Valentin L. Imaging in gynecological disease (5): clinical and ultrasound characteristics in fibroma and fibrothecoma of the ovary. Ultrasound Obstet Gynecol. 2009;34(2):188‐195.
Mohammed SA, Kumar A. Meigs Syndrome. 2023.
Moro F, Baima Poma C, Zannoni GF, et al. Imaging in gynecological disease (12): clinical and ultrasound features of invasive and non‐invasive malignant serous ovarian tumors. Ultrasound Obstet Gynecol. 2017;50(6):788‐799.
Pozzati F, Moro F, Pasciuto T, et al. Imaging in gynecological disease (14): clinical and ultrasound characteristics of ovarian clear cell carcinoma. Ultrasound Obstet Gynecol. 2018;52(6):792‐800.
Moro F, Magoga G, Pasciuto T, et al. Imaging in gynecological disease (13): clinical and ultrasound characteristics of endometrioid ovarian cancer. Ultrasound Obstet Gynecol. 2018;52(4):535‐543.
Ludovisi M, Foo X, Mainenti S, Testa AC, Arora R, Jurkovic D. Ultrasound diagnosis of serous surface papillary borderline ovarian tumor: a case series with a review of the literature. J Clin Ultrasound. 2015;43(9):573‐577.
Demidov VN, Lipatenkova J, Vikhareva O, Van Holsbeke C, Timmerman D, Valentin L. Imaging of gynecological disease (2): clinical and ultrasound characteristics of Sertoli cell tumors, Sertoli–Leydig cell tumors and Leydig cell tumors. Ultrasound Obstet Gynecol. 2008;31(1):85‐91.
Di Legge A, Pollastri P, Mancari R, et al. Clinical and ultrasound characteristics of surgically removed adnexal lesions with largest diameter≤ 2.5 cm: a pictorial essay. Ultrasound Obstet Gynecol. 2017;50(5):648‐656.
Bruno M, Capanna G, Di Florio C, Sollima L, Guido M, Ludovisi M. Sonographic characteristics of ovarian Leydig cell tumor. Ultrasound Obstet Gynecol. 2023;62:441‐442.
Testa AC, Ferrandina G, Timmerman D, et al. Imaging in gynecological disease (1): ultrasound features of metastases in the ovaries differ depending on the origin of the primary tumor. Ultrasound Obstet Gynecol. 2007;29(5):505‐511.
Arezzo F, Loizzi V, La Forgia D, et al. Radiomics analysis in ovarian cancer: a narrative review. Appl Sci. 2021;11(17):7833.
Jiang Y, Wang C, Zhou S. Artificial intelligence‐based risk stratification, accurate diagnosis and treatment prediction in gynecologic oncology. Seminars in Cancer Biology. Vol 96. Academic Press; 2023:82‐99.
Chiappa V, Bogani G, Interlenghi M, et al. The adoption of Radiomics and machine learning improves the diagnostic processes of women with ovarian MAsses (the AROMA pilot study). J Ultrasound. 2021;24:429‐437.
Chiappa V, Interlenghi M, Bogani G, et al. A decision support system based on radiomics and machine learning to predict the risk of malignancy of ovarian masses from transvaginal ultrasonography and serum CA‐125. European Radiology Experimental. 2021;5(1):1‐15.
Guo Y, Jiang T, Ouyang L, et al. A novel diagnostic nomogram based on serological and ultrasound findings for preoperative prediction of malignancy in patients with ovarian masses. Gynecol Oncol. 2021;160(3):704‐712.
Christiansen F, Epstein EL, Smedberg E, Åkerlund M, Smith K, Epstein E. Ultrasound image analysis using deep neural networks for discriminating between benign and malignant ovarian tumors: comparison with expert subjective assessment. Ultrasound Obstet Gynecol. 2021;57(1):155‐163.
Gao Y, Zeng S, Xu X, et al. Deep learning‐enabled pelvic ultrasound images for accurate diagnosis of ovarian cancer in China: a retrospective, multicentre, diagnostic study. The Lancet Digital Health. 2022;4(3):e179‐e187.
Chen H, Yang BW, Qian L, et al. Deep learning prediction of ovarian malignancy at US compared with O‐RADS and expert assessment. Radiology. 2022;304(1):106‐113.