Toward Precision Diagnosis: Machine Learning in Identifying Malignant Orbital Tumors With Multiparametric 3 T MRI.
Journal
Investigative radiology
ISSN: 1536-0210
Titre abrégé: Invest Radiol
Pays: United States
ID NLM: 0045377
Informations de publication
Date de publication:
11 Apr 2024
11 Apr 2024
Historique:
medline:
10
4
2024
pubmed:
10
4
2024
entrez:
10
4
2024
Statut:
aheadofprint
Résumé
Orbital tumors present a diagnostic challenge due to their varied locations and histopathological differences. Although recent advancements in imaging have improved diagnosis, classification remains a challenge. The integration of artificial intelligence in radiology and ophthalmology has demonstrated promising outcomes. This study aimed to evaluate the performance of machine learning models in accurately distinguishing malignant orbital tumors from benign ones using multiparametric 3 T magnetic resonance imaging (MRI) data. In this single-center prospective study, patients with orbital masses underwent presurgery 3 T MRI scans between December 2015 and May 2021. The MRI protocol comprised multiparametric imaging including dynamic contrast-enhanced (DCE), diffusion-weighted imaging (DWI), intravoxel incoherent motion (IVIM), as well as morphological imaging acquisitions. A repeated nested cross-validation strategy using random forest classifiers was used for model training and evaluation, considering 8 combinations of explanatory features. Shapley additive explanations (SHAP) values were used to assess feature contributions, and the model performance was evaluated using multiple metrics. One hundred thirteen patients were analyzed (57/113 [50.4%] were women; average age was 51.5 ± 17.5 years, range: 19-88 years). Among the 8 combinations of explanatory features assessed, the performance on predicting malignancy when using the most comprehensive model, which is the most exhaustive one incorporating all 46 explanatory features-including morphology, DWI, DCE, and IVIM, achieved an area under the curve of 0.9 [0.73-0.99]. When using the streamlined "10-feature signature" model, performance reached an area under the curve of 0.88 [0.71-0.99]. Random forest feature importance graphs measured by the mean of SHAP values pinpointed the 10 most impactful features, which comprised 3 quantitative IVIM features, 4 quantitative DCE features, 1 quantitative DWI feature, 1 qualitative DWI feature, and age. Our findings demonstrate that a machine learning approach, integrating multiparametric MRI data such as DCE, DWI, IVIM, and morphological imaging, offers high-performing models for differentiating malignant from benign orbital tumors. The streamlined 10-feature signature, with a performance close to the comprehensive model, may be more suitable for clinical application.
Sections du résumé
BACKGROUND
BACKGROUND
Orbital tumors present a diagnostic challenge due to their varied locations and histopathological differences. Although recent advancements in imaging have improved diagnosis, classification remains a challenge. The integration of artificial intelligence in radiology and ophthalmology has demonstrated promising outcomes.
PURPOSE
OBJECTIVE
This study aimed to evaluate the performance of machine learning models in accurately distinguishing malignant orbital tumors from benign ones using multiparametric 3 T magnetic resonance imaging (MRI) data.
MATERIALS AND METHODS
METHODS
In this single-center prospective study, patients with orbital masses underwent presurgery 3 T MRI scans between December 2015 and May 2021. The MRI protocol comprised multiparametric imaging including dynamic contrast-enhanced (DCE), diffusion-weighted imaging (DWI), intravoxel incoherent motion (IVIM), as well as morphological imaging acquisitions. A repeated nested cross-validation strategy using random forest classifiers was used for model training and evaluation, considering 8 combinations of explanatory features. Shapley additive explanations (SHAP) values were used to assess feature contributions, and the model performance was evaluated using multiple metrics.
RESULTS
RESULTS
One hundred thirteen patients were analyzed (57/113 [50.4%] were women; average age was 51.5 ± 17.5 years, range: 19-88 years). Among the 8 combinations of explanatory features assessed, the performance on predicting malignancy when using the most comprehensive model, which is the most exhaustive one incorporating all 46 explanatory features-including morphology, DWI, DCE, and IVIM, achieved an area under the curve of 0.9 [0.73-0.99]. When using the streamlined "10-feature signature" model, performance reached an area under the curve of 0.88 [0.71-0.99]. Random forest feature importance graphs measured by the mean of SHAP values pinpointed the 10 most impactful features, which comprised 3 quantitative IVIM features, 4 quantitative DCE features, 1 quantitative DWI feature, 1 qualitative DWI feature, and age.
CONCLUSIONS
CONCLUSIONS
Our findings demonstrate that a machine learning approach, integrating multiparametric MRI data such as DCE, DWI, IVIM, and morphological imaging, offers high-performing models for differentiating malignant from benign orbital tumors. The streamlined 10-feature signature, with a performance close to the comprehensive model, may be more suitable for clinical application.
Identifiants
pubmed: 38597586
doi: 10.1097/RLI.0000000000001076
pii: 00004424-990000000-00210
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
Copyright © 2024 Wolters Kluwer Health, Inc. All rights reserved.
Déclaration de conflit d'intérêts
Conflicts of interest and sources of funding: none declared.
Références
Shields JA, Shields CL, Scartozzi R. Survey of 1264 patients with orbital tumors and simulating lesions. Ophthalmology. 2004;111:997–1008.
Demirci H, Shields CL, Shields JA, et al. Orbital tumors in the older adult population. Ophthalmology. 2002;109:243–248.
Eissa L, Abdel Razek AAK, Helmy E. Arterial spin labeling and diffusion-weighted MR imaging: utility in differentiating idiopathic orbital inflammatory pseudotumor from orbital lymphoma. Clin Imaging. 2021;71:63–68.
ElKhamary SM, Galindo-Ferreiro A, AlGhafri L, et al. Characterization of diffuse orbital mass using apparent diffusion coefficient in 3-Tesla MRI. Eur J Radiol Open. 2018;5:52–57.
Lecler A, Duron L, Zmuda M, et al. Intravoxel incoherent motion (IVIM) 3 T MRI for orbital lesion characterization. Eur Radiol. 2021;31:14–23.
Shor N, Sené T, Zuber K, et al. Discriminating between IgG4-related orbital disease and other causes of orbital inflammation with intra voxel incoherent motion (IVIM) MR imaging at 3 T. Diagn Interv Imaging. 2021;102:727–734.
Yuan Y, Kuai XP, Chen XS, et al. Assessment of dynamic contrast-enhanced magnetic resonance imaging in the differentiation of malignant from benign orbital masses. Eur J Radiol. 2013;82:1506–1511.
Lecler A, Duron L, Charlson E, et al. Comparison between 7 Tesla and 3 Tesla MRI for characterizing orbital lesions. Diagn Interv Imaging. 2022;103:433–439.
Hagiwara A, Fujita S, Kurokawa R, et al. Multiparametric MRI: from simultaneous rapid acquisition methods and analysis techniques using scoring, machine learning, radiomics, and deep learning to the generation of novel metrics. Invest Radiol. 2023;58:548–560.
Yang L, Zhang H, Xie X, et al. MRI-based radiomics nomogram for preoperative differentiation between ocular adnexal lymphoma and idiopathic orbital inflammation. J Magn Reson Imaging. 2023;57:1594–1604.
Miller T. Explanation in artificial intelligence: insights from the social sciences. Artif Intell. 2019;267:1–38.
Scheda R, Diciotti S. Explanations of machine learning models in repeated nested cross-validation: an application in age prediction using brain complexity features. Appl Sci. 2022;12:6681.
Armstrong GW, Lorch AC. A(eye): a review of current applications of artificial intelligence and machine learning in ophthalmology. Int Ophthalmol Clin. 2020;60:57–71.
Wu JH, Nishida T, Weinreb RN, et al. Performances of machine learning in detecting glaucoma using fundus and retinal optical coherence tomography images: a meta-analysis. Am J Ophthalmol. 2022;237:1–12.
Li F, Su Y, Lin F, et al. A deep-learning system predicts glaucoma incidence and progression using retinal photographs. J Clin Invest. 2022;132:e157968.
Moraes G, Fu DJ, Wilson M, et al. Quantitative analysis of OCT for neovascular age-related macular degeneration using deep learning. Ophthalmology. 2021;128:693–705.
Nakagawa J, Fujima N, Hirata K, et al. Utility of the deep learning technique for the diagnosis of orbital invasion on CT in patients with a nasal or sinonasal tumor. Cancer Imaging. 2022;22:52.
Xie X, Yang L, Zhao F, et al. A deep learning model combining multimodal radiomics, clinical and imaging features for differentiating ocular adnexal lymphoma from idiopathic orbital inflammation. Eur Radiol. 2022;32:6922–6932.
Tooley AA, Tailor P, Tran AQ, et al. Differentiating intradiploic orbital dermoid and epidermoid cysts utilizing clinical features and machine learning. Indian J Ophthalmol. 2022;70:2102–2106.
Decoux A, Duron L, Habert P, et al. Comparative performances of machine learning algorithms in radiomics and impacting factors. Sci Rep. 2023;13:14069.
2017 Kaggle Machine Learning & Data Science Survey. Available at: https://www.kaggle.com/datasets/kaggle/kaggle-survey-2017. Accessed September 3, 2023.
Chen T, Guestrin C. XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2016:785–794. doi:10.1145/2939672.2939785
Wegmeth L, Vente T, Purucker L, Beel J. The effect of random seeds for data splitting on recommendation accuracy. Intelligent Systems Group: University of Siegen, Germany. Available at: https://ceur-ws.org/Vol-3476/paper4.pdf. Accessed January 2024.
Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3:32–35.
Shapley LS. 17. A value for n-person games. In: Kuhn HW, Tucker AW, eds. Contributions to the Theory of Games (AM-28), Volume II. Princeton, NJ: Princeton University Press; 1953:307–318. doi:10.1515/9781400881970-018
Lundberg SM, Erion G, Chen H, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2:56–67.
Lundberg S, Lee SI. A unified approach to interpreting model predictions. Published online 2017. Available at: https://arxiv.org/abs/1705.07874. Accessed September 2023.
Ren J, Yuan Y, Qi M, et al. MRI-based radiomics nomogram for distinguishing solitary fibrous tumor from schwannoma in the orbit: a two-center study. Eur Radiol. 2024;34:560–568.
Han Q, Du L, Mo Y, et al. Machine learning based non-enhanced CT radiomics for the identification of orbital cavernous venous malformations: an innovative tool. J Craniofac Surg. 2022;33:814–820.
Hou Y, Xie X, Chen J, et al. Bag-of-features-based radiomics for differentiation of ocular adnexal lymphoma and idiopathic orbital inflammation from contrast-enhanced MRI. Eur Radiol. 2021;31:24–33.
Duron L, Heraud A, Charbonneau F, et al. A magnetic resonance imaging radiomics signature to distinguish benign from malignant orbital lesions. Invest Radiol. 2021;56:173–180.
Ro SR, Asbach P, Siebert E, et al. Characterization of orbital masses by multiparametric MRI. Eur J Radiol. 2016;85:324–336.
Xu X-Q, Hu H, Liu H, et al. Benign and malignant orbital lymphoproliferative disorders: differentiating using multiparametric MRI at 3.0 T. J Magn Reson Imaging. 2017;45:167–176.
University of Washington–Diabetes Research Center Free Diabetic Retinopathy Risk Calculator. Available at: https://depts.washington.edu/diabetes/free-diabetic-retinopathy-risk-calculator/. Accessed September 10, 2023.
Advanced AMD Risk Calculator | National Eye Institute. Available at: https://www.nei.nih.gov/research/clinical-trials/advanced-amd-risk-calculator. Accessed September 10, 2023.
Boeken T, Feydy J, Lecler A, et al. Artificial intelligence in diagnostic and interventional radiology: where are we now? Diagn Interv Imaging. 2023;104:1–5.
Bellman R. Dynamic Programming. Princeton, NJ: Princeton University Press; 1984.
Kuo MD, Jamshidi N. Behind the numbers: decoding molecular phenotypes with radiogenomics—guiding principles and technical considerations. Radiology. 2014;270:320–325.
Shao J, Zhu J, Jin K, et al. End-to-end deep-learning-based diagnosis of benign and malignant orbital tumors on computed tomography images. J Pers Med. 2023;13:204.