3D-MCN: A 3D Multi-scale Capsule Network for Lung Nodule Malignancy Prediction.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
14 05 2020
14 05 2020
Historique:
received:
08
01
2020
accepted:
17
04
2020
entrez:
16
5
2020
pubmed:
16
5
2020
medline:
1
12
2020
Statut:
epublish
Résumé
Despite the advances in automatic lung cancer malignancy prediction, achieving high accuracy remains challenging. Existing solutions are mostly based on Convolutional Neural Networks (CNNs), which require a large amount of training data. Most of the developed CNN models are based only on the main nodule region, without considering the surrounding tissues. Obtaining high sensitivity is challenging with lung nodule malignancy prediction. Moreover, the interpretability of the proposed techniques should be a consideration when the end goal is to utilize the model in a clinical setting. Capsule networks (CapsNets) are new and revolutionary machine learning architectures proposed to overcome shortcomings of CNNs. Capitalizing on the success of CapsNet in biomedical domains, we propose a novel model for lung tumor malignancy prediction. The proposed framework, referred to as the 3D Multi-scale Capsule Network (3D-MCN), is uniquely designed to benefit from: (i) 3D inputs, providing information about the nodule in 3D; (ii) Multi-scale input, capturing the nodule's local features, as well as the characteristics of the surrounding tissues, and; (iii) CapsNet-based design, being capable of dealing with a small number of training samples. The proposed 3D-MCN architecture predicted lung nodule malignancy with a high accuracy of 93.12%, sensitivity of 94.94%, area under the curve (AUC) of 0.9641, and specificity of 90% when tested on the LIDC-IDRI dataset. When classifying patients as having a malignant condition (i.e., at least one malignant nodule is detected) or not, the proposed model achieved an accuracy of 83%, and a sensitivity and specificity of 84% and 81% respectively.
Identifiants
pubmed: 32409715
doi: 10.1038/s41598-020-64824-5
pii: 10.1038/s41598-020-64824-5
pmc: PMC7224210
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
7948Références
Bray, F., et al. CA: A Cancer Journal for Clinicians, Global Cancer Statistics 2018: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. 66, 7–30, https://doi.org/10.3322/caac.21492 (2018).
Siegel, R. L., Miller, K. D. & Jemal, A. Cancer statistics, 2016. CA: A Cancer J. Clinicians 68, 394–424, https://doi.org/10.3322/caac.21332 (2016).
doi: 10.3322/caac.21332
Aberle, D. R. et al. Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening. N. Engl. J. Med. 365, 395–409, https://doi.org/10.1056/NEJMoa1102873 (2011).
doi: 10.1056/NEJMoa1102873
pubmed: 21714641
Zhang, Y., Oikonomou, A., Wong, A., Haider, M. A. & Khalvati, F. Radiomics-based prognosis analysis for non-small cell lung cancer. Sci. Rep. 7, 481–487, https://doi.org/10.1038/srep46349 (2017).
doi: 10.1038/srep46349
Causey, J. L. et al. Highly accurate model for prediction of lung nodule malignancy with ct scans. Scientific Reports 8, https://doi.org/10.1038/s41598-018-27569-w (2018).
Aerts, H. J. W. L. et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nature Communications 5, https://doi.org/10.1038/ncomms5006 (2014).
Oikonomou, A. et al. Radiomics analysis at pet/ct contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy. Scientific Reports 8, https://doi.org/10.1038/s41598-018-22357-y (2018).
Afshar, A., Mohammadi, A., Konstantinos, N. P., Oikonomou, A. & Benali, H. From hand-crafted to deep learning-based cancer radiomics: Challenges and opportunities. IEEE Signal. Process. Mag. 36, 132–160, https://doi.org/10.1109/MSP.2019.2900993 (2019).
doi: 10.1109/MSP.2019.2900993
Gillies, R. J., Kinahan, P. E. & Hricak, H. Radiomics: Images are more than pictures, they are data. Radiology 278, 563–577, https://doi.org/10.1148/radiol.2015151169 (2015).
doi: 10.1148/radiol.2015151169
pubmed: 26579733
pmcid: 4734157
Lambin, P. et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur. J. Cancer 48, 441–446 (2012).
doi: 10.1016/j.ejca.2011.11.036
Chen, C. et al. Radiomic features analysis in computed tomography images of lung nodule classification. PLoS One 13, https://doi.org/10.1371/journal.pone.0192002 (2018).
Parmar, C. et al. Radiomic feature clusters and prognostic signatures specific for lung and head and neck cancer. Scientific Reports 5, https://doi.org/10.1038/srep11044 (2015).
Coroller, T. P. et al. Multiview convolutional neural networks for lung nodule classification. Radiotherapy Oncol. 119, 480–486, https://doi.org/10.1016/j.radonc.2016.04.004 (2016).
doi: 10.1016/j.radonc.2016.04.004
Huynh, E. et al. Ct-based radiomic analysis of stereotactic body radiation therapy patients with lung cancer. Radiotherapy Oncol. 120, 258–266, https://doi.org/10.1016/j.radonc.2016.05.024 (2016).
doi: 10.1016/j.radonc.2016.05.024
Yip, S. S. F. & Aerts, H. J. W. L. Applications and limitations of radiomics. Physics in Medicine and Biology 61, https://doi.org/10.1088/0031-9155/61/13/R150 (2016).
Park, J. E. et al. Reproducibility and Generalizability in Radiomics Modeling: Possible Strategies in Radiologic and Statistical Perspectives. Korean J. Radiology 20, 1124–1137, https://doi.org/10.3348/kjr.2018.0070 (2019).
doi: 10.3348/kjr.2018.0070
Lao, J. et al. A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Scientific Reports 7, https://doi.org/10.1038/s41598-017-10649-8 (2017).
Li, Z., Wang, Y., Yu, J., Guo, Y. & Cao, W. Deep learning based radiomics (dlr) and its usage in noninvasive idh1 prediction for low grade glioma. Scientific Reports 7, https://doi.org/10.1038/s41598-017-05848-2 (2017).
Oakden-Rayner, L. et al. Precision radiology: Predicting longevity using feature engineering and deep learning methods in a radiomics framework. Scientific Reports 7, https://doi.org/10.1038/s41598-017-01931-w (2017).
Cha, K. H. et al. Bladder cancer treatment response assessment in ct using radiomics with deep-learning. Scientific Reports 7, https://doi.org/10.1038/s41598-017-09315-w (2017).
Kuma, D. et al. Discovery radiomics for pathologically-proven computed tomography lung cancer prediction. Karray F., Campilho A., Cheriet F. (eds) Image Analysis and Recognition. ICIAR 2017. Lecture Notes in Computer Science, Springer, Cham 10317 (2017).
Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Neural Information Processing Systems (NIPS) (2012).
Yamashita, R., Nishio, M., Do, R. K. G. & Togashi, K. Convolutional neural networks: An overview and application in radiology. Insights into Imaging 9, 611–629, https://doi.org/10.1007/s13244-018-0639-9 (2018).
doi: 10.1007/s13244-018-0639-9
pubmed: 29934920
pmcid: 6108980
Sabour, S., Frosst, N. & Hinton, G. E. Dynamic routing between capsules. Neural Information Processing Systems (NIPS) (2017).
Afshar, P., Mohammadi, A., & Plataniotis, K. N. Brain tumor type classification via capsule networks. 25th IEEE International Conference on Image Processing (ICIP) 3129–3133 (2018).
Afshar, P., Plataniotis, K. N. & Mohammadi, A. Capsule networks for brain tumor classification based on MRI images and coarse tumor boundaries. 44th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 1368–1372 (2019).
Armato, S. G. III et al. Data from lidc-idri. The Cancer Imaging Archive. https://doi.org/10.7937/K9/TCIA.2015.LO9QL9SX (2015).
Armato, S. G. III et al. The lung image database consortium (lidc) and image database resource initiative (idri): A completed reference database of lung nodules on ct scans. Med. Phys. 38, 915–931 (2011).
doi: 10.1118/1.3528204
Clark, K. et al. The cancer imaging archive (tcia): Maintaining and operating a public information repository. J. Digital Imaging 26, 1045–1057 (2013).
doi: 10.1007/s10278-013-9622-7
Nibali, A., Zhen, H. & Wollersheim, D. Pulmonary nodule classification with deep residual networks. Int. J. Computer Assist. Radiology Surg. 12, 1799–1808 (2017).
doi: 10.1007/s11548-017-1605-6
Sun, W., Zheng, B. & Qian, W. Computer aided lung cancer diagnosis with deep learning algorithms. Proceedings of SPIE 9785, https://doi.org/10.1117/12.2216307 (2016).
Xie, Y., Zhang, J., Liu, S., Cai, W. & Xia, Y. Lung nodule classification by jointly using visual descriptors and deep features. Medical Computer Vision and Bayesian and Graphical Models for Biomedical Imaging. BAMBI 2016, MCV 2016. Lecture Notes in Computer Science, Springer, Cham 10081 (2017).
Shen, W. et al. Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognit. 61, 663–673, https://doi.org/10.1016/j.patcog.2016.05.029 (2017).
doi: 10.1016/j.patcog.2016.05.029
Lalkhen, A. G. & McCluskey, A. Clinical tests: sensitivity and specificity. Continuing Educ. Anaesth. Crit. Care Pain. 8, 221–223, https://doi.org/10.1093/bjaceaccp/mkn041 (2008).
doi: 10.1093/bjaceaccp/mkn041
Brosch, T. et al. Deep convolutional encoder networks for multiple sclerosis lesion segmentation. Medical Image Computing and Computer-Assisted Intervention, MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, Springer, Cham 9351 (2015).
Sudre, C. H., Li, W., Vercauteren, T., Ourselin, S. & Jorge Cardoso, M. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. DLMIA 2017, ML-CDS 2017. Lecture Notes in Computer Science, Springer, Cham 10553 (2017).
Jacobs, C. et al. Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images. Med. Image Anal. 18, 374–384, https://doi.org/10.1016/j.media.2013.12.001 (2014).
doi: 10.1016/j.media.2013.12.001
pubmed: 24434166
Maaten, L. V. D. & Hinton, G. Visualizing Data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008).
Liu, K. & Kang, G. Multiview convolutional neural networks for lung nodule classification. Int. J. Imaging Syst. Technol. 27, 12–22, https://doi.org/10.1002/ima.22206 (2017).
doi: 10.1002/ima.22206
Tafti, A. P., Bashiri, F. S., LaRose, E., & Peissig, P. Diagnostic Classification of Lung CT Images Using Deep 3D Multi-Scale Convolutional Neural Network. 2018 IEEE International Conference on Healthcare Informatics (ICHI), https://doi.org/10.1109/ICHI.2018.00078 (2018).
Hao, Z. et al. Multiscale superpixel classification for tumor segmentation in breast ultrasound images. 2012 19th IEEE International Conference on Image Processing, https://doi.org/10.1109/ICIP.2012.6467485 (2012).
Chaddad, A., Sabri, S., Niazi, T. & Abdulkarim, B. Prediction of survival with multi-scale radiomic analysis in glioblastoma patients. Med. Biol. Eng. Comput. 56, 2287–2300, https://doi.org/10.1007/s11517-018-1858-4 (2018).
doi: 10.1007/s11517-018-1858-4
pubmed: 29915951
Chollet, F. keras. GitHub repository, https://github.com/fchollet/keras (2015).