Thyroid nodule recognition using a joint convolutional neural network with information fusion of ultrasound images and radiofrequency data.
Deep learning
Diagnosis computer-assisted
Neural networks computer
Thyroid gland
Ultrasonography
Journal
European radiology
ISSN: 1432-1084
Titre abrégé: Eur Radiol
Pays: Germany
ID NLM: 9114774
Informations de publication
Date de publication:
Jul 2021
Jul 2021
Historique:
received:
27
08
2020
accepted:
01
12
2020
revised:
06
11
2020
pubmed:
8
1
2021
medline:
24
6
2021
entrez:
7
1
2021
Statut:
ppublish
Résumé
To develop a deep learning-based method with information fusion of US images and RF signals for better classification of thyroid nodules (TNs). One hundred sixty-three pairs of US images and RF signals of TNs from a cohort of adult patients were used for analysis. We developed an information fusion-based joint convolutional neural network (IF-JCNN) for the differential diagnosis of malignant and benign TNs. The IF-JCNN contains two branched CNNs for deep feature extraction: one for US images and the other one for RF signals. The extracted features are fused at the backend of IF-JCNN for TN classification. Across 5-fold cross-validation, the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) obtained by using the IF-JCNN with both US images and RF signals as inputs for TN classification were respectively 0.896 (95% CI 0.838-0.938), 0.885 (95% CI 0.804-0.941), 0.910 (95% CI 0.815-0.966), and 0.956 (95% CI 0.926-0.987), which were better than those obtained by using only US images: 0.822 (0.755-0.878; p = 0.0044), 0.792 (0.679-0.868, p = 0.0091), 0.866 (0.760-0.937, p = 0.197), and 0.901 (0.855-0.948, p = .0398), or RF signals: 0.767 (0.694-0.829, p < 0.001), 0.781 (0.685-0.859, p = 0.0037), 0.746 (0.625-0.845, p < 0.001), 0.845 (0.786-0.903, p < 0.001). The proposed IF-JCNN model filled the gap of just using US images in CNNs to characterize TNs, and it may serve as a promising tool for assisting the diagnosis of thyroid cancer. • Raw radiofrequency signals before ultrasound imaging of thyroid nodules provide useful information that is not carried by ultrasound images. • The information carried by raw radiofrequency signals and ultrasound images for thyroid nodules is complementary. • The performance of deep convolutional neural network for diagnosing thyroid nodules can be significantly improved by fusing US images and RF signals in the model as compared with just using US images.
Identifiants
pubmed: 33409774
doi: 10.1007/s00330-020-07585-z
pii: 10.1007/s00330-020-07585-z
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
5001-5011Subventions
Organisme : National Natural Science Foundation of China
ID : 81871429
Organisme : National Natural Science Foundation of China
ID : 61901282
Références
Durante C, Grani G, Lamartina L, Filetti S, Cooper DS (2018) The diagnosis and management of thyroid nodules: a review. JAMA. 319(9):914
doi: 10.1001/jama.2018.0898
Pacini F, Castagna M, Brilli L, Pentheroudakis G, ESMO Guidelines Working Group (2012) Thyroid cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol 23(suppl_7):vii110–vivii9
Larsen PR (2006) New guidelines for patients with thyroid nodules and differentiated thyroid cancer. Nat Clin Pract Endocrinol Metab 2(6):297
Davies L, Welch HG (2006) Increasing incidence of thyroid cancer in the United States, 1973-2002. JAMA. 295(18):2164–2167
doi: 10.1001/jama.295.18.2164
Tessler FN, Middleton WD, Grant EG et al (2017) ACR thyroid imaging, reporting and data system (TI-RADS): white paper of the ACR TI-RADS committee. J Am Coll Radiol 14(5):587–595
Russ G, Bonnema SJ, Erdogan MF, Durante C, Ngu R, Leenhardt L (2017) European Thyroid Association guidelines for ultrasound malignancy risk stratification of thyroid nodules in adults: the EU-TIRADS. Eur Thyroid J 6(5):225–237
doi: 10.1159/000478927
Cooper DS, Doherty GM, Haugen BR et al (2006) Management guidelines for patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Taskforce. Thyroid. 16(2):109–142
Singh N, Jindal A (2012) Ultra sonogram images for thyroid segmentation and texture classification in diagnosis of malignant (cancerous) or benign (non-cancerous) nodules. Int J Eng Innov Technol 1(5):202–206
Bibicu D, Moraru L, Biswas A (2013) Thyroid nodule recognition based on feature selection and pixel classification methods. J Digit Imaging 26(1):119–128
doi: 10.1007/s10278-012-9475-5
Legakis I, Savelonas MA, Maroulis D, Iakovidis DK (2011) Computer-based nodule malignancy risk assessment in thyroid ultrasound images. Int J Comput Appl 33(1):29–35
Tsantis S, Dimitropoulos N, Cavouras D, Nikiforidis G (2009) Morphological and wavelet features towards sonographic thyroid nodules evaluation. Comput Med Imaging Graph 33(2):91–99
doi: 10.1016/j.compmedimag.2008.10.010
Iakovidis DK, Keramidas EG, Maroulis D (2010) Fusion of fuzzy statistical distributions for classification of thyroid ultrasound patterns. Artif Intell Med 50(1):33–41
doi: 10.1016/j.artmed.2010.04.004
Acharya UR, Chowriappa P, Fujita H et al (2016) Thyroid lesion classification in 242 patient population using Gabor transform features from high resolution ultrasound images. Knowl Based Syst 107:235–245
Hong Y, Liu X, Li Z, Zhang X, Chen M, Luo Z (2009) Real-time ultrasound elastography in the differential diagnosis of benign and malignant thyroid nodules. J Ultrasound Med 28(7):861–867
doi: 10.7863/jum.2009.28.7.861
Ma J, Wu F, Jiang T, Zhu J, Kong D (2017) Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Med Phys 44(5):1678–1691
doi: 10.1002/mp.12134
Song W, Li S, Liu J et al (2018) Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition. IEEE J Biomed Health Inform 23(3):1215–1224
Wang L, Yang S, Yang S et al (2019) Automatic thyroid nodule recognition and diagnosis in ultrasound imaging with the YOLOv2 neural network. World J Surg Oncol 17(1):12
Li X, Zhang S, Zhang Q et al (2019) Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. Lancet Oncol 20(2):193–201
Durante C, Grani G, Lamartina L, Filetti S, Mandel SJ, Cooper DS (2018) The diagnosis and management of thyroid nodules: a review. JAMA. 319(9):914–924
doi: 10.1001/jama.2018.0898
Ospina NS, Iñiguez-Ariza NM, Castro MR (2020) Thyroid nodules: diagnostic evaluation based on thyroid cancer risk assessment. BMJ 368:l6670
doi: 10.1136/bmj.l6670
Cho H, Sung M, Jun B (2016) Canny text detector: Fast and robust scene text localization algorithm. In: Proc IEEE (ed) Comput Soc Conf Comput Vis Pattern Recognit; 2016 Jun 26-Jul 1. IEEE Computer Society Press, Las Vegas, NV: Silver Spring Md, pp 3566–3573
Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ (2018) EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. J Neural Eng 15(5):056013
doi: 10.1088/1741-2552/aace8c
Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proc IEEE (ed) Comput Soc Conf Comput Vis Pattern Recognit; 2017 Jul 21-26. IEEE Computer Society Press, Honolulu, HI: Silver Spring, Md, pp 1251–1258
Clopper CJ, Pearson ES (1934) The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika. 26(4):404–413
doi: 10.1093/biomet/26.4.404
DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845
doi: 10.2307/2531595
Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proc 13th European Conf Comput Vis; 2014 Sep. 6-12; Zurich, Switzerland: Springer. p 818–33
Xia J, Chen H, Li Q et al (2017) Ultrasound-based differentiation of malignant and benign thyroid nodules: an extreme learning machine approach. Comput Methods Programs Biomed 147:37–49
Ma J, Wu F, Zhu J, Xu D, Kong D (2017) A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics. 73:221–230
doi: 10.1016/j.ultras.2016.09.011
Pereira C, Dighe M, Alessio AM (2018) Comparison of machine learned approaches for thyroid nodule characterization from shear wave elastography images. In: Medical Imaging 2018: Computer-Aided Diagnosis. International Society for Optics and Photonics, Houston, p 105751X
Bao-Liang G, Fu-Sheng O, Li-Zhu, et al (2018) Development and validation of an ultrasound-based nomogram to improve the diagnostic accuracy for malignant thyroid nodules. Eur Radiol 29(3):1518–1526
Ma J, Wu F, Jiang T, Zhao Q, Kong D (2017) Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. Int J Comput Assist Radiol Surg 12(11):1895–1910
Jeong EY, Kim HL, Ha EJ, Park SY, Cho YJ, Han M (2019) Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol 29(4):1978–1985
doi: 10.1007/s00330-018-5772-9
Gillies RJ, Kinahan PE, Hricak H (2015) Radiomics: images are more than pictures, they are data. Radiology. 278(2):563–577
doi: 10.1148/radiol.2015151169
Oelze ML, Mamou J (2016) Review of quantitative ultrasound: envelope statistics and backscatter coefficient imaging and contributions to diagnostic ultrasound. IEEE Trans Ultrason Ferroelectr Freq Control 63(2):336–351
doi: 10.1109/TUFFC.2015.2513958
Shen Y, Xing Y, Lin H et al (2019) Quantitative analysis of non-alcoholic fatty liver in rats via combining multiple ultrasound parameters. Math Biosci Eng 16(5):4546–4558
Tsui P-H, Wan Y-L (2016) Application of ultrasound nakagami imaging for the diagnosis of fatty liver. J Med Ultrasound 24(2):47–49
doi: 10.1016/j.jmu.2016.03.005
Lin SC, Heba E, Wolfson T et al (2015) Noninvasive diagnosis of nonalcoholic fatty liver disease and quantification of liver fat using a new quantitative ultrasound technique. Clin Gastroenterol Hepatol 13(7):1337–1345 e6
Grani G, Lamartina L, Ramundo V et al (2020) Taller-Than-WideShape: a new definition improves the specificity of TIRADS systems. Eur Thyroid J 9(2):85–91