Accurate object localization facilitates automatic esophagus segmentation in deep learning.
Automatic segmentation
Deep learning
Esophagus
Object localization
Journal
Radiation oncology (London, England)
ISSN: 1748-717X
Titre abrégé: Radiat Oncol
Pays: England
ID NLM: 101265111
Informations de publication
Date de publication:
12 May 2024
12 May 2024
Historique:
received:
20
10
2023
accepted:
01
05
2024
medline:
13
5
2024
pubmed:
13
5
2024
entrez:
12
5
2024
Statut:
epublish
Résumé
Currently, automatic esophagus segmentation remains a challenging task due to its small size, low contrast, and large shape variation. We aimed to improve the performance of esophagus segmentation in deep learning by applying a strategy that involves locating the object first and then performing the segmentation task. A total of 100 cases with thoracic computed tomography scans from two publicly available datasets were used in this study. A modified CenterNet, an object location network, was employed to locate the center of the esophagus for each slice. Subsequently, the 3D U-net and 2D U-net_coarse models were trained to segment the esophagus based on the predicted object center. A 2D U-net_fine model was trained based on the updated object center according to the 3D U-net model. The dice similarity coefficient and the 95% Hausdorff distance were used as quantitative evaluation indexes for the delineation performance. The characteristics of the automatically delineated esophageal contours by the 2D U-net and 3D U-net models were summarized. Additionally, the impact of the accuracy of object localization on the delineation performance was analyzed. Finally, the delineation performance in different segments of the esophagus was also summarized. The mean dice coefficient of the 3D U-net, 2D U-net_coarse, and 2D U-net_fine models were 0.77, 0.81, and 0.82, respectively. The 95% Hausdorff distance for the above models was 6.55, 3.57, and 3.76, respectively. Compared with the 2D U-net, the 3D U-net has a lower incidence of delineating wrong objects and a higher incidence of missing objects. After using the fine object center, the average dice coefficient was improved by 5.5% in the cases with a dice coefficient less than 0.75, while that value was only 0.3% in the cases with a dice coefficient greater than 0.75. The dice coefficients were lower for the esophagus between the orifice of the inferior and the pulmonary bifurcation compared with the other regions. The 3D U-net model tended to delineate fewer incorrect objects but also miss more objects. Two-stage strategy with accurate object location could enhance the robustness of the segmentation model and significantly improve the esophageal delineation performance, especially for cases with poor delineation results.
Sections du résumé
BACKGROUND
BACKGROUND
Currently, automatic esophagus segmentation remains a challenging task due to its small size, low contrast, and large shape variation. We aimed to improve the performance of esophagus segmentation in deep learning by applying a strategy that involves locating the object first and then performing the segmentation task.
METHODS
METHODS
A total of 100 cases with thoracic computed tomography scans from two publicly available datasets were used in this study. A modified CenterNet, an object location network, was employed to locate the center of the esophagus for each slice. Subsequently, the 3D U-net and 2D U-net_coarse models were trained to segment the esophagus based on the predicted object center. A 2D U-net_fine model was trained based on the updated object center according to the 3D U-net model. The dice similarity coefficient and the 95% Hausdorff distance were used as quantitative evaluation indexes for the delineation performance. The characteristics of the automatically delineated esophageal contours by the 2D U-net and 3D U-net models were summarized. Additionally, the impact of the accuracy of object localization on the delineation performance was analyzed. Finally, the delineation performance in different segments of the esophagus was also summarized.
RESULTS
RESULTS
The mean dice coefficient of the 3D U-net, 2D U-net_coarse, and 2D U-net_fine models were 0.77, 0.81, and 0.82, respectively. The 95% Hausdorff distance for the above models was 6.55, 3.57, and 3.76, respectively. Compared with the 2D U-net, the 3D U-net has a lower incidence of delineating wrong objects and a higher incidence of missing objects. After using the fine object center, the average dice coefficient was improved by 5.5% in the cases with a dice coefficient less than 0.75, while that value was only 0.3% in the cases with a dice coefficient greater than 0.75. The dice coefficients were lower for the esophagus between the orifice of the inferior and the pulmonary bifurcation compared with the other regions.
CONCLUSION
CONCLUSIONS
The 3D U-net model tended to delineate fewer incorrect objects but also miss more objects. Two-stage strategy with accurate object location could enhance the robustness of the segmentation model and significantly improve the esophageal delineation performance, especially for cases with poor delineation results.
Identifiants
pubmed: 38735947
doi: 10.1186/s13014-024-02448-z
pii: 10.1186/s13014-024-02448-z
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
55Subventions
Organisme : Jiangsu Provincial Medical Key Discipline
ID : ZDXK202235
Organisme : Jiangsu Provincial Medical Key Discipline
ID : ZDXK202235
Informations de copyright
© 2024. The Author(s).
Références
Song Y, Hu J, Wu Q, Xu F, Nie S, Zhao Y, et al. Automatic delineation of the clinical target volume and organs at risk by deep learning for rectal cancer postoperative radiotherapy. Radiother Oncol. 2020;145:186–92.
doi: 10.1016/j.radonc.2020.01.020
pubmed: 32044531
Amjad A, Xu J, Thill D, Lawton C, Hall W, Awan MJ, et al. General and custom deep learning autosegmentation models for organs in head and neck, abdomen, and male pelvis. Med Phys. 2022;49(3):1686–700.
doi: 10.1002/mp.15507
pubmed: 35094390
Branco D, Mayadev J, Moore K, Ray X. Dosimetric and feasibility evaluation of a CBCT-based daily adaptive radiotherapy protocol for locally advanced cervical cancer. J Appl Clin Med Phys. 2022;24(1):e13783.
doi: 10.1002/acm2.13783
pubmed: 36208134
pmcid: 9859994
Rigaud B, Anderson BM, Yu ZH, Gobeli M, Cazoulat G, Söderberg J, et al. Automatic segmentation using deep learning to Enable Online Dose optimization during Adaptive Radiation Therapy of Cervical Cancer. Int J Radiat Oncol Biol Phys. 2021;109(4):1096–110.
doi: 10.1016/j.ijrobp.2020.10.038
pubmed: 33181248
Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods. 2020;18(2):203–11.
doi: 10.1038/s41592-020-01008-z
pubmed: 33288961
Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y et al. TransUNet: transformers make strong encoders for Medical Image Segmentation. ArXiv. 2021;abs/2102.04306.
Cao H, Wang Y, Chen J, Jiang D, Zhang X, Tian Q, et al. editors. Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation. ECCV Workshops; 2021.
Gao Y, Gao Y, Gao Y, Huang R, Yang Y, Zhang J, et al. FocusNetv2: Imbalanced large and small organ segmentation with adversarial shape constraint for head and neck CT images. Med Image Anal. 2020;67:101831.
doi: 10.1016/j.media.2020.101831
pubmed: 33129144
Milletari F, Navab N, Ahmadi S-A. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. arXiv e-prints [Internet]. 2016 June 01, 2016. https://ui.adsabs.harvard.edu/abs/2016arXiv160604797M .
Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. arXiv e-prints [Internet]. 2016 June 01, 2016. https://ui.adsabs.harvard.edu/abs/2016arXiv160606650C .
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv e-prints [Internet]. 2015 May 01, 2015. https://ui.adsabs.harvard.edu/abs/2015arXiv150504597R .
Shi F, Hu W, Wu J, Han M, Wang J, Zhang W, et al. Deep learning empowered volume delineation of whole-body organs-at-risk for accelerated radiotherapy. Nat Commun. 2022;13(1):6566.
doi: 10.1038/s41467-022-34257-x
pubmed: 36323677
pmcid: 9630370
Li Z, Kamnitsas K, Glocker B. Overfitting of neural nets under class imbalance: analysis and improvements for segmentation. ArXiv. 2019;abs/1907.10982.
Lin T-Y, Goyal P, Girshick RB, He K, Dollár P. Focal Loss for Dense Object Detection. 2017 IEEE International Conference on Computer Vision (ICCV). 2017:2999–3007.
Francis S, Jayaraj PB, Pournami PN, Thomas M, Jose AT, Binu AJ, et al. ThoraxNet: a 3D U-Net based two-stage framework for OAR segmentation on thoracic CT images. Phys Eng Sci Med. 2022;45(1):189–203.
doi: 10.1007/s13246-022-01101-x
pubmed: 35029804
Chung SY, Chang JS, Choi MS, Chang Y, Choi BS, Chun J, et al. Clinical feasibility of deep learning-based auto-segmentation of target volumes and organs-at-risk in breast cancer patients after breast-conserving surgery. Radiat Oncol. 2021;16(1):44.
doi: 10.1186/s13014-021-01771-z
pubmed: 33632248
pmcid: 7905884
Chen X, Sun S, Bai N, Han K, Liu Q, Yao S, et al. A deep learning-based auto-segmentation system for organs-at-risk on whole-body computed tomography images for radiation therapy. Radiother Oncol. 2021;160:175–84.
doi: 10.1016/j.radonc.2021.04.019
pubmed: 33961914
Liao W, Luo X, He Y, Dong Y, Li C, Li K et al. Comprehensive evaluation of a deep learning model for automatic organs at risk segmentation on heterogeneous computed tomography images for abdominal radiotherapy. Int J Radiat Oncol Biol Phys. 2023.
Johnston N, De Rycke J, Lievens Y, van Eijkeren M, Aelterman J, Vandersmissen E, et al. Dose-volume-based evaluation of convolutional neural network-based auto-segmentation of thoracic organs at risk. Phys Imaging Radiat Oncol. 2022;23:109–17.
doi: 10.1016/j.phro.2022.07.004
pubmed: 35936797
pmcid: 9352974
Vu CC, Siddiqui ZA, Zamdborg L, Thompson AB, Quinn TJ, Castillo E, et al. Deep convolutional neural networks for automatic segmentation of thoracic organs-at‐risk in radiation oncology – use of non‐domain transfer learning. J Appl Clin Med Phys. 2020;21(6):108–13.
doi: 10.1002/acm2.12871
pubmed: 32602187
pmcid: 7324695
Wittenstein O, Hiepe P, Sowa LH, Karsten E, Fandrich I, Dunst J. Automatic image segmentation based on synthetic tissue model for delineating organs at risk in spinal metastasis treatment planning. Strahlenther Onkol. 2019;195(12):1094–103.
doi: 10.1007/s00066-019-01463-4
pubmed: 31037351
pmcid: 6868111
Yang J, Veeraraghavan H, Armato SG, Farahani K, Kirby JS, Kalpathy-Kramer J, et al. Autosegmentation for thoracic radiation treatment planning: a grand challenge at AAPM 2017. Med Phys. 2018;45(10):4568–81.
doi: 10.1002/mp.13141
pubmed: 30144101
pmcid: 6714977
Lambert Z, Petitjean C, Dubray B, Kuan S, editors. SegTHOR: Segmentation of Thoracic Organs at Risk in CT images. 2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA); 2020 9–12 Nov. 2020.
Zhou X, Wang D, Krähenbühl P. Objects as Points ArXiv. 2019;abs/1904.07850.
He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. arXiv e-prints [Internet]. 2015 December 01, 2015. https://ui.adsabs.harvard.edu/abs/2015arXiv151203385H .
Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, et al. editors. Automatic differentiation in PyTorch2017.
Zhao Y, Wang X, Che T, Bao G, Li S. Multi-task deep learning for medical image computing and analysis: a review. Comput Biol Med. 2023;153:106496.
doi: 10.1016/j.compbiomed.2022.106496
pubmed: 36634599
Wei Z, Ren J, Korreman SS, Nijkamp J. Towards interactive deep-learning for tumour segmentation in head and neck cancer radiotherapy. Phys Imaging Radiat Oncol. 2023;25:100408.
doi: 10.1016/j.phro.2022.12.005
pubmed: 36655215
Jiang J, Veeraraghavan H, One Shot PACS. Patient specific anatomic context and shape prior Aware Recurrent Registration-Segmentation of Longitudinal thoracic cone Beam CTs. IEEE Trans Med Imaging. 2022;41(8):2021–32.
doi: 10.1109/TMI.2022.3154934
Dahiya N, Alam SR, Zhang P, Zhang SY, Li T, Yezzi A, et al. Multitask 3D CBCT-to‐CT translation and organs‐at‐risk segmentation using physics‐based data augmentation. Med Phys. 2021;48(9):5130–41.
doi: 10.1002/mp.15083
pubmed: 34245012
Dong X, Lei Y, Wang T, Thomas M, Tang L, Curran WJ, et al. Automatic multiorgan segmentation in thorax CT images using U-net‐GAN. Med Phys. 2019;46(5):2157–68.
doi: 10.1002/mp.13458
pubmed: 30810231
pmcid: 6510589
Singh J, Beeche C, Shi Z, Beale O, Rosin B, Leader J, et al. Batch-balanced focal loss: a hybrid solution to class imbalance in deep learning. J Med Imaging (Bellingham). 2023;10(05):051809.
pubmed: 37361550
Tappeiner E, Welk M, Schubert R. Tackling the class imbalance problem of deep learning-based head and neck organ segmentation. Int J Comput Assist Radiol Surg. 2022;17(11):2103–11.
doi: 10.1007/s11548-022-02649-5
pubmed: 35578086
pmcid: 9515025
Fernando KRM, Tsokos CP. Dynamically weighted balanced loss: Class Imbalanced Learning and confidence calibration of deep neural networks. IEEE Trans Neural Netw Learn Syst. 2022;33(7):2940–51.
doi: 10.1109/TNNLS.2020.3047335
pubmed: 33444149