A conventional-to-spectral CT image translation augmentation workflow for robust contrast injection-independent organ segmentation.
data augmentation
deep learning
image translation
noncontrast CT
spectral CT
whole heart multilabels segmentation
Journal
Medical physics
ISSN: 2473-4209
Titre abrégé: Med Phys
Pays: United States
ID NLM: 0425746
Informations de publication
Date de publication:
Feb 2022
Feb 2022
Historique:
revised:
07
10
2021
received:
29
04
2021
accepted:
11
10
2021
pubmed:
25
10
2021
medline:
12
2
2022
entrez:
24
10
2021
Statut:
ppublish
Résumé
In computed tomography (CT) cardiovascular imaging, the numerous contrast injection protocols used to enhance structures make it difficult to gather training datasets for deep learning applications supporting diverse protocols. Moreover, creating annotations on noncontrast scans is extremely tedious. Recently, spectral CT's virtual-noncontrast images (VNC) have been used as data augmentation to train segmentation networks performing on enhanced and true-noncontrast (TNC) scans alike, while improving results on protocols absent of their training dataset. However, spectral data are not widely available, making it difficult to gather specific datasets for each task. As a solution, we present a data augmentation workflow based on a trained image translation network, to bring spectral-like augmentation to any conventional CT dataset. The conventional CT-to-spectral image translation network (HUSpectNet) was first trained to generate VNC from conventional housnfied units images (HU), using an unannotated spectral dataset of 1830 patients. It was then tested on a second dataset of 300 spectral CT scans by comparing VNC generated through deep learning (VNC Tested on 300 full scans, our HUSpectNet translation network shows a mean absolute error of 6.70 ± 2.83 HU between VNC Using the proposed workflow, we trained versatile heart segmentation networks on a dataset of conventional enhanced CT scans, providing robust predictions on both enhanced scans with different contrast injection protocols and TNC scans. The performances obtained were not significantly inferior to training the model on a genuine spectral CT dataset, regardless of the architecture implemented. Using a general-purpose conventional-to-spectral CT translation network as data augmentation could therefore contribute to reducing data collection and annotation requirements for machine learning-based CT studies, while extending their range of application.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1108-1122Informations de copyright
© 2021 American Association of Physicists in Medicine.
Références
Shahzad R, Bos D, Budde RPJ, et al. Automatic segmentation and quantification of the cardiac structures from non-contrast-enhanced cardiac CT scans. Phys Med Biol. 2017;62(9):3798-3813. https://doi.org/10.1088/1361-6560/aa63cb
Zhuang X, Li L, Payer C, et al. Evaluation of algorithms for multi-modality whole heart segmentation: an open-access grand challenge. Med Image Anal. 2019;58:101537. https://doi.org/10.1016/j.media.2019.101537
Chen C, Qin C, Qiu H, et al. Deep learning for cardiac image segmentation: a review. Front Cardiovasc Med. 2020;7:25. https://doi.org/10.3389/fcvm.2020.00025
Singh G, Al'Aref S, van Assen M, et al. Machine learning in cardiac CT: basic concepts and contemporary data. J Cardiovasc Comput Tomogr 2018;12(3):192-201. https://doi.org/10.1016/j.jcct.2018.04.010
Kurugol S, San Jose Estepar R, Ross J, Washko GR. Aorta segmentation with a 3D level set approach and quantification of aortic calcifications in non-contrast chest CT. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012:2343-2346. https://doi.org/10.1109/EMBC.2012.6346433
Kurkure U, Avila-Montes OC & Kakadiaris IA Automated segmentation of thoracic aorta in non-contrast CT images. In: 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro. 2008:29-32. https://doi.org/10.1109/ISBI.2008.4540924
Bruns S, Wolterink JM, Takx RAP, et al. Deep learning from dual-energy information for whole-heart segmentation in dual-energy and single-energy non-contrast-enhanced cardiac CT. Med Phys. 2020;47(10):5048-5060. https://doi.org/10.1002/mp.14451
Sedghi Gamechi Z, Bons LR, Giordano M, et al. Automated 3D segmentation and diameter measurement of the thoracic aorta on non-contrast enhanced CT. Eur Radiol. 2019;29(9):4613-4623. https://doi.org/10.1007/s00330-018-5931-z
Wang C, Smedby Ö. Automatic multi-organ segmentation in non-enhanced CT datasets using hierarchical shape priors. In: 2014 22nd International Conference on Pattern Recognition. 2014:3327-3332. https://doi.org/10.1109/ICPR.2014.574
Xie Y, Padgett J, Biancardi AM, Reeves AP. Automated aorta segmentation in low-dose chest CT images. Int J Comput Assist Radiol Surg. 2014;9(2):211-219. https://doi.org/10.1007/s11548-013-0924-5
Li Q, Yu Z, Wang Y, Zheng H. TumorGAN: a multi-modal data augmentation framework for brain tumor segmentation. Sensors. 2020;20(15):E4203. https://doi.org/10.3390/s20154203
Ly B, Cochet H, Sermesant M. Style data augmentation for robust segmentation of multi-modality cardiac MRI. In: Pop M, Sermesant M, Camara O, eds. Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges. Lecture Notes in Computer Science. Springer International Publishing; 2020:197-208. https://doi.org/10.1007/978-3-030-39074-7_21
Lartaud P-J, Rouchaud A, Rouet J-M, Nempont O, Boussel L. Spectral CT based training dataset generation and augmentation for conventional CT vascular segmentation. In: Shen D, Liu T, Peters TM, eds. Medical Image Computing and Computer Assisted Intervention - MICCAI 2019. Springer International Publishing; 2019:768-775. https://doi.org/10.1007/s11548-021-02468-0
Bruns S, Wolterink JM, van Hamersvelt RW, Zreik M, Leiner T, Isgum I. Improving myocardium segmentation in cardiac CT angiography using spectral information. In: Medical Imaging: Image Processing. 2018. https://doi.org/10.1117/12.2512229
Bruns S, Wolterink JM, van Hamersvelt RW, Leiner T, Išgum I. CNN-based segmentation of the cardiac chambers and great vessels in non-contrast-enhanced cardiac CT. In: International Conference on Medical Imaging with Deep Learning - Extended Abstract Track. 2019. Accessed August 18, 2020. http://arxiv.org/abs/1908.07727
Chen S, Zhong X, Hu S, et al. Automatic multi-organ segmentation in dual-energy CT (DECT) with dedicated 3D fully convolutional DECT networks. Med Phys. 2020;47(2):552-562. https://doi.org/10.1002/mp.13950
JC González Sánchez, Magnusson M, Sandborg M, Å CarlssonTedgren, Malusek A. Segmentation of bones in medical dual-energy computed tomography volumes using the 3D U-Net. Phys Med. 2020;69:241-247. https://doi.org/10.1016/j.ejmp.2019.12.014
Lartaud P-J, Hallé D, Schleef A, et al. Spectral augmentation for heart chambers segmentation on conventional contrasted and unenhanced CT scans: an in-depth study. Int J Comput Assist Radiol Surg. Published online August 7, 2021. https://doi.org/10.1007/978-3-030-32245-8_85
Zhao W, Lv T & Gao P et al. Dual-energy CT imaging using a single-energy CT data is feasible via deep learning. 2019. https://arxiv.org/abs/1906.04874
Lyu T, Wu Z, Zhang Y, Chen Y, Xing L & Zhao W Dual-energy CT imaging from single-energy CT data with material decomposition convolutional neural network. 2020. https://arxiv.org/abs/2006.00149
Liugang G, Kai X, Chunying L, et al. Generation of virtual non-contrast CT from intravenous enhanced CT in radiotherapy using convolutional neural networks. Front Oncol. 2020;10:1715. https://doi.org/10.3389/fonc.2020.01715
Zhao W, Lyu T, Chen Y & Xing L A deep learning approach for virtual monochromatic spectral CT imaging with a standard single energy CT scanner. 2020. https://arxiv.org/abs/2005.09859
Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, eds. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015. Springer International Publishing; 2015:234-241. https://doi.org/10.1007/978-3-319-24574-4_28
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. UNet++: a nested U-Net architecture for medical image segmentation. ArXiv180710165 Cs Eess Stat. Published online July 18, 2018. Accessed March 29, 2021. http://arxiv.org/abs/1807.10165
Payer C, Štern D, Bischof H, Urschler M. Multi-label whole heart segmentation using CNNs and anatomical label configurations. In: Pop M, Sermesant M, Jodoin P-M, eds. Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges. Lecture Notes in Computer Science. Springer International Publishing; 2018:190-198. https://doi.org/10.1007/978-3-319-75541-0_20
Wang C, Smedby Ö. Automatic whole heart segmentation using deep learning and shape context. In: Pop M, Sermesant M, Jodoin P-M, eds. Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges. Lecture Notes in Computer Science. Springer International Publishing; 2018:242-249. https://doi.org/10.1007/978-3-319-75541-0_26
Tong Q, Ning M, Si W, Liao X, Qin J. 3D deeply-supervised U-Net based whole heart segmentation. In: Pop M, Sermesant M, Jodoin P-M, eds. Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges. Lecture Notes in Computer Science. Springer International Publishing; 2018:224-232. https://doi.org/10.1007/978-3-319-75541-0_24
Bratt A, Guenther Z, Hahn LD, et al. Left atrial volume as a biomarker of atrial fibrillation at routine chest CT: deep learning approach. Radiol Cardiothorac Imaging. 2019;1(5):e190057. https://doi.org/10.1148/ryct.2019190057
Wang C, MacGillivray T, Macnaught G, Yang G, Newby D. A two-stage 3D Unet framework for multi-class segmentation on full resolution image. ArXiv180404341 Cs. Published online April 12, 2018. Accessed August 20, 2020. http://arxiv.org/abs/1804.04341
Xu Z, Wu Z, Feng JCFUN. Combining faster R-CNN and U-net network for efficient whole heart segmentation. ArXiv181204914 Cs. Published online December 12, 2018. Accessed August 20, 2020. http://arxiv.org/abs/1812.04914
Ye C, Wang W, Zhang S, Wang K. Multi-depth fusion network for whole-heart CT image segmentation. IEEE Access. 2019;7:23421-23429. https://doi.org/10.1109/ACCESS.2019.2899635
Dormer JD, Ma L, Halicek M, Reilly CM, Schreibmann E, Fei B. Heart chamber segmentation from CT using convolutional neural networks. Proc SPIE- Int Soc Opt Eng. 2018. https://doi.org/10.1117/12.2293554
Yang X, Bian C, Yu L, Ni D, Heng P-A. Hybrid loss guided convolutional networks for whole heart parsing. In: Pop M, Sermesant M, Jodoin P-M, eds. Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges. Lecture Notes in Computer Science. Springer International Publishing; 2018:215-223. https://doi.org/10.1007/978-3-319-75541-0_23
Yang X, Bian C, Yu L, Ni D, Heng P-A. 3D convolutional networks for fully automatic fine-grained whole heart partition. In: Pop M, Sermesant M, Jodoin P-M, eds. Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges. Springer International Publishing; 2018:181-189. https://doi.org/10.1007/978-3-319-75541-0_19
Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O. 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W, eds. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016. Lecture Notes in Computer Science. Springer International Publishing; 2016:424-432. https://doi.org/10.1007/978-3-319-46723-8_49
Bullock J, Cuesta-Lázaro C, Quera-Bofarull A. XNet: a convolutional neural network (CNN) implementation for medical X-ray image segmentation suitable for small datasets. In: Medical Imaging 2019: Biomedical Applications in Molecular, Structural, and Functional Imaging. Vol. 10953. International Society for Optics and Photonics; 2019. https://doi.org/10.1117/12.2512451
He K, Sun J, Tang X. Guided image filtering. IEEE Trans Pattern Anal Mach Intell. 2013;35(6):1397-1409. https://doi.org/10.1109/TPAMI.2012.213
Taha AA, Hanbury A. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging. 2015;15:29. https://doi.org/10.1186/s12880-015-0068-x
Isola P, Zhu J, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:5967-5976. https://doi.org/10.1109/CVPR.2017.632
Tripathy S, Kannala J, Rahtu E. Learning image-to-image translation using paired and unpaired training samples. In: Jawahar CV, Li H, Mori G, Schindler K, eds. Computer Vision - ACCV 2018. Lecture Notes in Computer Science. Springer International Publishing; 2019:51-66. https://doi.org/10.1007/978-3-030-20890-5_4
Cong W, Xi Y, Fitzgerald P, De Man B, Wang G. Virtual monoenergetic CT imaging via deep learning. Patterns. 2020(1):100128. https://doi.org/10.1016/j.patter.2020.100128
Koike Y, Ohira S, Akino Y, et al. Deep learning-based virtual noncontrast CT for volumetric modulated arc therapy planning: comparison with a dual-energy CT-based approach. Med Phys. 2020;47(2):371-379. https://doi.org/10.1002/mp.13925
Sauter AP, Muenzel D, Dangelmaier J, et al. Dual-layer spectral computed tomography: virtual non-contrast in comparison to true non-contrast images. Eur J Radiol. 2018;104:108-114. https://doi.org/10.1016/j.ejrad.2018.05.007