IAS-NET: Joint intraclassly adaptive GAN and segmentation network for unsupervised cross-domain in neonatal brain MRI segmentation.
nneonatal brain segmentation
unsupervised domain adaptation (UDA)
Journal
Medical physics
ISSN: 2473-4209
Titre abrégé: Med Phys
Pays: United States
ID NLM: 0425746
Informations de publication
Date de publication:
Nov 2021
Nov 2021
Historique:
revised:
15
07
2021
received:
04
06
2021
accepted:
15
08
2021
pubmed:
9
9
2021
medline:
18
11
2021
entrez:
8
9
2021
Statut:
ppublish
Résumé
In neonatal brain magnetic resonance image (MRI) segmentation, the model we trained on the training set (source domain) often performs poorly in clinical practice (target domain). As the label of target-domain images is unavailable, this cross-domain segmentation needs unsupervised domain adaptation (UDA) to make the model adapt to the target domain. However, the shape and intensity distribution of neonatal brain MRI images across the domains are largely different from adults'. Current UDA methods aim to make synthesized images similar to the target domain as a whole. But it is impossible to synthesize images with intraclass similarity because of the regional misalignment caused by the cross-domain difference. This will result in generating intraclassly incorrect intensity information from target-domain images. To address this issue, we propose an IAS-NET (joint intraclassly adaptive generative adversarial network (GAN) (IA-NET) and segmentation) framework to bridge the gap between the two domains for intraclass alignment. Our proposed IAS-NET is an elegant learning framework that transfers the appearance of images across the domains from both image and feature perspectives. It consists of the proposed IA-NET and a segmentation network (S-NET). The proposed IA-NET is a GAN-based adaptive network that contains one generator (including two encoders and one shared decoder) and four discriminators for cross-domain transfer. The two encoders are implemented to extract original image, mean, and variance features from source and target domains. The proposed local adaptive instance normalization algorithm is used to perform intraclass feature alignment to the target domain in the feature-map level. S-NET is a U-net structure network that is used to provide semantic constraint by a segmentation loss for the training of IA-NET. Meanwhile, it offers pseudo-label images for calculating intraclass features of the target domain. Source code (in Tensorflow) is available at https://github.com/lb-whu/RAS-NET/. Extensive experiments are carried out on two different data sets (NeoBrainS12 and dHCP), respectively. There exist great differences in the shape, size, and intensity distribution of magnetic resonance (MR) images in the two databases. Compared to baseline, we improve the average dice score of all tissues on NeoBrains12 by 6% through adaptive training with unlabeled dHCP images. Besides, we also conduct experiments on dHCP and improved the average dice score by 4%. The quantitative analysis of the mean and variance of the synthesized images shows that the synthesized image by the proposed is closer to the target domain both in the full brain or within each class than that of the compared methods. In this paper, the proposed IAS-NET can improve the performance of the S-NET effectively by its intraclass feature alignment in the target domain. Compared to the current UDA methods, the synthesized images by IAS-NET are more intraclassly similar to the target domain for neonatal brain MR images. Therefore, it achieves state-of-the-art results in the compared UDA models for the segmentation task.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
6962-6975Informations de copyright
© 2021 American Association of Physicists in Medicine.
Références
Makropoulos A, Counsell SJ, Rueckert D. A review on automatic fetal and neonatal brain MRI segmentation. NeuroImage. 2018;170:231-248.
Illavarason P, Arokia Renjit J, Mohan Kumar P. A study on the quality of life of CP children requiring early rehabilitation interventions. International Conference on Science Technology Engineering and Mathematics (ICONSTEM), Chennai, India; 2019:91-97.
Hack M, Fanaroff AA. Outcomes of children of extremely low birth weight and gestational age in the 1990s. Seminars Neonatol. 2000;5:89-106.
Marlow N, Wolke D, Bracewell MA, Samara M, EPICure Study Group. Neurologic and developmental disability at six years of age after extremely preterm birth. N Engl J Med. 2005;352:9-19.
Delobel-Ayoub M, Arnaud C, White-Koning M, et al. Behavioral problems and cognitive performance at 5 Years of age after very preterm birth: the EPIPAGE study. Pediatrics. 2009;123:1485-1492.
Peterson BS, Anderson AW, Ehrenkranz R, et al. Regional brain volumes and their later neurodevelopmental correlates in term and preterm infants. Pediatrics. 2003;111:939-948.
Counsell SJ, Edwards AD, Chew ATM, et al. Specific relations between neurodevelopmental abilities and white matter microstructure in children born preterm. Brain. 2008;131:3201-3208.
Xu Y, Graud T, Bloch I. From neonatal to adult brain MR image segmentation in a few seconds using 3D-like fully convolutional network and transfer learning. IEEE International Conference on Image Processing (ICIP), Beijing; 2017:4417-4421.
Nie D, Wang L, Gao Y, et al. Fully convolutional networks for multi-modality isointense infant brain image segmentation. IEEE Int Symp Biomed Imaging. 2016;2016:1342-1345.
Moeskops P, Viergever MA, Mendrik AM, de Vries LS, Benders MJNL, Išgum I. Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans Med Imaging. 2016;35:1252-1261.
Ganin Y, Ustinova E, Ajakan H, et al. Domain-adversarial training of neural networks. J Mach Learn Res. 2016;17:2096-2030.
Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial discriminative domain adaptation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI; 2017:2962-2971.
Tajbakhsh N, Jeyaseelan L, Li Q, Chiang J, Wu Z, Ding X. Embracing imperfect datasets: a review of deep learning solutions for medical image segmentation. Med Image Anal. 2020;63:101693.
Bowles C, Chen L, Guerrero R, et al. Gan augmentation: augmenting training data using generative adversarial networks. 2018, arXiv preprint arXiv:1810.10863.
Karras T, Aila T, Laine S, Lehtinen J. Progressive growing of GANs for improved quality, stability, and variation. International Conference on Learning Representations (ICLR), Vancouver; 2018.
Zhao A, Balakrishnan G, Durand F, Guttag JV, Dalca AV. Data augmentation using learned transforms for one-shot medical image segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA; 2019:8543-8553.
Zhu JY, Park T, Isola P, Efros A. Unpaired image-to-image translation using cycle-consistent adversarial networks. IEEE International Conference on Computer Vision, Venice, Italy; 2017:2223-2232.
Huo Y, Xu Z, Moon H, et al. Synseg-net: synthetic segmentation without target modality ground truth. IEEE Trans Med Imaging. 2018;38:1016-1025.
Chen C, Dou Q, Chen H, Qin J, Heng P-A. Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation. Proceedings of The Thirty-Third Conference on Artificial Intelligence (AAAI). Association for the Advancement of Artificial Intelligence (AAAI); 2019:865-872.
Huang X, Liu M-Y, Belongie S, Kautz J. Multimodal unsupervised image-to-image translation. IEEE European Conference on Computer Vision (ECCV), Munich; 2018:172-189.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV; 2016:770-778.
Smith SM. Fast robust automated brain extraction. Hum Brain Mapp. 2002;17(3):143-155.
Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich; 2015:234-241.
Guerrero R, Qin C, Oktay O, et al. White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. Neuroimage. 2018;17:918-934.
Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV. Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging. 2019;38:1788-1800.
Nalepa J, Cwiek M, Mrukwa G, et al. Data augmentation via image registration. IEEE International Conference on Image Processing (ICIP), Taipei; 2019:4250-4254.
Zhang Z, Yang L, Zheng Y. Translating and segmenting multimodal medical volumes with cycle- and shape-consistency generative adversarial network. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT; 2018:9242-9251.
Hoffman J, Tzeng E, Park T, et al. Cycada: cycle-consistent adversarial domain adaptation. Int Conf Mach Learn (ICML), Vienna, Austria; 2018:1994-2003.
Toldo M, Maracani A, Michieli U, Zanuttigh P, et al. Unsupervised domain adaptation in semantic segmentation: a review. 2020, arXiv preprint arXiv:2005.10876.
Liu M-Y, Breuel T, Kautz J. Unsupervised image-to-image translation networks. Conference on Neural Information Processing Systems (NIPS), Long Beach, CA; 2017:700-708.
Huang X, Belongie S. Arbitrary style transfer in real-time with adaptive instance normalization. IEEE International Conference on Computer Vision (ICCV), Venice; 2017:1510-1519.
Choi J, Kim T, Kim C. Self-ensembling with GAN-based data augmentation for domain adaptation in semantic segmentation. IEEE International Conference on Computer Vision (ICCV), Seoul; 2019:6829-6839.
Yu B, Zhou L, Wang L, Shi Y, Fripp J, Bourgeat P. Ea-gans: Edge-aware generative adversarial networks for crossmodality MR image synthesis. IEEE Trans Med Imaging. 2019;38:1750-1762.
Išgum I, Benders MJNL, Avants B, et al. Evaluation of automatic neonatal brain segmentation algorithms: the NeoBrains12 challenge. Med Image Anal. 2015;20:135-151.
Makropoulos A, Robinson EC, Schuh A, et al. The developing Human Connectome Project: a minimal processing pipeline for neonatal cortical surface reconstruction. NeuroImage. 2018;173:88-112.
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA; 2015:3431-3440.
Chen C, Ouyang C, Tarroni G, et al. Unsupervised multi-modal style transfer for cardiac MR segmentation. International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Shenzhen, China; 2019.
Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Advances in Neural Information Processing Systems, Long Beach, CA; 2017: 6626-6637.
Zou Y, Yu Z, Vijaya Kumar BVK, Wang J. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. Proceedings of European Conference on Computer Vision (ECCV), Munich; 2018:289-305.
Pizzati F, de Charette R, Zaccaria M, Cerri P. Domain bridge for unpaired image-to-image translation and unsupervised domain adaptation. Proc. of the British Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO; 2020:2990-2998.
Zhu X, Goldberg AB. Introduction to Semi-Supervised Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool; 2009:1-130.
Oliver A, Odena A, Raffel C, Cubuk ED, Goodfellow IJ. Realistic evaluation of deep semi-supervised learning algorithms. 2018, arXiv preprint arXiv:1804.09170.
Qinmu P, Ouyang M, Wang J, et al. Regularized-Ncut: robust and homogeneous functional parcellation of neonate and adult brain networks. Artif Intell Medicine 2020;106:101872.