Simultaneous super-resolution and contrast synthesis of routine clinical magnetic resonance images of the knee for improving automatic segmentation of joint cartilage: data from the Osteoarthritis Initiative.
MR imaging
contrast synthesis
knee cartilage segmentation
super resolution
Journal
Medical physics
ISSN: 2473-4209
Titre abrégé: Med Phys
Pays: United States
ID NLM: 0425746
Informations de publication
Date de publication:
Oct 2020
Oct 2020
Historique:
received:
19
05
2020
revised:
07
07
2020
accepted:
24
07
2020
pubmed:
4
8
2020
medline:
15
5
2021
entrez:
4
8
2020
Statut:
ppublish
Résumé
High resolution three-dimensional (3D) magnetic resonance (MR) images are well suited for automated cartilage segmentation in the human knee joint. However, volumetric scans such as 3D Double-Echo Steady-State (DESS) images are not routinely acquired in clinical practice which limits opportunities for reliable cartilage segmentation using (fully) automated algorithms. In this work, a method for generating synthetic 3D MR (syn3D-DESS) images with better contrast and higher spatial resolution from routine, low resolution, two-dimensional (2D) Turbo-Spin Echo (TSE) clinical knee scans is proposed. A UNet convolutional neural network is employed for synthesizing enhanced artificial MR images suitable for automated knee cartilage segmentation. Training of the model was performed on a large, publically available dataset from the OAI, consisting of 578 MR examinations of knee joints from 102 healthy individuals and patients with knee osteoarthritis. The generated synthetic images have higher spatial resolution and better tissue contrast than the original 2D TSE, which allow high quality automated 3D segmentations of the cartilage. The proposed approach was evaluated on a separate set of MR images from 88 subjects with manual cartilage segmentations. It provided a significant improvement in automated segmentation of knee cartilages when using the syn3D-DESS images compared to the original 2D TSE images. The proposed method can successfully synthesize 3D DESS images from 2D TSE images to provide images suitable for automated cartilage segmentation.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
4939-4948Subventions
Organisme : Australian Research Councils Linkage Projects funding scheme
ID : LP100200422
Organisme : National Health and Medical Research Councils Development
ID : APP1091996
Organisme : NIH HHS
ID : N01-AR-2-2258
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2259
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2260
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2261
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2262
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2258
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2259
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2260
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2261
Pays : United States
Organisme : NIH HHS
ID : N01-AR-2-2262
Pays : United States
Informations de copyright
© 2020 American Association of Physicists in Medicine.
Références
Gold GE, Chen CA, Koo S, Hargreaves BA, Bangerter NK. Recent advances in MRI of articular cartilage. Am J Roentgenol. 2009;193:628-638.
Peterfy CG, Schneider E, Nevitt M. The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. Osteoarthr Cartil. 2008;16:1433-1441.
Wirth W, Nevitt M, Hellio Le Graverand M-P, et al. Sensitivity to change of cartilage morphometry using coronal FLASH, sagittal DESS, and coronal MPR DESS protocols-comparative data from the Osteoarthritis Initiative (OAI). Osteoarthr cartil. 2010;18:547-554.
Ristow O, Steinbach L, Sabo G, et al. Isotropic 3D fast spin-echo imaging versus standard 2D imaging at 3.0 T of the knee-image quality and diagnostic performance. Eur Radiol. 2009;19:1263-1272.
Fripp J, Crozier S, Warfield SK, Ourselin S. Automatic segmentation and quantitative analysis of the articular cartilages from magnetic resonance images of the knee. IEEE Trans Med Image. 2010;29:55-64.
Eckstein F, Kwoh CK, Link TM. Imaging research results from the Osteoarthritis Initiative (OAI): a review and lessons learned 10 years after start of enrolment. Ann Rheum Dis. 2014;2006:1289-1300.
Paproki A, Engstrom C, Chandra SS, Neubert A, Fripp J, Crozier S. Automated segmentation and analysis of normal and osteoarthritic knee menisci from magnetic resonance images - data from the Osteoarthritis Initiative. Osteoarthr Cartil. 2014;22:1259-1270.
Ambellan F, Tack A, Ehlke M, Zachow S. Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: data from the osteoarthritis initiative. Med Image Anal. 2019;52:109-118.
Brem MH, Lang PK, Neumann G, et al. Magnetic resonance image segmentation using semi-automated software for quantification of knee articular cartilage-initial evaluation of a technique for paired scans. Skeletal Radiol. 2009;38:505-511.
Dam EB, Lillholm M, Marques J, Nielsen M. Automatic segmentation of high- and low-field knee MRIs using knee image quantification with data from the osteoarthritis initiative. J Med Imaging (Bellingham). 2015;2:024001.
Wang Z, Liu D, Yang J, Han W, Huang T. Deep Networks for Image Super-Resolution with Sparse Prior. arXiv. 2015;1507.08905:1-10.
Dong C, Loy CC, He K, Tang X. image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell. 2016;38:295-307.
Bahrami K, Shi F, Zong X, Won Shin H, An H, Shen D. Reconstruction of 7T-like Images from 3T MRI. IEEE Trans Med Imaging. 2016;35:2085-2097.
Jog A, Carass A, Roy S, Pham DL, Prince JL. Random forest regression for magnetic resonance image synthesis. Med Image Anal. 2017;35:475-488.
Sevetlidis V, Giuffrida MV, Tsaftaris SA. Whole image synthesis using a deep encoder-decoder network. In: Tsaftaris SA, Gooya A, Frangi AF, Prince JL, eds. Simulation and Synthesis in Medical Imaging (SASHIMI 2016) - Lecture Notes in Computer Science. Vol. 9968. Lecture Notes in Computer Science. Berlin: Springer International Publishing; 2016:127-137.
Roy S, Carass A, Prince JL. Magnetic resonance image example-based contrast synthesis. IEEE Trans Med Imaging. 2013;32:2348-2363.
Van Nguyen H, Zhou K, Vemulapalli R.Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer International Publishing; 2015:677-684.
Nie D, Trullo R, Lian J, et al. Medical image synthesis with context-aware generative adversarial networks. In: Descoteaux M, Maier-Hein L, Franz A, Jannin P, Collins DL, Duchesne S, eds. Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. Lecture Notes in Computer Science. Berlin: Springer International Publishing; 2017:417-425.
Wolterink JM, Leiner T, Viergever MA, Išgum I. Generative adversarial networks for noise reduction in low-dose CT. IEEE Trans Med Imaging. 2017;36:2536-2545.
Chartsias A, Joyce T, Giuffrida MV, Tsaftaris SA. Multimodal MR synthesis via modality-invariant latent representation. IEEE Trans Med Imaging. 2018;37:803-814.
Wang C, Papanastasiou G, Tsaftaris S, et al. TPSDicyc: Improved Deformation Invariant Cross-domain Medical Image Synthesis. In: Knoll F, Maier A, Rueckert D, Ye JC, eds. Machine Learning for Medical Image Reconstruction., Lecture Notes in Computer Science. Berlin: Springer International Publishing; 2019:245-254.
Ben-Cohen A, Klang E, Diamant I, et al. CT image-based decision support system for categorization of liver metastases into primary cancer sites: initial results. Acad Radiol. 2017;24:1501-1509.
Bi L, Kim J, Kumar A, Feng D, Fulham M, et al. Synthesis of positron emission tomography (PET) images via multi-channel generative adversarial networks (GANs). In: Cardoso MJ, Arbel T, Gao F, eds. Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment. Lecture Notes in Computer Science. Berlin: Springer International Publishing; 2017:43-51.
Li J, Madry A, Peebles J, Schmidt L. On the Limitations of First-Order Approximation in GAN Dynamics. arXiv:170609884 [cs]. Published online June 29, 2017. Accessed March 8, 2019. http://arxiv.org/abs/1706.09884
Kodali N, Abernethy J, Hays J, Kira Z. On Convergence and Stability of GANs. Published online May 19, 2017. Accessed March 8, 2019. https://arxiv.org/abs/1705.07215v5
Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. arXiv:170107875 [cs, stat]. Published online December 6, 2017. Accessed July 3, 2020. http://arxiv.org/abs/1701.07875
Mescheder L, Geiger A, Nowozin S. Which Training Methods for GANs do actually Converge? In: International Conference on Machine Learning; 2018:3481-3490. Accessed July 3, 2020. http://proceedings.mlr.press/v80/mescheder18a.html
Arora S, Ge R, Liang Y, Ma T, Zhang Y.Generalization and Equilibrium in Generative Adversarial Nets (GANs). In: International Conference on Machine Learning; 2017:224-232. Accessed July 3, 2020. http://proceedings.mlr.press/v70/arora17a.html
Chaudhari AS, Fang Z, Kogan F, et al. Super-resolution musculoskeletal MRI using deep learning. Magn Reson Med. 2018;80:2139-2154.
Yu B, Zhou L, Wang L, Shi Y, Fripp J, Bourgeat P. Ea-GANs: edge-aware generative adversarial networks for cross-modality MR image synthesis. IEEE Trans Med Imaging. 2019;38:1750-1762.
Sanchez I, Vilaplana V. Brain MRI super-resolution using 3D generative adversarial networks. arXiv:181211440 [cs, stat]. Published online December 29, 2018. Accessed July 3, 2020. http://arxiv.org/abs/1812.11440
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: International Conference on Medical Image Computing and Computer Assisted Intervention; 2015:234-241.
Rivest-Hénault D, Dowson N, Greer PB, Fripp J, Dowling JA. Robust inverse-consistent affine CT-MR registration in MRI-assisted and MRI-alone prostate radiation therapy. Med Image Anal. 2015;23:56-69.
Bollmann S, Rasmussen KGB, Kristensen M, et al. DeepQSM - using deep learning to solve the dipole inversion for quantitative susceptibility mapping. NeuroImage. 2019;195:373-383.
Cootes TF, Taylor CJ, Cooper DH, Graham J. Active shape models-their training and application. Comput vis image und. 1995;61:38-59.
Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M.Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Mori K, Sakuma I, Sato Y, Barillot C, Navab N, eds. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2013. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer; 2013:246-253.
Zhou Z, Zhao G, Kijowski R, Liu F. Deep convolutional neural network for segmentation of knee joint anatomy. Magn Reson Med. 2018;80:2759-2770.
Raj A, Vishwanathan S, Ajani B, Krishnan K, Agarwal H. Automatic knee cartilage segmentation using fully volumetric convolutional neural networks for evaluation of osteoarthritis. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018); 2018:851-854.