Noise2Recon: Enabling SNR-robust MRI reconstruction with semi-supervised and self-supervised learning.
data efficiency
deep learning
denoising
distribution shift
image reconstruction
self-supervised learning
semi-supervised learning
Journal
Magnetic resonance in medicine
ISSN: 1522-2594
Titre abrégé: Magn Reson Med
Pays: United States
ID NLM: 8505245
Informations de publication
Date de publication:
11 2023
11 2023
Historique:
revised:
23
05
2023
received:
05
05
2022
accepted:
24
05
2023
medline:
31
8
2023
pubmed:
10
7
2023
entrez:
10
7
2023
Statut:
ppublish
Résumé
To develop a method for building MRI reconstruction neural networks robust to changes in signal-to-noise ratio (SNR) and trainable with a limited number of fully sampled scans. We propose Noise2Recon, a consistency training method for SNR-robust accelerated MRI reconstruction that can use both fully sampled (labeled) and undersampled (unlabeled) scans. Noise2Recon uses unlabeled data by enforcing consistency between model reconstructions of undersampled scans and their noise-augmented counterparts. Noise2Recon was compared to compressed sensing and both supervised and self-supervised deep learning baselines. Experiments were conducted using retrospectively accelerated data from the mridata three-dimensional fast-spin-echo knee and two-dimensional fastMRI brain datasets. All methods were evaluated in label-limited settings and among out-of-distribution (OOD) shifts, including changes in SNR, acceleration factors, and datasets. An extensive ablation study was conducted to characterize the sensitivity of Noise2Recon to hyperparameter choices. In label-limited settings, Noise2Recon achieved better structural similarity, peak signal-to-noise ratio, and normalized-RMS error than all baselines and matched performance of supervised models, which were trained with Noise2Recon is a label-efficient reconstruction method that is robust to distribution shifts, such as changes in SNR, acceleration factors, and others, with limited or no fully sampled training data.
Types de publication
Journal Article
Research Support, U.S. Gov't, Non-P.H.S.
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
2052-2070Subventions
Organisme : NIAMS NIH HHS
ID : K24 AR06206
Pays : United States
Organisme : NIAMS NIH HHS
ID : R01 AR063643
Pays : United States
Organisme : NIBIB NIH HHS
ID : P41EB015891
Pays : United States
Organisme : NIBIB NIH HHS
ID : R01EB009690
Pays : United States
Organisme : NIBIB NIH HHS
ID : R01EB002524
Pays : United States
Organisme : NIBIB NIH HHS
ID : R01EB026136
Pays : United States
Informations de copyright
© 2023 International Society for Magnetic Resonance in Medicine.
Références
Pruessmann KP, Weiger M, Scheidegger MB, Boesiger P. Sense: sensitivity encoding for fast mri. Magn Reson Med. 1999;42:952-962.
Griswold MA, Jakob PM, Heidemann RM, et al. Generalized autocalibrating partially parallel acquisitions (grappa). Magn Reson Med. 2002;47:1202-1210.
Lustig M, Donoho DL, Santos JM, Pauly JM. Compressed sensing mri. IEEE Signal Process Mag. 2008;25:72-82.
Usman M, Atkinson D, Odille F, et al. Motion corrected compressed sensing for free-breathing dynamic cardiac mri. Magn Reson Med. 2013;70:504-516.
Feng L, Benkert T, Block KT, Sodickson DK, Otazo R, Chandarana H. Compressed sensing for body mri. J Magn Reson Imaging. 2017;45:966-987.
Notohamiprodjo M, Glaser C, Herrmann KA, et al. Diffusion tensor imaging of the kidney with parallel imaging: initial clinical experience. Invest Radiol. 2008;43:677-685.
Skare S, Newbould RD, Clayton DB, Albers GW, Nagle S, Bammer R. Clinical multishot dw-epi through parallel imaging with considerations of susceptibility, motion, and noise. Magn Reson Med. 2007;57:881-890.
Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated mri data. Magn Reson Med. 2018;79:3055-3071.
Sandino CM, Cheng JY, Chen F, Mardani M, Pauly JM, Vasanawala SS. Compressed sensing: from research to clinical practice with deep neural networks: shortening scan times for magnetic resonance imaging. IEEE Signal Process Mag. 2020;37:117-127.
Wang S, Su Z, Ying L, et al. Accelerating magnetic resonance imaging via deep learning. Paper presented at: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI); 2016; Prague, Czech Republic:514-517.
Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a Gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process. 2017;26:3142-3155.
Jin KH, McCann MT, Froustey E, Unser M. Deep convolutional neural network for inverse problems in imaging. IEEE Trans Image Process. 2017;26:4509-4522.
Akçakaya M, Moeller S, Weingärtner S, Uğurbil K. Scan-specific robust artificial-neural-networks for k-space interpolation (raki) reconstruction: database-free deep learning for fast imaging. Magn Reson Med. 2019;81:439-453.
Quan TM, NguyenDuc T, Jeong WK. Compressed sensing mri reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging. 2018;37:1488-1497.
Pezzotti N, Yousefi S, Elmahdy MS, et al. An adaptive intelligence algorithm for undersampled knee mri reconstruction: application to the 2019 fastmri challenge. arXiv preprint arXiv:2004.07339. 2020.
Adler J, Öktem O. Learned primal-dual reconstruction. IEEE Trans Med Imaging. 2018;37:1322-1332.
Meinhardt T, Moeller M, Hazirbas C, Cremers D. Learning proximal operators: using denoising networks for regularizing inverse imaging problems. Paper presented at: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy; October 2017:1799-1808.
Aggarwal HK, Mani MP, Jacob M. Modl: model-based deep learning architecture for inverse problems. IEEE Trans Med Imaging. 2019;38:394-405.
Yaman B, Hosseini SAH, Moeller S, Ellermann J, Uğurbil K, Akçakaya M. Self-supervised learning of physics-guided reconstruction neural networks without fully sampled reference data. Magn Reson Med. 2020;84:3172-3191.
Lehtinen J, Munkberg J, Hasselgren J, et al. Noise2Noise: learning image restoration without clean data. In: Dy J, Krause A, eds. Proceedings of the 35th International Conference on Machine Learning. Cambridge University Press; 2018:2965-2974.
Hu BSC, Cheng JY. System and method for noise-based training of a prediction model. US Patent 10,997,501. May 4, 2021.
Batson J, Royer L. Noise2self: blind denoising by self-supervision. CoRR 2019; abs/1901.11365.
Hendriksen AA, Pelt DM, Batenburg KJ. Noise2inverse: self-supervised deep convolutional denoising for tomography. IEEE Trans Comput Imaging. 2020;6:1320-1335.
Romano Y, Elad M, Milanfar P. The little engine that could: regularization by denoising (red). SIAM J Imaging Sci. 2017;10:1804-1844.
Liu J, Sun Y, Eldeniz C, Gan W, An H, Kamilov US. Rare: image reconstruction using deep priors learned without groundtruth. IEEE J Select Topics Signal Process. 2020;14:1088-1099.
Darestani MZ, Heckel R. Accelerated mri with un-trained neural networks. IEEE Trans Comput Imaging. 2021;7:724-733.
Lei K, Mardani M, Pauly JM, Vasanawala SS. Wasserstein gans for mr imaging: from paired to unpaired training. IEEE Trans Med Imaging. 2021;40:105-115.
Cole EK, Ong F, Vasanawala SS, Pauly JM. Fast unsupervised mri reconstruction without fully-sampled ground truth data using generative adversarial networks. Paper presented at: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW); 2021; Montreal, Canada:3971-3980.
Yaman B, Hosseini SAH, Akçakaya M. Zero-shot self-supervised learning for mri reconstruction. arXiv preprint arXiv:2102.07737 2021.
Demirel OB, Yaman B, Dowdle L, et al. 20-fold accelerated 7t fmri using referenceless self-supervised deep learning reconstruction. Paper presented at: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Guadalaraja, Mexico; 2021:3765-3769.
Lahiri A, Wang G, Ravishankar S, Fessler JA. Blind primed supervised (blips) learning for mr image reconstruction. IEEE Trans Med Imaging. 2021;40(11):3113-3124.
Ravishankar S, Nadakuditi RR, Fessler JA. Efficient sum of outer products dictionary learning (soup-dil) and its application to inverse problems. IEEE Trans Comput Imaging. 2017;3:694-709.
Song P, Weizman L, Mota JF, Eldar YC, Rodrigues MR. Coupled dictionary learning for multi-contrast mri reconstruction. IEEE Trans Med Imaging. 2019;39:621-633.
Chaudhari AS, Sandino CM, Cole EK, et al. Prospective deployment of deep learning in mri: a framework for important considerations, challenges, and recommendations for best practices. J Magn Reson Imaging. 2021;54:357-371.
Hammernik K, Schlemper J, Qin C, Duan J, Summers RM, Rueckert D. Systematic evaluation of iterative deep neural networks for fast parallel mri reconstruction with sensitivity-weighted coil combination. Magn Reson Med. 2021;86:1859-1872.
Darestani MZ, Chaudhari AS, Heckel R. Measuring robustness in deep learning based compressive sensing. Paper presented at: Proceedings of the 38th International Conference on Machine Learning; July 18-24, 2021:2433-2444.
Lustig M, Donoho D, Pauly JM. Sparse mri: the application of compressed sensing for rapid mr imaging. Magn Reson Med. 2007;58:1182-1195.
Antun V, Renna F, Poon C, Adcock B, Hansen AC. On instabilities of deep learning in image reconstruction and the potential costs of ai. Proc Natl Acad Sci. 2020;117:30088-30095.
Donoho DL, Maleki A, Montanari A. The noise-sensitivity phase transition in compressed sensing. IEEE Trans Inform Theory. 2011;57:6920-6941.
Virtue P, Lustig M. The empirical effect of gaussian noise in undersampled mri reconstruction. Tomography. 2017;3:211-221.
Knoll F, Hammernik K, Kobler E, Pock T, Recht MP, Sodickson DK. Assessment of the generalization of learned image reconstruction and the potential for transfer learning. Magn Reson Med. 2019;81:116-128.
Jalal A, Arvinte M, Daras G, Price E, Dimakis AG, Tamir J. Robust compressed sensing mri with deep generative priors. Advances in Neural Information Processing Systems. 2021;34:14938-14954.
Raj A, Bresler Y, Li B. Improving robustness of deep-learning-based image reconstruction. Paper presented at: International Conference on Machine Learning; 2020:7932-7942.
Xie Q, Dai Z, Hovy E, Luong T, Le Q. Unsupervised data augmentation for consistency training. Paper presented at: Advances in Neural Information Processing Systems; 2020:6256-6268.
Sohn K, Berthelot D, Carlini N, et al. Fixmatch: simplifying semi-supervised learning with consistency and confidence. Adv Neural Inform Process Syst. 2020;33:596-608.
Moran N, Schmidt D, Zhong Y, Coady P. Noisier2noise: learning to denoise from unpaired noisy data. Paper presented at: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2020:12064-12072.
Macovski A. Noise in mri. Magn Reson Med. 1996;36:494-497.
Fabian Z, Heckel R, Soltanolkotabi M. Data augmentation for deep learning based accelerated mri reconstruction with limited data. Paper presented at: International Conference on Machine Learning; 2021:3057-3067.
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15:1929-1958.
Redpath TW. Signal-to-noise ratio in mri. Br J Radiol. 1998;71:704-707.
Hansen MS, Kellman P. Image reconstruction: an overview for clinicians. J Magn Reson Imaging. 2015;41:573-585.
Ong F, Amin S, Vasanawala S, Lustig M. Mridata.org: an open archive for sharing mri raw data. Paper presented at: Proceedings of International Society of Magnetic Resonance Medicine, Paris, France; 2018. Abstract 3425.
Ong F, Lustig M. Sigpy: a python package for high performance iterative reconstruction. Paper presented at: Proceedings of the ISMRM 27th Annual Meeting; 2019; Montreal, Quebec, Canada. Abstract 4819.
Ying L, Sheng J. Joint image reconstruction and sensitivity estimation in sense (jsense). Magn Reson Med. 2007;57:1196-1202.
Zbontar J, Knoll F, Sriram A, et al. fastmri: an open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.08839. 2018.
Mataev G, Milanfar P, Elad M. Deepred: deep image prior powered by red. Paper presented at: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops; October 2019.
Reehorst ET, Schniter P. Regularization by denoising: clarifications and new interpretations. IEEE Trans Comput Imaging. 2019;5:52-67.
Muckley MJ, Riemenschneider B, Radmanesh A, et al. State-of-the-art machine learning mri reconstruction in 2020: results of the second fastmri challenge. arXiv preprint arXiv:2012.06318. 2020.
Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. Paper presented at: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2015:234-241.
Paszke A, Gross S, Massa F, et al. Pytorch: an imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems. 2019;32. Accessed June 4 2023. https://proceedings.neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004;13:600-612.
Hestness J, Narang S, Ardalani N, et al. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409. 2017.
Ratner AJ, DeSa CM, Wu S, Selsam D, Ré C. Data programming: creating large training sets, quickly. Adv Neural Inform Process Syst. 2016;29:3567-3575.
Wu S, Zhang HR, Ré C. Understanding and improving information transfer in multi-task learning. arXiv preprint arXiv:2005.00944. 2020.
Braun S, Neil D, Liu SC. A curriculum learning method for improved noise robustness in automatic speech recognition. Paper presented at: 2017 25th European Signal Processing Conference (EUSIPCO), Kos, Greece; 2017:548-552.
Gnanasambandam A, Chan S. One size fits all: can we train one denoiser for all noise levels? Paper presented at: International Conference on Machine Learning; 2020:3576-3586.
Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.
Xin L, Wang D, Shi W. FISTA-CSNet: a deep compressed sensing network by unrolling iterative optimization algorithm. The Visual Computer. 2022;1-17. Accessed June 4 2023. https://link.springer.com/article/10.1007/s00371-022-02583-2