Deep learning based synthetic-CT generation in radiotherapy and PET: A review.
artificial intelligence
convolutional neural networks
deep learning
image synthesis
machine learning
pseudo-CT
radiotherapy
Journal
Medical physics
ISSN: 2473-4209
Titre abrégé: Med Phys
Pays: United States
ID NLM: 0425746
Informations de publication
Date de publication:
Nov 2021
Nov 2021
Historique:
revised:
06
06
2021
received:
09
02
2021
accepted:
13
07
2021
pubmed:
19
8
2021
medline:
18
11
2021
entrez:
18
8
2021
Statut:
ppublish
Résumé
Recently,deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: (i) to replace computed tomography in magnetic resonance (MR) based treatment planning, (ii) facilitate cone-beam computed tomography based image-guided adaptive radiotherapy, and (iii) derive attenuation maps for the correction of positron emission tomography. Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarizing the achievements. Lastly, the statistics of all the cited works from various aspects were analyzed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.
Types de publication
Journal Article
Review
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
6537-6566Informations de copyright
© 2021 The Authors. Medical Physics published by Wiley Periodicals LLC on behalf of American Association of Physicists in Medicine.
Références
Husband J, Reznek RH, Husband JE. Imaging in Oncology. CRC Press; 2016. https://doi.org/10.1201/9780203732656
Beaton L, Bandula S, Gaze MN, Sharma RA. How rapid advances in imaging are defining the future of precision radiation oncology. Br J Cancer. 2019;120:779-790.
Verellen D, De Ridder M, Linthout N, Tournel K, Soete G, Storme G. Innovations in image-guided radiotherapy. Nat Rev Cancer. 2007;7:949-960.
Jaffray DA. Image-guided radiotherapy: from current concept to future perspectives. Nat Rev Clin Oncol. 2012;9:688.
Seco J, Spadea MF. Imaging in particle therapy: state of the art and future perspective. Acta Oncol. 2015;54:1254-1258.
IAEA. Radiotherapy in Cancer Care: Facing the Global Challenge. International Atomic Energy Agency; 2017. http://www-pub.iaea.org/MTCD/Publications/PDF/P1638_web.pdf
Seco J, Evans PM. Assessing the effect of electron density in photon dose calculations. Med Phys. 2006;33:540-552.
Unterrainer M, Eze C, Ilhan H, et al. Recent advances of PET imaging in clinical radiation oncology. Radiat Oncol. 2020;15:1:15.
Dirix P, Haustermans K, Vandecaveye V. The value of magnetic resonance imaging for radiotherapy planning. Semin Radiat Oncol. 2014;24:151-159.
Schmidt MA, Payne GS. Radiotherapy planning using MRI. Phys Med Biol. 2015;60:R323.
Devic S. MRI simulation for radiotherapy treatment planning. Med Phys. 2012;39:6701.
Nyholm T, Nyberg M, Karlsson MG, Karlsson M. Systematisation of spatial uncertainties for comparison between a MR and a CT-based radiotherapy workflow for prostate treatments. Radiat Oncol. 2009;4:1-9.
Ulin K, Urie MM, Cherlow JM. Results of a multi-institutional benchmark test for cranial CT/MR image registration. Int J Radiat Oncol Biol Phys. 2010;77:1584-1589.
Fraass BA, McShan DL, Diaz RF, et al. Integration of magnetic resonance imaging into radiation therapy treatment planning: I. Technical considerations. Int J Radiat Oncol Biol Phys. 1987;13:1897-908.
Lee YK, Bollet M, Charles-Edwards G, et al. Radiotherapy treatment planning of prostate cancer using magnetic resonance imaging alone. Radiother Oncol. 2003;66:203-216.
Nyholm T, Jonsson J. Counterpoint: opportunities and challenges of a magnetic resonance imaging-only radiotherapy work flow. Semin Radiat Oncol. 2014;24:175-80.
Kapanen M, Collan J, Beule A, Seppälä T, Saarilahti K, Tenhunen M. Commissioning of MRI-only based treatment planning procedure for external beam radiotherapy of prostate. Magn Reson Med. 2013;70:127-35.
Owrangi AM, Greer PB, Glide-Hurst CK. MRI-only treatment planning: benefits and challenges. Phys Med Biol. 2018;63:05TR01.
Karlsson M, Karlsson MG, Nyholm T, Amies C, Zackrisson B. Dedicated magnetic resonance imaging in the radiotherapy clinic. Int. J. Radiat. Oncol. Biol. Phys. 2009;74:644-651.
Lagendijk JJW, Raaymakers BW, Berg CAT, Moerland MA, Philippens ME, Van Vulpen M. MR guidance in radiotherapy. Phys Med Biol. 2014;59:R349.
Jonsson JH, Karlsson MG, Karlsson M, Nyholm T. Treatment planning using MRI data: an analysis of the dose calculation accuracy for different treatment regions. Radiat Oncol. 2010;5:62.
Edmund JM, Nyholm T. A review of substitute CT generation for MRI-only radiation therapy. Radiat Oncol. 2017;12:28.
Johnstone E, Wyatt JJ, Henry AM, et al. Systematic review of synthetic computed tomography generation methodologies for use in magnetic resonance imaging-only radiation therapy. Int J Radiat Oncol Biol Phys. 2018;100:199-217.
Wafa B, Moussaoui A. A review on methods to estimate a CT from MRI data in the context of MRI-alone RT. Med Tech J. 2018;2:150-178.
Kerkmeijer LGW, Maspero M, Meijer GJ, van der Voort van Zyp JRN, de Boer HCJ, van den Berg CAT. Magnetic Resonance Imaging only Workflow for Radiotherapy Simulation and Planning in Prostate Cancer. Clin Oncol. 2018;30:(11):692-701. https://doi.org/10.1016/j.clon.2018.08.009
Bird D, Henry AM, Sebag-Montefiore D, Buckley DL, Al-Qaisieh B, Speight R. A systematic review of the clinical implementation of pelvic magnetic resonance imaging-only planning for external beam radiation therapy. Int J Radiat Oncol Biol Phys. 2019;105:479-492.
Thorwarth D, Low DA. Technical challenges of real-time adaptive MR-guided radiotherapy. Front Oncol. 2021;11.
Hoffmann A, Oborn B, Moteabbed M, et al. MR-guided proton therapy: a review and a preview. Radiat Oncol. 2020;15.
Taasti VT, Klages P, Parodi K, Muren LP. Developments in deep learning based corrections of cone beam computed tomography to enable dose calculations for adaptive radiotherapy. Physics and Imaging in Radiat Oncol. 2020;15:77-79.
Zhu L, Wang J, Xing L. Noise suppression in scatter correction for cone-beam CT. Med Phys. 2009;36:(3):741-752. https://doi.org/10.1118/1.3063001
Zhu L, Xie Y, Wang J, Xing L. Scatter correction for cone-beam CT in radiation therapy. Med Phys. 2009;36:(6Part1):2258-2268. https://doi.org/10.1118/1.3130047
Mehranian A, Arabi H, Zaidi H. Vision 20/20: magnetic resonance imaging-guided attenuation correction in PET/MRI: challenges, solutions, and opportunities. Med Phys. 2016;43:1130-1155.
Mecheter I, Alic L, Abbod M, Amira A, Ji J. MR image-based attenuation correction of brain PET imaging: review of literature on machine learning approaches for segmentation. Journal of Digital Imaging. 2020;1-18.
Catana C. Attenuation correction for human PET/MRI studies. Phys Med Biol. 2020;65:TR02.
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436-444.
Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep Learning. Vol. 2 in Adaptive Computation and Machine Learning. MIT Press; 2016.
Meyer P, Noblet V, Mazzara C, Lallement A. Survey on deep learning for radiotherapy. Comp Biol Med. 2018;98:126-146.
Sahiner B, Pezeshk A, Hadjiiski LM, et al. Deep learning in medical imaging and radiation therapy. Med Phys. 2018;46:e1-e36.
Boon I, Yong TA, Boon C. Assessing the role of artificial intelligence (AI) in clinical oncology: utility of machine learning in radiotherapy target volume delineation. Medicines. 2018;5:131.
Wang C, Zhu X, Hong JC, Zheng D. Artificial intelligence in radiotherapy treatment planning: present and future. Tech Cancer Res Treat. 2019;18:153303381987392.
Boldrini L, Bibault JE, Masciocchi C, Shen Y, Bittner M-I. Deep learning: a review for the radiation oncologist. Front Oncol. 2019;9.
Jarrett D, Stride E, Vallis K, Gooding MJ. Applications and limitations of machine learning in radiation oncology. Brit J Radiol. 2019;92:20190001.
Kiser KJ, Fuller CD, Reed VK. Artificial intelligence in radiation oncology treatment planning: a brief overview. J Med Art Intell. 2019;2:9.
Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neur Inf Proc Syst. 2012;25:1097-1105.
Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60-88.
Nie D, Cao X, Gao Y, Wang L, Shen D. Estimating CT image from MRI data using 3D fully convolutional networks. 2016;2016:170-178.
Lee JS. A Review of Deep-Learning-Based Approaches for Attenuation Correction in Positron Emission Tomography. IEEE Trans Radiat Plasma Med Sci. 2021;5:(2):160-184. https://doi.org/10.1109/trpms.2020.3009269
Yu B, Wang Y, Wang L, Shen D, Zhou L. Medical Image Synthesis via Deep Learning. Springer International Publishing; 2020:23-44.
Wang T, Lei Y, Fu Y, et al. A review on medical imaging synthesis using deep learning and its clinical applications. J Appl Clin Med Phys. 2021;22:11-36.
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436-444.
Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2015:234-241.
Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. Adv Neural Inform Process Syst. 2014;27:2672-2680.
Isola P, Zhu J-Y, Zhou T & Efros AA. Image-to-image translation with conditional adversarial networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017; Honolulu, HI:1125-1134. https://doi.org/10.1109/CVPR.2017.632
Wu X, Xu K, Hall P. A survey of image synthesis and editing with generative adversarial networks. Tsinghua Sci Technol. 2017;22:(6):660-674. https://doi.org/10.23919/tst.2017.8195348
Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA. Generative Adversarial Networks: An Overview. IEEE Signal Process Mag. 2018;35:(1):53-65. https://doi.org/10.1109/msp.2017.2765202
Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: a review. Med Image Anal. 2019;58:101552.
Zhu J-Y, Park T, Isola P & Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision; 2017:2223-2232. https://doi.org/10.1109/ICCV.2017.244
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Imag Proc. 2004;13:600-612.
Low DA. Gamma dose distribution evaluation tool. J Phys. 2010;250:012071.
Clasie BM, Sharp GC, Seco J, Flanz JB, Kooy HM. Numerical solutions of the γ-index in two and three dimensions. Phys Med Biol. 2012;57:6981.
Hussein M, Clark CH, Nisbet A. Challenges in calculation of the gamma index in radiotherapy-towards good practice. Phys Med. 2017;36:1-11.
Drzymala RE, Mohan R., Brewster L., et al. Dose-volume histograms. Int J Radiat Oncol Biol Phys. 1991;21:71-78.
Paganetti H. Range uncertainties in proton therapy and the role of Monte Carlo simulations. Phys Med Biol. 2012;57:R99.
Pileggi G, Speier C, Sharp GC, et al. Proton range shift analysis on brain pseudo-CT generated from T1 and T2 MR. Acta Oncol. 2018;57:1521-1531.
Andres EA, Fidon L, Vakalopoulou M, et al. Dosimetry-driven quality measure of brain pseudo computed tomography generated from deep learning for MRI-only radiotherapy treatment planning. Int J Radiat Oncol Biol Phys. 2020;108:813-823.
Eckl M, Hoppen L, Sarria GR, et al. Evaluation of a cycle-generative adversarial network-based cone-beam CT to synthetic CT conversion algorithm for adaptive radiation therapy. Phys Med. 2020;80:308-316.
Peng Y, Chen S, Qin A, et al. Magnetic resonance-based synthetic computed tomography images generated using generative adversarial networks for nasopharyngeal carcinoma radiotherapy treatment planning. Radiother Oncol. 2020;150:217-224.
Qian P, Xu K, Wang T, Zheng Q, Yang H, Baydoun A, Zhu J, Traughber B, Muzic RF. Estimating CT from MR Abdominal Images Using Novel Generative Adversarial Networks. J Grid Comput. 2020;18:(2):211-226. https://doi.org/10.1007/s10723-020-09513-3
Xu K, Cao J, Xia K, et al. Multichannel residual conditional GAN-leveraged abdominal pseudo-CT generation via Dixon MR images. IEEE Access. 2019;7:163823-163830.
Ladefoged CN, Marner L, Hindsholm A, Law I, Højgaard L, Andersen FL. Deep learning based attenuation correction of PET/MRI in pediatric brain tumor patients: evaluation in a clinical setting. Front Neurosci. 2019;12:1005.
Maspero M, Bentvelzen LG, Savenije MHF, et al. Deep learning-based synthetic CT generation for paediatric brain MR-only photon and proton radiotherapy. Radiother Oncol. 2020;153:197-204. https://doi.org/10.1016/j.radonc.2020.09.029
Florkow MC, Guerreiro F, Zijlstra F, et al. Deep learning-enabled MRI-only photon and proton therapy treatment planning for paediatric abdominal tumours. Radiother Oncol. 2020;153:220-227.
Jeon W, An HJ, Kim J-i, et al. Preliminary application of synthetic computed tomography image generation from magnetic resonance image using deep-learning in breast cancer patients. J Radiat Prot Res. 2019;44:149-155.
Bradshaw TJ, Zhao G, Jang H, Liu F, McMillan AB. Feasibility of deep learning-based PET/MR attenuation correction in the pelvis using only diagnostic MR images. Tomography. 2018;4:138.
Fu J, Singhrao K, Cao M, et al. Generation of abdominal synthetic CTs from 035 T MR images using generative adversarial networks for MR-only liver radiotherapy. Biom Phys Eng Express. 2020;6:015033.
Li Y, Li W, Xiong J, Xia J, Xie Y. Comparison of Supervised and Unsupervised Deep Learning Methods for Medical Image Synthesis between Computed Tomography and Magnetic Resonance Images. Biomed Res Int. 2020;2020:1-9. https://doi.org/10.1155/2020/5193707
Xu L, Zeng X, Zhang H, Li W, Lei J, Huang Z. BPGAN: bidirectional CT-to-MRI prediction using multi-generative multi-adversarial nets with spectral normalization and localization. Neural Netw. 2020;128:82-98.
Fu J, Yang Y, Singhrao K, et al. Deep learning approaches using 2D and 3D convolutional neural networks for generating male pelvic synthetic computed tomography from magnetic resonance imaging. Med Phys. 2019;46:3788-3798.
Neppl S, Landry G, Kurz C, et al. Evaluation of proton and photon dose distributions recalculated on 2D and 3D Unet-generated pseudo-CTs from T1-weighted MR head scans. Acta Oncol. 2019;58:1429-1434.
Fetty L, Bylund M, Kuess P, et al. Latent space manipulation for high-resolution medical image synthesis via the StyleGAN. Zeits Med Phys. 2020;30.
Xiang L, Wang Q, Nie D, et al. Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image. Med Imag Anal. 2018;47:31-44.
Cusumano D, Lenkowicz J, Votta C, et al. A deep learning approach to generate synthetic CT in low field MR-guided adaptive radiotherapy for abdominal and pelvic cases. Radiother Oncol. 2020;153:205-212.
Harms J, Lei Y, Wang T, et al. Paired cycle-GAN-based image correction for quantitative cone-beam computed tomography. Med Phys. 2019;46:3998-4009.
Maspero M, Houweling AC, Savenije MHF, et al. A single neural network for cone-beam computed tomography-based radiotherapy of head-and-neck, lung and breast cancer. Phys Imag Radiat Oncol. 2020;14:24-31.
Zhang Y, Yue N, Su MY, Liu B, Ding Y, Zhou Y, Wang H, Kuang Y, Nie K. Improving CBCT quality to CT level using deep learning with generative adversarial network. Med Phys. 2021;48:(6):2816-2826. https://doi.org/10.1002/mp.14624
Maspero M, Savenije MHF, Dinkla AM, Seevinck PR, Intven MPW, Jurgenliemk-Schulz IM, Kerkmeijer LGW, van den Berg CAT. Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy. Phys Med Biol. 2018;63:(18):185001. https://doi.org/10.1088/1361-6560/aada6d
Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Med Phys. 2017;44:1408-1419.
Emami H, Dong M, Nejad-Davarani SP, Glide-Hurst CK. Generating synthetic CTs from magnetic resonance images using generative adversarial networks. Med Phys. 2018;45:3627-3636.
Jin C-B, Kim H, Liu M, et al. Deep CT to MR synthesis using paired and unpaired data. Sensors. 2019;19:2361.
Lei Y, Harms J, Wang T, et al. MRI-only based synthetic CT generation using dense cycle consistent generative adversarial networks. Med Phys. 2019;46:3565-3581.
Yang H, Sun J, Carass A, et al. Unsupervised MR-to-CT synthesis using structure-constrained CycleGAN. IEEE Trans Med Imag. 2020;39:4249-4261.
Massa H, Johnson J, McMillan A. Comparison of deep learning synthesis of synthetic CTs using clinical MRI inputs. Phys Med Biol. 2020;65:NT03.
Wang Y, Liu C, Zhang X, Deng W. Synthetic CT generation based on T2 weighted MRI of nasopharyngeal carcinoma (NPC) using a deep convolutional neural network (DCNN). Front Oncol. 2019;9.
Tie X, Lam S-K, Zhang Y, Lee K-H, Au K-H, Cai J. Pseudo-CT generation from multi-parametric MRI using a novel multi-channel multi-path conditional generative adversarial network for nasopharyngeal carcinoma patients. Med Phys. 2020;47:1750-1762.
Kearney V, Ziemer BP, Perry A, Wang Y, Chan JW, Ma L, Morin O, Yom SS, Solberg TD. Attention-Aware Discrimination for MR-to-CT Image Translation Using Cycle-Consistent Generative Adversarial Networks. Radiol Artif Intell. 2020;2:(2):e190027. https://doi.org/10.1148/ryai.2020190027
Largent A, Marage L, Gicquiau I, et al. Head-and-neck MRI-only radiotherapy treatment planning: from acquisition in treatment position to pseudo-CT generation. Cancer Radiother. 2020;24:288-297.
Su P, Guo S, Roys S, et al. Transcranial MR imaging-guided focused ultrasound interventions using deep learning synthesized CT. Am J Neurorad. 2020;41:1841-1848.
Florkow MC, Zijlstra F, Willemsen K, Maspero M, Berg CAT, Kerkmeijer LGW, Castelein RM, Weinans H, Viergever MA, Stralen M, Seevinck PR. Deep learning-based MR-to-CT synthesis: The influence of varying gradient echo-based MR images as input channels. Magn Reson Med. 2020;83:(4):1429-1441. https://doi.org/10.1002/mrm.28008
Bahrami A, Karimian A, Fatemizadeh E, Arabi H, Zaidi H. A new deep convolutional neural network design with efficient learning capability: application to CT image synthesis from MRI. Med Phys. 2020;47:5158-5171.
Liu Y, Lei Y, Wang Y, et al. MRI-based treatment planning for proton radiotherapy: dosimetric validation of a deep learning-based liver synthetic CT generation method. Phys Med Biol. 2019;64:145015.
Liu L, Johansson A, Cao Y, Dow J, Lawrence TS, Balter JM. Abdominal synthetic CT generation from MR Dixon images using a U-net trained with “semi-synthetic” CT data. Phys Med Biol. 2020;65:125001.
Dinkla AM, Wolterink JM, Maspero M, et al. MR-only brain radiation therapy: dosimetric evaluation of synthetic CTs generated by a dilated convolutional neural network. Int J Radiat Oncol Biol Phys. 2018;102:801-812.
Liu F, Yadav P, Baschnagel AM, McMillan AB. MR-based treatment planning in radiation therapy using a deep learning approach. J Appl Clin Med Phys. 2019;20:105-114.
Kazemifar S, McGuire S, Timmerman R, et al. MRI-only brain radiotherapy: assessing the dosimetric accuracy of synthetic CT images generated using a deep learning approach. Radiother Oncol. 2019;136:56-63.
Shafai-Erfani G, Lei Y, Liu Y, et al. MRI-based proton treatment planning for base of skull tumors. Int J Part Ther. 2019;6:12-25.
Gupta D, Kim M, Vineberg KA, Balter JM. Generation of synthetic CT images from MRI for treatment planning and patient positioning using a 3-channel U-net trained on sagittal images. Front Oncol. 2019;9:964.
Spadea MF, Pileggi G, Zaffino P, et al. Deep convolution neural network (DCNN) multiplane approach to synthetic CT generation from MR images-application in brain proton therapy. Int J Radiat Oncol Biol Phys. 2019;105:495-503.
Koike Y, Akino Y, Sumida I, et al. Feasibility of synthetic computed tomography generated with an adversarial network for multi-sequence magnetic resonance-based brain radiotherapy. J Radiat Res. 2020;61:92-103.
Kazemifar S, Barragán MAM, Souris K, et al. Dosimetric evaluation of synthetic CT generated with GANs for MRI-only proton therapy treatment planning of brain tumors. J Appl Clin Med Phys. 2020;21:76-86.
Chen S, Qin A, Zhou D, Yan D. U-net-generated synthetic CT images for magnetic resonance imaging-only prostate intensity-modulated radiation therapy treatment planning. Med Phys. 2018;45:5659-5665.
Arabi H, Dowling JA, Burgos N, et al. Comparative study of algorithms for synthetic CT generation from MRI: consequences for MRI-guided radiation planning in the pelvic region. Med Phys. 2018;45:5218-5233.
Liu Y, Lei Y, Wang Y, et al. Evaluation of a deep learning-based pelvic synthetic CT generation technique for MRI-based prostate proton treatment planning. Phys Med Biol. 2019;64:205022.
Largent A, Barateau A, Nunes J-C, et al. Comparison of deep learning-based and patch-based methods for pseudo-CT generation in MRI-based prostate dose planning. Int J Radiat Oncol Biol Phys. 2019;105:1137-1150.
Boni KNB, Klein J, Vanquin L, et al. MR to CT synthesis with multicenter data in the pelvic area using a conditional generative adversarial network. Phys Med Biol. 2020;65:075002.
Fetty L, Löfstedt T, Heilemann G, et al. Investigating conditional GAN performance with different generator architectures, an ensemble model, and different MR scanners for MR-sCT conversion. Phys Med Biol. 2020;65:5004.
Bird D, Nix MG, McCallum H, et al. Multicentre, deep learning, synthetic-CT generation for ano-rectal MR-only radiotherapy treatment planning. Radiother Oncol. 2021;156:23-28.
Dinkla AM, Florkow MC, Maspero M, Savenije MHF, Zijlstra F, Doornaert PAH, Stralen M, Philippens MEP, Berg CAT, Seevinck PR. Dosimetric evaluation of synthetic CT for head and neck radiotherapy generated by a patch-based three-dimensional convolutional neural network. Med Phys. 2019;46:(9):4095-4104. https://doi.org/10.1002/mp.13663
Klages P, Benslimane I, Riyahi S, et al. Patch-based generative adversarial neural network models for head and neck MR-only planning. Med Phys. 2020;47:626-642.
Qi M, Li Y, Wu A, et al. Multi-sequence MR image-based synthetic CT generation using a generative adversarial network for head and neck MRI-only radiotherapy. Med Phys. 2020;47:1880-1894.
Thummerer A, Jong BA, Zaffino P, et al. Comparison of the suitability of CBCT-and MR-based synthetic CTs for daily adaptive proton therapy in head and neck patients. Phys Med Biol. 2020;65:235036.
Olberg S, Zhang H, Kennedy WR, et al. Synthetic CT reconstruction using a deep spatial pyramid convolutional framework for MR-only breast radiotherapy. Med Phys. 2019;46:4135-4147.
Florkow MC, Zijlstra F, Kerkmeijer LGW, Maspero M, van den Berg CAT, van Stralen M, Seevinck PR, et al. The impact of MRI-CT registration errors on deep learning-based synthetic CT generation. In: Medical Imaging 2019: Image Processing, Vol. 10949. International Society for Optics and Photonics; 2019:1094938. https://doi.org/10.1117/12.2512747
Reinhold JC, Dewey BE, Carass A, Prince JL. Evaluating the impact of intensity normalization on MR image synthesis. In: Angelini ED, Landman BA, eds. Medical Imaging 2019: Image Processing. SPIE; 2019.
Kida S, Nakamoto T, Nakano M, et al. Cone beam computed tomography image quality improvement using a deep convolutional neural network. Cureus. 2018;10:e2548.
Chen L, Liang X, Shen C, Jiang S, Wang J. Synthetic CT generation from CBCT images via deep learning. Med Phys. 2020;47:1115-1125.
Kida S, Kaji S, Nawa K, et al. Visual enhancement of cone-beam CT by use of CycleGAN. Med Phys. 2020;47:998-1010.
Yuan N, Dyer B, Rao S, et al. Convolutional neural network enhancement of fast-scan low-dose cone-beam CT images for head and neck radiotherapy. Phys Med Biol. 2020;65:035003.
Liang X, Chen L, Nguyen D, et al. Generating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy. Phys Med Biol. 2019;64:125002.
Li Y, Zhu J, Liu Z, et al. A preliminary study of using a deep convolution neural network to generate synthesized CT images based on CBCT for adaptive radiotherapy of nasopharyngeal carcinoma. Phys Med Biol. 2019;64:145010.
Barateau A, De Crevoisier R, Largent A, et al. Comparison of CBCT-based dose calculation methods in head and neck cancer radiotherapy: from Hounsfield unit to density calibration curve to deep learning. Med Phys. 2020;47:4683-4693.
Liu Y, Lei Y, Wang T, et al. CBCT-based synthetic CT generation using deep-attention cycleGAN for pancreatic adaptive radiotherapy. Med Phys. 2020;47:2472-2483.
Landry G, Hansen D, Kamp F, et al. Comparing Unet training with three different datasets to correct CBCT images for prostate radiotherapy dose calculations. Phys Med Biol. 2019;64. https://doi.org/10.1088/1361-6560/aaf496.
Kurz C, Maspero M, Savenije MHF, et al. CBCT correction using a cycle-consistent generative adversarial network and unpaired training to enable photon and proton dose calculation. Phys Med Biol. 2019;64:225004.
Thummerer A, Zaffino P, Meijers A, et al. Comparison of CBCT based synthetic CT methods suitable for proton dose calculations in adaptive proton therapy. Phys Med Biol. 2020;65:095002.
Radford A, Metz L & Chintala S Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint 2015. arXiv:1511.06434.
Karras T, Aila T, Laine S & Lehtinen J Progressive growing of gans for improved quality, stability, and variation. Preprint 2017. arXiv:1710.10196.
Oktay O, Schlemper J & Folgoc LL et al. Attention U-net: learning where to look for the pancreas. Preprint 2018. arXiv:1804.03999.
Leynes AP, Yang J, Wiesinger F, et al. Direct pseudo-CT generation for pelvis PET/MRI attenuation correction using deep convolutional neural networks with multi-parametric MRI: zero echo-time and Dixon deep pseudo-CT (ZeDD-CT). J Nucl Med. 2017;59:852-858.
Baydoun A, Xu K, Yang H, Zhou F, Heo JU, Jones RS, Avril N, Traughber MS, Traughber BJ, Qian P, Muzic RF. Dixon-based thorax synthetic CT generation using Generative Adversarial Network. Artif Intell Med. 2020;3-4:100010. https://doi.org/10.1016/j.ibmed.2020.100010
Gong K, Yang J, Kim K, El FG, Seo Y, Li Q. Attenuation correction for brain PET imaging using deep neural network based on Dixon and ZTE MR images. Phys Med Biol. 2018;63:125011.
Jang H, Liu F, Zhao G, Bradshaw T, McMillan AB. Deep learning based MRAC using rapid ultrashort echo time imaging. Med Phys. 2018;45:3697-3704.
Torrado-Carvajal A, Vera-Olmos J, Izquierdo-Garcia D, et al. Dixon-VIBE deep learning (DIVIDE) pseudo-CT synthesis for pelvis PET/MR attenuation correction. J Nucl Med. 2019;60:429-435.
Blanc-Durand P, Khalife M, Sgard B, et al. Attenuation correction using 3D deep convolutional neural network for brain 18F-FDG PET/MR: comparison with Atlas, ZTE and CT based attenuation correction. PloS One. 2019;14:e0223141.
Gong K, Han PK, Johnson K, El Fakhri G, Ma C, Li Q. Attenuation correction using deep learning and integrated UTE/multi-echo Dixon sequence: evaluation in amyloid and tau PET imaging. Eur J Nucl Med Mol Imaging. 2021;48:1-11.
Pozaruk A, Pawar K, Li S, et al. Augmented deep learning model for improved quantitative accuracy of MR-based PET attenuation correction in PSMA PET-MRI prostate imaging. Eur J Nucl Med Mol Imaging. 2021;48:9-20.
Gong K, Yang J, Larson PEZ, et al. MR-based attenuation correction for brain PET using 3D cycle-consistent adversarial network. IEEE Trans Radiat Plasma Med Sci. 2021;5:185-192.
Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology. 2018;286:676-684.
Arabi H, Zeng G, Zheng G, Zaidi H. Novel adversarial semantic structure deep learning for MRI-guided attenuation correction in brain PET/MRI. Eur J Nucl Med Mol Imaging. 2019;46:2746-2759.
Spuhler KD, Gardus J, Gao Y, DeLorenzo C, Parsey R, Huang C. Synthesis of patient-specific transmission data for PET attenuation correction for PET/MRI neuroimaging using a convolutional neural network. J Nucl Med. 2019;60:555-560.
Liu F, Jang H, Kijowski R, Zhao G, Bradshaw T, McMillan AB. A deep learning approach for 18F-FDG PET attenuation correction. EJNMMI Phys. 2018;5:(1). https://doi.org/10.1186/s40658-018-0225-8
Dong X, Wang T, Lei Y, et al. Synthetic CT generation from non-attenuation corrected PET images for whole-body PET imaging. Phys Med Biol. 2019;64:215016.
Armanious K, Hepp T, Küstner T, et al. Independent attenuation correction of whole body [18F] FDG-PET using a deep learning approach with generative adversarial networks. EJNMMI Res. 2020;10:1-9.
Simonyan K & Zisserman A Very deep convolutional networks for large-scale image recognition. Preprint 2014. arXiv:1409.1556.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016:770-778.
Van Dyk JJ, ed. The Modern Technology of Radiation Oncology. Medical Physics Publisher; 2020.
Stemkens B, Paulson ES, Tijssen RHN. Nuts and bolts of 4D-MRI for radiotherapy. Phys Med Biol. 2018;63:21TR01.
Paganelli C, Whelan B, Peroni M, et al. MRI-guidance for motion management in external beam radiotherapy: current status and future challenges. Phys Med Biol. 2018;63:22TR03.
Freedman JN, Bainbridge HE, Nill S, et al. Synthetic 4D-CT of the thorax for treatment plan adaptation on MR-guided radiotherapy systems. Phys Med Biol. 2019;64:115005.
Goodman TR, Mustafa A, Rowe E. Pediatric CT radiation exposure: where we were, and where we are now. Pediatric Radiol. 2019;49:469-478.
Walker A, Metcalfe P, Liney G, et al. MRI geometric distortion: impact on tangential whole-breast IMRT. J Appl Clin Med Phys. 2016;17:7-19.
Gustafsson C, Nordström F, Persson E, Brynolfsson J, Olsson LE. Assessment of dosimetric impact of system specific geometric distortion in an MRI only based radiotherapy workflow for prostate. Phys Med Biol. 2017;62:2976-2989.
Maspero M, Tyyger MD, Tijssen RHN, Seevinck PR, Intven MPW, van den Berg CAT. Feasibility of magnetic resonance imaging-only rectum radiotherapy with a commercial synthetic computed tomography generation solution. Phys Imaging Radiat Oncol. 2018;7:58-64. https://doi.org/10.1016/j.phro.2018.09.002
Lagendijk JJW, Raaymakers BW, Raaijmakers AJE, et al. MRI/linac integration. Radiother Oncol. 2008;86:25-29.
Fallone BG. The rotating biplanar linac-magnetic resonance imaging system. 2014;24:200-202.
Mutic S, Dempsey JF. The ViewRay system: magnetic resonance-guided and controlled radiotherapy. 2014;24:196-199.
Keall PJ, Barton M, Crozier S, et al. The Australian magnetic resonance imaging-linac program. Semin Radiat Oncol. 2014;24:203-206.
Jaffray DA, Carlone MC, Milosevic MF, et al. A facility for magnetic resonance-guided radiation therapy. Semin Radiat Oncol. 2014;24:193-195.
Winkel D, Bol GH, Kroon PS, et al. Adaptive radiotherapy: the Elekta unity MR-linac concept. Clin Transl Radiat Oncol. 2019;18:54-59.
Hall WA, Paulson ES, Heide UA, et al. The transformation of radiation oncology using real-time magnetic resonance guidance: a review. Eur J Cancer. 2019;122:42-52.
Groot Koerkamp ML, de Hond YJM, Maspero M, Kontaxis C, Mandija S, Vasmel JE, Charaghvandi RK, Philippens MEP, van Asselen B, van den Bongard HJGD, Hackett SS, Houweling AC. Synthetic CT for single-fraction neoadjuvant partial breast irradiation on an MRI-linac. Phys Med Biol. 2021;66:(8):085010. https://doi.org/10.1088/1361-6560/abf1ba
Boda-Heggemann J, Lohr F, Wenz F, Flentje M, Guckenberger M. kV cone-beam CT-based IGRT. Strahlen Onkol. 2011;187:284-291.
Elstrøm UV, Muren LP, Petersen JB, Grau C. Evaluation of image quality for different kV cone-beam CT acquisition and reconstruction methods in the head and neck region. Acta Oncol. 2011;50:908-917.
Peroni M, Ciardo D, Spadea MF, et al. Automatic segmentation and online virtualCT in head-and-neck adaptive radiation therapy. Int J Radiat Oncol Biol Phys. 2012;84:e427-e433.
Veiga C, Alshaikhi J, Amos R, et al. Cone-beam computed tomography and deformable registration-based “dose of the day” calculations for adaptive proton therapy. Int J Part Ther. 2015;2:404-414.
Park Y-K, Sharp GC, Phillips J, Winey BA. Proton dose calculation on scatter-corrected CBCT image: feasibility study for adaptive proton therapy. Med Phys. 2015;42:4449-4459.
Kurz C, Nijhuis R, Reiner M, et al. Feasibility of automated proton therapy plan adaptation for head and neck tumors using cone beam CT images. Radiat Oncol. 2016;11:1-9.
Arai K, Kadoya N, Kato T, et al. Feasibility of CBCT-based proton dose calculation using a histogram-matching algorithm in proton beam therapy. Phys Med. 2017;33:68-76.
Gomà C, Almeida IP, Verhaegen F. Revisiting the single-energy CT calibration for proton therapy treatment planning: a critical look at the stoichiometric method. Phys Med Biol. 2018;63:235011.
Harms J, Lei Y, Wang T, et al. Cone-beam CT-derived relative stopping power map generation via deep learning for proton radiotherapy. Med Phys. 2020;47:4416-4427.
Hansen DC, Landry G, Kamp F, et al. ScatterNet: a convolutional neural network for cone-beam CT intensity correction. Med Phys. 2018;45:4916-4926.
Wang G, Ye JC, Mueller K, Fessler JA. Image reconstruction is a new frontier of machine learning. IEEE Trans Med Imag. 2018;37:1289-1296.
Wang G, Zhang Y, Ye X, Mou X. Machine Learning for Tomographic Imaging. IOP Publishing; 2019.
Li Y, Li K, Zhang C, Montoya J, Chen G-H. Learning to reconstruct computed tomography images directly from sinogram data under a variety of data acquisition conditions. IEEE Trans Med Imag. 2019;38:2469-2481.
Maier AK, Syben C, Stimpel B, et al. Learning with known operators reduces maximum error bounds. Nat Machine Intell. 2019;1:373-380.
Lønning K, Putzky P, Sonke J-J, Reneman L, Caan MWA, Welling M. Recurrent inference machines for reconstructing heterogeneous MRI data. Med Image Anal. 2019;53:64-78.
Izquierdo-Garcia D, Sawiak SJ, Knesaurek K, et al. Comparison of MR-based attenuation correction and CT-based attenuation correction of whole-body PET/MR imaging. Eur J Nucl Med Mol Imag. 2014;41:1574-1584.
Shiri I, Arabi H, Geramifar P, et al. Deep-JASC: joint attenuation and scatter correction in whole-body 18F-FDG PET using a deep residual network. Eur J Nucl Med Mol Imag. 2020;47:2533-2548.
Shorten C, Khoshgoftaar TM. A survey on Image Data Augmentation for Deep Learning. J Big Data. 2019;6:(1). https://doi.org/10.1186/s40537-019-0197-0
Li Z, Kamnitsas K, Glocker B. Overfitting of neural nets under class imbalance: analysis and improvements for segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019:402-410.
Hang Z, Orazio G, Iuri F, Jan K. Loss Functions for neural networks for image processing. CoRR. 2015;abs/1511.08861.
Wolterink JM, Dinkla AM, Savenije Mark H, Seevinck PR, Berg Cornelis AT, Išgum Ivana. Deep MR to CT Synthesis Using Unpaired Data. Springer; 2017:14-23.
Rehman A, Khan FG. A deep learning based review on abdominal images. Multimed Tools Appl. 2020;https://doi.org/10.1007/s11042-020-09592-0
Singh SP, Wang L, Gupta S, Goli H, Padmanabhan P, Gulyás B. 3D deep learning on medical images: a review. Sensors. 2020;20:5097.
Kamnitsas K, Ferrante E, Parisot S, et al. DeepMedic for brain tumor segmentation. In: International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Springer; 2016:138-149.
Schlemper J, Oktay O, Schaap M, et al. Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal. 2019;53:197-207.
Keeling P, Clark J, Finucane S. Challenges in the clinical implementation of precision medicine companion diagnostics. Expert Rev Mol Diagn. 2020;20:593-599.
Bertholet J, Anastasi G, Noble D, et al. Patterns of practice for adaptive and real-time radiation therapy (POP-ART RT) part II: offline and online plan adaption for interfractional changes. Radiother Oncol. 2020;153:88-96.
Palmér E, Karlsson A, Nordström F, et al. Synthetic computed tomography data allows for accurate absorbed dose calculations in a magnetic resonance imaging only workflow for head and neck radiotherapy. Phys Imag Radiat Oncol. 2021;17:36-42.
Council of European Union. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC; 2017. http://data.europa.eu/eli/reg/2017/745/oj.
Fiorino C, Guckenberger M, Schwarz M, Heide UA, Heijmen B. Technology-driven research for radiotherapy innovation. Mol Oncol. 2020;14:1500-1513.
Beckers R, Kwade Z, Zanca F. The EU medical device regulation: implications for artificial intelligence-based medical device software in medical physics. Phys Med. 2021;83:1-8.
Liesbeth V, Michaël C, Anna M D, et al. Overview of artificial intelligence-based applications in radiotherapy: recommendations for implementation and quality assurance. Radiother Oncol. 2020;153:55-66.
Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Brit Med J. 2020;370:m3164.
Dowling JA, Korhonen J. MR-only methodology. In: MRI for Radiotherapy. Springer International Publishing; 2019:131-151. https://doi.org/10.1007/978-3-030-14442-5_9
Teuho J, Johansson J, Linden J, et al. Effect of attenuation correction on regional quantification between PET/MR and PET/CT: a multicenter study using a 3-dimensional brain phantom. J Nucl Med. 2016;57:818-824.
Wyatt JJ, Dowling JA, Kelly CG, et al. Investigating the generalisation of an atlas-based synthetic-CT algorithm to another centre and MR scanner for prostate MR-only radiotherapy. Phys Med Biol. 2017;62:N548-N560.
Persson E, Gustafsson C, Nordström F, et al. MR-OPERA: a multicenter/multivendor validation of magnetic resonance imaging-only prostate treatment planning using synthetic computed tomography images. Int J Radiat Oncol Biol Phys. 2017;99:692-700.
Greer P, Martin J, Sidhom M, et al. A multi-center prospective study for implementation of an MRI-only prostate treatment planning workflow. Front Oncol. 2019;9:826.
Loi G, Fusella M, Vecchi C, et al. Computed tomography to cone beam computed tomography deformable image registration for contour propagation using head and neck, patient-based computational phantoms: a multicenter study. Pract Radiat Oncol. 2020;10:125-132.
Pan SJ Jialin, Yang Q. A Survey on Transfer Learning. IEEE Trans Knowl Data Eng. 2010;22:(10):1345-1359. https://doi.org/10.1109/tkde.2009.191
Cheplygina V, Bruijne M, Pluim JPW. Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med Image Anal. 2019;54:280-296.
Li W, Kazemifar S, Bai T, et al. Synthesizing CT images from MR images with deep learning: model generalization for different datasets through transfer learning. 2021;7:025020.
Mutic S, Palta JR, Butker EK, et al. Quality assurance for computed-tomography simulators and the computed-tomography-simulation process: report of the AAPM Radiation Therapy Committee Task Group No. 66. Med Phys. 2003;30:2762-2792.
Gallas RR, Hünemohr N, Runz A, Niebuhr NI, Jäkel O, Greilich S. An anthropomorphic multimodality (CT/MRI) head phantom prototype for end-to-end tests in ion radiotherapy. Zeitsch Mediz Phys. 2015;25:391-399.
Niebuhr NI, Johnen W, Echner G, et al. The ADAM-pelvis phantom-an anthropomorphic, deformable and multimodal phantom for MRgRT. Phys Med Biol. 2019;64:04NT05.
Singhrao K, Fu J, Wu HH, et al. A novel anthropomorphic multimodality phantom for MRI-based radiotherapy quality assurance testing. Med physics. 2020;47:1443-1451.
Colvill E, Krieger M, Bosshard P, et al. Anthropomorphic phantom for deformable lung and liver CT and MR imaging for radiotherapy. Phys Med Biol. 2020;65:07NT02.
Chen X, Men K, Chen B, et al. CNN-based quality assurance for automatic segmentation of breast cancer in radiotherapy. Front Oncol. 2020;10.
Bragman FJ, Tanno R & Eaton-Rosen Z et al. Uncertainty in multitask learning: joint representations for probabilistic MR-only radiotherapy planning. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2018:3-11. https://doi.org/10.1007/978-3-030-00937-3_1
Hemsley M, Chugh B, Ruschin M, et al. Deep generative model for synthetic-CT generation with uncertainty predictions. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2020:834-844.
Abdar M, Pourpanah F, Hussain S, Rezazadegan D, Liu L, Ghavamzadeh M, Fieguth P, Cao X, Khosravi A, Acharya UR, Makarenkov V & Nahavandi S. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Inf Fusion. 2021;76:243-297. https://doi.org/10.1016/j.inffus.2021.05.008
Kawahara D, Saito A, Ozawa S, Nagata Y. Image synthesis with deep convolutional generative adversarial networks for material decomposition in dual-energy CT from a kilovoltage CT. Comp Biol Med. 2020;128:104111.
Jans LBO, Chen M, Elewaut D, et al. MRI-based synthetic CT in the detection of structural lesions in patients with suspected sacroiliitis: comparison with MRI. Radiol. 2020;298:343-349.
Staartjes VE, Seevinck PR, Vandertop WP, Stralen M, Schröder ML. Magnetic resonance imaging-based synthetic computed tomography of the lumbar spine for surgical planning: a clinical proof-of-concept. Neurosurg Focus. 2021;50:E13.
McKenzie EM, Santhanam A, Ruan D, O'Connor D, Cao M, Sheng K. Multimodality image registration in the head-and-neck using a deep learning-derived synthetic CT as a bridge. Med Phys. 2020;47:1094-1104.
Siedek F, Yeo SY, Heijman E, Grinstein O, Bratke G, Heneweer C, Puesken M, Persigehl T, Maintz D, Grüll H. Magnetic Resonance-Guided High-Intensity Focused Ultrasound (MR-HIFU): Technical Background and Overview of Current Clinical Applications (Part 1). RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren. 2019;191:(06):522-530. https://doi.org/10.1055/a-0817-5645
Jiang J, Hu Y-C, Tyagi N, et al. Cross-modality (CT-MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets. Med Phys. 2019;46:4392-4404.
Kieselmann JP, Fuller CD, Gurney-Champion OJ, Oelfke U. Cross-modality deep learning: Contouring of MRI data from annotated CT data only. Med Phys. 2020;48:1673-1684.