Deep learning-based markerless lung tumor tracking in stereotactic radiotherapy using Siamese networks.
deep learning
radiotherapy
tumor tracking
Journal
Medical physics
ISSN: 2473-4209
Titre abrégé: Med Phys
Pays: United States
ID NLM: 0425746
Informations de publication
Date de publication:
Nov 2023
Nov 2023
Historique:
revised:
27
03
2023
received:
27
10
2022
accepted:
27
04
2023
medline:
6
11
2023
pubmed:
23
5
2023
entrez:
23
5
2023
Statut:
ppublish
Résumé
Radiotherapy (RT) is involved in about 50% of all cancer patients, making it a very important treatment modality. The most common type of RT is external beam RT, which consists of delivering the radiation to the tumor from outside the body. One novel treatment delivery method is volumetric modulated arc therapy (VMAT), where the gantry continuously rotates around the patient during the radiation delivery. Accurate tumor position monitoring during stereotactic body radiotherapy (SBRT) for lung tumors can help to ensure that the tumor is only irradiated when it is inside the planning target volume. This can maximize tumor control and reduce uncertainty margins, lowering organ-at-risk dose. Conventional tracking methods are prone to errors, or have a low tracking rate, especially for small tumors that are in close vicinity to bony structures. We investigated patient-specific deep Siamese networks for real-time tumor tracking, during VMAT. Due to lack of ground truth tumor locations in the kilovoltage (kV) images, each patient-specific model was trained on synthetic data (DRRs), generated from the 4D planning CT scans, and evaluated on clinical data (x-rays). Since there are no annotated datasets with kV images, we evaluated the model on a 3D printed anthropomorphic phantom but also on six patients by computing the correlation coefficient with the breathing-related vertical displacement of the surface-mounted marker (RPM). For each patient/phantom, we used 80% of DRRs for training and 20% for validation. The proposed Siamese model outperformed the conventional benchmark template matching-based method (RTR): (1) when evaluating both methods on the 3D phantom, the Siamese model obtained a 0.57-0.79-mm mean absolute distance to the ground truth tumor locations, compared to 1.04-1.56 mm obtained by RTR; (2) on patient data, the Siamese-determined longitudinal tumor position had a correlation coefficient of 0.71-0.98 with the RPM, compared to 0.07-0.85 for RTR; (3) the Siamese model had a 100% tracking rate, compared to 62%-82% for RTR. Based on these results, we argue that Siamese-based real-time 2D markerless tumor tracking during radiation delivery is possible. Further investigation and development of 3D tracking is warranted.
Sections du résumé
BACKGROUND
BACKGROUND
Radiotherapy (RT) is involved in about 50% of all cancer patients, making it a very important treatment modality. The most common type of RT is external beam RT, which consists of delivering the radiation to the tumor from outside the body. One novel treatment delivery method is volumetric modulated arc therapy (VMAT), where the gantry continuously rotates around the patient during the radiation delivery.
PURPOSE
OBJECTIVE
Accurate tumor position monitoring during stereotactic body radiotherapy (SBRT) for lung tumors can help to ensure that the tumor is only irradiated when it is inside the planning target volume. This can maximize tumor control and reduce uncertainty margins, lowering organ-at-risk dose. Conventional tracking methods are prone to errors, or have a low tracking rate, especially for small tumors that are in close vicinity to bony structures.
METHODS
METHODS
We investigated patient-specific deep Siamese networks for real-time tumor tracking, during VMAT. Due to lack of ground truth tumor locations in the kilovoltage (kV) images, each patient-specific model was trained on synthetic data (DRRs), generated from the 4D planning CT scans, and evaluated on clinical data (x-rays). Since there are no annotated datasets with kV images, we evaluated the model on a 3D printed anthropomorphic phantom but also on six patients by computing the correlation coefficient with the breathing-related vertical displacement of the surface-mounted marker (RPM). For each patient/phantom, we used 80% of DRRs for training and 20% for validation.
RESULTS
RESULTS
The proposed Siamese model outperformed the conventional benchmark template matching-based method (RTR): (1) when evaluating both methods on the 3D phantom, the Siamese model obtained a 0.57-0.79-mm mean absolute distance to the ground truth tumor locations, compared to 1.04-1.56 mm obtained by RTR; (2) on patient data, the Siamese-determined longitudinal tumor position had a correlation coefficient of 0.71-0.98 with the RPM, compared to 0.07-0.85 for RTR; (3) the Siamese model had a 100% tracking rate, compared to 62%-82% for RTR.
CONCLUSIONS
CONCLUSIONS
Based on these results, we argue that Siamese-based real-time 2D markerless tumor tracking during radiation delivery is possible. Further investigation and development of 3D tracking is warranted.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
6881-6893Subventions
Organisme : Varian Medical Systems
Informations de copyright
© 2023 The Authors. Medical Physics published by Wiley Periodicals LLC on behalf of American Association of Physicists in Medicine.
Références
Citrin DE. Recent developments in radiotherapy. N Engl J Med. 2017;377:1065-1075.
Ong CL, Verbakel WF, Dahele M, et al. Fast arc delivery for stereotactic body radiotherapy of vertebral and lung tumors. Int J Radiat Oncol Biol Phys. 2012;83:e137-e143.
Keall PJ, Mageras GS, Balter JM, et al. The management of respiratory motion in radiation oncology report of AAPM Task Group 76. Med Phys. 2006;33:3874-3900.
Bertholet J, Knopf A, Eiben B, et al. Real-time intrafraction motion monitoring in external beam radiotherapy. Phys Med Biol. 2019;64:15TR01.
Hazelaar C, Dahele M, Mostafavi H, van der Weide L, Slotman B, Verbakel W. Markerless positional verification using template matching and triangulation of kV images acquired during irradiation for lung tumors treated in breath-hold. Phys Med Biol. 2018;63:115005.
van Sörnsen de Koste JR, Dahele M, Mostafavi Sloutsky A, Senan S, Slotman BJ, Verbakel WF. Markerless tracking of small lung tumors for stereotactic radiotherapy. Med Phys. 2015;42:1640-1652.
de Bruin K, Dahele M, Mostafavi H, Slotman BJ, Verbakel WF. Markerless real-time 3-dimensional kV tracking of lung tumors during free breathing stereotactic radiation therapy. Adv Radiat Oncol. 2021;6:100705.
Roeske JC, Mostafavi H, Haytmyradov M, et al. Characterization of Markerless Tumor Tracking Using the On-Board Imager of a Commercial Linear Accelerator Equipped With Fast-kV Switching Dual-Energy Imaging. Adv Radiat Oncol. 2020;5:1006-1013.
Terunuma T, Tokui A, Sakae T. Novel real-time tumor-contouring method using deep learning to prevent mistracking in X-ray fluoroscopy. Radiol Phys Technol. 2018;11:43-53.
Zhao W, Shen L, Han B, et al. Markerless pancreatic tumor target localization enabled by deep learning. Int J Radiat Oncol Biol Phys. 2019;105:432-439.
Takahashi W, Oshikawa S, Mori S. Real-time markerless tumour tracking with patient-specific deep learning using a personalised data generation strategy: proof of concept by phantom study. Brit J Radiol. 2020;93:20190420.
Hirai R, Sakata Y, Tanizawa A, Mori S. Real-time tumor tracking using fluoroscopic imaging with deep neural network analysis. Physica Medica. 2019;59:22-29.
Wei R, Zhou F, Liu B, et al. Convolutional neural network (CNN) based three dimensional tumor localization using single X-ray projection. IEEE Access. 2019;7:37026-37038.
Tao R, Gavves E, Smeulders AW. Siamese instance search for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016:1420-1429.
Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PH. Fully-convolutional siamese networks for object tracking. In: European Conference on Computer Vision. Springer; 2016:850-865.
Li B, Wu W, Wang Q, Zhang F, Xing J, Yan J. SiamRPN++: evolution of siamese visual tracking with very deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019:4282-4291.
Wang Q, Zhang L, Bertinetto L, Hu W, Torr PH. Fast online object tracking and segmentation: a unifying approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019:1328-1338.
Hazelaar C, van Eijnatten M, Dahele M, et al. Using 3D printing techniques to create an anthropomorphic thorax phantom for medical imaging purposes. Med Phys. 2018;45:92-100.
Chi PCM, Balter P, Luo D, Mohan R, Pan T. Relation of external surface to internal tumor motion studied with cine CT. Med Phys. 2006;33:3116-3123.
Lee T, Ewald A, Schultz T, Park S. SU-E-J-253: evaluation of 4DCT images with correlation of RPM signals to tumor motion for respiratory-gated radiotherapy. Med Phys. 2014;41:216.
Mueller M, Poulsen P, Hansen R, et al. The markerless lung target tracking AAPM Grand Challenge (MATCH) results. Med Phys. 2022;49:1161-1180.
Unberath M, Zaech JN, Lee SC, et al. DeepDRR-a catalyst for machine learning in fluoroscopy-guided procedures. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2018:98-106.
Bromley J, Bentz JW, Bottou L, et al. Signature verification using a “siamese” time delay neural network. Int J Pattern Recognit Artif. 1993;7:669-688.
Li B, Yan J, Wu W, Zhu Z, Hu X. High performance visual tracking with siamese region proposal network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018:8971-8980.
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention. Springer; 2015:234-241.
Caruana R. Multitask learning. Mach Learn. 1997;28:41-75.
He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2017:2961-2969.
Ulyanov D, Vedaldi A, & Lempitsky V. Instance Normalization: The Missing Ingredient for Fast Stylization (Version 3). arXiv. 2016. https://doi.org/10.48550/ARXIV.1607.08022
Tian Z, Shen C, Chen H, He T. Fcos: Fully convolutional one-stage object detection. In: Proceedings of the IEEE International Conference on Computer Vision. 2019:9627-9636.
Lin TY, Goyal P, Girshick R, He K, Dollár P. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision. 2017:2980-2988.
Zheng Z, Wang P, Liu W, Li J, Ye R, & Ren D. Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression (Version 1). arXiv. 2019. https://doi.org/10.48550/ARXIV.1911.08287
Feng Q, Ablavsky V, Bai Q, & Sclaroff S. Siamese Natural Language Tracker: Tracking by Natural Language Descriptions with Siamese Trackers (Version 2). arXiv. 2019. https://doi.org/10.48550/ARXIV.1912.02048
Remmerts de Vries I, Dahele M, Mostafavi H, Slotman B, Verbakel w. Markerless 3D tumor tracking during single-fraction free-breathing 10MV flattening-filter-free stereotactic lung radiotherapy. Radiother Oncol. 2021;164:6-12.
Xu Q, Hamilton RJ, Schowengerdt RA, Alexander B, Jiang SB. Lung tumor tracking in fluoroscopic video based on optical flow. Med Phys. 2008;35:5351-5359.
Shieh CC, Caillet V, Dunbar M, et al. A Bayesian approach for three-dimensional markerless tumor tracking using kV imaging during lung radiotherapy. Phys Med Biol. 2017;62:3065.
Wang C, Hunt M, Zhang L, et al. 3D localization of lung tumors on cone beam CT projections via a convolutional recurrent neural network. Med Phys. 2020;47:1161-1166.