A Cascaded Deep-Learning Framework for Segmentation of Metastatic Brain Tumors Before and After Stereotactic Radiation Therapy

Radiation therapy is a major treatment option for brain metastasis. For radiation treatment planning and outcome evaluation, magnetic resonance (MR) images are acquired before and at multiple sessions after the treatment. Accurate segmentation of brain tumors on MR images is crucial for treatment planning, response evaluation, and developing data-driven models for outcome prediction. Due to the high volume of imaging data acquired from each patient at multiple follow-up sessions, manual tumor segmentation is resource- and time-consuming in clinic, hence developing an automatic segmentation framework is highly desirable. In this work, we proposed a cascaded 2D-3D Unet framework to segment brain tumors automatically on contrast-enhanced T1- weighted images acquired before and at multiple scan sessions after radiotherapy. 2D Unet is a well-known structure for medical image segmentation. 3D Unet is an extension of 2D Unet with a volumetric input image to provide richer spatial information. The limitation of 3D Unet is that it is memory consuming and cannot process large volumetric images. To address this limitation, a large volumetric input of 3D Unet is often patched to smaller volumes which leads to loss of context. To overcome this problem, we proposed using two cascaded 2D Unets to crop the input volume around the tumor area and reduce the input size of the 3D Unet, obviating the need to patch the input images. The framework was trained using images acquired from 96 patients before radiation therapy and tested using images acquired from 10 patients before and at four follow-up scans after radiotherapy. The segmentation results for the images of independent test set demonstrated that the cascaded framework outperformed the 2D and 3D Unets alone, with an average Dice score of 0.9 versus 0.86 and 0.88 for the baseline, and 0.87 versus 0.83 and 0.84 for the first followup. Similar results were obtained for the other follow-up scans.