Fluoroscopic 3D Images Generation Using A GAN Method

Abstract

Respiratory motion is a major source of error in radiation therapy for thorax and lung cancer. Image-Guided Radiation Therapy (IGRT) is the use of frequent imaging during radiation therapy to enhance the precision and accuracy of the treatment delivery. Cone Beam Computed Tomography (CBCT) is an imaging modality that is used in radiotherapy delivery rooms to account for respiratory motion uncertainties. One CBCT scan results in a set of projections (2D images), which is taken in a couple of minutes while the patient is normally breathing. The known procedure for utilizing these projections is to sort them into breathing phases and reconstruct a 3D image from each, which results in making 4D-CBCT images. These 4D-CBCT images are used to estimate fluoroscopic images right before the treatment delivery. The objective of this work is to generate 3D fluoroscopic images of the lungs from two orthogonal projections using a Generative Adversarial Network (GAN) method. GAN is a powerful deep learning-based technique that allows two independent models to develop and compete with each other, which are named the generator (G) and the discriminator (D). This method has the potential of generating a 3D volume image representing patient anatomy at the time of treatment delivery from any two orthogonal projections. Using multiple pairs of projections from multiple pairs of angles, the model can generate a 3D fluoroscopic image representing the respiratory motion while the patient is in the treatment position. The resulting generated 3D images give very good results in terms of generating quality images representing the same breathing phase as the orthogonal projections. The quantitative results include the measures of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), which equal 26.01 and 0.572, respectively. Although the proposed method does not intend to replace the used motion model-based methods, its goal is to add niche applications in radiation therapy, such as tumor tracking and dose verification.