Deep Learning-based Style Transfer of Multimodal Medical images for Enhanced Surgical planning
Rp Nirush, U Vishnu Kanna, Jothiraj Selvaraj, Snekhalatha Umapathy · 2024
This study introduces a novel application of neural style transfer techniques in medical imaging, specifically focusing on Digitally Reconstructed Radiograph (DRR) and Xray images. DRRs, crucial for surgical planning, are generated from CT scans, providing detailed spinal views. By employing neural style transfer, the content of X-ray images can be combined with the stylistic features of DRRs, resulting in synthetic X-ray images that retain structural information while incorporating DRR characteristics. This process involves minimizing the difference between the input image’s content and the X-ray content, while incorporating DRR style. Content and style losses are computed using feature representations, with the total loss being a combination of these losses. The Limited memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) optimization technique is utilized for updating pixel values iteratively. This approach enhances visualization and interpretation of medical imaging data, aiding in surgical navigation and reducing surgeon time in identifying surgical sites.