Surgical Simulation of Cerebrovascular Disease With Multimodal Fusion 3-Dimensional Computer Graphics
Nobuhito Saito, Taichi Kin, Hiroshi Ōyama, Masanori Yoshino, Daichi Nakagawa, Masaaki Shojima, Hideaki Imai, Hirofumi Nakatomi · Neurosurgery · 2013
Although recent advancements in medical imaging technology have allowed detailed preoperative examinations, neurosurgeons still have to interpret large amounts of medical imaging data. In various modalities such as computed tomography (CT), magnetic resonance imaging (MRI), and angiography, there are multiple sequences and 3-dimensional (3-D) images, and it is not uncommon for there to be several hundred to several thousand section images per case.1 Clinicians have to interpret each of these multimodalities/sequences individually and consolidate this information in their heads to form a 3-D image that can be used in preoperative planning. From the perspectives of accuracy, reproducibility, and sharing information with other people, it is hard to ensure sufficient precision.2 Furthermore, the spatial resolution of the 3-D images used in today’s clinical settings is inferior to that in 2-dimensional (2-D) imaging because the processing methods are limited. Consequently, ascertaining detailed findings from 3-D images alone is unsatisfactory; clinicians must additionally interpret 2-D section images of the same site.3 In this report, we describe the fusion of all image data required for preoperative examination and the construction of 3-D computer graphics (3-DCG) with a high spatial resolution using our own image processing technique. We then apply this to surgical strategies in cerebral vascular disease and report our experience and the usefulness of the technique. PATIENTS AND METHODS Patient Population Subjects included patients with cerebral vascular disease who underwent preoperative examination using 3-DCG with the proposed fusion technique at the University of Tokyo, Department of Neurosurgery. The internal review board of University of Tokyo Hospital approved the study protocol, and written informed consent was obtained from all patients before participation in this study. Image Processing The image processing technique was as per the technique reported earlier and is outlined briefly below.1-5 CT, magnetic resonance imaging, and 3-D rotational angiography (3-DRA) images were used for 3-D reconstruction. Preoperatively recorded images were output in the DICOM (digital imaging communication in medicine) format and imported into the image processing software Avizo 6.3 (Visualization Science Group, Bordeaux, France). For registration of different modalities, automatic registration was performed with the normalized mutual information method using our original initialization method that we have devised.1 Images were presented using a hybrid rendering method in which the surface-rendering method was combined with the volume-rendering method.2 For surface rendering, we used the method of setting multiple modalities and multiple thresholds for a target tissue (multimodal individualizing tissue threshold method) to obtain high spatial resolution.1 The 3-D images (fusion 3-DCG) completed on the workstation were used to carry out the interactive virtual reality operative simulation. RESULTS The proposed 3-DCG fusion technique facilitates the rendering of 1 fusion 3-DCG with a high spatial resolution. Errors in the simulation did not pose any clinical problems, and there were no diagnostic errors caused by fusion 3-DCG. In the surgical simulation, fusing each modality made it possible to comprehensively understand the pathological condition in 3-D. We describe here typical cases in which fusion 3-DCG with a high spatial resolution was beneficial in terms of surgical strategy because many findings were observed in fusion 3-DCG that could not be found in nonfused images. Case 1: Aneurysm of the Left Posterior Cerebral Artery Various virtual surgical sites were examined preoperatively with the proposed 3-DCG fusion technique. In this case, the 3-D reconstruction was created using a 3-DRA of the 3-D model of the vessel, fast imaging employing steady-state acquisition of the 3-D model of the cortex, and CT of the 3-D model of the skull. In this case, the posterior transpetrosal approach (Figure 1A) or subtemporal approach (Figure 1B) was examined as a virtual surgical site, and the aneurysm was actually reached by a subtemporal approach. Fusion 3-DCG with the proposed method makes it easy to understand the lesion and the anatomy around the lesion in detail and in 3-D, even in surgical sites that are relatively uncommon such as in this case (Figure 1C and 1D).FIGURE 1: Aneurysm of left posterior cerebral artery. A, virtual surgical view of left posterior transpetrosal approach by fusion 3-dimensional computer graphics (3-DCG). B, virtual surgical view of left subtemporal approach by fusion 3-DCG. C, virtual surgical view of left subtemporal approach. D, actual operative view. AN, aneurysm; P2, P2 segment of the posterior cerebral artery; P3, P3 segment of the posterior cerebral artery.Case 2: Left Paraclinoid Aneurysm It is possible to manipulate various surgical simulations with fusion 3-DCG. This is a case of double bilobular paraclinoid aneurysms (Figure 2A). The anterior clinoid process was virtually removed (Figure 2B); the optic nerve and flow of the ophthalmic artery were confirmed through the semitransparency of the skull (Figure 2C); and virtual reality simulation of neck clipping with a virtual clip was conducted (Figure 2D).FIGURE 2: Surgical simulation of a left paraclinoid aneurysm by fusion 3-dimensional computer graphics (3-DCG). A, virtual surgical view by fusion 3-DCG. The positional relationship between the aneurysm, optic nerve, and anterior clinoid process (ACP) can be seen. B, surgical view of the virtual clinoidectomy. Part of the 2 tubercular paraclinoid aneurysms is hidden directly under the ACP. C, virtual surgical view of the semitransparent bone. The optic nerve of the optic tract and ophthalmic artery flow can be verified. D, surgical simulation with a virtual clip. A1, A1 segment of the anterior cerebral artery, AN, aneurysm; IC, internal carotid artery; M1, M1 segment of the middle cerebral artery.Case 3: Aneurysm of the Distal Anterior Cerebral Artery The characteristics of the aneurysmal wall are often obtained from multiple modalities/sequences. This is a case of an aneurysm associated with thrombus, hematoma, and calcification (Figure 3A-3C). With just time-of-flight MR angiography, only the part of the aneurysm with blood flow can be visualized in 3-D (Figure 3D), whereas fusion 3-DCG allows the thrombus, hematoma, and calcification parts of the aneurysm to be displayed as 1 CG (Figure 3E), thus allowing the correct shape of the aneurysm to be found (Figure 3F).FIGURE 3: Aneurysm of left distal anterior cerebral artery. A, in the time-of-flight magnetic resonance angiography (TOF-MRA), only the part with aneurysmal blood flow is shown. B, in the contrast-enhanced TOF-MRA, we can see the enhanced hematoma and thrombus, which is the signal void part. C, calcification that was unclear on computed tomography and magnetic resonance imaging is seen as an aneurysm. D, 3-dimensional image reconstructed from TOF-MRA only. E, wall information such as calcification (*), hematoma (**), and thrombus (***) were rendered in fusion 3-DCG, and the precise shape of the aneurysm was found. F, actual surgical findings.Case 4: Arteriovenous Malformation In arteriovenous malformations, identifying the feeder and drainer is an extremely important surgical strategy. No matter how detailed an interpretation is made of medical images of vascular information such as in 2-D digital subtraction angiography, it is difficult to understand the positional relationship of cerebral parenchyma and blood vessels. Fusion 3-DCG allows a detailed understanding of the positional relationship between the arteriovenous malformation and the cortical sulcus and gyrus (Figure 4A), and even with blood vessels that can be only partially identified from the brain surface, it is easy to distinguish the feeder from the drainer. Fusion 3-DCG is very useful when identifying the feeder and drainer in actual surgery (Figure 4B).FIGURE 4: Left frontal arteriovenous malformation. A, virtual surgical view by fusion 3-dimensional computer graphics. The positional relationship between the feeder (red), drainer (blue), nidus (gray) and cortical sulcus, and cortical gyrus can be clearly seen. B, actual surgical findings.Case 5: Brainstem Cavernous Malformation Brainstem cavernous malformations associated with developmental venous anomalies often cannot be identified in preoperative image findings (Figure 5A). In fusion 3-DCG with high-resolution imaging through 3-DRA, developmental venous anomalies that could not be identified by 2-D digital subtraction angiography were successfully rendered in detail (Figure 5B). In this case, the most accessible point was identified by use of the 2-point method6 on fusion 3-DCG, and the virtual surgical view showing retraction of the cerebellum (Figure 5C) was very similar to actual surgical findings (Figure 5D).FIGURE 5: Brainstem cavernous malformation (CM). A, in the 2-dimesional digital subtraction angiography, the rate of visualization of the developmental venous anomalies (DVAs) associated with brain CM is not high. B, dorsal view of the brainstem by fusion 3-dimensional (3-D) computer graphics. The positional relationship of the brain CM and DVAs and the 3-D positional relationship of the important anatomical structure of the cerebellar peduncle, dentate nucleus, and dorsal brainstem can be seen in detail, and the most accessible point in the surgical approach can be examined in 3-D. C, virtual surgical view by virtual cerebellum retraction. D, actual surgical findings.Case 6. Left Temporal Pial Arteriovenous Fistula Fusion 3-DCG using the proposed method was possible with the fusion of functional images and fluid analysis image data.7 This is a case of pial arteriovenous fistula with the shunting point on the brain surface; in this case, diagnosis was very difficult on 2-D digital subtraction angiography (Figure 6A). Localized in the left frontal lobe (Figure 6B) and using fusion 3-DCG to ascertain the positional relationship between the lesion with the corticospinal tract, arcuate fiber and optic radiation was useful in surgical examinations (Figure 6C). Blood flow analysis data using the mesh from the finite-element method7,8 were fused to create CG, which helped to identify the shunting point (Figure 6D).FIGURE 6: Left temporal pial arteriovenous fistula. A, on 2-D digital subtraction angiography, it is difficult to identify the shunting point and to differentiate it from the arteriovenous malformation. B, in fusion 3-dimensional computer graphics (3-DCG), the shunting point was accurately identified. C, fusion 3-DCG in which functional imaging data and the bleeding area are fused. D, fluid analysis using the finite-element method with mesh. A, anterior; P, posterior.Case 7: Spinal Dural Arteriovenous Fistula Spinal angiography is the gold standard for the diagnosis of spinal dural arteriovenous fistula, in which it is often difficult to identify the shunting point and to ascertain the 3-D positional relationship with the lesion (Figure 7A). The positional relationship of the arteriovenous fistula, bone, and dura mater is a very important factor in surgical planning. In this case, fusion 3-DCG images were created that reconstructed a 3-D model of the bone from plain CT, a 3-D model of the dura mater from the postmyelography CT (Figure 7B), and a 3-D model of the vessels from spinal 3-DRA (Figure 7C).4 In this fusion 3-DCG, the abnormal vessel and the dura mater penetration site were easily identified, and the virtual surgical view (Figure 7D) of the virtual bone removal was consistent with actual surgical findings (Figure 7E).FIGURE 7: Left, the spinal dural arteriovenous fistula (AVF). A, the AVF shunting point was identified on 2-dimensional digital subtraction angiography, but it was difficult to understand the 3-dimensional positional relationship of the lesion and the surrounding anatomical structure. B, on postmyelography computed tomography (CT), the boundary between the dura mater and extradura was identified, but the shunting point could not be identified. C, On 3-dimensional rotational angiography and postmyelography CT fused 3-dimensional computer graphics (3-DCG), the shunting point dura mater penetration site was easily identified. D, virtual operative area by fusion 3-DCG. E, operative view.DISCUSSION We reported that surgical simulations of cerebrovascular disease with high-resolution multimodal fusion 3-dimensional images are useful. Although the normalized mutual information method is widely used in medical image registration methods,9,10 there is a problem in that, unless the proper initial values are set, correct interpretation cannot be obtained, and in registration of > 2 modalities, there are often difficulties.10 In the technique proposed here, multimodal registration was made possible by focusing on the luminance range from the initial voxel size, the distribution, and the image range.1 Visualization of medical 3-D imaging can be broadly divided into the surface-rendering method and the volume-rendering method. The former divides the object surface into micropolygons to render an image, and the latter displays the entire selected threshold for an image as a 3-D image. When complex 3-D images that fuse multiple modalities are visualized, there are limitations to both rendering methods. In our proposed method, we used a hybrid rendering method in which we used mainly the surface-rendering method and displayed 3-DCG simultaneously as required using the volume-rendering method. This way, we succeeded in visually simplifying highly complicated 3-D information.2 In the surface-rendering method, as a general rule, 3-D images are created by selecting 1 threshold pair, although in the data of 1 image the signal intensity differs even with the same tissue, and the threshold with little noise is generally selected even if a certain degree of spatial resolution must be sacrificed. In our proposed method, we divided multimodal images for 1 target tissue into random regions of interest, and by setting the threshold so that each region of interest had a high spatial resolution and low noise, we were able to reconstruct 3-D images with a high spatial resolution.3 Problems with this method include the need for a few hours to process images and the fact that the accuracy verification method for registration and manual segmentation is not completely established. Moreover, the original objective of multimodal fusion 3-DCG is to improve patient prognosis and surgical outcomes. In this respect, further cases should be gathered and examined. For related video content, please access the Supplemental Digital Content: http://www.youtube.com/watch?v=94M4V2EMCUI Disclosures The internal review board at University of Tokyo Hospital approved the study protocol, and written informed consent was obtained from all subjects. The authors have no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.