Plenary lecture 4: visualization and interaction of the virtual human organs for the pre-operative planning
Lucio Tommaso De Paolis · International Conference on Mathematical Methods and Computational Techniques in Electrical Engineering · 2009
The visualization of 3D models of the patient's body emerges as a priority in surgery both in preoperative planning and during surgical procedures and the introduction of new modalities of interaction with the 3D models of the human organs could also be required. In this paper is present a virtual interface and a low cost multi-touch screen. Both systems are able to interpret in real-time the user's finger movements and can be used in the pre-operative planning for the navigation and manipulation of 3D models of the human body built from real patient's CT images. The developed virtual interface, based on the use of an optical tracking system, is the first prototype of a system designed to avoid contact with the computer so that the surgeon will be able to visualize models of the patient' organs more effectively. In particular, the surgeon will be able to rotate, to translate and to zoom in on the 3D models simply by moving his finger in the free space; in addition, it is possible to choose to visualize all of the organs or only some of them. In the system have been utilized the NDI Polaris Vicra as optical tracker and two open-source and multiplatform programs for the construction of the graphic environment (OpenSceneGraph toolkit) and for the building of the 3D models (3D Slicer package). A 3D model of the abdominal area has been reconstructed from CT images using segmentation and classification algorithms. All of the interactions with the models happen in real-time using the virtual interface which appears as a touch-screen suspended in the free space and situated in a location chosen by the user at the starting up of the application. The finger movements are detected by means of the optical tracker and are used to simulate the touch with some buttons present in the developed virtual interface. The doctor, using this system, is able to rotate, to translate and to zoom in on the 3D models of the patient's organs simply by moving his finger in free space and he can select the visualization of all of the organs or only some of them. The optical tracker has already been used in computer aided systems and, for this reason, it is easy to integrate the described virtual interface with these systems. The introduction of other functionalities of interaction with the models is in progress, after further investigation and consideration of surgeons' requirements. The developed multi-touch screen provides a user interface customized for doctor requirements. It is possible to use one or more fingers in order to interact with the complete 3D models of the human body or with some parts of these; in addition it is provided the possibility to visualize the CT slice sets used to build the virtual models. The interaction results very simple and evident for the user and the system can be a helpful tool for the diagnosis and the surgical operation planning.