Usefulness of a glass-free medical three-dimensional autostereoscopic display in neurosurgery

Autostereoscopic 3D Display System for 3D Medical Images

Recent advances in autostereoscopic three-dimensional (3D) display systems have led to innovations in consumer electronics and vehicle systems (e.g., head-up displays). However, medical images with stereoscopic depth provided by 3D displays have yet to be developed sufficiently for widespread adoption in diagnostics. Indeed, many stereoscopic 3D displays necessitate special 3D glasses that are unsuitable for clinical environments. This paper proposes a novel glasses-free 3D autostereoscopic display system based on an eye tracking algorithm and explores its viability as a 3D navigator for cardiac computed tomography (CT) images. The proposed method uses a slit-barrier with a backlight unit, which is combined with an eye tracking method that exploits multiple machine learning techniques to display 3D images. To obtain high-quality 3D images with minimal crosstalk, the light field 3D directional subpixel rendering method combined with the eye tracking module is applied using a user’s 3D eye positions. Three-dimensional coronary CT angiography images were volume rendered to investigate the performance of the autostereoscopic 3D display systems. The proposed system was trialed by expert readers, who identified key artery structures faster than with a conventional two-dimensional display without reporting any discomfort or 3D fatigue. With the proposed autostereoscopic 3D display systems, the 3D medical image navigator system has the potential to facilitate faster diagnoses with improved accuracy.

Interactive Holographic Display Based on Finger Gestures

In this paper, we demonstrate an interactive, finger-sensitive system which enables an observer to intuitively handle electro-holographic images in real time. In this system, a motion sensor detects finger gestures (swiping and pinching) and translates them into the rotation and enlargement/reduction of the holographic image, respectively. By parallelising the hologram calculation using a graphics processing unit, we realised the interactive handling of the holographic image in real time. In a demonstration of the system, we used a Leap Motion sensor and a phase modulation-type spatial light modulator with 1,920 × 1,080 pixels and a pixel pitch of 8.0 µm × 8.0 µm. The constructed interactive finger-sensitive system was able to rotate a holographic image composed of 4,096 point light sources using a swiping gesture and enlarge or reduce it using a pinching gesture in real time. The average calculation speed was 27.6 ms per hologram. Finally, we extended the constructed system to a full-colour reconstruction system that generates a more realistic three-dimensional image. The extended system successfully allowed the handling of a full-colour holographic image composed of 1,709 point light sources with a calculation speed of 22.6 ms per hologram.

Usefulness and capability of three-dimensional, full high-definition movies for surgical education

Background

Because of changing surgical procedures in the fields of oral and maxillofacial surgery, new methods for surgical education are needed and could include recent advances in digital technology. Many doctors have attempted to use digital technology as educational tools for surgical training, and movies have played an important role in these attempts. We have been using a 3D full high-definition (full-HD) camcorder to record movies of intra-oral surgeries.

Method

The subjects were medical students and doctors receiving surgical training who did not have actual surgical experience (n = 67). Participants watched an 8-min, 2D movie of orthognathic surgery and subsequently watched the 3D version. After watching the 3D movie, participants were asked to complete a questionnaire.

Result

A lot of participants (84%) felt a 3D movie excellent or good and answered that the advantages of a 3D movie were their appearance of solidity or realism. Almost all participants (99%) answered that 3D movies were quite useful or useful for medical practice.

Conclusions

Three-dimensional full-HD movies have the potential to improve the quality of medical education and clinical practice in oral and maxillofacial surgery.

Floating autostereoscopic 3D display with multidimensional images for telesurgical visualization

PURPOSE

We propose a combined floating autostereoscopic three-dimensional (3D) display approach for telesurgical visualization, which could reproduce live surgical scene in a realistic and intuitive manner.

METHODS

A polyhedron-shaped 3D display device is developed for spatially floating autostereoscopic 3D image. Integral videography (IV) technique is adopted to generate real-time 3D images. Combined two-dimensional (2D) and 3D displays are presented floatingly around the center of the display device through reflection of semitransparent mirrors. Intra-operative surgery information is fused and updated in the 3D display, so that telesurgical visualization could be enhanced remotely.

RESULTS

The experimental results showed that our approach can achieve a combined floating autostereoscopic display that presents 2D and 3D fusion images. The glasses-free IV 3D display has full parallax and can be observed by multiple persons from surrounding areas at the same time. Furthermore, real-time surgical scene could be presented and updated in a realistic and intuitive visualization platform. It is shown that the proposed method is feasible for facilitating telesurgical visualization.

CONCLUSION

The proposed floating autostereoscopic display device presents surgical information in an efficient form, so as to enhance operative cooperation and efficiency during operation. Combined presentation of imaging information is promising for medical applications.

Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback

BACKGROUND

With the decrease in the number of cerebral aneurysms treated surgically and the increase of complexity of those treated surgically, there is a need for simulation-based tools to teach future neurosurgeons the operative techniques of aneurysm clipping.

OBJECTIVE

To develop and evaluate the usefulness of a new haptic-based virtual reality simulator in the training of neurosurgical residents.

METHODS

A real-time sensory haptic feedback virtual reality aneurysm clipping simulator was developed using the ImmersiveTouch platform. A prototype middle cerebral artery aneurysm simulation was created from a computed tomographic angiogram. Aneurysm and vessel volume deformation and haptic feedback are provided in a 3-dimensional immersive virtual reality environment. Intraoperative aneurysm rupture was also simulated. Seventeen neurosurgery residents from 3 residency programs tested the simulator and provided feedback on its usefulness and resemblance to real aneurysm clipping surgery.

RESULTS

Residents thought that the simulation would be useful in preparing for real-life surgery. About two-thirds of the residents thought that the 3-dimensional immersive anatomic details provided a close resemblance to real operative anatomy and accurate guidance for deciding surgical approaches. They thought the simulation was useful for preoperative surgical rehearsal and neurosurgical training. A third of the residents thought that the technology in its current form provided realistic haptic feedback for aneurysm surgery.

CONCLUSION

Neurosurgical residents thought that the novel immersive VR simulator is helpful in their training, especially because they do not get a chance to perform aneurysm clippings until late in their residency programs.

Improving workflows of neuro-interventional procedures with autostereoscopic 3D visualization of multi-modality imaging in hybrid interventional suites

PURPOSE

Recent developments in interventional neuroradiology techniques, medical imaging modalities, endovascular stenting and embolization materials lead to an increasing number of patients with cerebral aneurysms and arteriovenous malformations that are eligible for endovascular treatment and have opened new perspectives for novel ways for patient treatment in general. In this paper, we describe a software tool for 3D image fusion of multi-modal acquisitions to assist endovascular treatment of cerebral malformations. The software and an autostereoscopic 3D display were implemented and tested in clinical applications in a hybrid interventional suite that is used for radio-interventional as well as neurosurgical procedures. Our hypothesis is that fusion of image data acquired prior to intervention procedures with images acquired during those procedures should allow better visualizing and navigating through complex cerebral vasculature. This should also improve workflows of neuro-interventional procedures.

METHODS

Preoperative and intra-operative acquisitions of vascular images of the brain were performed and transferred to a dedicated imaging workstation to be processed with our image fusion and visualization software tool. The tool was developed as a plugin extension to the open-source DICOM viewer OsiriX and is based on a modular and scalable architecture. Several processing modules were implemented to allow spatial co-registration and fusion of preoperative and intra-operative modalities. A special extension was also implemented for interactive autostereosopic, glass-free 3D visualization of fused results.

RESULTS

The software platform was validated and evaluated in nine in vivo procedures by expert users. All patient cases were related to interventional treatment of neuro-vascular diseases. The emphasis was laid on the added value of spatial co-registration and fusion of preoperative and intra-operative modalities, as well as the overall impact on workflow during the intervention. The co-registered and fused images were visualized on an autostereoscopic 3D monitor installed in hybrid interventional suite. All experiments were evaluated and scored by interventional physicians and technicians.

CONCLUSIONS

Displaying 3D-4D representations of brain vascular anomalies based on multi-modal acquisitions on a 3D autostereoscopic display is beneficial for the workflow and efficiency of interventional radiologists. The implemented software tool fulfills the premise of applicability of an open-source platform for more advanced, multi-modal visualization and processing of brain vascular structures for image-guided therapeutic interventions.