Hybrid Spine Simulator Prototype for X-ray Free Pedicle Screws Fixation Training

Registration Sanity Check for AR-guided Surgical Interventions: Experience From Head and Face Surgery

Achieving and maintaining proper image registration accuracy is an open challenge of image-guided surgery. This work explores and assesses the efficacy of a registration sanity check method for augmented reality-guided navigation (AR-RSC), based on the visual inspection of virtual 3D models of landmarks. We analyze the AR-RSC sensitivity and specificity by recruiting 36 subjects to assess the registration accuracy of a set of 114 AR images generated from camera images acquired during an AR-guided orthognathic intervention. Translational or rotational errors of known magnitude up to ±1.5 mm/±15.5°, were artificially added to the image set in order to simulate different registration errors. This study analyses the performance of AR-RSC when varying (1) the virtual models selected for misalignment evaluation (e. g., the model of brackets, incisor teeth, and gingival margins in our experiment), (2) the type (translation/rotation) of registration error, and (3) the level of user experience in using AR technologies. Results show that: 1) the sensitivity and specificity of the AR-RSC depends on the virtual models (globally, a median true positive rate of up to 79.2% was reached with brackets, and a median true negative rate of up to 64.3% with incisor teeth), 2) there are error components that are more difficult to identify visually, 3) the level of user experience does not affect the method. In conclusion, the proposed AR-RSC, tested also in the operating room, could represent an efficient method to monitor and optimize the registration accuracy during the intervention, but special attention should be paid to the selection of the AR data chosen for the visual inspection of the registration accuracy.

Single feature constrained manual registration method for Augmented Reality applications in gynecological laparoscopic interventions

Augmented Reality (AR) can avoid some of the drawbacks of Minimally Invasive Surgery and may provide opportunities for developing innovative tools to assist surgeons. In laparoscopic surgery, the achievement of easy and sufficiently accurate registration is an open challenge. This is particularly true in procedures, such as laparoscopic abdominal Sacro-Colpopexy, where there is a lack of a sufficient number of visible anatomical landmarks to be used as a reference for registration. In an attempt to address the above limitations, we developed and preliminarily testes a constrained manual procedure based on the identification of a single anatomical landmark in the laparoscopic images, and the intraoperative measurement of the laparoscope orientation. Tests in a rigid in-vitro environment show good accuracy (median error 2.4 mm obtained in about 4 min) and good preliminary feedback from the technical staff who tested the system. Further experimentation in a more realistic environment is needed to validate these positive results. Clinical Relevance - This paper provides a new registration method for the development of AR educational videos and AR-based navigation systems for laparoscopic interventions.

Augmented Reality: Mapping Methods and Tools for Enhancing the Human Role in Healthcare HMI

Background: Augmented Reality (AR) represents an innovative technology to improve data visualization and strengthen the human perception. Among Human–Machine Interaction (HMI), medicine can benefit most from the adoption of these digital technologies. In this perspective, the literature on orthopedic surgery techniques based on AR was evaluated, focusing on identifying the limitations and challenges of AR-based healthcare applications, to support the research and the development of further studies. Methods: Studies published from January 2018 to December 2021 were analyzed after a comprehensive search on PubMed, Google Scholar, Scopus, IEEE Xplore, Science Direct, and Wiley Online Library databases. In order to improve the review reporting, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used. Results: Authors selected sixty-two articles meeting the inclusion criteria, which were categorized according to the purpose of the study (intraoperative, training, rehabilitation) and according to the surgical procedure used. Conclusions: AR has the potential to improve orthopedic training and practice by providing an increasingly human-centered clinical approach. Further research can be addressed by this review to cover problems related to hardware limitations, lack of accurate registration and tracking systems, and absence of security protocols.

Serious Games and Mixed Reality Applications for Healthcare

Serious games are games in which the main goal is not entertainment, but a serious purpose ranging from the acquisition of knowledge to interactive training, to name just a few [...]

Development and Validation of a Mixed Reality Configuration of a Simulator for a Minimally Invasive Spine Surgery Using the Workspace of a Haptic Device and Simulator Users

Most surgical simulators leverage virtual or bench models to simulate reality. This study proposes and validates a method for workspace configuration of a surgical simulator which utilizes a haptic device for interaction with a virtual model and a bench model to provide additional tactile feedback based on planned surgical manoeuvers. Numerical analyses were completed to determine the workspace and position of a haptic device, relative to the bench model, used in the surgical simulator, and the determined configuration was validated using device limitations and user data from surgical and nonsurgical users. For the validation, surgeons performed an identical surgery on a cadaver prior to using the simulator, and their trajectories were then compared to the determined workspace for the haptic device. The configuration of the simulator was determined appropriate through workspace analysis and the collected user trajectories. Statistical analyses suggest differences in trajectories between the participating surgeons which were not affected by the imposed haptic workspace. This study, therefore, demonstrates a method to optimally position a haptic device with respect to a bench model while meeting the manoeuverability needs of a surgical procedure. The validation method identified workspace position and user trajectory towards ideal configuration of a mixed reality simulator.

Hybrid Simulation and Planning Platform for Cryosurgery with Microsoft HoloLens

Cryosurgery is a technique of growing popularity involving tissue ablation under controlled freezing. Technological advancement of devices along with surgical technique improvements have turned cryosurgery from an experimental to an established option for treating several diseases. However, cryosurgery is still limited by inaccurate planning based primarily on 2D visualization of the patient’s preoperative images. Several works have been aimed at modelling cryoablation through heat transfer simulations; however, most software applications do not meet some key requirements for clinical routine use, such as high computational speed and user-friendliness. This work aims to develop an intuitive platform for anatomical understanding and pre-operative planning by integrating the information content of radiological images and cryoprobe specifications either in a 3D virtual environment (desktop application) or in a hybrid simulator, which exploits the potential of the 3D printing and augmented reality functionalities of Microsoft HoloLens. The proposed platform was preliminarily validated for the retrospective planning/simulation of two surgical cases. Results suggest that the platform is easy and quick to learn and could be used in clinical practice to improve anatomical understanding, to make surgical planning easier than the traditional method, and to strengthen the memorization of surgical planning.

DICOM 3D viewers, virtual reality or 3D printing - a pilot usability study for assessing the preference of orthopedic surgeons

As standard practice in orthopedic surgery, the information gathered by analyzing Computer Tomography (CT) 2D images is used for patient diagnosis and planning surgery. Lately, these virtual slices are the input for generating 3D virtual models using DICOM viewers, facilitating spatial orientation, and diagnosis. Virtual Reality (VR) and 3D printing (3DP) technologies are also reported for use in anatomy visualization, medical training, and diagnosis. However, it has not been yet investigated whether the surgeons consider that the advantages offered by 3DP and VR outweigh their development efforts. Moreover, no comparative evaluation for understanding surgeon's preference in using these investigation tools has been performed so far. Therefore, in this paper, a pilot usability test was conducted for collecting surgeons' opinions. 3D models of knee, hip and foot were displayed using DICOM 3D viewer, two VR environments and as 3D-printed replicas. These tools adequacy for diagnosis was comparatively assessed in three cases scenarios, the time for completing the diagnosis tasks was recorded and questionnaires filled in. The time for preparing the models for VR and 3DP, the resources needed and the associated costs were presented in order to provide surgeons with the whole context. Results showed a preference in using desktop DICOM viewer with 3D capabilities along with the information provided by Unity-based VR solution for visualizing the virtual model from various angles challenging to analyze on the computer screen. 3D-printed replicas were considered more useful for physically simulating the surgery than for diagnosis. For the VR and 3DP models, the lack of information on bone quality was considered an important drawback. The following order of using the tools was preferred: DICOM viewer, followed by Unity VR and 3DP.