Augmented Reality Based Surgical Navigation of Complex Pelvic Osteotomies—A Feasibility Study on Cadavers

Tablet-based Augmented reality and 3D printed templates in fully guided Microtia Reconstruction: a clinical workflow

BACKGROUND

Microtia is a congenital malformation of the auricle that affects approximately 4 of every 10,000 live newborns. Radiographic film paper is traditionally employed to bidimensionally trace the structures of the contralateral healthy ear in a quasi-artistic manner. Anatomical points provide linear and angular measurements. However, this technique proves time-consuming, subjectivity-rich, and greatly dependent on surgeon expertise. Hence, it's susceptible to shape errors and misplacement.

METHODS

We present an innovative clinical workflow that combines 3D printing and augmented reality (AR) to increase objectivity and reproducibility of these procedures. Specifically, we introduce patient-specific 3D cutting templates and remodeling molds to carve and construct the cartilaginous framework that will conform the new ear. Moreover, we developed an in-house AR application compatible with any commercial Android tablet. It precisely guides the positioning of the new ear during surgery, ensuring symmetrical alignment with the healthy one and avoiding time-consuming intraoperative linear or angular measurements. Our solution was evaluated in one case, first with controlled experiments in a simulation scenario and finally during surgery.

RESULTS

Overall, the ears placed in the simulation scenario had a mean absolute deviation of 2.2 ± 1.7 mm with respect to the reference plan. During the surgical intervention, the reconstructed ear was 3.1 mm longer and 1.3 mm wider with respect to the ideal plan and had a positioning error of 2.7 ± 2.4 mm relative to the contralateral side. Note that in this case, additional morphometric variations were induced from inflammation and other issues intended to be addressed in a subsequent stage of surgery, which are independent of our proposed solution.

CONCLUSIONS

In this work we propose an innovative workflow that combines 3D printing and AR to improve ear reconstruction and positioning in microtia correction procedures. Our implementation in the surgical workflow showed good accuracy, empowering surgeons to attain consistent and objective outcomes.

Augmented reality-guided pelvic osteotomy of Ganz: feasibility in cadavers

INTRODUCTION

The periacetabular osteotomy is a technically demanding procedure with the goal to improve the osseous containment of the femoral head. The options for controlled execution of the osteotomies and verification of the acetabular reorientation are limited. With the assistance of augmented reality, new possibilities are emerging to guide this intervention. However, the scientific knowledge regarding AR navigation for PAO is sparse.

METHODS

In this cadaveric study, we wanted to find out, if the execution of this complex procedure is feasible with AR guidance, quantify the accuracy of the execution of the three-dimensional plan, and find out what has to be done to proceed to real surgery. Therefore, an AR guidance for the PAO was developed and applied on 14 human hip cadavers. The guidance included performance of the four osteotomies and reorientation of the acetabular fragment. The osteotomy starting points, the orientation of the osteotomy planes, as well as the reorientation of the acetabular fragment were compared to the 3D planning.

RESULTS

The mean 3D distance between planned and performed starting points was between 9 and 17 mm. The mean angle between planned and performed osteotomies was between 6° and 7°. The mean reorientation error between the planned and performed rotation of the acetabular fragment was between 2° and 11°.

CONCLUSION

The planned correction can be achieved with promising accuracy and without serious errors. Further steps for a translation from the cadaver to the patient have been identified and must be addressed in future work.

Management of chondral and osteochondral lesions of the hip : A comprehensive review

Chondral and osteochondral lesions encompass several acute or chronic defects of the articular cartilage and/or subchondral bone. These lesions can result from several different diseases and injuries, including osteochondritis dissecans, osteochondral defects, osteochondral fractures, subchondral bone osteonecrosis, and insufficiency fractures. As the cartilage has a low capacity for regeneration and self-repair, these lesions can progress to osteoarthritis. This study provides a comprehensive overview of the subject matter that it covers. PubMed, Scopus and Google Scholar were accessed using the following keywords: "chondral lesions/defects of the femoral head", "chondral/cartilage lesions/defects of the acetabulum", "chondral/cartilage lesions/defects of the hip", "osteochondral lesions of the femoral head", "osteochondral lesions of the acetabulum", "osteochondral lesions of the hip", "osteochondritis dissecans," "early osteoarthritis of the hip," and "early stage avascular necrosis". Hip osteochondral injuries can cause significant damage to the articular surface and diminish the quality of life. It can be difficult to treat such injuries, especially in patients who are young and active. Several methods are used to treat chondral and osteochondral injuries of the hip, such as mesenchymal stem cells and cell-based treatment, surgical repair, and microfractures. Realignment of bony anatomy may also be necessary for optimal outcomes. Despite several treatments being successful, there is a lack of head-to-head comparisons and large sample size studies in the current literature. Additional research will be required to provide appropriate clinical recommendations for treating chondral/osteochondral injuries of the hip joint.

Automatic 3D Postoperative Evaluation of Complex Orthopaedic Interventions

In clinical practice, image-based postoperative evaluation is still performed without state-of-the-art computer methods, as these are not sufficiently automated. In this study we propose a fully automatic 3D postoperative outcome quantification method for the relevant steps of orthopaedic interventions on the example of Periacetabular Osteotomy of Ganz (PAO). A typical orthopaedic intervention involves cutting bone, anatomy manipulation and repositioning as well as implant placement. Our method includes a segmentation based deep learning approach for detection and quantification of the cuts. Furthermore, anatomy repositioning was quantified through a multi-step registration method, which entailed a coarse alignment of the pre- and postoperative CT images followed by a fine fragment alignment of the repositioned anatomy. Implant (i.e., screw) position was identified by 3D Hough transform for line detection combined with fast voxel traversal based on ray tracing. The feasibility of our approach was investigated on 27 interventions and compared against manually performed 3D outcome evaluations. The results show that our method can accurately assess the quality and accuracy of the surgery. Our evaluation of the fragment repositioning showed a cumulative error for the coarse and fine alignment of 2.1 mm. Our evaluation of screw placement accuracy resulted in a distance error of 1.32 mm for screw head location and an angular deviation of 1.1° for screw axis. As a next step we will explore generalisation capabilities by applying the method to different interventions.

Exploring the Performance of Geometry-Based Markerless Registration in a Simulated Surgical Environment: A Comparative Study of Registration Algorithms in Medical Augmented Reality

Augmented Reality (AR) has been utilized in multiple applications in the medical field, such as augmenting Computed Tomography (CT) images onto the patient's body during surgery. However, one of the challenges in its utilization is to register the pre-operative CT images to the patient's body accurately. The current registration process requires prior attachment of tracking markers, and their localization within the body and CT images. This process can be cumbersome, error-prone, and dependent on the surgeon's experience. Moreover, there are cases where medical instruments, drapes, or the body may occlude the markers. In light of these limitations, markerless registration algorithms have the potential to aid the registration process in the clinical setting. While those algorithms have been successfully used in other sectors, such as multimedia, they have not yet been thoroughly investigated in a clinical setting, especially in surgery, where there are more challenging cases with different positions of the patients in the image and the surgical environment. In this paper, we benchmarked and evaluated the performance of 6 state-of-the-art markerless registration algorithms from the multimedia sector by registering a CT image onto the whole-body phantom dataset acquired from a simulated surgical environment. We also analyzed the suitability of these algorithms for use in the surgical setting and discussed their potential for the advancement of AR-assisted surgery.Clinical Relevance-Our study provides insight into the potential of AR-assisted surgery and helps practitioners in choosing the most suitable registration algorithm for their needs to improve patient outcomes, reduce the risk of surgical errors and shorten the time of preoperative planning.

Visualization, registration and tracking techniques for augmented reality guided surgery: a review

Augmented reality (AR) surgical navigation has developed rapidly in recent years. This paper reviews and analyzes the visualization, registration, and tracking techniques used in AR surgical navigation systems, as well as the application of these AR systems in different surgical fields. The types of AR visualization are divided into two categories ofin situvisualization and nonin situvisualization. The rendering contents of AR visualization are various. The registration methods include manual registration, point-based registration, surface registration, marker-based registration, and calibration-based registration. The tracking methods consist of self-localization, tracking with integrated cameras, external tracking, and hybrid tracking. Moreover, we describe the applications of AR in surgical fields. However, most AR applications were evaluated through model experiments and animal experiments, and there are relatively few clinical experiments, indicating that the current AR navigation methods are still in the early stage of development. Finally, we summarize the contributions and challenges of AR in the surgical fields, as well as the future development trend. Despite the fact that AR-guided surgery has not yet reached clinical maturity, we believe that if the current development trend continues, it will soon reveal its clinical utility.

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.

Augmented Reality in Surgery: A Scoping Review

Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.

Augmented Reality as a Tool to Guide Patient-Specific Templates Placement in Pelvic Resections

Patient-specific templates (PST) have become a useful tool for guiding osteotomy in complex surgical scenarios such as pelvic resections. The design of the surgical template results in sharper, less jagged resection margins than freehand cuts. However, their correct placement can become difficult in some anatomical regions and cannot be verified during surgery. Conventionally, pelvic resections are performed using Computer Assisted Surgery (CAS), and in recent years Augmented Reality (AR) has been proposed in the literature as an additional tool to support PST placement. This work presents an AR task to simplify and improve the accuracy of the positioning of the template by displaying virtual content. The focus of the work is the creation of the virtual guides displayed during the AR task. The system was validated on a patient-specific phantom designed to provide a realistic setup. Encouraging results have been achieved. The use of the AR simplifies the surgical task and optimizes the correct positioning of the cutting template: an average error of 2.19 mm has been obtained, lower than obtained with state-of-the-art solutions. In addition, supporting PST placement through AR guidance is less time-consuming than the standard procedure that solely relies on anatomical landmarks as reference.

Augmented Reality: Mapping Methods and Tools for Enhancing the Human Role in Healthcare HMI

Background: Augmented Reality (AR) represents an innovative technology to improve data visualization and strengthen the human perception. Among Human–Machine Interaction (HMI), medicine can benefit most from the adoption of these digital technologies. In this perspective, the literature on orthopedic surgery techniques based on AR was evaluated, focusing on identifying the limitations and challenges of AR-based healthcare applications, to support the research and the development of further studies. Methods: Studies published from January 2018 to December 2021 were analyzed after a comprehensive search on PubMed, Google Scholar, Scopus, IEEE Xplore, Science Direct, and Wiley Online Library databases. In order to improve the review reporting, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used. Results: Authors selected sixty-two articles meeting the inclusion criteria, which were categorized according to the purpose of the study (intraoperative, training, rehabilitation) and according to the surgical procedure used. Conclusions: AR has the potential to improve orthopedic training and practice by providing an increasingly human-centered clinical approach. Further research can be addressed by this review to cover problems related to hardware limitations, lack of accurate registration and tracking systems, and absence of security protocols.

The "true" acetabular anteversion angle (AV angle): 2D CT versus 3D model

INTRODUCTION

Different factors can lead to inconsistencies in measurement for the acetabular version using 2D axial CT-cuts. We have defined a "true" anteversion angle (AV angle) in the physiological position of the pelvis in 3D with the largest European population measured to our knowledge.

MATERIAL AND METHODS

We analyzed 258 hemipelvises and created 3D models. We compared the results of our AV angle 3D method with the cross-sectional cuts of the same acetabula. We included factors like side, sex, body mass index, and patient positioning.

RESULTS

Overall, the mean (SD) AV angle was 16.1 (5.9)° as measured with the 3D method and 22.0 (6.0)° as measured with the 2D method (p < 0.0001). Measured with both the 3D and the 2D method, the AV angle was significantly larger in female than in male individuals (p < 0.0001). In the 2D method, the AV angle estimation was influenced by the pelvic tilt.

CONCLUSION

We propose a more accurate method for the measurement of the AV angle of the acetabulum in a 3D model that is not influenced by patient positioning or pelvic tilt. We provide a computational model that will facilitate operative decisions and improve preoperative planning. We confirm that 3D measurement should be the gold standard in measuring the acetabular anteversion.

Augmented Reality as a Tool to Guide PSI Placement in Pelvic Tumor Resections

Patient-specific instruments (PSIs) have become a valuable tool for osteotomy guidance in complex surgical scenarios such as pelvic tumor resection. They provide similar accuracy to surgical navigation systems but are generally more convenient and faster. However, their correct placement can become challenging in some anatomical regions, and it cannot be verified objectively during the intervention. Incorrect installations can result in high deviations from the planned osteotomy, increasing the risk of positive resection margins. In this work, we propose to use augmented reality (AR) to guide and verify PSIs placement. We designed an experiment to assess the accuracy provided by the system using a smartphone and the HoloLens 2 and compared the results with the conventional freehand method. The results showed significant differences, where AR guidance prevented high osteotomy deviations, reducing maximal deviation of 54.03 mm for freehand placements to less than 5 mm with AR guidance. The experiment was performed in two versions of a plastic three-dimensional (3D) printed phantom, one including a silicone layer to simulate tissue, providing more realism. We also studied how differences in shape and location of PSIs affect their accuracy, concluding that those with smaller sizes and a homogeneous target surface are more prone to errors. Our study presents promising results that prove AR's potential to overcome the present limitations of PSIs conveniently and effectively.