Three‐dimensional evaluation of the holographic projection in digital dental model superimposition using HoloLens device

Simulated augmented reality-based calibration of optical see-through head mound display for surgical navigation

PURPOSE

Calibration of an optical see-through head-mounted display is critical for augmented reality-based surgical navigation. While conventional methods have advanced, calibration errors remain significant. Moreover, prior research has focused primarily on calibration accuracy and procedure, neglecting the impact on the overall surgical navigation system. Consequently, these enhancements do not necessarily translate to accurate augmented reality in the optical see-through head mount due to systemic errors, including those in calibration.

METHOD

This study introduces a simulated augmented reality-based calibration to address these issues. By replicating the augmented reality that appeared in the optical see-through head mount, the method achieves calibration that compensates for augmented reality errors, thereby reducing them. The process involves two distinct calibration approaches, followed by adjusting the transformation matrix to minimize displacement in the simulated augmented reality.

RESULTS

The efficacy of this method was assessed through two accuracy evaluations: registration accuracy and augmented reality accuracy. Experimental results showed an average translational error of 2.14 mm and rotational error of 1.06° across axes in both approaches. Additionally, augmented reality accuracy, measured by the overlay regions' ratio, increased to approximately 95%. These findings confirm the enhancement in both calibration and augmented reality accuracy with the proposed method.

CONCLUSION

The study presents a calibration method using simulated augmented reality, which minimizes augmented reality errors. This approach, requiring minimal manual intervention, offers a more robust and precise calibration technique for augmented reality applications in surgical navigation.

Taxonomic discordance of immersive realities in dentistry: A systematic scoping review

OBJECTIVES

This review aimed to map taxonomy frameworks, descriptions, and applications of immersive technologies in the dental literature.

DATA

The Preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) guidelines was followed, and the protocol was registered at open science framework platform (https://doi.org/10.17605/OSF.IO/H6N8M).

SOURCES

Systematic search was conducted in MEDLINE (via PubMed), Scopus, and Cochrane Library databases, and complemented by manual search.

STUDY SELECTION

A total of 84 articles were included, with 81 % between 2019 and 2023. Most studies were experimental (62 %), including education (25 %), protocol feasibility (20 %), in vitro (11 %), and cadaver (6 %). Other study types included clinical report/technique article (24 %), clinical study (9 %), technical note/tip to reader (4 %), and randomized controlled trial (1 %). Three-quarters of the included studies were published in oral and maxillofacial surgery (38 %), dental education (26 %), and implant (12 %) disciplines. Methods of display included head mounted display device (HMD) (55 %), see through screen (32 %), 2D screen display (11 %), and projector display (2 %). Descriptions of immersive realities were fragmented and inconsistent with lack of clear taxonomy framework for the umbrella and the subset terms including virtual reality (VR), augmented reality (AR), mixed reality (MR), augmented virtuality (AV), extended reality, and X reality.

CONCLUSIONS

Immersive reality applications in dentistry are gaining popularity with a notable surge in the number of publications in the last 5 years. Ambiguities are apparent in the descriptions of immersive realities. A taxonomy framework based on method of display (full or partial) and reality class (VR, AR, or MR) is proposed.

CLINICAL SIGNIFICANCE

Understanding different reality classes can be perplexing due to their blurred boundaries and conceptual overlapping. Immersive technologies offer novel educational and clinical applications. This domain is fast developing. With the current fragmented and inconsistent terminologies, a comprehensive taxonomy framework is necessary.

Augmented Reality-Guided Apicoectomy Based on Maxillofacial CBCT Scans

Implementation of augmented reality (AR) image guidance systems using preoperative cone beam computed tomography (CBCT) scans in apicoectomies promises to help surgeons overcome iatrogenic complications associated with this procedure. This study aims to evaluate the intraoperative feasibility and usability of HoloLens 2, an established AR image guidance device, in the context of apicoectomies. Three experienced surgeons carried out four AR-guided apicoectomies each on human cadaver head specimens. Preparation and operating times of each procedure, as well as the subjective usability of HoloLens for AR image guidance in apicoectomies using the System Usability Scale (SUS), were measured. In total, twelve AR-guided apicoectomies on six human cadaver head specimens were performed (n = 12). The average preparation time amounted to 162 (±34) s. The surgical procedure itself took on average 9 (±2) min. There was no statistically significant difference between the three surgeons. Quantification of the usability of HoloLens revealed a mean SUS score of 80.4 (±6.8), indicating an "excellent" usability level. In conclusion, this study implies the suitability, practicality, and simplicity of AR image guidance systems such as the HoloLens in apicoectomies and advocates their routine implementation.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

User's image perception improved strategy and application of augmented reality systems in smart medical care: A review

BACKGROUND

Augmented reality (AR) is a new human-computer interaction technology that combines virtual reality, computer vision, and computer networks. With the rapid advancement of the medical field towards intelligence and data visualisation, AR systems are becoming increasingly popular in the medical field because they can provide doctors with clear enough medical images and accurate image navigation in practical applications. However, it has been discovered that different display types of AR systems have different effects on doctors' perception of the image after virtual-real fusion during the actual medical application. If doctors cannot correctly perceive the image, they may be unable to correctly match the virtual information with the real world, which will have a significant impact on their ability to recognise complex structures.

METHODS

This paper uses Citespace, a literature analysis tool, to visualise and analyse the research hotspots when AR systems are used in the medical field.

RESULTS

A visual analysis of the 1163 articles retrieved from the Web of Science Core Collection database reveals that display technology and visualisation technology are the key research directions of AR systems at the moment.

CONCLUSION

This paper categorises AR systems based on their display principles, reviews current image perception optimisation schemes for various types of systems, and analyses and compares different display types of AR systems based on their practical applications in the field of smart medical care so that doctors can select the appropriate display types based on different application scenarios. Finally, the future development direction of AR display technology is anticipated in order for AR technology to be more effectively applied in the field of smart medical care. The advancement of display technology for AR systems is critical for their use in the medical field, and the advantages and disadvantages of various display types should be considered in different application scenarios to select the best AR system.

Application of HoloLens-based augmented reality and three-dimensional printed anatomical tooth reference models in dental education

Tooth anatomy is fundamental knowledge used in everyday dental practice to reconstruct the occlusal surface during cavity fillings. The main objective of this project was to evaluate the suitability of two types of anatomical tooth reference models used to support reconstruction of the occlusal anatomy of the teeth: (1) a three-dimensional (3D)-printed model and (2) a model displayed in augmented reality (AR) using Microsoft HoloLens. The secondary objective was to evaluate three aspects impacting the outcome: clinical experience, comfort of work, and other variables. The tertiary objective was to evaluate the usefulness of AR in dental education. Anatomical models of crowns of three different molars were made using cone beam computed tomography image segmentation, printed with a stereolithographic 3D-printer, and then displayed in the HoloLens. Each participant reconstructed the occlusal anatomy of three teeth. One without any reference materials and two with an anatomical reference model, either 3D-printed or holographic. The reconstruction work was followed by the completion of an evaluation questionnaire. The maximum Hausdorff distances (Hmax) between the superimposed images of the specimens after the procedures and the anatomical models were then calculated. The results showed that the most accurate but slowest reconstruction was achieved with the use of 3D-printed reference models and that the results were not affected by other aspects considered. For this method, the Hmax was observed to be 630 μm (p = 0.004). It was concluded that while AR models can be helpful in dental anatomy education, they are not suitable replacements for physical models.

Evaluation of the impact of reference tooth morphology and alignment on model measurement accuracy

Background

The development of personalized and high-precision dental treatment is inseparable from the accurate measurement and analysis of the model. Compared with traditional plaster models, digital models allow dentists to obtain richer and more detailed inspection results. However, the measurement of digital models in clinical practice usually ignores the influence of the overall three-dimensional (3D) structure of teeth and tooth arrangement on the measurement results. The purpose of this study was to evaluate the effect of calibrated tooth axis and tooth arrangement on tooth width and arch length.

Methods

A total of 110 digital models from 80 participants were used to measure teeth width and dental arch length using the following methods: Method A, simple positioning of the proximal and distal of teeth; Method B: calibration of the clinical crown axis; and Method C: calibration of the overall 3D axis of the teeth. The Measurand model included pre- and post-orthodontic models of the same patients to assess the impact of tooth alignment on outcomes.

Results

In the aligned dentition, whether the tooth axis was calibrated had no effect on the measurement results. On unaligned dentitions, calibrating the pinion allowed for more accurate measurements, with Method C the closest to the true size. Furthermore, the arrangement of teeth affected the measurement, but there was no continuous linear correlation with arch length discrepancy (ALD).

Conclusions

Clinicians should choose appropriate measurement methods according to actual needs when performing model measurement, and should pay attention to the influence of tooth axis, tooth shape, and arrangement on the measurement results.

Application of holography and augmented reality based technology to visualize the internal structure of the dental root - a proof of concept

BACKGROUND

The Augmented Reality (AR) blends digital information with the real world. Thanks to cameras, sensors, and displays it can supplement the physical world with holographic images. Nowadays, the applications of AR range from navigated surgery to vehicle navigation.

DEVELOPMENT

The purpose of this feasibility study was to develop an AR holographic system implementing Vertucci's classification of dental root morphology to facilitate the study of tooth anatomy. It was tailored to run on the AR HoloLens 2 (Microsoft) glasses. The 3D tooth models were created in Autodesk Maya and exported to Unity software. The holograms of dental roots can be projected in a natural setting of the dental office. The application allowed to display 3D objects in such a way that they could be rotated, zoomed in/out, and penetrated. The advantage of the proposed approach was that students could learn a 3D internal anatomy of the teeth without environmental visual restrictions.

CONCLUSIONS

It is feasible to visualize internal dental root anatomy with AR holographic system. AR holograms seem to be attractive adjunct for learning of root anatomy.

Technikgestützte zahnmedizinische Hausbesuche durch nicht-ärztliches Fachpersonal zur Minderung des Ansteckungsrisikos

Ziel des Beitrags ist die Identifikation von Problemen, Meta-Anforderungen und Designprinzipien für den Einsatz von Mixed und Virtual Reality Brillen zur Unterstützung nicht-ärztlichen Fachpersonals bei zahnmedizinischen Hausbesuchen. Im Rahmen von zwei Gruppendiskussionen und einem Experteninterview wurden zunächst mögliche Einsatzszenarien identifiziert. Anschließend wurde eine systematische Literaturrecherche in den Datenbanken CINAHL, Business Source Premier und MEDLINE durchgeführt. In der gefundenen Literatur konnten 14 Probleme bei der Anwendung von Mixed und Virtual Reality Brillen identifiziert werden. Darauf basierend wurden 14 Meta-Anforderungen abgeleitet und in fünf Designprinzipien zusammengefasst. Abschließend wurden die Ergebnisse mit den Spezifikationen der Microsoft HoloLens 2 abgeglichen, um eine Eignung für die Unterstützung der geplanten Hausbesuche festzustellen. Zudem wurde ein Umsetzungskonzept skizziert. Die Ergebnisse dienen als wichtige Empfehlungen für die praxisnahe Umsetzung zukünftiger Konzepte bezüglich der Anwendung von Mixed und Virtual Reality Brillen im (zahn-)medizinischen Kontext. Die Literaturrecherche zeigt eine Forschungslücke im Bereich zahnmedizinischer Hausbesuche auf. Die Ergebnisse dieses Beitrags schaffen daher eine solide Basis für die zukünftige Forschung.

A novel visualization system of using augmented reality in knee replacement surgery: Enhanced bidirectional maximum correntropy algorithm

BACKGROUND AND AIM

Image registration and alignment are the main limitations of augmented reality (AR)-based knee replacement surgery. This research aims to decrease the registration error, eliminate outcomes that are trapped in local minima to improve the alignment problems, handle the occlusion and maximize the overlapping parts.

METHODOLOGY

Markerless image registration method was used for AR-based knee replacement surgery to guide and visualize the surgical operation. While weight least square algorithm was used to enhance stereo camera-based tracking by filling border occlusion in right-to-left direction and non-border occlusion from left-to-right direction.

RESULTS

This study has improved video precision to 0.57-0.61 mm alignment error. Furthermore, with the use of bidirectional points, that is, forward and backward directional cloud point, the iteration on image registration was decreased. This has led to improve the processing time as well. The processing time of video frames was improved to 7.4-11.74 frames per second.

CONCLUSIONS

It seems clear that this proposed system has focused on overcoming the misalignment difficulty caused by the movement of patient and enhancing the AR visualization during knee replacement surgery. The proposed system was reliable and favourable which helps in eliminating alignment error by ascertaining the optimal rigid transformation between two cloud points and removing the outliers and non-Gaussian noise. The proposed AR system helps in accurate visualization and navigation of anatomy of knee such as femur, tibia, cartilage, blood vessels and so forth.

Feasibility of the application of mixed reality in mandible reconstruction with fibula flap: A cadaveric specimen study

BACKGROUND

In recent years, a new technology, mixed reality (MR), has emerged and surpassed the limitations of augmented reality (AR) with its inability to interact with hologram. This study aimed to investigate the feasibility of the application of MR in mandible reconstruction with fibula flap.

METHODS

Computed tomography (CT) examination was performed for one cadaveric mandible and ten fibula bones. Using professional software Proplan CMF 3.0 (Materialize, Leuven, Belgium), we created a defected mandibular model and simulated the reconstruction design with these 10 fibula bones. The surgical plans were transferred to the HoloLens. We used HoloLens to guide the osteotomy and shaping of the fibular bone. After fixing the fibular segments using the Ti template, all segments underwent a CT examination. Before and after objects were compared for measurements of the location of fibular osteotomies, angular deviation of fibular segments, and intergonial angle distances.

RESULTS

The mean location of the fibular osteotomies, angular deviation of the fibular segments, and intergonial angle distances were 2.11 ± 1.31 mm, 2.85°± 1.97°, and 7.24 ± 3.42 mm, respectively.

CONCLUSION

The experimental results revealed that slight deviations remained in the accuracy of fibular osteotomy. With the further development of technology, it has the potential to improve the efficiency and precision of the reconstructive surgery.