Augmented reality-guided positioning system for radiotherapy patients

Head model dataset for mixed reality navigation in neurosurgical interventions for intracranial lesions

Mixed reality navigation (MRN) technology is emerging as an increasingly significant and interesting topic in neurosurgery. MRN enables neurosurgeons to "see through" the head with an interactive, hybrid visualization environment that merges virtual- and physical-world elements. Offering immersive, intuitive, and reliable guidance for preoperative and intraoperative intervention of intracranial lesions, MRN showcases its potential as an economically efficient and user-friendly alternative to standard neuronavigation systems. However, the clinical research and development of MRN systems present challenges: recruiting a sufficient number of patients within a limited timeframe is difficult, and acquiring low-cost, commercially available, medically significant head phantoms is equally challenging. To accelerate the development of novel MRN systems and surmount these obstacles, the study presents a dataset designed for MRN system development and testing in neurosurgery. It includes CT and MRI data from 19 patients with intracranial lesions and derived 3D models of anatomical structures and validation references. The models are available in Wavefront object (OBJ) and Stereolithography (STL) formats, supporting the creation and assessment of neurosurgical MRN applications.

Developing simulation-based learning application for radiation therapy students at pre-clinical stage

INTRODUCTION

Simulation-based education has been particularly valuable as a preclinical training method that adequately prepares students for clinical practice, including simulation in educational programs enhances the quality of learning outcomes. However, relevant previous research has exhibited several crucial limitations, with most of them having focused solely on the setup procedures. This study aimed to outline the development of an educational application in radiationtherapy and emphasizes the essential factors that radiation therapist technologists(RTTs) must consider in the treatment room from the perspective of experienced RTTs.

METHOD

We connected the virtual pendants to the linear accelerator components using C# programming and Unity. Customized scripts were assigned to specific linear accelerator (LINAC) functions, and the patient and RTT avatars were developed. We also included audio feedback for the realistic gantry movement sounds.

RESULT

This study outlines various aspects of radiotherapy procedures duringtreatment, such as the simulation of patient positioning, treatment fields, and pendantfunctions, aimed toward enabling the effective use of virtual reality technology inradiation therapy.

DISCUSSION

This study explores the potential of an avatar-based app for radiotherapy education, providing foundational data for future trials.

CONCLUSION

Simulation learning is the most advantageous pre-clinical instrument for equipping students with the skills necessary for clinical practice. This study's resultsare expected to facilitate radiotherapy students' adoption of clinical replacement applications and improve collaborative partnerships and knowledge sharing. Notably, this application complements traditional learning methods, further enhancing the overall educational experience.

Feasibility evaluation of radiotherapy positioning system guided by augmented reality and point cloud registration

PURPOSE

To develop a radiotherapy positioning system based on Point Cloud Registration (PCR) and Augmented Reality (AR), and to verify its feasibility.

METHODS

The optimal steps of PCR were investigated, and virtual positioning experiments were designed to evaluate its accuracy and speed. AR was implemented by Unity 3D and Vuforia for initial position correction, and PCR for precision registration, to achieve the proposed radiotherapy positioning system. Feasibility of the proposed system was evaluated through phantom positioning tests as well as real human positioning tests. Real human tests involved breath-holding positioning and free-breathing positioning tests. Evaluation metrics included 6 Degree of Freedom (DOF) deviations and Distance (D) errors. Additionally, the interaction between CBCT and the proposed system was envisaged through CBCT and optical cross-source PCR.

RESULTS

Point-to-plane iterative Closest Point (ICP), statistical filtering, uniform down-sampling, and optimal sampling ratio were determined for PCR procedure. In virtual positioning tests, a single registration took only 0.111 s, and the average D error for 15 patients was 0.015 ± 0.029 mm., Errors of phantom tests were at the sub-millimeter level, with an average D error of 0.6 ± 0.2 mm. In the real human positioning tests, the average accuracy of breath-holding positioning was still at the sub-millimeter level, where the errors of X, Y and Z axes were 0.59 ± 0.12 mm, 0.54 ± 0.12 mm, and 0.52 ± 0.09 mm, and the average D error was 0.96 ± 0.22 mm. In the free-breathing positioning, the average errors in X, Y, and Z axes were still less than 1 mm. Although the mean D error was 1.93 ± 0.36 mm, it still falls within a clinically acceptable error margin.

CONCLUSION

The AR and PCR-guided radiotherapy positioning system enables markerless, radiation-free and high-accuracy positioning, which is feasible in real-world scenarios.

The Feasibility and Accuracy of Holographic Navigation with Laser Crosshair Simulator Registration on a Mixed-Reality Display

Addressing conventional neurosurgical navigation systems' high costs and complexity, this study explores the feasibility and accuracy of a simplified, cost-effective mixed reality navigation (MRN) system based on a laser crosshair simulator (LCS). A new automatic registration method was developed, featuring coplanar laser emitters and a recognizable target pattern. The workflow was integrated into Microsoft's HoloLens-2 for practical application. The study assessed the system's precision by utilizing life-sized 3D-printed head phantoms based on computed tomography (CT) or magnetic resonance imaging (MRI) data from 19 patients (female/male: 7/12, average age: 54.4 ± 18.5 years) with intracranial lesions. Six to seven CT/MRI-visible scalp markers were used as reference points per case. The LCS-MRN's accuracy was evaluated through landmark-based and lesion-based analyses, using metrics such as target registration error (TRE) and Dice similarity coefficient (DSC). The system demonstrated immersive capabilities for observing intracranial structures across all cases. Analysis of 124 landmarks showed a TRE of 3.0 ± 0.5 mm, consistent across various surgical positions. The DSC of 0.83 ± 0.12 correlated significantly with lesion volume (Spearman rho = 0.813, p < 0.001). Therefore, the LCS-MRN system is a viable tool for neurosurgical planning, highlighting its low user dependency, cost-efficiency, and accuracy, with prospects for future clinical application enhancements.

A Novel Registration Method for a Mixed Reality Navigation System Based on a Laser Crosshair Simulator: A Technical Note

Mixed Reality Navigation (MRN) is pivotal in augmented reality-assisted intelligent neurosurgical interventions. However, existing MRN registration methods face challenges in concurrently achieving low user dependency, high accuracy, and clinical applicability. This study proposes and evaluates a novel registration method based on a laser crosshair simulator, evaluating its feasibility and accuracy. A novel registration method employing a laser crosshair simulator was introduced, designed to replicate the scanner frame's position on the patient. The system autonomously calculates the transformation, mapping coordinates from the tracking space to the reference image space. A mathematical model and workflow for registration were designed, and a Universal Windows Platform (UWP) application was developed on HoloLens-2. Finally, a head phantom was used to measure the system's target registration error (TRE). The proposed method was successfully implemented, obviating the need for user interactions with virtual objects during the registration process. Regarding accuracy, the average deviation was 3.7 ± 1.7 mm. This method shows encouraging results in efficiency and intuitiveness and marks a valuable advancement in low-cost, easy-to-use MRN systems. The potential for enhancing accuracy and adaptability in intervention procedures positions this approach as promising for improving surgical outcomes.

Machine learning-based radiotherapy time prediction and treatment scheduling management

PURPOSE

The utility efficiency of medical devices is important, especially for countries such as China, which have a large population and shortage of medical care resources. Radiotherapy devices are among the most valuable and specialized equipment categories and carry enormous treatment loads. In this study, a novel method is proposed to improve the efficiency of a radiotherapy device (linac). Although scheduling management with accurate prediction of the entire treatment time included in each appointment, arrange a reasonable time duration for appointments and save time between patient shifts effectively. Tasks belonging to the treatment and non-treatment groups can be assigned more flexibly based on the availability of time.

MATERIAL AND METHODS

Data from 1665 patients, including patient positioning time (PT) and treatment time (TT), were collected in collaboration with the Radiotherapy Center of the Department of Oncology at the Second Affiliated Hospital of Kunming Medical University from November 2020 to August 2021. The features related to PT and TT were extracted and used to train the machine learning-based model to predict PT and TT in independent patients. The prediction results were subsequently applied to a minute-based scheduling tool.

CONCLUSION

Artificial intelligence is a promising approach to solve abstract problems with a specialized knowledge background. The results of this study show encouraging prediction outcomes in relation to effective scheduling management and could improve the efficiency of the linac. This successful trial broadens the meaning of medical data and potential future research directions in radiotherapy.

Innovative integration of augmented reality and optical surface imaging: A coarse-to-precise system for radiotherapy positioning

BACKGROUND

Traditional methods of radiotherapy positioning have shortcomings such as fragile skin-markers, additional doses, and lack of information integration. Emerging technologies may provide alternatives for the relevant clinical practice.

PURPOSE

To propose a noninvasive radiotherapy positioning system integrating augmented reality (AR) and optical surface, and to evaluate its feasibility in clinical workflow.

METHODS

AR and structured light-based surface were integrated to implement the coarse-to-precise positioning through two coherent steps, the AR-based coarse guidance and the optical surface-based precise verification. To implement quality assurance, recognition of face and pattern was used for patient authentication, case association, and accessory validation in AR scenes. The holographic images reconstructed from simulation computed tomography (CT) images, guided the initial posture correction by virtual-real alignment. The point clouds of body surface were fused, with the calibration and pose estimation of structured light cameras, and segmented according to the preset regions of interest (ROIs). The global-to-local registration for cross-source point clouds was achieved to calculate couch shifts in six degrees-of-freedom (DoF), which were ultimately transmitted to AR scenes. The evaluation based on phantom and human-body (4 volunteers) included, (i) quality assurance workflow, (ii) errors of both steps and correlation analysis, (iii) receiver operating characteristic (ROC), (iv) distance characteristics of accuracy, and (v) clinical positioning efficiency.

RESULTS

The maximum errors in phantom evaluation were 3.4 ± 2.5 mm in Vrt and 1.4 ± 1.0° in Pitch for the coarse guidance step, while 1.6 ± 0.9 mm in Vrt and 0.6 ± 0.4° in Pitch for the precise verification step. The Pearson correlation coefficients between precise verification and cone beam CT (CBCT) results were distributed in the interval [0.81, 0.85]. In ROC analysis, the areas under the curve (AUC) were 0.87 and 0.89 for translation and rotation, respectively. In human body-based evaluation, the errors of thorax and abdomen (T&A) were significantly greater than those of head and neck (H&N) in Vrt (2.6 ± 1.1 vs. 1.7 ± 0.8, p < 0.01), Lng (2.3 ± 1.1 vs. 1.4 ± 0.9, p < 0.01), and Rtn (0.8 ± 0.4 vs. 0.6 ± 0.3, p = 0.01) while relatively similar in Lat (1.8 ± 0.9 vs. 1.7 ± 0.8, p = 0.07). The translation displacement range, after coarse guidance step, required for high accuracy of the optical surface component of the integrated system was 0-42 mm, and the average positioning duration of the integrated system was significantly less than that of conventional workflow (355.7 ± 21.7 vs. 387.7 ± 26.6 s, p < 0.01).

CONCLUSIONS

The combination of AR and optical surface has utility and feasibility for patient positioning, in terms of both safety and accuracy.

Development and feasibility evaluation of an AR-assisted radiotherapy positioning system

Purpose

The aim of this study is to develop an augmented reality (AR)–assisted radiotherapy positioning system based on HoloLens 2 and to evaluate the feasibility and accuracy of this method in the clinical environment.

Methods

The obtained simulated computed tomography (CT) images of an “ISO cube”, a cube phantom, and an anthropomorphic phantom were reconstructed into three-dimensional models and imported into the HoloLens 2. On the basis of the Vuforia marker attached to the “ISO cube” placed at the isocentric position of the linear accelerator, the correlation between the virtual and real space was established. First, the optimal conditions to minimize the deviation between virtual and real objects were explored under different conditions with a cube phantom. Then, the anthropomorphic phantom–based positioning was tested under the optimal conditions, and the positioning errors were evaluated with cone-beam CT.

Results

Under the normal light intensity, the registration and tracking angles are 0°, the distance is 40 cm, and the deviation reached a minimum of 1.4 ± 0.3 mm. The program would not run without light. The hologram drift caused by the light change, camera occlusion, and head movement were 0.9 ± 0.7 mm, 1.0 ± 0.6 mm, and 1.5 ± 0.9 mm, respectively. The anthropomorphic phantom–based positioning errors were 3.1 ± 1.9 mm, 2.4 ± 2.5 mm, and 4.6 ± 2.8 mm in the X (lateral), Y (vertical), and Z (longitudinal) axes, respectively, and the angle deviation of Rtn was 0.26 ± 0.14°.

Conclusion

The AR-assisted radiotherapy positioning based on HoloLens 2 is a feasible method with certain advantages, such as intuitive visual guidance, radiation-free position verification, and intelligent interaction. Hardware and software upgrades are expected to further improve accuracy and meet clinicalbrendaannmae requirements.

Brain Tumor and Augmented Reality: New Technologies for the Future

In recent years, huge progress has been made in the management of brain tumors, due to the availability of imaging devices, which provide fundamental anatomical and pathological information not only for diagnostic purposes [...].