1
|
Zhang G, Jiang Z, Zhu J, Dai T, He X, Liu X, Chang Y, Wang L. Innovative integration of augmented reality and optical surface imaging: A coarse-to-precise system for radiotherapy positioning. Med Phys 2023. [PMID: 37060328 DOI: 10.1002/mp.16417] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Revised: 04/01/2023] [Accepted: 04/02/2023] [Indexed: 04/16/2023] Open
Abstract
BACKGROUND Traditional methods of radiotherapy positioning have shortcomings such as fragile skin-markers, additional doses, and lack of information integration. Emerging technologies may provide alternatives for the relevant clinical practice. PURPOSE To propose a noninvasive radiotherapy positioning system integrating augmented reality (AR) and optical surface, and to evaluate its feasibility in clinical workflow. METHODS AR and structured light-based surface were integrated to implement the coarse-to-precise positioning through two coherent steps, the AR-based coarse guidance and the optical surface-based precise verification. To implement quality assurance, recognition of face and pattern was used for patient authentication, case association, and accessory validation in AR scenes. The holographic images reconstructed from simulation computed tomography (CT) images, guided the initial posture correction by virtual-real alignment. The point clouds of body surface were fused, with the calibration and pose estimation of structured light cameras, and segmented according to the preset regions of interest (ROIs). The global-to-local registration for cross-source point clouds was achieved to calculate couch shifts in six degrees-of-freedom (DoF), which were ultimately transmitted to AR scenes. The evaluation based on phantom and human-body (4 volunteers) included, (i) quality assurance workflow, (ii) errors of both steps and correlation analysis, (iii) receiver operating characteristic (ROC), (iv) distance characteristics of accuracy, and (v) clinical positioning efficiency. RESULTS The maximum errors in phantom evaluation were 3.4 ± 2.5 mm in Vrt and 1.4 ± 1.0° in Pitch for the coarse guidance step, while 1.6 ± 0.9 mm in Vrt and 0.6 ± 0.4° in Pitch for the precise verification step. The Pearson correlation coefficients between precise verification and cone beam CT (CBCT) results were distributed in the interval [0.81, 0.85]. In ROC analysis, the areas under the curve (AUC) were 0.87 and 0.89 for translation and rotation, respectively. In human body-based evaluation, the errors of thorax and abdomen (T&A) were significantly greater than those of head and neck (H&N) in Vrt (2.6 ± 1.1 vs. 1.7 ± 0.8, p < 0.01), Lng (2.3 ± 1.1 vs. 1.4 ± 0.9, p < 0.01), and Rtn (0.8 ± 0.4 vs. 0.6 ± 0.3, p = 0.01) while relatively similar in Lat (1.8 ± 0.9 vs. 1.7 ± 0.8, p = 0.07). The translation displacement range, after coarse guidance step, required for high accuracy of the optical surface component of the integrated system was 0-42 mm, and the average positioning duration of the integrated system was significantly less than that of conventional workflow (355.7 ± 21.7 vs. 387.7 ± 26.6 s, p < 0.01). CONCLUSIONS The combination of AR and optical surface has utility and feasibility for patient positioning, in terms of both safety and accuracy.
Collapse
Affiliation(s)
- Gongsen Zhang
- Artificial Intelligence Laboratory, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, Shandong, China
| | - Zejun Jiang
- Department of Radiation Oncology Physics and Technology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, Shandong, China
| | - Jian Zhu
- Department of Radiation Oncology Physics and Technology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, Shandong, China
| | - Tianyuan Dai
- Department of Radiation Oncology Physics and Technology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, Shandong, China
| | - Xiaolong He
- Artificial Intelligence Laboratory, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, Shandong, China
| | - Xinchao Liu
- Cheeloo College of Medicine, Shandong University, Ji'nan, Shandong, China
| | - Yankui Chang
- School of Nuclear Science and Technology, University of Science and Technology of China, Hefei, China
| | - Linlin Wang
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, Shandong, China
| |
Collapse
|
2
|
Zhang G, Liu X, Wang L, Zhu J, Yu J. Development and feasibility evaluation of an AR-assisted radiotherapy positioning system. Front Oncol 2022; 12:921607. [PMID: 36267969 PMCID: PMC9577500 DOI: 10.3389/fonc.2022.921607] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2022] [Accepted: 09/13/2022] [Indexed: 11/28/2022] Open
Abstract
Purpose The aim of this study is to develop an augmented reality (AR)–assisted radiotherapy positioning system based on HoloLens 2 and to evaluate the feasibility and accuracy of this method in the clinical environment. Methods The obtained simulated computed tomography (CT) images of an “ISO cube”, a cube phantom, and an anthropomorphic phantom were reconstructed into three-dimensional models and imported into the HoloLens 2. On the basis of the Vuforia marker attached to the “ISO cube” placed at the isocentric position of the linear accelerator, the correlation between the virtual and real space was established. First, the optimal conditions to minimize the deviation between virtual and real objects were explored under different conditions with a cube phantom. Then, the anthropomorphic phantom–based positioning was tested under the optimal conditions, and the positioning errors were evaluated with cone-beam CT. Results Under the normal light intensity, the registration and tracking angles are 0°, the distance is 40 cm, and the deviation reached a minimum of 1.4 ± 0.3 mm. The program would not run without light. The hologram drift caused by the light change, camera occlusion, and head movement were 0.9 ± 0.7 mm, 1.0 ± 0.6 mm, and 1.5 ± 0.9 mm, respectively. The anthropomorphic phantom–based positioning errors were 3.1 ± 1.9 mm, 2.4 ± 2.5 mm, and 4.6 ± 2.8 mm in the X (lateral), Y (vertical), and Z (longitudinal) axes, respectively, and the angle deviation of Rtn was 0.26 ± 0.14°. Conclusion The AR-assisted radiotherapy positioning based on HoloLens 2 is a feasible method with certain advantages, such as intuitive visual guidance, radiation-free position verification, and intelligent interaction. Hardware and software upgrades are expected to further improve accuracy and meet clinicalbrendaannmae requirements.
Collapse
Affiliation(s)
- Gongsen Zhang
- 1Department of Radiology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, China
| | - Xinchao Liu
- Cancer Center, Shandong University, Jinan, China
| | - Linlin Wang
- 1Department of Radiology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, China
- Cancer Center, Shandong University, Jinan, China
- *Correspondence: Linlin Wang, ; Jinming Yu,
| | - Jian Zhu
- 1Department of Radiology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, China
| | - Jinming Yu
- 1Department of Radiology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, China
- *Correspondence: Linlin Wang, ; Jinming Yu,
| |
Collapse
|
3
|
Li C, Lu Z, He M, Sui J, Lin T, Xie K, Sun J, Ni X. Augmented reality-guided positioning system for radiotherapy patients. J Appl Clin Med Phys 2022; 23:e13516. [PMID: 34985188 PMCID: PMC8906221 DOI: 10.1002/acm2.13516] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Revised: 11/18/2021] [Accepted: 12/15/2021] [Indexed: 01/22/2023] Open
Abstract
In modern radiotherapy, error reduction in the patients’ daily setup error is important for achieving accuracy. In our study, we proposed a new approach for the development of an assist system for the radiotherapy position setup by using augmented reality (AR). We aimed to improve the accuracy of the position setup of patients undergoing radiotherapy and to evaluate the error of the position setup of patients who were diagnosed with head and neck cancer, and that of patients diagnosed with chest and abdomen cancer. We acquired the patient's simulation CT data for the three‐dimensional (3D) reconstruction of the external surface and organs. The AR tracking software detected the calibration module and loaded the 3D virtual model. The calibration module was aligned with the Linac isocenter by using room lasers. And then aligned the virtual cube with the calibration module to complete the calibration of the 3D virtual model and Linac isocenter. Then, the patient position setup was carried out, and point cloud registration was performed between the patient and the 3D virtual model, such the patient's posture was consistent with the 3D virtual model. Twenty patients diagnosed with head and neck cancer and 20 patients diagnosed with chest and abdomen cancer in the supine position setup were analyzed for the residual errors of the conventional laser and AR‐guided position setup. Results show that for patients diagnosed with head and neck cancer, the difference between the two positioning methods was not statistically significant (P > 0.05). For patients diagnosed with chest and abdomen cancer, the residual errors of the two positioning methods in the superior and inferior direction and anterior and posterior direction were statistically significant (t = −5.80, −4.98, P < 0.05). The residual errors in the three rotation directions were statistically significant (t = −2.29 to −3.22, P < 0.05). The experimental results showed that the AR technology can effectively assist in the position setup of patients undergoing radiotherapy, significantly reduce the position setup errors in patients diagnosed with chest and abdomen cancer, and improve the accuracy of radiotherapy.
Collapse
Affiliation(s)
- Chunying Li
- Department of Radiotherapy, Changzhou Second People's Hospital, Nanjing Medical University, Changzhou, China.,Laboratory of Medical Physics Center, Nanjing Medical University, Jiangning District, Nanjing, China.,Changzhou Key Laboratory of Medical Physics, Changzhou, China
| | - Zhengda Lu
- Department of Radiotherapy, Changzhou Second People's Hospital, Nanjing Medical University, Changzhou, China.,Laboratory of Medical Physics Center, Nanjing Medical University, Jiangning District, Nanjing, China.,Changzhou Key Laboratory of Medical Physics, Changzhou, China
| | - Mu He
- Department of Radiotherapy, Changzhou Second People's Hospital, Nanjing Medical University, Changzhou, China.,Laboratory of Medical Physics Center, Nanjing Medical University, Jiangning District, Nanjing, China.,Changzhou Key Laboratory of Medical Physics, Changzhou, China
| | - Jianfeng Sui
- Department of Radiotherapy, Changzhou Second People's Hospital, Nanjing Medical University, Changzhou, China.,Laboratory of Medical Physics Center, Nanjing Medical University, Jiangning District, Nanjing, China.,Changzhou Key Laboratory of Medical Physics, Changzhou, China
| | - Tao Lin
- Department of Radiotherapy, Changzhou Second People's Hospital, Nanjing Medical University, Changzhou, China.,Laboratory of Medical Physics Center, Nanjing Medical University, Jiangning District, Nanjing, China.,Changzhou Key Laboratory of Medical Physics, Changzhou, China
| | - Kai Xie
- Department of Radiotherapy, Changzhou Second People's Hospital, Nanjing Medical University, Changzhou, China.,Laboratory of Medical Physics Center, Nanjing Medical University, Jiangning District, Nanjing, China.,Changzhou Key Laboratory of Medical Physics, Changzhou, China
| | - Jiawei Sun
- Department of Radiotherapy, Changzhou Second People's Hospital, Nanjing Medical University, Changzhou, China.,Laboratory of Medical Physics Center, Nanjing Medical University, Jiangning District, Nanjing, China.,Changzhou Key Laboratory of Medical Physics, Changzhou, China
| | - Xinye Ni
- Department of Radiotherapy, Changzhou Second People's Hospital, Nanjing Medical University, Changzhou, China.,Laboratory of Medical Physics Center, Nanjing Medical University, Jiangning District, Nanjing, China.,Changzhou Key Laboratory of Medical Physics, Changzhou, China
| |
Collapse
|
4
|
Sarmadi H, Muñoz-Salinas R, Álvaro Berbís M, Luna A, Medina-Carnicer R. Joint scene and object tracking for cost-Effective augmented reality guided patient positioning in radiation therapy. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 209:106296. [PMID: 34380076 DOI: 10.1016/j.cmpb.2021.106296] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Accepted: 07/17/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVE The research is done in the field of Augmented Reality (AR) for patient positioning in radiation therapy is scarce. We propose an efficient and cost-effective algorithm for tracking the scene and the patient to interactively assist the patient's positioning process by providing visual feedback to the operator. Up to our knowledge, this is the first framework that can be employed for mobile interactive AR to guide patient positioning. METHODS We propose a pointcloud processing method that, combined with a fiducial marker-mapper algorithm and the generalized ICP algorithm, tracks the patient and the camera precisely and efficiently only using the CPU unit. The 3D reference model and body marker map alignment is calculated employing an efficient body reconstruction algorithm. RESULTS Our quantitative evaluation shows that the proposed method achieves a translational and rotational error of 4.17 mm/0.82∘ at 9 fps. Furthermore, the qualitative results demonstrate the usefulness of our algorithm in patient positioning on different human subjects. CONCLUSION Since our algorithm achieves a relatively high frame rate and accuracy employing a regular laptop (without a dedicated GPU), it is a very cost-effective AR-based patient positioning method. It also opens the way for other researchers by introducing a framework that could be improved upon for better mobile interactive AR patient positioning solutions in the future.
Collapse
Affiliation(s)
- Hamid Sarmadi
- Instituto Maimónides de Investigación en Biomedicina (IMIBIC). Avenida Menéndez Pidal s/n, Córdoba, 14004, Spain.
| | - Rafael Muñoz-Salinas
- Computing and Numerical Analysis Department, Edificio Einstein. Campus de Rabanales, Córdoba University, Córdoba, 14071, Spain; Instituto Maimónides de Investigación en Biomedicina (IMIBIC). Avenida Menéndez Pidal s/n, Córdoba, 14004, Spain.
| | - M Álvaro Berbís
- HT Médica, Hospital San Juan de Dios. Avda Brillante 106, Córdoba, 14012, Spain.
| | - Antonio Luna
- HT Médica, Clínica las Nieves, Carmelo Torres 2, Jaén, 23007, Spain.
| | - R Medina-Carnicer
- Computing and Numerical Analysis Department, Edificio Einstein. Campus de Rabanales, Córdoba University, Córdoba, 14071, Spain; Instituto Maimónides de Investigación en Biomedicina (IMIBIC). Avenida Menéndez Pidal s/n, Córdoba, 14004, Spain.
| |
Collapse
|
5
|
Cardan R, Covington EL, Popple R. A Holographic Augmented Reality Guidance System for Patient Alignment: A Feasibility Study. Cureus 2021; 13:e14695. [PMID: 34055539 PMCID: PMC8153089 DOI: 10.7759/cureus.14695] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
Purpose To evaluate the accuracy of an augmented reality holographic guidance system for potential use in patient alignment in radiotherapy applications. Methods A cubic phantom was scanned on a CT simulator and a 3D mesh was extracted using the Eclipse Scripting API. An application was created for the Microsoft HoloLens to allow users to see the scanned mesh as a hologram overlaid in the treatment vault. Six therapists were equipped with the HoloLens glasses and instructed to move the real phantom to align with the perceived spatial hologram using only couch controls. The initial couch coordinates were recorded and then recorded at each step as the therapist moved the phantom to each new location. The application varied the position of the virtual phantom to 10 preprogrammed locations within a 40-cm cubic volume in a combination of vertical, longitudinal, and lateral axis shifts. The absolute position difference between the holographic world and real-world phantom was recorded at each step. Also, the relative position from one position to the next was recorded. Results Fifty shifts were collected across the six therapists. The mean difference between the physical position and instructed holographic position was 0.58 ± 0.31 cm for relative shifts and 0.51 ± 0.33 cm for absolute position. The maximum difference between the holographic position and the actual post shift position was 1.53 cm for relative and 1.58 cm for absolute. Conclusion Holographic augmented reality guidance using the Microsoft HoloLens provides adequate accuracy for initial treatment alignment but lacks the fine alignment accuracy of X-ray imaging systems.
Collapse
Affiliation(s)
- Rex Cardan
- Radiation Oncology, University of Alabama at Birmingham, Birmingham, USA
| | | | - Richard Popple
- Radiation Oncology, University of Alabama at Birmingham, Birmingham, USA
| |
Collapse
|
6
|
Batista V, Meyer J, Kügele M, Al-Hallaq H. Clinical paradigms and challenges in surface guided radiation therapy: Where do we go from here? Radiother Oncol 2020; 153:34-42. [PMID: 32987044 DOI: 10.1016/j.radonc.2020.09.041] [Citation(s) in RCA: 35] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Revised: 09/17/2020] [Accepted: 09/18/2020] [Indexed: 12/26/2022]
Abstract
Surface guided radiotherapy (SGRT) is becoming a routine tool for patient positioning for specific clinical sites in many clinics. However, it has not yet gained its full potential in terms of widespread adoption. This vision paper first examines some of the difficulties in transitioning to SGRT before exploring the current and future role of SGRT alongside and in concert with other imaging techniques. Finally, future horizons and innovative ideas that may shape and impact the direction of SGRT going forward are reviewed.
Collapse
Affiliation(s)
- Vania Batista
- Department of Radiation Oncology, Heidelberg University Hospital, Germany; Heidelberg Institute of Radiation Oncology (HIRO), Germany; National Center for Tumor Diseases (NCT), Heidelberg, Germany.
| | - Juergen Meyer
- Seattle Cancer Care Alliance, University of Washington, Department of Radiation Oncology, United States.
| | - Malin Kügele
- Department of Hematology, Oncology and Radiation Physics, Skåne University Hospital, Lund, Sweden; Medical Radiation Physics, Department of Clinical Sciences, Lund University, Sweden.
| | - Hania Al-Hallaq
- The University of Chicago, Department of Radiation and Cellular Oncology, United States.
| |
Collapse
|
7
|
An overview of augmented and virtual reality applications in radiotherapy and future developments enabled by modern tablet devices. JOURNAL OF RADIOTHERAPY IN PRACTICE 2013. [DOI: 10.1017/s1460396913000277] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
AbstractPurposeWe review augmented (AR) and virtual reality (VR) applications in radiotherapy as found in the scientific literature and highlight future developments enabled by the use of small mass-produced devices and portability of techniques developed in other fields to radiotherapy.AnalysisThe application of AR and VR within radiotherapy is still in its infancy, with the notable exception of training and teaching applications. The relatively high cost of equipment needed to generate a realistic 3D effect seems one factor that has slowed down its use, but also the sheer amount of image data is relatively recent, were radiotherapy professionals are only beginning to explore how to use this to its full potential. This increased availability of 3D data in radiotherapy will drive the application of AR and VR in radiotherapy to efficiently recognise and extract key features in the data to act on in clinical decision making.ConclusionThe development of small mass-produced tablet devices coming on the market will allow the user to interact with computer-generated information more easily, facilitating the application of AR and VR. The increased connectivity enabling virtual presence of remote multidisciplinary team meetings heralds significant changes to how radiotherapy professionals will work, to the benefit of our patients.
Collapse
|
8
|
Portela Sotelo MA, Desserée É, Moreau JM, Shariat B, Beuve M. 3-D model-based multiple-object video tracking for treatment room supervision. IEEE Trans Biomed Eng 2011; 59:562-70. [PMID: 22127989 DOI: 10.1109/tbme.2011.2176940] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
We present a method to monitor a patient and the equipment in a radiotherapy treatment room, by exploiting the information in the treatment plan, enriched with other elements such as visual, geometric, and "semantic" information. Using all these information items, and a generic model, a virtual environment of the scene is created, with maximum precision. The images resulting from video sequences with several cameras are also used to confront the filmed information on the scene and its numerical representation. The method is based on the features of the scene elements, and on a fuzzy formalism. The feasibility of the method is being quantitatively evaluated in the absence of treatment, to be further exploited in a module for external control by video in real conditions.
Collapse
|
9
|
Hoegele W, Loeschel R, Dobler B, Hesser J, Koelbl O, Zygmanski P. Stochastic formulation of patient positioning using linac-mounted cone beam imaging with prior knowledge. Med Phys 2011; 38:668-81. [DOI: 10.1118/1.3532959] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
|