|
1
|
Su Y, Sun Y, Hosny M, Gao W, Fu Y. Facial landmark-guided surface matching for image-to-patient registration with an RGB-D camera. Int J Med Robot 2022; 18:e2373. [PMID: 35133715 DOI: 10.1002/rcs.2373] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Revised: 01/17/2022] [Accepted: 01/29/2022] [Indexed: 01/01/2023]
Abstract
BACKGROUND Fiducial marker-based image-to-patient registration is the most common way in image-guided neurosurgery, which is labour-intensive, time consuming, invasive and error prone. METHODS We proposed a method of facial landmark-guided surface matching for image-to-patient registration using an RGB-D camera. Five facial landmarks are localised from preoperative magnetic resonance (MR) images using deep learning and RGB image using Adaboost with multi-scale block local binary patterns, respectively. The registration of two facial surface point clouds derived from MR images and RGB-D data is initialised by aligning these five landmarks and further refined by weighted iterative closest point algorithm. RESULTS Phantom experiment results show the target registration error is less than 3 mm when the distance from the camera to the phantom is less than 1000 mm. The registration takes less than 10 s. CONCLUSIONS The proposed method is comparable to the state-of-the-arts in terms of the accuracy yet more time-saving and non-invasive.
Collapse
Affiliation(s)
- Yixian Su
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China
| | - Yu Sun
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China
| | - Mohamed Hosny
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China.,Department of Electrical Engineering, Benha Faculty of Engineering, Benha University, Benha, Egypt
| | - Wenpeng Gao
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China
| | - Yili Fu
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China
| |
Collapse
|
|
2
|
Li J, Deng Z, Shen N, He Z, Feng L, Li Y, Yao J. A fully automatic surgical registration method for percutaneous abdominal puncture surgical navigation. Comput Biol Med 2021; 136:104663. [PMID: 34375903 DOI: 10.1016/j.compbiomed.2021.104663] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2021] [Revised: 07/12/2021] [Accepted: 07/17/2021] [Indexed: 01/16/2023]
Abstract
Surgical registration that maps surgical space onto image space plays an important role in surgical navigation. Accurate surgical registration can help surgeons efficiently locate surgical instruments. The complicated marker-based surgical registration method is highly accurate, but it is time-consuming. Therefore, a marker-less surgical registration method with high-precision and high-efficiency is proposed without human intervention. Firstly, the surgical navigation system based on the multi-vision system is calibrated by using a specially-designed calibration board. When extracting the abdominal point cloud acquired by the structured light vision system, the constraint is constructed by using Computed Tomography (CT) image to filter out the points in irrelevant areas to improve the computational efficiency. The Coherent Point Drift (CPD) algorithm based on Gaussian Mixture Model (GMM) is applied in the registration of abdominal point cloud with lack of surface features. To enhance the efficiency of the CPD algorithm, firstly, the system calibration result is used in rough registration of the point cloud, and then the proper point cloud pretreatment method and its parameters are studied through experiments. Finally, the puncturing simulation experiments were carried out by using the abdominal phantom. The experimental results show that the proposed surgical registration method has high accuracy and efficiency, and has potential clinical application value.
Collapse
Affiliation(s)
- Jing Li
- Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China
| | - Zongqian Deng
- Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China
| | - Nanyan Shen
- Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China.
| | - Zhou He
- Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China
| | - Lanyun Feng
- Department of Integrative Oncology, Fudan University Shanghai Cancer Center, Shanghai, China; Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China
| | - Yingjie Li
- Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China
| | - Jia Yao
- Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China
| |
Collapse
|
|
4
|
Xiao D, Li Y, Luo H, Zhang Y, Guo X, Zheng H, Hu Q, Jia F. In vivo comparison of two navigation systems for abdominal percutaneous needle intervention. Abdom Radiol (NY) 2017; 42:1993-2000. [PMID: 28217826 DOI: 10.1007/s00261-017-1083-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
Abstract
PURPOSE To compare the accuracy of a Kinect-Optical navigation system with an electromagnetic (EM) navigation system for percutaneous liver needle intervention. MATERIALS AND METHODS Five beagles with nine artificial tumors were used for validation. The Veran IG4 EM navigation system and a custom-made Kinect-Optical navigation system were used. Needle insertions into each tumor were conducted with these two guidance methods. The target positioning error (TPE) and the time cost of the puncture procedures were evaluated. RESULTS A total of 18 needle insertions were performed to evaluate the navigation accuracy of the two guidance approaches. The targeting error was 6.78 ± 3.22 mm and 8.72 ± 3.5 mm for the Kinect-Optical navigation system and the EM navigation system, respectively. There is no statistically significant difference in the TPE between the Kinect-Optical navigation system and the EM navigation system (p = 0.229). The processing time with the Kinect-Optical system (10 min) is similar to that of the Veran IG4 system (12 min). CONCLUSIONS The accuracy of the Kinect-Optical navigation system is comparable to that of the EM navigation system.
Collapse
Affiliation(s)
- Deqiang Xiao
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China
| | - Yong Li
- Department of Interventional Radiology, Shenzhen People's Hospital, No. 1017, Dongmen North Rd., Luohu, Shenzhen, China
| | - Huoling Luo
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China
| | - Yanfang Zhang
- Department of Interventional Radiology, Shenzhen People's Hospital, No. 1017, Dongmen North Rd., Luohu, Shenzhen, China.
| | - Xuejun Guo
- Department of Radiology, Peking University Shenzhen Hospital, No. 1120, Lianhua Rd, Futian, Shenzhen, China
| | - Huimin Zheng
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China
| | - Qingmao Hu
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China
| | - Fucang Jia
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China.
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, No. 1068, Xueyuan Avenue, Xili Nanshan, Shenzhen, China.
| |
Collapse
|
|
5
|
Ma K, Wang J, Singh V, Tamersoy B, Chang YJ, Wimmer A, Chen T. Multimodal Image Registration with Deep Context Reinforcement Learning. MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION − MICCAI 2017 2017. [DOI: 10.1007/978-3-319-66182-7_28] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|