1
|
Xu X, Zhao S, Gong L, Zuo S. A novel contact optimization algorithm for endomicroscopic surface scanning. Int J Comput Assist Radiol Surg 2024:10.1007/s11548-024-03223-x. [PMID: 38970745 DOI: 10.1007/s11548-024-03223-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 06/24/2024] [Indexed: 07/08/2024]
Abstract
PURPOSE Probe-based confocal laser endomicroscopy (pCLE) offers real-time, cell-level imaging and holds promise for early cancer diagnosis. However, a large area surface scanning for image acquisition is needed to overcome the limitation of field-of-view. Obtaining high-quality images during scanning requires maintaining a stable contact distance between the tissue and probe. This work presents a novel contact optimization algorithm to acquire high-quality pCLE images. METHODS The contact optimization algorithm, based on swarm intelligence of whale optimization algorithm, is designed to optimize the probe position, according to the quality of the image acquired by probe. An accurate image quality assessment of total co-occurrence entropy is introduced to evaluate the pCLE image quality. The algorithm aims to maintain a consistent probe-tissue contact, resulting in high-quality images acquisition. RESULTS Scanning experiments on sponge, ex vivo swine skin tissue and stomach tissue demonstrate the effectiveness of the contact optimization algorithm. Scanning results of the sponge with three different trajectories (spiral trajectory, circle trajectory, and raster trajectory) reveal high-quality mosaics with clear details in every part of the image and no blurred sections. CONCLUSION The contact optimization algorithm successfully identifies the optimal distance between probe and tissue, improving the quality of pCLE images. Experimental results confirm the high potential of this method in endomicroscopic surface scanning.
Collapse
Affiliation(s)
- Xingfeng Xu
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Shengzhe Zhao
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Lun Gong
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Siyang Zuo
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China.
| |
Collapse
|
2
|
Xu X, Tang R, Zuo S. A novel OCT servoing control method for endomicroscopy surface scanning. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
|
3
|
Intensity-based nonrigid endomicroscopic image mosaicking incorporating texture relevance for compensation of tissue deformation. Comput Biol Med 2021; 142:105169. [PMID: 34974384 DOI: 10.1016/j.compbiomed.2021.105169] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2021] [Revised: 12/12/2021] [Accepted: 12/20/2021] [Indexed: 12/09/2022]
Abstract
Image mosaicking has emerged as a universal technique to broaden the field-of-view of the probe-based confocal laser endomicroscopy (pCLE) imaging system. However, due to the influence of probe-tissue contact forces and optical components on imaging quality, existing mosaicking methods remain insufficient to deal with practical challenges. In this paper, we present the texture encoded sum of conditional variance (TESCV) as a novel similarity metric, and effectively incorporate it into a sequential mosaicking scheme to simultaneously correct rigid probe shift and nonrigid tissue deformation. TESCV combines both intensity dependency and texture relevance to quantify the differences between pCLE image frames, where a discriminative binary descriptor named fully cross-detected local derivative pattern (FCLDP) is designed to extract more detailed structural textures. Furthermore, we also analytically derive the closed-form gradient of TESCV with respect to the transformation variables. Experiments on the circular dataset highlighted the advantage of the TESCV metric in improving mosaicking performance compared with the other four recently published metrics. The comparison with the other four state-of-the-art mosaicking methods on the spiral and manual datasets indicated that the proposed TESCV-based method not only worked stably at different contact forces, but was also suitable for both low- and high-resolution imaging systems. With more accurate and delicate mosaics, the proposed method holds promises to meet clinical demands for intraoperative optical biopsy.
Collapse
|
4
|
Xu X, Tang R, Gong L, Chen B, Zuo S. Two Dimensional Position-Based Visual Servoing for Soft Tissue Endomicroscopy. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3084885] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
|
5
|
Gong L, Zheng J, Ping Z, Wang Y, Wang S, Zuo S. Robust Mosaicing of Endomicroscopic Videos via Context-Weighted Correlation Ratio. IEEE Trans Biomed Eng 2021; 68:579-591. [PMID: 32746056 DOI: 10.1109/tbme.2020.3007768] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2022]
Abstract
Probe-based confocal laser endomicroscopy (pCLE) is a promising imaging tool that provides in situ and in vivo optical imaging to perform real-time pathological assessments. However, due to limited field of view, it is difficult for clinicians to get a full understanding of the scanned tissues. In this paper, we develop a novel mosaicing framework to assemble all frame sequences into a full view image. First, a hybrid rigid registration that combines feature matching and template matching is presented to achieve a global alignment of all frames. Then, the parametric free-form deformation (FFD) model with a multiresolution architecture is implemented to accommodate non-rigid tissue distortions. More importantly, we devise a robust similarity metric called context-weighted correlation ratio (CWCR) to promote registration accuracy, where spatial and geometric contexts are incorporated into the estimation of functional intensity dependence. Experiments on both robotic setup and manual manipulation have demonstrated that the proposed scheme significantly precedes some state-of-the-art mosaicing schemes in the presence of intensity fluctuations, insufficient overlap and tissue distortions. Moreover, the comparisons of the proposed CWCR metric and two other metrics have validated the effectiveness of the context-weighted strategy in quantifying the differences between two frames. Benefiting from more rational and delicate mosaics, the proposed scheme is more suitable to instruct diagnosis and treatment during optical biopsies.
Collapse
|
6
|
Liu PR, Lu L, Zhang JY, Huo TT, Liu SX, Ye ZW. Application of Artificial Intelligence in Medicine: An Overview. Curr Med Sci 2021; 41:1105-1115. [PMID: 34874486 PMCID: PMC8648557 DOI: 10.1007/s11596-021-2474-3] [Citation(s) in RCA: 62] [Impact Index Per Article: 20.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Accepted: 12/01/2020] [Indexed: 02/06/2023]
Abstract
Artificial intelligence (AI) is a new technical discipline that uses computer technology to research and develop the theory, method, technique, and application system for the simulation, extension, and expansion of human intelligence. With the assistance of new AI technology, the traditional medical environment has changed a lot. For example, a patient's diagnosis based on radiological, pathological, endoscopic, ultrasonographic, and biochemical examinations has been effectively promoted with a higher accuracy and a lower human workload. The medical treatments during the perioperative period, including the preoperative preparation, surgical period, and postoperative recovery period, have been significantly enhanced with better surgical effects. In addition, AI technology has also played a crucial role in medical drug production, medical management, and medical education, taking them into a new direction. The purpose of this review is to introduce the application of AI in medicine and to provide an outlook of future trends.
Collapse
Affiliation(s)
- Peng-ran Liu
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022 China
| | - Lin Lu
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022 China
| | - Jia-yao Zhang
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022 China
| | - Tong-tong Huo
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022 China
| | - Song-xiang Liu
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022 China
| | - Zhe-wei Ye
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022 China
| |
Collapse
|
7
|
Review of surgical robotic systems for keyhole and endoscopic procedures: state of the art and perspectives. Front Med 2020; 14:382-403. [DOI: 10.1007/s11684-020-0781-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2019] [Accepted: 03/05/2020] [Indexed: 02/06/2023]
|
8
|
Perperidis A, Dhaliwal K, McLaughlin S, Vercauteren T. Image computing for fibre-bundle endomicroscopy: A review. Med Image Anal 2020; 62:101620. [PMID: 32279053 PMCID: PMC7611433 DOI: 10.1016/j.media.2019.101620] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2018] [Accepted: 11/18/2019] [Indexed: 12/12/2022]
Abstract
Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.
Collapse
Affiliation(s)
- Antonios Perperidis
- Institute of Sensors, Signals and Systems (ISSS), Heriot Watt University, EH14 4AS, UK; EPSRC IRC "Hub" in Optical Molecular Sensing & Imaging, MRC Centre for Inflammation Research, Queen's Medical Research Institute (QMRI), University of Edinburgh, EH16 4TJ, UK.
| | - Kevin Dhaliwal
- EPSRC IRC "Hub" in Optical Molecular Sensing & Imaging, MRC Centre for Inflammation Research, Queen's Medical Research Institute (QMRI), University of Edinburgh, EH16 4TJ, UK.
| | - Stephen McLaughlin
- Institute of Sensors, Signals and Systems (ISSS), Heriot Watt University, EH14 4AS, UK.
| | - Tom Vercauteren
- School of Biomedical Engineering and Imaging Sciences, King's College London, WC2R 2LS, UK.
| |
Collapse
|
9
|
Ping Z, Wang H, Chen X, Wang S, Zuo S. Modular Robotic Scanning Device for Real-Time Gastric Endomicroscopy. Ann Biomed Eng 2018; 47:563-575. [PMID: 30362081 DOI: 10.1007/s10439-018-02156-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2018] [Accepted: 10/13/2018] [Indexed: 01/15/2023]
Abstract
Optical biopsy methods, such as probe-based endomicroscopy, can be used for the identification of early mucosal dysplasia in various gastrointestinal conditions and have potential applications in the screening of early-stage gastric cancer in vivo. However, it is difficult to scan a large area of the gastric mucosa for mosaicing during standard endoscopy. This paper proposes a novel 'snap-on' robotic scanning device that can integrate distally with a commercial endoscope. A customized low-cost endomicroscopy system is used for obtaining micro imaging. The developed device could scan a large area of gastric tissue during standard endoscopy. The device achieves positioning accuracy that is less than 0.23 mm. Experimental results showed that the device could achieve large area mosaicing (15.8-18.6 mm2) and demonstrated the potential clinical value of the device for real-time gastric tissue identification and margin assessment. This approach presents an important alternative to current histology techniques for gastrointestinal tract diagnosis.
Collapse
Affiliation(s)
- Zhongyuan Ping
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Haibo Wang
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Xin Chen
- Department of Gastroenterology and Hepatology, Tianjin Medical University General Hospital, Tianjin, 300070, China
| | - Shuxin Wang
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Siyang Zuo
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China.
| |
Collapse
|
10
|
Wang H, Wang S, Li J, Zuo S. Robotic Scanning Device for Intraoperative Thyroid Gland Endomicroscopy. Ann Biomed Eng 2018; 46:543-554. [PMID: 29340932 DOI: 10.1007/s10439-018-1978-x] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2017] [Accepted: 01/04/2018] [Indexed: 01/02/2023]
Abstract
Probe-based confocal laser endomicroscopy has a high potential to be a promising tool that can provide intraoperative high-resolution in vivo morphological imaging at cellular and subcellular levels for thyroidectomy, and allow real-time assessment of tumor margins. However, the typical images acquired with this technique cover a very small area limited by the field of view of the probe, accompanied by tissue deformation and inconsistent probe-tissue contact when operated manually. In this paper, a novel compact robotic device for large area scanning has been developed. The device can scan a large surface in a spiral trajectory by rotating the tip frame along the spiral groove of the base frame. The fiber Bragg grating sensor with a passive linear structure is used to detect and maintain a stable probe-tissue contact force during scanning. An active linear actuation is also integrated for adjusting the probe-tissue contact level prior to each scan. Results demonstrate that the scanning device ensures a suitable probe-tissue contact force and compensates for simulated hand tremor. Mosaicing results of lens tissue paper and porcine belly tissue with both bench and hand-held experiments show the effectiveness and usability of the device, demonstrating the potential clinical value of the system.
Collapse
Affiliation(s)
- Haibo Wang
- The Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Shuxin Wang
- The Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Jinhua Li
- The Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Siyang Zuo
- The Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China.
| |
Collapse
|
11
|
Pang Z, Yang G, Khedri R, Zhang YT. Introduction to the Special Section: Convergence of Automation Technology, Biomedical Engineering, and Health Informatics Toward the Healthcare 4.0. IEEE Rev Biomed Eng 2018. [DOI: 10.1109/rbme.2018.2848518] [Citation(s) in RCA: 62] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|