1
|
Taleb A, Guigou C, Leclerc S, Lalande A, Bozorg Grayeli A. Image-to-Patient Registration in Computer-Assisted Surgery of Head and Neck: State-of-the-Art, Perspectives, and Challenges. J Clin Med 2023; 12:5398. [PMID: 37629441 PMCID: PMC10455300 DOI: 10.3390/jcm12165398] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Revised: 08/08/2023] [Accepted: 08/14/2023] [Indexed: 08/27/2023] Open
Abstract
Today, image-guided systems play a significant role in improving the outcome of diagnostic and therapeutic interventions. They provide crucial anatomical information during the procedure to decrease the size and the extent of the approach, to reduce intraoperative complications, and to increase accuracy, repeatability, and safety. Image-to-patient registration is the first step in image-guided procedures. It establishes a correspondence between the patient's preoperative imaging and the intraoperative data. When it comes to the head-and-neck region, the presence of many sensitive structures such as the central nervous system or the neurosensory organs requires a millimetric precision. This review allows evaluating the characteristics and the performances of different registration methods in the head-and-neck region used in the operation room from the perspectives of accuracy, invasiveness, and processing times. Our work led to the conclusion that invasive marker-based methods are still considered as the gold standard of image-to-patient registration. The surface-based methods are recommended for faster procedures and applied on the surface tissues especially around the eyes. In the near future, computer vision technology is expected to enhance these systems by reducing human errors and cognitive load in the operating room.
Collapse
Affiliation(s)
- Ali Taleb
- Team IFTIM, Institute of Molecular Chemistry of University of Burgundy (ICMUB UMR CNRS 6302), Univ. Bourgogne Franche-Comté, 21000 Dijon, France; (C.G.); (S.L.); (A.L.); (A.B.G.)
| | - Caroline Guigou
- Team IFTIM, Institute of Molecular Chemistry of University of Burgundy (ICMUB UMR CNRS 6302), Univ. Bourgogne Franche-Comté, 21000 Dijon, France; (C.G.); (S.L.); (A.L.); (A.B.G.)
- Otolaryngology Department, University Hospital of Dijon, 21000 Dijon, France
| | - Sarah Leclerc
- Team IFTIM, Institute of Molecular Chemistry of University of Burgundy (ICMUB UMR CNRS 6302), Univ. Bourgogne Franche-Comté, 21000 Dijon, France; (C.G.); (S.L.); (A.L.); (A.B.G.)
| | - Alain Lalande
- Team IFTIM, Institute of Molecular Chemistry of University of Burgundy (ICMUB UMR CNRS 6302), Univ. Bourgogne Franche-Comté, 21000 Dijon, France; (C.G.); (S.L.); (A.L.); (A.B.G.)
- Medical Imaging Department, University Hospital of Dijon, 21000 Dijon, France
| | - Alexis Bozorg Grayeli
- Team IFTIM, Institute of Molecular Chemistry of University of Burgundy (ICMUB UMR CNRS 6302), Univ. Bourgogne Franche-Comté, 21000 Dijon, France; (C.G.); (S.L.); (A.L.); (A.B.G.)
- Otolaryngology Department, University Hospital of Dijon, 21000 Dijon, France
| |
Collapse
|
2
|
Mailhot N, Cheriton R, Vyas K, Cook J, Prawer S, Hinzer K, Spinello D. Eighty-Five Percent of Improved Optical Power Delivery to Epiretinal Prostheses Using Rigid Body Compensation Algorithm. J Biomech Eng 2021; 143:1097182. [PMID: 33537711 DOI: 10.1115/1.4050026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2019] [Indexed: 11/08/2022]
Abstract
Vision impairment caused by degenerative retinal pathologies such as age-related macular degeneration can be treated using retinal implants. Such devices receive power and data using cables passing through a permanent surgical incision in the eye wall (sclera), which increases the risk to patients and surgical costs. A recently developed retinal implant design eliminates the necessity of the implant cable using a photonic power converter (PPC), which receives optical power and data through the pupil and is directed by an ellipsoidal reflector and micro-electromechanical mirror. We present a misalignment compensation algorithm model that accounts for rigid-body motions of the reflector relative to the eye and applies the correction to the mirror coordinates in the presence of angular misalignment of the reflector. We demonstrate that up to 85% of the nominal optical power can be delivered to the implant with axial reflector misalignments up to 30 deg using the compensation algorithm.
Collapse
Affiliation(s)
- Nathaniel Mailhot
- Department of Mechanical Engineering, University of Ottawa, Ottawa, ON K1N 6M6, Canada
| | - Ross Cheriton
- National Research Council of Canada, Ottawa, ON K1N 6M6, Canada
| | - Kaustubh Vyas
- Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6M6, Canada
| | - John Cook
- Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6M6, Canada
| | - Steven Prawer
- Department of Physics, University of Melbourne, Melbourne VIC 3010, Australia
| | - Karin Hinzer
- Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6M6, Canada
| | - Davide Spinello
- Department of Mechanical Engineering, University of Ottawa, Ottawa, ON K1N 6M6, Canada
| |
Collapse
|
3
|
Li Q, Li S, Wu Y, Guo W, Qi S, Huang G, Chen S, Liu Z, Chen X. Orientation-independent Feature Matching (OIFM) for Multimodal Retinal Image Registration. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.101957] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
4
|
Braun D, Yang S, Martel JN, Riviere CN, Becker BC. EyeSLAM: Real-time simultaneous localization and mapping of retinal vessels during intraocular microsurgery. Int J Med Robot 2017; 14. [PMID: 28719002 DOI: 10.1002/rcs.1848] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2016] [Revised: 05/20/2017] [Accepted: 05/23/2017] [Indexed: 12/17/2022]
Abstract
BACKGROUND Fast and accurate mapping and localization of the retinal vasculature is critical to increasing the effectiveness and clinical utility of robot-assisted intraocular microsurgery such as laser photocoagulation and retinal vessel cannulation. METHODS The proposed EyeSLAM algorithm delivers 30 Hz real-time simultaneous localization and mapping of the human retina and vasculature during intraocular surgery, combining fast vessel detection with 2D scan-matching techniques to build and localize a probabilistic map of the vasculature. RESULTS In the harsh imaging environment of retinal surgery with high magnification, quick shaky motions, textureless retina background, variable lighting and tool occlusion, EyeSLAM can map 75% of the vessels within two seconds of initialization and localize the retina in real time with a root mean squared (RMS) error of under 5.0 pixels (translation) and 1° (rotation). CONCLUSIONS EyeSLAM robustly provides retinal maps and registration that enable intelligent surgical micromanipulators to aid surgeons in simulated retinal vessel tracing and photocoagulation tasks.
Collapse
Affiliation(s)
- Daniel Braun
- The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Sungwook Yang
- The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Joseph N Martel
- The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Cameron N Riviere
- The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Brian C Becker
- The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| |
Collapse
|
5
|
Prokopetc K, Bartoli A. A comparative study of transformation models for the sequential mosaicing of long retinal sequences of slit-lamp images obtained in a closed-loop motion. Int J Comput Assist Radiol Surg 2016; 11:2163-2172. [PMID: 27325139 DOI: 10.1007/s11548-016-1439-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2016] [Accepted: 05/31/2016] [Indexed: 11/28/2022]
Abstract
PURPOSE Navigated panretinal photocoagulation is a standard care for proliferative diabetic retinopathy. Slit-lamp-based systems used for this treatment provide a narrow view of the retina. Retinal mosaics are used for view expansion and treatment planning. Mosaicing slit-lamp images is a hard task due to the absence of a physical model of the imaging process, large textureless regions and imaging artifacts, mostly reflections. METHODS We present a comparative study of various geometric transformation models applied to retinal image mosaicing in computer-assisted slit-lamp imaging. We propose an efficient point correspondence-based framework for transformation model evaluation in a typical closed-loop motion scenario. We compare the performance of multiple linear and nonlinear models of different complexities and assess the effect of the number of points used for parameter estimation. We use a local fitting error (LFE) metric to estimate the models' performance in pairwise registration. Because LFE alone is not conclusive regarding the problem of accumulated drift, we propose a loop closure error (LCE) metric to quantify the effect of accumulated local registration errors. We also provide a new normalization procedure for the quadratic transformation model, widely used in retinal image registration. RESULTS In total, seven transformation models were evaluated on three datasets of long image sequences. LFE decreases with increasing complexity of the model, while LCE, in contrast, shows superior performance of simple models. Varying the number of point correspondences did not reveal a common trend for the LCE metric, showing an increase in the error for simple models and an unstable behavior of the complex models. CONCLUSION Our results show that simple models are less sensitive to drift and preferable for sequential mosaicing in slit-lamp imaging, while more complex models are the best choice for short-term registration.
Collapse
Affiliation(s)
- Kristina Prokopetc
- ALCoV-ISIT, UMR 6284 CNRS / Université d'Auvergne, 28 place Henri Dunant, 63001, Clermont-Ferrand, France.
- QuantelMedical, 11 Rue du Bois Joli, 63808, Cournon-d' Auvergne, France.
| | - Adrien Bartoli
- ALCoV-ISIT, UMR 6284 CNRS / Université d'Auvergne, 28 place Henri Dunant, 63001, Clermont-Ferrand, France
| |
Collapse
|
6
|
Retinal image registration using topological vascular tree segmentation and bifurcation structures. Biomed Signal Process Control 2015. [DOI: 10.1016/j.bspc.2014.10.009] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
7
|
De Zanet SI, Ciller C, Rudolph T, Maeder P, Munier F, Balmer A, Cuadra MB, Kowal JH. Landmark detection for fusion of fundus and MRI toward a patient-specific multimodal eye model. IEEE Trans Biomed Eng 2014; 62:532-40. [PMID: 25265602 DOI: 10.1109/tbme.2014.2359676] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Ophthalmologists typically acquire different image modalities to diagnose eye pathologies. They comprise, e.g., Fundus photography, optical coherence tomography, computed tomography, and magnetic resonance imaging (MRI). Yet, these images are often complementary and do express the same pathologies in a different way. Some pathologies are only visible in a particular modality. Thus, it is beneficial for the ophthalmologist to have these modalities fused into a single patient-specific model. The goal of this paper is a fusion of Fundus photography with segmented MRI volumes. This adds information to MRI that was not visible before like vessels and the macula. This paper contributions include automatic detection of the optic disc, the fovea, the optic axis, and an automatic segmentation of the vitreous humor of the eye.
Collapse
|
8
|
Richa R, Linhares R, Comunello E, von Wangenheim A, Schnitzler JY, Wassmer B, Guillemot C, Thuret G, Gain P, Hager G, Taylor R. Fundus image mosaicking for information augmentation in computer-assisted slit-lamp imaging. IEEE TRANSACTIONS ON MEDICAL IMAGING 2014; 33:1304-1312. [PMID: 24718569 DOI: 10.1109/tmi.2014.2309440] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Laser photocoagulation is currently the standard treatment for sight-threatening diseases worldwide, namely diabetic retinopathy and retinal vein occlusions. The slit lamp biomicroscope is the most commonly used device for this procedure, specially for the treatment of the eye periphery. However, only a small portion of the retina can be visualized through the biomicroscope, complicating the task of localizing and identifying surgical targets, increasing treatment duration and patient discomfort. In order to assist surgeons, we propose a method for creating intraoperative retina maps for view expansion using a slit-lamp device. Based on the mosaicking method described by Richa et al, 2012, the proposed method is a combination of direct and feature-based methods, suitable for the textured nature of the human retina. In this paper, we describe three major enhancements to the original formulation. The first is a visual tracking method using local illumination compensation to cope with the challenging visualization conditions. The second is an efficient pixel selection scheme for increased computational efficiency. The third is an entropy-based mosaic update method to dynamically improve the retina map during exploration. To evaluate the performance of the proposed method, we conducted several experiments on human subjects with a computer-assisted slit-lamp prototype. We also demonstrate the practical value of the system for photo documentation, diagnosis and intraoperative navigation.
Collapse
|
9
|
Richa R, Vágvölgyi B, Balicki M, Hager GD, Taylor RH. Hybrid tracking and mosaicking for information augmentation in retinal surgery. ACTA ACUST UNITED AC 2013; 15:397-404. [PMID: 23285576 DOI: 10.1007/978-3-642-33415-3_49] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/24/2023]
Abstract
Current technical limitations in retinal surgery hinder the ability of surgeons to identify and localize surgical targets, increasing operating times and risks of surgical error. In this paper we present a hybrid tracking and mosaicking method for augmented reality in retinal surgery. The system is a combination of direct and feature-based tracking methods. A novel extension for direct visual tracking using a robust image similarity measure in color images is also proposed. Several experiments conducted on phantom, in vivo rabbit and human images attest the ability of the method to cope with the challenging retinal surgery scenario. Applications of the proposed method for tele-mentoring and intra-operative guidance are demonstrated.
Collapse
|
10
|
Becker BC, Riviere CN. Real-Time Retinal Vessel Mapping and Localization for Intraocular Surgery. IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION : ICRA : [PROCEEDINGS]. IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION 2013:5360-5365. [PMID: 24488000 PMCID: PMC3905955 DOI: 10.1109/icra.2013.6631345] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Computer-aided intraocular surgery requires precise, real-time knowledge of the vasculature during retinal procedures such as laser photocoagulation or vessel cannulation. Because vitreoretinal surgeons manipulate retinal structures on the back of the eye through ports in the sclera, voluntary and involuntary tool motion rotates the eye in the socket and causes movement to the microscope view of the retina. The dynamic nature of the surgical workspace during intraocular surgery makes mapping, tracking, and localizing vasculature in real time a challenge. We present an approach that both maps and localizes retinal vessels by temporally fusing and registering individual-frame vessel detections. On video of porcine and human retina, we demonstrate real-time performance, rapid convergence, and robustness to variable illumination and tool occlusion.
Collapse
Affiliation(s)
- Brian C Becker
- Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA
| | - Cameron N Riviere
- Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA
| |
Collapse
|
11
|
Xiao D, Vignarajan J, Lock J, Frost S, Tay-Kearney ML, Kanagasingam Y. Retinal image registration and comparison for clinical decision support. Australas Med J 2012. [PMID: 23115586 DOI: 10.4066/amj.2012.1364.] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
BACKGROUND For eye diseases, such as glaucoma and age-related macular degeneration (ARMD), involved in long-term degeneration procedure, longitudinal comparison of retinal images is a common step for reliable diagnosis of these kinds of diseases. AIMS To provide a retinal image registration approach for longitudinal retinal image alignment and comparison. METHOD Two image registration solutions were proposed for facing different image qualities of retinal images to make the registration methods more robust and feasible in a clinical application system. RESULTS Thirty pairs of longitudinal retinal images were used for the registration test. The experiments showed both solutions provided good performance for the accurate image registrations with efficiency. CONCLUSION We proposed a set of retinal image registration solutions for longitudinal retinal image observation and comparison targeting a clinical application environment.
Collapse
Affiliation(s)
- Di Xiao
- The Australian e-Health Research Centre, CSIRO
| | | | | | | | | | | |
Collapse
|