1
|
Guan P, Luo H, Guo J, Zhang Y, Jia F. Intraoperative laparoscopic liver surface registration with preoperative CT using mixing features and overlapping region masks. Int J Comput Assist Radiol Surg 2023:10.1007/s11548-023-02846-w. [PMID: 36787037 DOI: 10.1007/s11548-023-02846-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Accepted: 01/27/2023] [Indexed: 02/15/2023]
Abstract
PURPOSE Laparoscopic liver resection is a minimal invasive surgery. Augmented reality can map preoperative anatomy information extracted from computed tomography to the intraoperative liver surface reconstructed from stereo 3D laparoscopy. However, liver surface registration is particularly challenging as the intraoperative surface is only partially visible and suffers from large liver deformations due to pneumoperitoneum. This study proposes a deep learning-based robust point cloud registration network. METHODS This study proposed a low overlap liver surface registration algorithm combining local mixed features and global features of point clouds. A learned overlap mask is used to filter the non-overlapping region of the point cloud, and a network is used to predict the overlapping region threshold to regulate the training process. RESULTS We validated the algorithm on the DePoLL (the Deformable Porcine Laparoscopic Liver) dataset. Compared with the baseline method and other state-of-the-art registration methods, our method achieves minimum target registration error (TRE) of 19.9 ± 2.7 mm. CONCLUSION The proposed point cloud registration method uses the learned overlapping mask to filter the non-overlapping areas in the point cloud, then the extracted overlapping area point cloud is registered according to the mixed features and global features, and this method is robust and efficient in low-overlap liver surface registration.
Collapse
Affiliation(s)
- Peidong Guan
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.,Shenzhen College of Advanced Technology, University of Chinese Academy and Sciences, Shenzhen, China
| | - Huoling Luo
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Jianxi Guo
- Department of Interventional Radiology, Shenzhen People's Hospital, Shenzhen, China
| | - Yanfang Zhang
- Department of Interventional Radiology, Shenzhen People's Hospital, Shenzhen, China.
| | - Fucang Jia
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. .,Shenzhen College of Advanced Technology, University of Chinese Academy and Sciences, Shenzhen, China. .,Pazhou Lab, Guangzhou, China.
| |
Collapse
|
2
|
Automatic, global registration in laparoscopic liver surgery. Int J Comput Assist Radiol Surg 2021; 17:167-176. [PMID: 34697757 PMCID: PMC8739294 DOI: 10.1007/s11548-021-02518-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Accepted: 10/04/2021] [Indexed: 11/13/2022]
Abstract
Purpose The initial registration of a 3D pre-operative CT model to a 2D laparoscopic video image in augmented reality systems for liver surgery needs to be fast, intuitive to perform and with minimal interruptions to the surgical intervention. Several recent methods have focussed on using easily recognisable landmarks across modalities. However, these methods still need manual annotation or manual alignment. We propose a novel, fully automatic pipeline for 3D–2D global registration in laparoscopic liver interventions. Methods Firstly, we train a fully convolutional network for the semantic detection of liver contours in laparoscopic images. Secondly, we propose a novel contour-based global registration algorithm to estimate the camera pose without any manual input during surgery. The contours used are the anterior ridge and the silhouette of the liver. Results We show excellent generalisation of the semantic contour detection on test data from 8 clinical cases. In quantitative experiments, the proposed contour-based registration can successfully estimate a global alignment with as little as 30% of the liver surface, a visibility ratio which is characteristic of laparoscopic interventions. Moreover, the proposed pipeline showed very promising results in clinical data from 5 laparoscopic interventions. Conclusions Our proposed automatic global registration could make augmented reality systems more intuitive and usable for surgeons and easier to translate to operating rooms. Yet, as the liver is deformed significantly during surgery, it will be very beneficial to incorporate deformation into our method for more accurate registration.
Collapse
|
3
|
Wang Y, Cao D, Chen SL, Li YM, Zheng YW, Ohkohchi N. Current trends in three-dimensional visualization and real-time navigation as well as robot-assisted technologies in hepatobiliary surgery. World J Gastrointest Surg 2021; 13:904-922. [PMID: 34621469 PMCID: PMC8462083 DOI: 10.4240/wjgs.v13.i9.904] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Revised: 04/19/2021] [Accepted: 08/02/2021] [Indexed: 02/06/2023] Open
Abstract
With the continuous development of digital medicine, minimally invasive precision and safety have become the primary development trends in hepatobiliary surgery. Due to the specificity and complexity of hepatobiliary surgery, traditional preoperative imaging techniques such as computed tomography and magnetic resonance imaging cannot meet the need for identification of fine anatomical regions. Imaging-based three-dimensional (3D) reconstruction, virtual simulation of surgery and 3D printing optimize the surgical plan through preoperative assessment, improving the controllability and safety of intraoperative operations, and in difficult-to-reach areas of the posterior and superior liver, assistive robots reproduce the surgeon’s natural movements with stable cameras, reducing natural vibrations. Electromagnetic navigation in abdominal surgery solves the problem of conventional surgery still relying on direct visual observation or preoperative image assessment. We summarize and compare these recent trends in digital medical solutions for the future development and refinement of digital medicine in hepatobiliary surgery.
Collapse
Affiliation(s)
- Yun Wang
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
| | - Di Cao
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
| | - Si-Lin Chen
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
| | - Yu-Mei Li
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
| | - Yun-Wen Zheng
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
- Department of Gastrointestinal and Hepato-Biliary-Pancreatic Surgery, Faculty of Medicine, University of Tsukuba, Tsukuba 305-8575, Ibaraki, Japan
- Guangdong Provincial Key Laboratory of Large Animal Models for Biomedicine, and School of Biotechnology and Heath Sciences, Wuyi University, Jiangmen 529020, Guangdong Province, China
- School of Medicine, Yokohama City University, Yokohama 234-0006, Kanagawa, Japan
| | - Nobuhiro Ohkohchi
- Department of Gastrointestinal and Hepato-Biliary-Pancreatic Surgery, Faculty of Medicine, University of Tsukuba, Tsukuba 305-8575, Ibaraki, Japan
| |
Collapse
|
4
|
Schneider C, Allam M, Stoyanov D, Hawkes DJ, Gurusamy K, Davidson BR. Performance of image guided navigation in laparoscopic liver surgery - A systematic review. Surg Oncol 2021; 38:101637. [PMID: 34358880 DOI: 10.1016/j.suronc.2021.101637] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2021] [Revised: 07/04/2021] [Accepted: 07/24/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND Compared to open surgery, minimally invasive liver resection has improved short term outcomes. It is however technically more challenging. Navigated image guidance systems (IGS) are being developed to overcome these challenges. The aim of this systematic review is to provide an overview of their current capabilities and limitations. METHODS Medline, Embase and Cochrane databases were searched using free text terms and corresponding controlled vocabulary. Titles and abstracts of retrieved articles were screened for inclusion criteria. Due to the heterogeneity of the retrieved data it was not possible to conduct a meta-analysis. Therefore results are presented in tabulated and narrative format. RESULTS Out of 2015 articles, 17 pre-clinical and 33 clinical papers met inclusion criteria. Data from 24 articles that reported on accuracy indicates that in recent years navigation accuracy has been in the range of 8-15 mm. Due to discrepancies in evaluation methods it is difficult to compare accuracy metrics between different systems. Surgeon feedback suggests that current state of the art IGS may be useful as a supplementary navigation tool, especially in small liver lesions that are difficult to locate. They are however not able to reliably localise all relevant anatomical structures. Only one article investigated IGS impact on clinical outcomes. CONCLUSIONS Further improvements in navigation accuracy are needed to enable reliable visualisation of tumour margins with the precision required for oncological resections. To enhance comparability between different IGS it is crucial to find a consensus on the assessment of navigation accuracy as a minimum reporting standard.
Collapse
Affiliation(s)
- C Schneider
- Department of Surgical Biotechnology, University College London, Pond Street, NW3 2QG, London, UK.
| | - M Allam
- Department of Surgical Biotechnology, University College London, Pond Street, NW3 2QG, London, UK; General surgery Department, Tanta University, Egypt
| | - D Stoyanov
- Department of Computer Science, University College London, London, UK; Centre for Medical Image Computing (CMIC), University College London, London, UK
| | - D J Hawkes
- Centre for Medical Image Computing (CMIC), University College London, London, UK; Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK
| | - K Gurusamy
- Department of Surgical Biotechnology, University College London, Pond Street, NW3 2QG, London, UK
| | - B R Davidson
- Department of Surgical Biotechnology, University College London, Pond Street, NW3 2QG, London, UK
| |
Collapse
|
5
|
Schneider C, Thompson S, Totz J, Song Y, Allam M, Sodergren MH, Desjardins AE, Barratt D, Ourselin S, Gurusamy K, Stoyanov D, Clarkson MJ, Hawkes DJ, Davidson BR. Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study. Surg Endosc 2020; 34:4702-4711. [PMID: 32780240 PMCID: PMC7524854 DOI: 10.1007/s00464-020-07807-x] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2020] [Accepted: 07/10/2020] [Indexed: 02/06/2023]
Abstract
BACKGROUND The laparoscopic approach to liver resection may reduce morbidity and hospital stay. However, uptake has been slow due to concerns about patient safety and oncological radicality. Image guidance systems may improve patient safety by enabling 3D visualisation of critical intra- and extrahepatic structures. Current systems suffer from non-intuitive visualisation and a complicated setup process. A novel image guidance system (SmartLiver), offering augmented reality visualisation and semi-automatic registration has been developed to address these issues. A clinical feasibility study evaluated the performance and usability of SmartLiver with either manual or semi-automatic registration. METHODS Intraoperative image guidance data were recorded and analysed in patients undergoing laparoscopic liver resection or cancer staging. Stereoscopic surface reconstruction and iterative closest point matching facilitated semi-automatic registration. The primary endpoint was defined as successful registration as determined by the operating surgeon. Secondary endpoints were system usability as assessed by a surgeon questionnaire and comparison of manual vs. semi-automatic registration accuracy. Since SmartLiver is still in development no attempt was made to evaluate its impact on perioperative outcomes. RESULTS The primary endpoint was achieved in 16 out of 18 patients. Initially semi-automatic registration failed because the IGS could not distinguish the liver surface from surrounding structures. Implementation of a deep learning algorithm enabled the IGS to overcome this issue and facilitate semi-automatic registration. Mean registration accuracy was 10.9 ± 4.2 mm (manual) vs. 13.9 ± 4.4 mm (semi-automatic) (Mean difference - 3 mm; p = 0.158). Surgeon feedback was positive about IGS handling and improved intraoperative orientation but also highlighted the need for a simpler setup process and better integration with laparoscopic ultrasound. CONCLUSION The technical feasibility of using SmartLiver intraoperatively has been demonstrated. With further improvements semi-automatic registration may enhance user friendliness and workflow of SmartLiver. Manual and semi-automatic registration accuracy were comparable but evaluation on a larger patient cohort is required to confirm these findings.
Collapse
Affiliation(s)
- C. Schneider
- Division of Surgery & Interventional Science, Royal Free Campus, University College London, Pond Street, London, NW3 2QG UK
| | - S. Thompson
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - J. Totz
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - Y. Song
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - M. Allam
- Division of Surgery & Interventional Science, Royal Free Campus, University College London, Pond Street, London, NW3 2QG UK
| | - M. H. Sodergren
- Centre for Medical Image Computing (CMIC), University College London, London, UK
| | - A. E. Desjardins
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - D. Barratt
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - S. Ourselin
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - K. Gurusamy
- Division of Surgery & Interventional Science, Royal Free Campus, University College London, Pond Street, London, NW3 2QG UK ,Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Department of Hepatopancreatobiliary and Liver Transplant Surgery, Royal Free Hospital, London, UK
| | - D. Stoyanov
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Computer Science, University College London, London, UK
| | - M. J. Clarkson
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - D. J. Hawkes
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - B. R. Davidson
- Division of Surgery & Interventional Science, Royal Free Campus, University College London, Pond Street, London, NW3 2QG UK ,Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Department of Hepatopancreatobiliary and Liver Transplant Surgery, Royal Free Hospital, London, UK
| |
Collapse
|
6
|
Oldhafer KJ, Peterhans M, Kantas A, Schenk A, Makridis G, Pelzl S, Wagner KC, Weber S, Stavrou GA, Donati M. [Navigated liver surgery : Current state and importance in the future]. Chirurg 2019; 89:769-776. [PMID: 30225532 DOI: 10.1007/s00104-018-0713-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
The preoperative computer-assisted resection planning is the basis for every navigation. Thanks to modern algorithms, the prerequisites have been created to carry out a virtual resection planning and a risk analysis. Thus, individual segment resections can be precisely planned in any conceivable combination. The transfer of planning information and resection suggestions to the operating theater is still problematic. The so-called stereotactic liver navigation supports the exact intraoperative implementation of the planned resection strategy and provides the surgeon with real-time three-dimensional information on resection margins and critical structures during the resection. This is made possible by a surgical navigation system that measures the position of surgical instruments and then presents them together with the preoperative surgical planning data. Although surgical navigation systems have been indispensable in neurosurgery and spinal surgery for many years, these procedures have not yet become established as standard in liver surgery. This is mainly due to the technical challenge of navigating a moving organ. As the liver is constantly moving and deforming during surgery due to respiration and surgical manipulation, the surgical navigation system must be able to measure these alterations in order to adapt the preoperative navigation data to the current situation. Despite these advances, further developments are required until navigated liver resection enters clinical routine; however, it is already clear that laparoscopic liver surgery and robotic surgery will benefit most from navigation technology.
Collapse
Affiliation(s)
- K J Oldhafer
- Klinik für Allgemein- und Viszeralchirurgie, Asklepios Klinik Barmbek, Hamburg, Deutschland. .,Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland.
| | | | - A Kantas
- Klinik für Allgemein- und Viszeralchirurgie, Asklepios Klinik Barmbek, Hamburg, Deutschland.,Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland
| | - A Schenk
- Fraunhofer-Institut für Bildgestützte Medizin MEVIS, Bremen, Deutschland
| | - G Makridis
- Klinik für Allgemein- und Viszeralchirurgie, Asklepios Klinik Barmbek, Hamburg, Deutschland.,Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland
| | - S Pelzl
- apoQlar, Hamburg, Deutschland
| | - K C Wagner
- Klinik für Allgemein- und Viszeralchirurgie, Asklepios Klinik Barmbek, Hamburg, Deutschland.,Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland
| | - S Weber
- University of Bern, ARTORG Center for Biomedical Engineering Research, Bern, Schweiz
| | - G A Stavrou
- Klinik für Allgemein‑, Viszeralchirurgie, Thorax- und Kinderchirurgie, Klinikum Saarbrücken, Saarbrücken, Deutschland
| | - M Donati
- Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland.,Department of Surgery and Medical Surgical Specialties, University of Catania, Catania, Italien
| |
Collapse
|
7
|
Shang HT, Bao JH, Zhang XB, Wang HB, Zhang HT, Li ZL. Comparison of Clinical Efficacy and Complications Between Laparoscopic Partial and Open Partial Hepatectomy for Liver Carcinoma: A Meta-Analysis. J Laparoendosc Adv Surg Tech A 2019; 29:225-232. [PMID: 30653396 DOI: 10.1089/lap.2018.0346] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023] Open
Abstract
OBJECTIVE To contrast the clinical effects and complications for the treatment of liver carcinoma in laparoscopic partial hepatectomy (LPH) and open partial hepatectomy (OPH). METHODS The multiple databases were adopted to search relevant studies, and the articles eventually satisfying the inclusion criteria were included. All the meta-analyses were conducted with the Review Manager 5.3, and to estimate the quality of each article risk of bias table was performed. RESULTS In the end, 17 studies including 3897 patients were involved, which eventually satisfied the eligibility criteria. The number of samples in LPH group and OPH group were 1723 and 2174, respectively. The results of heterogeneity test suggested that recurrence rate (odds ratio [OR] = -20.11, 95% confidence interval, CI [-35.93 to -4.29], P = .01; P for heterogeneity <.00001, I2 = 100%), hospital days (mean difference (MD) = -2.21, 95% CI [-2.53 to -1.88], P < .000001; P for heterogeneity = .41, I2 = 58%), and blood loss (MD = -68.09, 95% CI [-85.07 to -51.11], P < .00001; P for heterogeneity = .13, I2 = 37%) were significantly different, whereas operating time (MD = 4.00, 95% CI [-17.50 to 25.49], P = .72; P for heterogeneity <.00001, I2 = 99%) and complication events (OR = 0.68, 95% CI [0.46 to 1.01], P = .05; P for heterogeneity = .34, I2 = 11%) between LPH and OPH were insignificantly different. CONCLUSION This study demonstrated that clinical efficacy of OPH was better than that of LPH to some extent, but LPH was a quicker recovery and less harmful therapy.
Collapse
Affiliation(s)
- Hai-Tao Shang
- Department of Hepatopancreatobiliary Surgery, Nan-Kai Hospital, Tianjin, China
| | - Jian-Heng Bao
- Department of Hepatopancreatobiliary Surgery, Nan-Kai Hospital, Tianjin, China
| | - Xi-Bo Zhang
- Department of Hepatopancreatobiliary Surgery, Nan-Kai Hospital, Tianjin, China
| | - Hai-Bo Wang
- Department of Hepatopancreatobiliary Surgery, Nan-Kai Hospital, Tianjin, China
| | - Hong-Tao Zhang
- Department of Hepatopancreatobiliary Surgery, Nan-Kai Hospital, Tianjin, China
| | - Zhong-Lian Li
- Department of Hepatopancreatobiliary Surgery, Nan-Kai Hospital, Tianjin, China
| |
Collapse
|
8
|
Paydarfar JA, Wu X, Halter RJ. Initial experience with image-guided surgical navigation in transoral surgery. Head Neck 2018; 41:E1-E10. [PMID: 30556235 DOI: 10.1002/hed.25380] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2018] [Revised: 05/08/2018] [Accepted: 05/28/2018] [Indexed: 12/19/2022] Open
Abstract
BACKGROUND Surgical navigation using image guidance may improve the safety and efficacy of transoral surgery (TOS); however, preoperative imaging cannot be accurately registered to the intraoperative state due to deformations resulting from placement of the laryngoscope or retractor. This proof of concept study explores feasibility and registration accuracy of surgical navigation for TOS by utilizing intraoperative imaging. METHODS Four patients undergoing TOS were recruited. Suspension laryngoscopy was performed with a CT-compatible laryngoscope. An intraoperative contrast enhanced CT scan was obtained and registered to fiducials placed on the neck, face, and laryngoscope. RESULTS All patients were successfully scanned and registered. Registration accuracy within the pharynx and larynx was 1 mm or less. Target registration was confirmed by localizing endoscopic and surface structures to the CT images. Successful tracking was performed in all 4 patients. CONCLUSION For surgical navigation during TOS, although a high level of registration accuracy can be achieved by utilizing intraoperative imaging, significant limitations of the existing technology have been identified. These limitations, as well as areas for future investigation, are discussed.
Collapse
Affiliation(s)
- Joseph A Paydarfar
- Section of Otolaryngology, Audiology, and Maxillofacial Surgery, Department of Surgery, Dartmouth-Hitchcock Medical Center, Geisel School of Medicine, Lebanon, New Hampshire
- Thayer School of Engineering at Dartmouth, Hanover, New Hampshire
| | - Xiaotian Wu
- Thayer School of Engineering at Dartmouth, Hanover, New Hampshire
| | - Ryan J Halter
- Thayer School of Engineering at Dartmouth, Hanover, New Hampshire
- Dartmouth College Geisel School of Medicine, Department of Surgery, Hanover, New Hampshire
| |
Collapse
|
9
|
Augmented visualization with depth perception cues to improve the surgeon's performance in minimally invasive surgery. Med Biol Eng Comput 2018; 57:995-1013. [PMID: 30511205 DOI: 10.1007/s11517-018-1929-6] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2018] [Accepted: 11/03/2018] [Indexed: 01/14/2023]
Abstract
Minimally invasive techniques, such as laparoscopy and radiofrequency ablation of tumors, bring important advantages in surgery: by minimizing incisions on the patient's body, they can reduce the hospitalization period and the risk of postoperative complications. Unfortunately, they come with drawbacks for surgeons, who have a restricted vision of the operation area through an indirect access and 2D images provided by a camera inserted in the body. Augmented reality provides an "X-ray vision" of the patient anatomy thanks to the visualization of the internal organs of the patient. In this way, surgeons are free from the task of mentally associating the content from CT images to the operative scene. We present a navigation system that supports surgeons in preoperative and intraoperative phases and an augmented reality system that superimposes virtual organs on the patient's body together with depth and distance information. We implemented a combination of visual and audio cues allowing the surgeon to improve the intervention precision and avoid the risk of damaging anatomical structures. The test scenarios proved the good efficacy and accuracy of the system. Moreover, tests in the operating room suggested some modifications to the tracking system to make it more robust with respect to occlusions. Graphical Abstract Augmented visualization in minimally invasive surgery.
Collapse
|
10
|
Leonard S, Sinha A, Reiter A, Ishii M, Gallia GL, Taylor RH, Hager GD. Evaluation and Stability Analysis of Video-Based Navigation System for Functional Endoscopic Sinus Surgery on In Vivo Clinical Data. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2185-2195. [PMID: 29993881 DOI: 10.1109/tmi.2018.2833868] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Functional endoscopic sinus surgery (FESS) is one of the most common outpatient surgical procedures performed in the head and neck region. It is used to treat chronic sinusitis, a disease characterized by inflammation in the nose and surrounding paranasal sinuses, affecting about 15% of the adult population. During FESS, the nasal cavity is visualized using an endoscope, and instruments are used to remove tissues that are often within a millimeter of critical anatomical structures, such as the optic nerve, carotid arteries, and nasolacrimal ducts. To maintain orientation and to minimize the risk of damage to these structures, surgeons use surgical navigation systems to visualize the 3-D position of their tools on patients' preoperative Computed Tomographies (CTs). This paper presents an image-based method for enhanced endoscopic navigation. The main contributions are: (1) a system that enables a surgeon to asynchronously register a sequence of endoscopic images to a CT scan with higher accuracy than other reported solutions using no additional hardware; (2) the ability to report the robustness of the registration; and (3) evaluation on in vivo human data. The system also enables the overlay of anatomical structures, visible, or occluded, on top of video images. The methods are validated on four different data sets using multiple evaluation metrics. First, for experiments on synthetic data, we observe a mean absolute position error of 0.21mm and a mean absolute orientation error of 2.8° compared with ground truth. Second, for phantom data, we observe a mean absolute position error of 0.97mm and a mean absolute orientation error of 3.6° compared with the same motion tracked by an electromagnetic tracker. Third, for cadaver data, we use fiducial landmarks and observe an average reprojection distance error of 0.82mm. Finally, for in vivo clinical data, we report an average ICP residual error of 0.88mm in areas that are not composed of erectile tissue and an average ICP residual error of 1.09mm in areas that are composed of erectile tissue.
Collapse
|
11
|
In vivo estimation of target registration errors during augmented reality laparoscopic surgery. Int J Comput Assist Radiol Surg 2018; 13:865-874. [PMID: 29663273 PMCID: PMC5973973 DOI: 10.1007/s11548-018-1761-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2018] [Accepted: 04/02/2018] [Indexed: 11/02/2022]
Abstract
PURPOSE Successful use of augmented reality for laparoscopic surgery requires that the surgeon has a thorough understanding of the likely accuracy of any overlay. Whilst the accuracy of such systems can be estimated in the laboratory, it is difficult to extend such methods to the in vivo clinical setting. Herein we describe a novel method that enables the surgeon to estimate in vivo errors during use. We show that the method enables quantitative evaluation of in vivo data gathered with the SmartLiver image guidance system. METHODS The SmartLiver system utilises an intuitive display to enable the surgeon to compare the positions of landmarks visible in both a projected model and in the live video stream. From this the surgeon can estimate the system accuracy when using the system to locate subsurface targets not visible in the live video. Visible landmarks may be either point or line features. We test the validity of the algorithm using an anatomically representative liver phantom, applying simulated perturbations to achieve clinically realistic overlay errors. We then apply the algorithm to in vivo data. RESULTS The phantom results show that using projected errors of surface features provides a reliable predictor of subsurface target registration error for a representative human liver shape. Applying the algorithm to in vivo data gathered with the SmartLiver image-guided surgery system shows that the system is capable of accuracies around 12 mm; however, achieving this reliably remains a significant challenge. CONCLUSION We present an in vivo quantitative evaluation of the SmartLiver image-guided surgery system, together with a validation of the evaluation algorithm. This is the first quantitative in vivo analysis of an augmented reality system for laparoscopic surgery.
Collapse
|
12
|
Luo X, Mori K, Peters TM. Advanced Endoscopic Navigation: Surgical Big Data, Methodology, and Applications. Annu Rev Biomed Eng 2018; 20:221-251. [PMID: 29505729 DOI: 10.1146/annurev-bioeng-062117-120917] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.
Collapse
Affiliation(s)
- Xiongbiao Luo
- Department of Computer Science, Fujian Key Laboratory of Computing and Sensing for Smart City, Xiamen University, Xiamen 361005, China;
| | - Kensaku Mori
- Department of Intelligent Systems, Graduate School of Informatics, Nagoya University, Nagoya 464-8601, Japan;
| | - Terry M Peters
- Robarts Research Institute, Western University, London, Ontario N6A 3K7, Canada;
| |
Collapse
|