1
|
Schmidt A, Mohareri O, DiMaio S, Yip MC, Salcudean SE. Tracking and mapping in medical computer vision: A review. Med Image Anal 2024; 94:103131. [PMID: 38442528 DOI: 10.1016/j.media.2024.103131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Revised: 02/08/2024] [Accepted: 02/29/2024] [Indexed: 03/07/2024]
Abstract
As computer vision algorithms increase in capability, their applications in clinical systems will become more pervasive. These applications include: diagnostics, such as colonoscopy and bronchoscopy; guiding biopsies, minimally invasive interventions, and surgery; automating instrument motion; and providing image guidance using pre-operative scans. Many of these applications depend on the specific visual nature of medical scenes and require designing algorithms to perform in this environment. In this review, we provide an update to the field of camera-based tracking and scene mapping in surgery and diagnostics in medical computer vision. We begin with describing our review process, which results in a final list of 515 papers that we cover. We then give a high-level summary of the state of the art and provide relevant background for those who need tracking and mapping for their clinical applications. After which, we review datasets provided in the field and the clinical needs that motivate their design. Then, we delve into the algorithmic side, and summarize recent developments. This summary should be especially useful for algorithm designers and to those looking to understand the capability of off-the-shelf methods. We maintain focus on algorithms for deformable environments while also reviewing the essential building blocks in rigid tracking and mapping since there is a large amount of crossover in methods. With the field summarized, we discuss the current state of the tracking and mapping methods along with needs for future algorithms, needs for quantification, and the viability of clinical applications. We then provide some research directions and questions. We conclude that new methods need to be designed or combined to support clinical applications in deformable environments, and more focus needs to be put into collecting datasets for training and evaluation.
Collapse
Affiliation(s)
- Adam Schmidt
- Department of Electrical and Computer Engineering, University of British Columbia, 2329 West Mall, Vancouver V6T 1Z4, BC, Canada.
| | - Omid Mohareri
- Advanced Research, Intuitive Surgical, 1020 Kifer Rd, Sunnyvale, CA 94086, USA
| | - Simon DiMaio
- Advanced Research, Intuitive Surgical, 1020 Kifer Rd, Sunnyvale, CA 94086, USA
| | - Michael C Yip
- Department of Electrical and Computer Engineering, University of California San Diego, 9500 Gilman Dr, La Jolla, CA 92093, USA
| | - Septimiu E Salcudean
- Department of Electrical and Computer Engineering, University of British Columbia, 2329 West Mall, Vancouver V6T 1Z4, BC, Canada
| |
Collapse
|
2
|
Zhou L, Wang W, Li Z, Wei H, Cai Q, Chen Q, Liu Z, Ye H, Song P, Cheng L, Lei P, Chen S. Clinical application of 3D-Slicer + 3D printing guide combined with transcranial neuroendoscopic in minimally invasive neurosurgery. Sci Rep 2022; 12:20421. [PMID: 36443477 PMCID: PMC9705550 DOI: 10.1038/s41598-022-24876-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Accepted: 11/22/2022] [Indexed: 11/29/2022] Open
Abstract
To explore the clinical advantages of 3D-Slicer + 3D printing guide combined with transcranial neuroendoscopic in minimally invasive neurosurgery. By collecting the datum of patients who underwent craniotomy under 3D-Slicer + 3D printing guide plate positioning combined with transcranial neuroendoscopic in our hospital from October 2021 to February 2022, this paper introduces the accurate planning and positioning lesions of patients before operation and the minimally invasive operation of intraoperative neuroendoscopic and analyses clinical data such as lesion size and surgical bone window size. We collected the case datum of 16 patients who underwent craniocerebral surgery with 3D-Slicer + 3D printing guide combined with transcranial neuroendoscopic, including 5 males and 11 females, aged 46-76 years, including 6 brain tumors (3 meningiomas, 1 glioblastoma, 2 lung cancer brain metastases), 2 cavernous hemangioma, 7 hydrocephalus and 1 chronic subdural hematoma. The lesions of the 16 patients were located accurately before operation and the target areas were reached quickly during operation. Postoperative imaging datum confirmed that the lesions was removed fully, and the ventricular end of shunt tube was in good position. The technology of 3D-Slicer + 3D printing guide plate combined with transcranial neuroendoscopic is not difficult, which has many advantages such as inexpensive equipment, simple operation, easy learning, accurate positioning, and minimally invasive surgery. It is considered to be a practical technology that is feasible, reliable, convenient for diagnosis, preoperative planning and minimally invasive surgery. It is suitable for promotion in neurosurgery and other surgical departments of all medical institutions.
Collapse
Affiliation(s)
- Long Zhou
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Wenju Wang
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Zhiyang Li
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Hangyu Wei
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Qiang Cai
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Qianxue Chen
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Zaiming Liu
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Hui Ye
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Ping Song
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Li Cheng
- grid.412632.00000 0004 1758 2270Department of Critical Care Medicine, Renmin Hospital of Wuhan University, Eastern Campus, Wuhan, China
| | - Pan Lei
- grid.412632.00000 0004 1758 2270Department of Neurosurgery, Renmin Hospital of Wuhan University, No. 238, Jiefang Road, Wuchang District, Wuhan, 430060 China
| | - Sheng Chen
- Department of Neurosurgery, Dangyang Renmin Hospital of Hubei Province, Yichang, China
| |
Collapse
|
3
|
Clinical application of 3D Slicer combined with Sina/MosoCam multimodal system in preoperative planning of brain lesions surgery. Sci Rep 2022; 12:19258. [PMID: 36357434 PMCID: PMC9649692 DOI: 10.1038/s41598-022-22549-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 10/17/2022] [Indexed: 11/12/2022] Open
Abstract
To explore the clinical advantages of 3D Slicer combined with Sina/MosoCam multimodal system in preoperative planning of brain lesions surgery. By collecting the data of brain lesions patients undergoing craniotomy under the preoperative positioning of 3D Slicer combined Sina/MosoCam multimodal system in the people's Hospital of Wuhan University from January 2021 to October 2021, the preoperative planning of patients was introduced, and the size of surgical bone window, operation time, preoperative and postoperative neurological dysfunction were counted. We collected the case data of 35 patients who were reconstructed by 3D Slicer and located by Sina/MosoCam projection. There were 14 cases of malignant tumors (7 cases of glioma, 2 cases of diffuse large B-cell lymphoma, 5 cases of metastatic cancer) and 21 cases of benign tumors (17 cases of meningioma, 1 case of central neurocytoma, 2 cases of cavernous hemangioma and 1 case of arachnoid cyst). All 35 patients were located accurately before operation, the lesions were found quickly during operation, and the postoperative imaging data confirmed that the lesions were removed completely, of which 28 cases (80%) had significantly improved neurological symptoms one month after operation. 3D Slicer combined with Sina/MosoCam multimodal system has many advantages, such as simple and easy to learn, convenient operation, accurate positioning and free. It is considered to be a new technology that is practical, reliable, convenient for diagnosis and preoperative planning. It is suitable for popularization and use in neurosurgery and other operating rooms of all medical institutions.
Collapse
|
4
|
Mulita F, Verras GI, Anagnostopoulos CN, Kotis K. A Smarter Health through the Internet of Surgical Things. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22124577. [PMID: 35746359 PMCID: PMC9231158 DOI: 10.3390/s22124577] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2022] [Revised: 06/10/2022] [Accepted: 06/14/2022] [Indexed: 05/14/2023]
Abstract
(1) Background: In the last few years, technological developments in the surgical field have been rapid and are continuously evolving. One of the most revolutionizing breakthroughs was the introduction of the IoT concept within surgical practice. Our systematic review aims to summarize the most important studies evaluating the IoT concept within surgical practice, focusing on Telesurgery and surgical Telementoring. (2) Methods: We conducted a systematic review of the current literature, focusing on the Internet of Surgical Things in Telesurgery and Telementoring. Forty-eight (48) studies were included in this review. As secondary research questions, we also included brief overviews of the use of IoT in image-guided surgery, and patient Telemonitoring, by systematically analyzing fourteen (14) and nineteen (19) studies, respectively. (3) Results: Data from 219 patients and 757 healthcare professionals were quantitively analyzed. Study designs were primarily observational or based on model development. Palpable advantages from the IoT incorporation mainly include less surgical hours, accessibility to high quality treatment, and safer and more effective surgical education. Despite the described technological advances, and proposed benefits of the systems presented, there are still identifiable gaps in the literature that need to be further explored in a systematic manner. (4) Conclusions: The use of the IoT concept within the surgery domain is a widely incorporated but less investigated concept. Advantages have become palpable over the past decade, yet further research is warranted.
Collapse
Affiliation(s)
- Francesk Mulita
- Intelligent Systems Lab, Department of Cultural Technology and Communication, University of the Aegean, 81100 Mytilene, Greece;
- Department of Surgery, General University Hospital of Patras, 26504 Rio, Greece;
- Correspondence: (F.M.); (K.K.); Tel.: +30-6974822712 (K.K.)
| | | | | | - Konstantinos Kotis
- Intelligent Systems Lab, Department of Cultural Technology and Communication, University of the Aegean, 81100 Mytilene, Greece;
- Correspondence: (F.M.); (K.K.); Tel.: +30-6974822712 (K.K.)
| |
Collapse
|
5
|
Segato A, Di Vece C, Zucchelli S, Marzo MD, Wendler T, Azampour MF, Galvan S, Secoli R, De Momi E. Position-Based Dynamics Simulator of Brain Deformations for Path Planning and Intra-Operative Control in Keyhole Neurosurgery. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3090016] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
6
|
Riva M, Hiepe P, Frommert M, Divenuto I, Gay LG, Sciortino T, Nibali MC, Rossi M, Pessina F, Bello L. Intraoperative Computed Tomography and Finite Element Modelling for Multimodal Image Fusion in Brain Surgery. Oper Neurosurg (Hagerstown) 2021; 18:531-541. [PMID: 31342073 DOI: 10.1093/ons/opz196] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2019] [Accepted: 04/16/2019] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND intraoperative computer tomography (iCT) and advanced image fusion algorithms could improve the management of brainshift and the navigation accuracy. OBJECTIVE To evaluate the performance of an iCT-based fusion algorithm using clinical data. METHODS Ten patients with brain tumors were enrolled; preoperative MRI was acquired. The iCT was applied at the end of microsurgical resection. Elastic image fusion of the preoperative MRI to iCT data was performed by deformable fusion employing a biomechanical simulation based on a finite element model. Fusion accuracy was evaluated: the target registration error (TRE, mm) was measured for rigid and elastic fusion (Rf and Ef) and anatomical landmark pairs were divided into test and control structures according to distinct involvement by the brainshift. Intraoperative points describing the stereotactic position of the brain were also acquired and a qualitative evaluation of the adaptive morphing of the preoperative MRI was performed by 5 observers. RESULTS The mean TRE for control and test structures with Rf was 1.81 ± 1.52 and 5.53 ± 2.46 mm, respectively. No significant change was observed applying Ef to control structures; the test structures showed reduced TRE values of 3.34 ± 2.10 mm after Ef (P < .001). A 32% average gain (range 9%-54%) in accuracy of image registration was recorded. The morphed MRI showed robust matching with iCT scans and intraoperative stereotactic points. CONCLUSIONS The evaluated method increased the registration accuracy of preoperative MRI and iCT data. The iCT-based non-linear morphing of the preoperative MRI can potentially enhance the consistency of neuronavigation intraoperatively.
Collapse
Affiliation(s)
- Marco Riva
- Department of Medical Biotechnology and Translational Medicine, Università degli Studi di Milano, Milan, Italy.,Unit of Oncological Neurosurgery, Humanitas Clinical and Research Center - IRCCS, Rozzano, Italy
| | | | | | - Ignazio Divenuto
- Unit of Neuroradiology, Humanitas Clinical and Research Center - IRCCS, Rozzano, Italy
| | - Lorenzo G Gay
- Unit of Oncological Neurosurgery, Humanitas Clinical and Research Center - IRCCS, Rozzano, Italy
| | - Tommaso Sciortino
- Unit of Oncological Neurosurgery, Humanitas Clinical and Research Center - IRCCS, Rozzano, Italy
| | - Marco Conti Nibali
- Unit of Oncological Neurosurgery, Humanitas Clinical and Research Center - IRCCS, Rozzano, Italy
| | - Marco Rossi
- Unit of Oncological Neurosurgery, Humanitas Clinical and Research Center - IRCCS, Rozzano, Italy
| | - Federico Pessina
- Unit of Oncological Neurosurgery, Humanitas Clinical and Research Center - IRCCS, Rozzano, Italy.,Department of Biomedical Sciences, Humanitas University, Rozzano, Italy
| | - Lorenzo Bello
- Unit of Oncological Neurosurgery, Humanitas Clinical and Research Center - IRCCS, Rozzano, Italy.,Department of Oncology and Hemato-oncology, Università degli Studi di Milano, Milan, Italy
| |
Collapse
|
7
|
Wittek A, Bourantas G, Zwick BF, Joldes G, Esteban L, Miller K. Mathematical modeling and computer simulation of needle insertion into soft tissue. PLoS One 2020; 15:e0242704. [PMID: 33351854 PMCID: PMC7755224 DOI: 10.1371/journal.pone.0242704] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Accepted: 11/08/2020] [Indexed: 01/25/2023] Open
Abstract
In this study we present a kinematic approach for modeling needle insertion into soft tissues. The kinematic approach allows the presentation of the problem as Dirichlet-type (i.e. driven by enforced motion of boundaries) and therefore weakly sensitive to unknown properties of the tissues and needle-tissue interaction. The parameters used in the kinematic approach are straightforward to determine from images. Our method uses Meshless Total Lagrangian Explicit Dynamics (MTLED) method to compute soft tissue deformations. The proposed scheme was validated against experiments of needle insertion into silicone gel samples. We also present a simulation of needle insertion into the brain demonstrating the method's insensitivity to assumed mechanical properties of tissue.
Collapse
Affiliation(s)
- Adam Wittek
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - George Bourantas
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Benjamin F Zwick
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Grand Joldes
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Lionel Esteban
- Commonwealth Science and Industry Research Organization CSIRO, Medical XCT Facility, Kensington, Western Australia, Australia
| | - Karol Miller
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| |
Collapse
|
8
|
Minkin K, Gabrovski K, Sirakov S, Penkov M, Todorov Y, Karakostov V, Dimova P. Three-dimensional neuronavigation in SEEG-guided epilepsy surgery. Acta Neurochir (Wien) 2019; 161:917-923. [PMID: 30937608 DOI: 10.1007/s00701-019-03874-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2019] [Accepted: 03/06/2019] [Indexed: 10/27/2022]
Abstract
OBJECTIVES Epilepsy surgery is mainly cortical surgery and the precise definition of the epileptogenic zone on the complex cortical surface is of paramount importance. Stereoelectroencephalography (SEEG) may delineate the epileptogenic zone even in cases of non-lesional epilepsy. The aim of our study was to present a technique of 3D neuronavigation based on the brain surface and SEEG electrodes reconstructions using FSL and 3DSlicer software. PATIENTS AND METHODS Our study included 26 consecutive patients operated on for drug-resistant epilepsy after SEEG exploration between January 2015 and December 2017. All patients underwent 1.5 T pre-SEEG MRI, post-SEEG CT, DICOM data post-processing using FSL and 3DSlicer, preoperative planning on 3DSlicer, and intraoperative 3D neuronavigation. Accuracy and precision of 3D SEEG reconstruction and 3D neuronavigation was assessed. RESULTS We identified 125 entry points of SEEG electrodes during 26 operations. The accuracy of 3D reconstruction was 0.8 mm (range, 0-2 mm) with a precision of 1.5 mm. The accuracy of 3D SEEG neuronavigation was 2.68 mm (range, 0-6 mm). The precision of 3D neuronavigation was 1.48 mm. CONCLUSION 3D neuronavigation for SEEG-guided epilepsy surgery using free software for post-processing of common MRI sequences is possible and a reliable method even with navigation systems without a brain extraction tool.
Collapse
|
9
|
Riva M, Hennersperger C, Milletari F, Katouzian A, Pessina F, Gutierrez-Becker B, Castellano A, Navab N, Bello L. 3D intra-operative ultrasound and MR image guidance: pursuing an ultrasound-based management of brainshift to enhance neuronavigation. Int J Comput Assist Radiol Surg 2017; 12:1711-1725. [DOI: 10.1007/s11548-017-1578-5] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2016] [Accepted: 03/20/2017] [Indexed: 12/01/2022]
|