1
|
Peng C, Tian CX, Mu Y, Ma M, Zhang Z, Wan M, Liu J, Li Z, Zuo X, Li W, Li Y. Hyperspectral imaging facilitating resect-and-discard strategy through artificial intelligence-assisted diagnosis of colorectal polyps: A pilot study. Cancer Med 2024; 13:e70195. [PMID: 39320133 PMCID: PMC11423483 DOI: 10.1002/cam4.70195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 08/15/2024] [Accepted: 08/25/2024] [Indexed: 09/26/2024] Open
Abstract
BACKGROUND AND AIMS The resect-and-discard strategy for colorectal polyps based on accurate optical diagnosis remains challenges. Our aim was to investigate the feasibility of hyperspectral imaging (HSI) for identifying colorectal polyp properties and diagnosis of colorectal cancer in fresh tissues during colonoscopy. METHODS 144,900 two dimensional images generated from 161 hyperspectral images of colorectal polyp tissues were prospectively obtained from patients undergoing colonoscopy. A residual neural network model was trained with transfer learning to automatically differentiate colorectal polyps, validated by histopathologic diagnosis. The diagnostic performances of the HSI-AI model and endoscopists were calculated respectively, and the auxiliary efficiency of the model was evaluated after a 2-week interval. RESULTS Quantitative HSI revealed histological differences in colorectal polyps. The HSI-AI model showed considerable efficacy in differentiating nonneoplastic polyps, non-advanced adenomas, and advanced neoplasia in vitro, with sensitivities of 96.0%, 94.0%, and 99.0% and specificities of 99.0%, 99.0%, and 96.5%, respectively. With the assistance of the model, the median negative predictive value of neoplastic polyps increased from 50.0% to 88.2% (p = 0.013) in novices. CONCLUSION This study demonstrated the feasibility of using HSI as a diagnostic tool to differentiate neoplastic colorectal polyps in vitro and the potential of AI-assisted diagnosis synchronized with colonoscopy. The tool may improve the diagnostic performance of novices and facilitate the application of resect-and-discard strategy to decrease the cost.
Collapse
Affiliation(s)
- Cheng Peng
- Department of Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
| | - Chong Xuan Tian
- Department of Biomedical Engineering Institute, School of Control Science and EngineeringShandong UniversityJinanChina
| | - Yijun Mu
- Department of Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
| | - Mingjun Ma
- Department of Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
| | - Zhenlei Zhang
- Department of Biomedical Engineering Institute, School of Control Science and EngineeringShandong UniversityJinanChina
| | - Meng Wan
- Department of Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
| | - Jing Liu
- Department of Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
| | - Zhen Li
- Department of Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
| | - Xiuli Zuo
- Department of Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
- Laboratory of Translational Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
- Robot Engineering Laboratory for Precise Diagnosis and Therapy of GI Tumor, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
- Shandong Provincial Clinical Research Center for Digestive DiseaseJinanChina
| | - Wei Li
- Department of Biomedical Engineering Institute, School of Control Science and EngineeringShandong UniversityJinanChina
| | - Yanqing Li
- Department of Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
- Laboratory of Translational Gastroenterology, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
- Robot Engineering Laboratory for Precise Diagnosis and Therapy of GI Tumor, Qilu Hospital, Cheeloo College of MedicineShandong UniversityJinanChina
- Shandong Provincial Clinical Research Center for Digestive DiseaseJinanChina
| |
Collapse
|
2
|
Bannone E, Collins T, Esposito A, Cinelli L, De Pastena M, Pessaux P, Felli E, Andreotti E, Okamoto N, Barberio M, Felli E, Montorsi RM, Ingaglio N, Rodríguez-Luna MR, Nkusi R, Marescaux J, Hostettler A, Salvia R, Diana M. Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial. Surg Endosc 2024; 38:3758-3772. [PMID: 38789623 DOI: 10.1007/s00464-024-10880-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Accepted: 04/23/2024] [Indexed: 05/26/2024]
Abstract
BACKGROUND Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting. METHODS Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized. RESULTS A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%. CONCLUSION To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.
Collapse
Affiliation(s)
- Elisa Bannone
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France.
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy.
| | - Toby Collins
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | - Alessandro Esposito
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Lorenzo Cinelli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of Gastrointestinal Surgery, San Raffaele Hospital IRCCS, Milan, Italy
| | - Matteo De Pastena
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Patrick Pessaux
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, Strasbourg, France
- Institut of Viral and Liver Disease, Inserm U1110, University of Strasbourg, Strasbourg, France
| | - Emanuele Felli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, Strasbourg, France
- Institut of Viral and Liver Disease, Inserm U1110, University of Strasbourg, Strasbourg, France
| | - Elena Andreotti
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Nariaki Okamoto
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
| | - Manuel Barberio
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- General Surgery Department, Ospedale Cardinale G. Panico, Tricase, Italy
| | - Eric Felli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of Visceral Surgery and Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Roberto Maria Montorsi
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Naomi Ingaglio
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - María Rita Rodríguez-Luna
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
| | - Richard Nkusi
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | - Jacque Marescaux
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | | | - Roberto Salvia
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Michele Diana
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
- Department of Surgery, University Hospital of Geneva, Geneva, Switzerland
| |
Collapse
|
3
|
Schulz T, Köhler H, Kohler LH, Langer S, Nuwayhid R. Hyperspectral Imaging Detects Clitoral Vascular Issues in Gender-Affirming Surgery. Diagnostics (Basel) 2024; 14:1252. [PMID: 38928666 PMCID: PMC11202724 DOI: 10.3390/diagnostics14121252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2024] [Revised: 06/02/2024] [Accepted: 06/11/2024] [Indexed: 06/28/2024] Open
Abstract
The aim of this study was to assess the efficacy of hyperspectral imaging (HSI) as an intraoperative perfusion imaging modality during gender affirmation surgery (GAS). The hypothesis posited that HSI could quantify perfusion to the clitoral complex, thereby enabling the prediction of either uneventful wound healing or the occurrence of necrosis. In this non-randomised prospective clinical study, we enrolled 30 patients who underwent GAS in the form of vaginoplasty with the preparation of a clitoral complex from 2020 to 2024 and compared patients' characteristics as well as HSI data regarding clitoris necrosis. Individuals demonstrating uneventful wound healing pertaining to the clitoral complex were designated as Group A. Patients with complete necrosis of the neo-clitoris were assigned to Group B. Patient characteristics were collected and subsequently a comparative analysis carried out. No significant difference in patient characteristics was observed between the two groups. Necrosis occurred when both StO2 and NIR PI parameters fell below 40%. For the simultaneous occurrence of StO2 and NIR PI of 40% or less, a sensitivity of 92% and specificity of 72% was calculated. Intraoperatively, the onset of necrosis in the clitoral complex can be reliably predicted with the assistance of HSI.
Collapse
Affiliation(s)
- Torsten Schulz
- Department of Orthopaedic, Trauma and Plastic Surgery, University Hospital Leipzig, 04103 Leipzig, Germany; (S.L.); (R.N.)
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany;
| | | | - Stefan Langer
- Department of Orthopaedic, Trauma and Plastic Surgery, University Hospital Leipzig, 04103 Leipzig, Germany; (S.L.); (R.N.)
| | - Rima Nuwayhid
- Department of Orthopaedic, Trauma and Plastic Surgery, University Hospital Leipzig, 04103 Leipzig, Germany; (S.L.); (R.N.)
| |
Collapse
|
4
|
Qasim AB, Motta A, Studier-Fischer A, Sellner J, Ayala L, Hübner M, Bressan M, Özdemir B, Kowalewski KF, Nickel F, Seidlitz S, Maier-Hein L. Test-time augmentation with synthetic data addresses distribution shifts in spectral imaging. Int J Comput Assist Radiol Surg 2024; 19:1021-1031. [PMID: 38483702 PMCID: PMC11178652 DOI: 10.1007/s11548-024-03085-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Accepted: 02/22/2024] [Indexed: 06/15/2024]
Abstract
PURPOSE Surgical scene segmentation is crucial for providing context-aware surgical assistance. Recent studies highlight the significant advantages of hyperspectral imaging (HSI) over traditional RGB data in enhancing segmentation performance. Nevertheless, the current hyperspectral imaging (HSI) datasets remain limited and do not capture the full range of tissue variations encountered clinically. METHODS Based on a total of 615 hyperspectral images from a total of 16 pigs, featuring porcine organs in different perfusion states, we carry out an exploration of distribution shifts in spectral imaging caused by perfusion alterations. We further introduce a novel strategy to mitigate such distribution shifts, utilizing synthetic data for test-time augmentation. RESULTS The effect of perfusion changes on state-of-the-art (SOA) segmentation networks depended on the organ and the specific perfusion alteration induced. In the case of the kidney, we observed a performance decline of up to 93% when applying a state-of-the-art (SOA) network under ischemic conditions. Our method improved on the state-of-the-art (SOA) by up to 4.6 times. CONCLUSION Given its potential wide-ranging relevance to diverse pathologies, our approach may serve as a pivotal tool to enhance neural network generalization within the realm of spectral imaging.
Collapse
Affiliation(s)
- Ahmad Bin Qasim
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany.
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany.
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany.
| | - Alessandro Motta
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Studier-Fischer
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Jan Sellner
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, A Partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - Leonardo Ayala
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Marco Hübner
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - Marc Bressan
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Berkin Özdemir
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Karl Friedrich Kowalewski
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
- Department of Urology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Felix Nickel
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, A Partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, A Partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
| |
Collapse
|
5
|
Kumar A, Goyal A. Emerging molecules, tools, technology, and future of surgical knife in gastroenterology. World J Gastrointest Surg 2024; 16:988-998. [PMID: 38690056 PMCID: PMC11056674 DOI: 10.4240/wjgs.v16.i4.988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Revised: 02/18/2024] [Accepted: 04/03/2024] [Indexed: 04/22/2024] Open
Abstract
The 21st century has started with several innovations in the medical sciences, with wide applications in health care management. This development has taken in the field of medicines (newer drugs/molecules), various tools and technology which has completely changed the patient management including abdominal surgery. Surgery for abdominal diseases has moved from maximally invasive to minimally invasive (laparoscopic and robotic) surgery. Some of the newer medicines have its impact on need for surgical intervention. This article focuses on the development of these emerging molecules, tools, and technology and their impact on present surgical form and its future effects on the surgical intervention in gastroenterological diseases.
Collapse
Affiliation(s)
- Ashok Kumar
- Department of Surgical Gastroenterology, Sanjay Gandhi Post Graduate Institute of Medical Sciences, Lucknow 226014, Uttar Pradesh, India
| | - Anirudh Goyal
- Department of Surgical Gastroenterology, Sanjay Gandhi Post Graduate Institute of Medical Sciences, Lucknow 226014, Uttar Pradesh, India
| |
Collapse
|
6
|
Zhang Y, Chen Z, Yang X. Light-M: An efficient lightweight medical image segmentation framework for resource-constrained IoMT. Comput Biol Med 2024; 170:108088. [PMID: 38320339 DOI: 10.1016/j.compbiomed.2024.108088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Revised: 12/22/2023] [Accepted: 01/27/2024] [Indexed: 02/08/2024]
Abstract
The Internet of Medical Things (IoMT) is being incorporated into current healthcare systems. This technology intends to connect patients, IoMT devices, and hospitals over mobile networks, allowing for more secure, quick, and convenient health monitoring and intelligent healthcare services. However, existing intelligent healthcare applications typically rely on large-scale AI models, and standard IoMT devices have significant resource constraints. To alleviate this paradox, in this paper, we propose a Knowledge Distillation (KD)-based IoMT end-edge-cloud orchestrated architecture for medical image segmentation tasks, called Light-M, aiming to deploy a lightweight medical model in resource-constrained IoMT devices. Specifically, Light-M trains a large teacher model in the cloud server and employs computation in local nodes through imitation of the performance of the teacher model using knowledge distillation. Light-M contains two KD strategies: (1) active exploration and passive transfer (AEPT) and (2) self-attention-based inter-class feature variation (AIFV) distillation for the medical image segmentation task. The AEPT encourages the student model to learn undiscovered knowledge/features of the teacher model without additional feature layers, aiming to explore new features and outperform the teacher. To improve the distinguishability of the student for different classes, the student learns the self-attention-based feature variation (AIFV) between classes. Since the proposed AEPT and AIFV only appear in the training process, our framework does not involve any additional computation burden for a student model during the segmentation task deployment. Extensive experiments on cardiac images and public real-scene datasets demonstrate that our approach improves student model learning representations and outperforms state-of-the-art methods by combining two knowledge distillation strategies. Moreover, when deployed on the IoT device, the distilled student model takes only 29.6 ms for one sample at the inference step.
Collapse
Affiliation(s)
- Yifan Zhang
- Shenzhen University, 3688 Nanhai Ave., Shenzhen, 518060, Guangdong, China
| | - Zhuangzhuang Chen
- Shenzhen University, 3688 Nanhai Ave., Shenzhen, 518060, Guangdong, China
| | - Xuan Yang
- Shenzhen University, 3688 Nanhai Ave., Shenzhen, 518060, Guangdong, China.
| |
Collapse
|
7
|
Uchikov P, Khalid U, Kraev K, Hristov B, Kraeva M, Tenchev T, Chakarov D, Sandeva M, Dragusheva S, Taneva D, Batashki A. Artificial Intelligence in the Diagnosis of Colorectal Cancer: A Literature Review. Diagnostics (Basel) 2024; 14:528. [PMID: 38472999 DOI: 10.3390/diagnostics14050528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2024] [Revised: 02/26/2024] [Accepted: 02/27/2024] [Indexed: 03/14/2024] Open
Abstract
BACKGROUND The aim of this review is to explore the role of artificial intelligence in the diagnosis of colorectal cancer, how it impacts CRC morbidity and mortality, and why its role in clinical medicine is limited. METHODS A targeted, non-systematic review of the published literature relating to colorectal cancer diagnosis was performed with PubMed databases that were scouted to help provide a more defined understanding of the recent advances regarding artificial intelligence and their impact on colorectal-related morbidity and mortality. Articles were included if deemed relevant and including information associated with the keywords. RESULTS The advancements in artificial intelligence have been significant in facilitating an earlier diagnosis of CRC. In this review, we focused on evaluating genomic biomarkers, the integration of instruments with artificial intelligence, MR and hyperspectral imaging, and the architecture of neural networks. We found that these neural networks seem practical and yield positive results in initial testing. Furthermore, we explored the use of deep-learning-based majority voting methods, such as bag of words and PAHLI, in improving diagnostic accuracy in colorectal cancer detection. Alongside this, the autonomous and expansive learning ability of artificial intelligence, coupled with its ability to extract increasingly complex features from images or videos without human reliance, highlight its impact in the diagnostic sector. Despite this, as most of the research involves a small sample of patients, a diversification of patient data is needed to enhance cohort stratification for a more sensitive and specific neural model. We also examined the successful application of artificial intelligence in predicting microsatellite instability, showcasing its potential in stratifying patients for targeted therapies. CONCLUSIONS Since its commencement in colorectal cancer, artificial intelligence has revealed a multitude of functionalities and augmentations in the diagnostic sector of CRC. Given its early implementation, its clinical application remains a fair way away, but with steady research dedicated to improving neural architecture and expanding its applicational range, there is hope that these advanced neural software could directly impact the early diagnosis of CRC. The true promise of artificial intelligence, extending beyond the medical sector, lies in its potential to significantly influence the future landscape of CRC's morbidity and mortality.
Collapse
Affiliation(s)
- Petar Uchikov
- Department of Special Surgery, Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Usman Khalid
- Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Krasimir Kraev
- Department of Propaedeutics of Internal Diseases "Prof. Dr. Anton Mitov", Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Bozhidar Hristov
- Section "Gastroenterology", Second Department of Internal Diseases, Medical Faculty, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Maria Kraeva
- Department of Otorhinolaryngology, Medical Faculty, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Tihomir Tenchev
- Department of Special Surgery, Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Dzhevdet Chakarov
- Department of Propaedeutics of Surgical Diseases, Section of General Surgery, Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Milena Sandeva
- Department of Midwifery, Faculty of Public Health, Medical University of Plovdiv, 4000 Plovdiv, Bulgaria
| | - Snezhanka Dragusheva
- Department of Nursing Care, Faculty of Public Health, Medical University of Plovdiv, 4000 Plovdiv, Bulgaria
| | - Daniela Taneva
- Department of Nursing Care, Faculty of Public Health, Medical University of Plovdiv, 4000 Plovdiv, Bulgaria
| | - Atanas Batashki
- Department of Special Surgery, Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| |
Collapse
|
8
|
Rai HM, Yoo J, Atif Moqurrab S, Dashkevych S. Advancements in traditional machine learning techniques for detection and diagnosis of fatal cancer types: Comprehensive review of biomedical imaging datasets. MEASUREMENT 2024; 225:114059. [DOI: 10.1016/j.measurement.2023.114059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/11/2024]
|
9
|
Rai HM, Yoo J. Analysis of Colorectal and Gastric Cancer Classification: A Mathematical Insight Utilizing Traditional Machine Learning Classifiers. MATHEMATICS 2023; 11:4937. [DOI: 10.3390/math11244937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/11/2024]
Abstract
Cancer remains a formidable global health challenge, claiming millions of lives annually. Timely and accurate cancer diagnosis is imperative. While numerous reviews have explored cancer classification using machine learning and deep learning techniques, scant literature focuses on traditional ML methods. In this manuscript, we undertake a comprehensive review of colorectal and gastric cancer detection specifically employing traditional ML classifiers. This review emphasizes the mathematical underpinnings of cancer detection, encompassing preprocessing techniques, feature extraction, machine learning classifiers, and performance assessment metrics. We provide mathematical formulations for these key components. Our analysis is limited to peer-reviewed articles published between 2017 and 2023, exclusively considering medical imaging datasets. Benchmark and publicly available imaging datasets for colorectal and gastric cancers are presented. This review synthesizes findings from 20 articles on colorectal cancer and 16 on gastric cancer, culminating in a total of 36 research articles. A significant focus is placed on mathematical formulations for commonly used preprocessing techniques, features, ML classifiers, and assessment metrics. Crucially, we introduce our optimized methodology for the detection of both colorectal and gastric cancers. Our performance metrics analysis reveals remarkable results: 100% accuracy in both cancer types, but with the lowest sensitivity recorded at 43.1% for gastric cancer.
Collapse
Affiliation(s)
- Hari Mohan Rai
- School of Computing, Gachon University, Seongnam-si 13120, Republic of Korea
| | - Joon Yoo
- School of Computing, Gachon University, Seongnam-si 13120, Republic of Korea
| |
Collapse
|
10
|
Young E, Edwards L, Singh R. The Role of Artificial Intelligence in Colorectal Cancer Screening: Lesion Detection and Lesion Characterization. Cancers (Basel) 2023; 15:5126. [PMID: 37958301 PMCID: PMC10647850 DOI: 10.3390/cancers15215126] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Revised: 10/14/2023] [Accepted: 10/14/2023] [Indexed: 11/15/2023] Open
Abstract
Colorectal cancer remains a leading cause of cancer-related morbidity and mortality worldwide, despite the widespread uptake of population surveillance strategies. This is in part due to the persistent development of 'interval colorectal cancers', where patients develop colorectal cancer despite appropriate surveillance intervals, implying pre-malignant polyps were not resected at a prior colonoscopy. Multiple techniques have been developed to improve the sensitivity and accuracy of lesion detection and characterisation in an effort to improve the efficacy of colorectal cancer screening, thereby reducing the incidence of interval colorectal cancers. This article presents a comprehensive review of the transformative role of artificial intelligence (AI), which has recently emerged as one such solution for improving the quality of screening and surveillance colonoscopy. Firstly, AI-driven algorithms demonstrate remarkable potential in addressing the challenge of overlooked polyps, particularly polyp subtypes infamous for escaping human detection because of their inconspicuous appearance. Secondly, AI empowers gastroenterologists without exhaustive training in advanced mucosal imaging to characterise polyps with accuracy similar to that of expert interventionalists, reducing the dependence on pathologic evaluation and guiding appropriate resection techniques or referrals for more complex resections. AI in colonoscopy holds the potential to advance the detection and characterisation of polyps, addressing current limitations and improving patient outcomes. The integration of AI technologies into routine colonoscopy represents a promising step towards more effective colorectal cancer screening and prevention.
Collapse
Affiliation(s)
- Edward Young
- Faculty of Health and Medical Sciences, University of Adelaide, Lyell McEwin Hospital, Haydown Rd, Elizabeth Vale, SA 5112, Australia
| | - Louisa Edwards
- Faculty of Health and Medical Sciences, University of Adelaide, Queen Elizabeth Hospital, Port Rd, Woodville South, SA 5011, Australia
| | - Rajvinder Singh
- Faculty of Health and Medical Sciences, University of Adelaide, Lyell McEwin Hospital, Haydown Rd, Elizabeth Vale, SA 5112, Australia
| |
Collapse
|
11
|
Chalopin C, Pfahl A, Köhler H, Knospe L, Maktabi M, Unger M, Jansen-Winkeln B, Thieme R, Moulla Y, Mehdorn M, Sucher R, Neumuth T, Gockel I, Melzer A. Alternative intraoperative optical imaging modalities for fluorescence angiography in gastrointestinal surgery: spectral imaging and imaging photoplethysmography. MINIM INVASIV THER 2023; 32:222-232. [PMID: 36622288 DOI: 10.1080/13645706.2022.2164469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 11/29/2022] [Indexed: 01/10/2023]
Abstract
INTRODUCTION Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA. MATERIAL AND METHODS The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works. RESULTS The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method. CONCLUSIONS Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Luise Knospe
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Department of Electrical, Mechanical and Industrial Engineering, Anhalt University of Applied Science, Köthen (Anhalt), Germany
| | - Michael Unger
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
- Department of General, Visceral and Oncological Surgery, St. Georg Hospital, Leipzig, Germany
| | - René Thieme
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Matthias Mehdorn
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Robert Sucher
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Institute of Medical Science and Technology (IMSAT), University of Dundee, Dundee, UK
| |
Collapse
|
12
|
Gabralla LA, Hussien AM, AlMohimeed A, Saleh H, Alsekait DM, El-Sappagh S, Ali AA, Refaat Hassan M. Automated Diagnosis for Colon Cancer Diseases Using Stacking Transformer Models and Explainable Artificial Intelligence. Diagnostics (Basel) 2023; 13:2939. [PMID: 37761306 PMCID: PMC10529133 DOI: 10.3390/diagnostics13182939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2023] [Revised: 08/23/2023] [Accepted: 08/31/2023] [Indexed: 09/29/2023] Open
Abstract
Colon cancer is the third most common cancer type worldwide in 2020, almost two million cases were diagnosed. As a result, providing new, highly accurate techniques in detecting colon cancer leads to early and successful treatment of this disease. This paper aims to propose a heterogenic stacking deep learning model to predict colon cancer. Stacking deep learning is integrated with pretrained convolutional neural network (CNN) models with a metalearner to enhance colon cancer prediction performance. The proposed model is compared with VGG16, InceptionV3, Resnet50, and DenseNet121 using different evaluation metrics. Furthermore, the proposed models are evaluated using the LC25000 and WCE binary and muticlassified colon cancer image datasets. The results show that the stacking models recorded the highest performance for the two datasets. For the LC25000 dataset, the stacked model recorded the highest performance accuracy, recall, precision, and F1 score (100). For the WCE colon image dataset, the stacked model recorded the highest performance accuracy, recall, precision, and F1 score (98). Stacking-SVM achieved the highest performed compared to existing models (VGG16, InceptionV3, Resnet50, and DenseNet121) because it combines the output of multiple single models and trains and evaluates a metalearner using the output to produce better predictive results than any single model. Black-box deep learning models are represented using explainable AI (XAI).
Collapse
Affiliation(s)
- Lubna Abdelkareim Gabralla
- Department of Computer Science and Information Technology, Applied College, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| | - Ali Mohamed Hussien
- Department of Computer Science, Faculty of Science, Aswan University, Aswan 81528, Egypt
| | - Abdulaziz AlMohimeed
- College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 13318, Saudi Arabia
| | - Hager Saleh
- Faculty of Computers and Artificial Intelligence, South Valley University, Hurghada 84511, Egypt
| | - Deema Mohammed Alsekait
- Department of Computer Science and Information Technology, Applied College, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| | - Shaker El-Sappagh
- Faculty of Computer Science and Engineering, Galala University, Suez 34511, Egypt
- Information Systems Department, Faculty of Computers and Artificial Intelligence, Benha University, Banha 13518, Egypt
| | - Abdelmgeid A. Ali
- Faculty of Computers and Information, Minia University, Minia 61519, Egypt
| | - Moatamad Refaat Hassan
- Department of Computer Science, Faculty of Science, Aswan University, Aswan 81528, Egypt
| |
Collapse
|
13
|
Liao WC, Mukundan A, Sadiaza C, Tsao YM, Huang CW, Wang HC. Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging. BIOMEDICAL OPTICS EXPRESS 2023; 14:4383-4405. [PMID: 37799695 PMCID: PMC10549751 DOI: 10.1364/boe.492635] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 10/07/2023]
Abstract
One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.
Collapse
Affiliation(s)
- Wei-Chih Liao
- Department of Internal Medicine, National Taiwan University Hospital, National Taiwan University College of Medicine, Taipei, Taiwan
- Graduate Institute of Epidemiology and Preventive Medicine, National Taiwan University, Taipei, Taiwan
| | - Arvind Mukundan
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Cleorita Sadiaza
- Department of Mechanical Engineering, Far Eastern University, P. Paredes St., Sampaloc, Manila, 1015, Philippines
| | - Yu-Ming Tsao
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Chien-Wei Huang
- Department of Gastroenterology, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st.Rd., Lingya District, Kaohsiung City 80284, Taiwan
- Department of Nursing, Tajen University, 20, Weixin Rd., Yanpu Township, Pingtung County 90741, Taiwan
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
- Department of Medical Research, Dalin Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, No. 2, Minsheng Road, Dalin, Chiayi, 62247, Taiwan
- Director of Technology Development, Hitspectra Intelligent Technology Co., Ltd., 4F., No. 2, Fuxing 4th Rd., Qianzhen Dist., Kaohsiung City 80661, Taiwan
| |
Collapse
|
14
|
Jong LJS, Post AL, Veluponnar D, Geldof F, Sterenborg HJCM, Ruers TJM, Dashtbozorg B. Tissue Classification of Breast Cancer by Hyperspectral Unmixing. Cancers (Basel) 2023; 15:2679. [PMID: 37345015 DOI: 10.3390/cancers15102679] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 04/26/2023] [Accepted: 05/04/2023] [Indexed: 06/23/2023] Open
Abstract
(1) Background: Assessing the resection margins during breast-conserving surgery is an important clinical need to minimize the risk of recurrent breast cancer. However, currently there is no technique that can provide real-time feedback to aid surgeons in the margin assessment. Hyperspectral imaging has the potential to overcome this problem. To classify resection margins with this technique, a tissue discrimination model should be developed, which requires a dataset with accurate ground-truth labels. However, establishing such a dataset for resection specimens is difficult. (2) Methods: In this study, we therefore propose a novel approach based on hyperspectral unmixing to determine which pixels within hyperspectral images should be assigned to the ground-truth labels from histopathology. Subsequently, we use this hyperspectral-unmixing-based approach to develop a tissue discrimination model on the presence of tumor tissue within the resection margins of ex vivo breast lumpectomy specimens. (3) Results: In total, 372 measured locations were included on the lumpectomy resection surface of 189 patients. We achieved a sensitivity of 0.94, specificity of 0.85, accuracy of 0.87, Matthew's correlation coefficient of 0.71, and area under the curve of 0.92. (4) Conclusion: Using this hyperspectral-unmixing-based approach, we demonstrated that the measured locations with hyperspectral imaging on the resection surface of lumpectomy specimens could be classified with excellent performance.
Collapse
Affiliation(s)
- Lynn-Jade S Jong
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Anouk L Post
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Centers, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
| | - Dinusha Veluponnar
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Freija Geldof
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Henricus J C M Sterenborg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Centers, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
| | - Theo J M Ruers
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Behdad Dashtbozorg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| |
Collapse
|
15
|
Muniz FB, Baffa MDFO, Garcia SB, Bachmann L, Felipe JC. Histopathological diagnosis of colon cancer using micro-FTIR hyperspectral imaging and deep learning. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 231:107388. [PMID: 36773592 DOI: 10.1016/j.cmpb.2023.107388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Revised: 01/26/2023] [Accepted: 01/30/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Current studies based on digital biopsy images have achieved satisfactory results in detecting colon cancer despite their limited visual spectral range. Such methods may be less accurate when applied to samples taken from the tumor margin region or to samples containing multiple diagnoses. In contrast with the traditional computer vision approach, micro-FTIR hyperspectral images quantify the tissue-light interaction on a histochemical level and characterize different tissue pathologies, as they present a unique spectral signature. Therefore, this paper investigates the possibility of using hyperspectral images acquired over micro-FTIR absorbance spectroscopy to characterize healthy, inflammatory, and tumor colon tissues. METHODS The proposed method consists of modeling hyperspectral data into a voxel format to detect the patterns of each voxel using fully connected deep neural network. A web-based computer-aided diagnosis tool for inference is also provided. RESULTS Our experiments were performed using the K-fold cross-validation protocol in an intrapatient approach and achieved an overall accuracy of 99% using a deep neural network and 96% using a linear support vector machine. Through the experiments, we noticed the high performance of the method in characterizing such tissues using deep learning and hyperspectral images, indicating that the infrared spectrum contains relevant information and can be used to assist pathologists during the diagnostic process.
Collapse
Affiliation(s)
- Frederico Barbosa Muniz
- Department of Computing and Mathematics, University of São Paulo, Bandeirantes Av. 3900, Monte Alegre, Ribeirão Preto, SP 14040-901, Brazil
| | - Matheus de Freitas Oliveira Baffa
- Department of Computing and Mathematics, University of São Paulo, Bandeirantes Av. 3900, Monte Alegre, Ribeirão Preto, SP 14040-901, Brazil
| | - Sergio Britto Garcia
- Department of Pathology and Legal Medicine, Medical School of Ribeirão Preto, University of São Paulo, Ribeirão Preto, SP, Brazil
| | - Luciano Bachmann
- Department of Physics, University of São Paulo, Ribeirão Preto, SP, Brazil
| | - Joaquim Cezar Felipe
- Department of Computing and Mathematics, University of São Paulo, Bandeirantes Av. 3900, Monte Alegre, Ribeirão Preto, SP 14040-901, Brazil.
| |
Collapse
|
16
|
Ellebrecht DB. Hyperspectral imaging enables the differentiation of differentially inflated and perfused pulmonary tissue: a proof-of-concept study in pulmonary lobectomies for intersegmental plane mapping. BIOMED ENG-BIOMED TE 2023:bmt-2022-0389. [PMID: 36932645 DOI: 10.1515/bmt-2022-0389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Accepted: 03/02/2023] [Indexed: 03/19/2023]
Abstract
OBJECTIVES The identification of the intersegmental plane is a major interoperative challenges during pulmonary segmentectomies. The objective of this pilot study is to test the feasibility of lung perfusion assessment by Hyperspectral Imaging for identification of the intersegmental plane. METHODS A pilot study (clinicaltrials.org: NCT04784884) was conducted in patients with lung cancer. Measuring tissue oxygenation (StO2; upper tissue perfusion), organ hemoglobin index (OHI), near-infrared index (NIR; deeper tissue perfusion) and tissue water index (TWI), the Hyperspectral Imaging measurements were carried out in inflated (Pvent) and deflated pulmonary lobes (PnV) as well as in deflated pulmonary lobes with divided circulation (PnVC) before dissection of the lobar bronchus. RESULTS A total of 341 measuring points were evaluated during pulmonary lobectomies. Pulmonary lobes showed a reduced StO2 (Pvent: 84.56% ± 3.92 vs. PnV: 63.62% ± 11.62 vs. PnVC: 39.20% ± 23.57; p<0.05) and NIR-perfusion (Pvent: 50.55 ± 5.62 vs. PnV: 47.55 ± 3.38 vs. PnVC: 27.60 ± 9.33; p<0.05). There were no differences of OHI and TWI between the three groups. CONCLUSIONS This pilot study demonstrates that HSI enables differentiation between different ventilated and perfused pulmonary tissue as a precondition for HSI segment mapping.
Collapse
Affiliation(s)
- David B Ellebrecht
- Department of Thoracic Surgery, LungClinic Großhansdorf, Großhansdorf, Germany
| |
Collapse
|
17
|
Ellebrecht DB, Kugler C. Intraoperative Determination of Bronchus Stump and Anastomosis Perfusion with Hyperspectral Imaging. Surg Innov 2023:15533506231157165. [PMID: 36802983 DOI: 10.1177/15533506231157165] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2023]
Abstract
BACKGROUND The intraoperative evaluation of bronchus perfusion is limited. Hyperspectral Imaging (HSI) is a newly established intraoperative imaging technique that enables a non-invasive, real-time perfusion analysis. Therefore, the purpose of this study was to determine the intraoperative perfusion of bronchus stump and anastomosis during pulmonary resections with HSI. METHODS In this prospective, IDEAL Stage 2a study (Clinicaltrials.gov: NCT04784884) HSI measurements were carried out before bronchial dissection and after bronchial stump formation or bronchial anastomosis, respectively. Tissue oxygenation (StO2; upper tissue perfusion), organ hemoglobin index (OHI), near-infrared index (NIR; deeper tissue perfusion) and tissue water index (TWI) were calculated. RESULTS Bronchus stumps showed a reduced NIR (77.82 ± 10.27 vs 68.01 ± 8.95; P = 0,02158) and OHI (48.60 ± 1.39 vs 38.15 ± 9.74; P = <.0001), although the perfusion of the upper tissue layers was equivalent before and after resection (67.42% ± 12.53 vs 65.91% ± 10.40). In the sleeve resection group, we found both a significant decrease in StO 2 and NIR between central bronchus and anastomosis region (StO2: 65.09% ± 12.57 vs 49.45 ± 9.94; P = .044; NIR: 83.73 ± 10.92 vs 58.62 ± 3.01; P = .0063). Additionally, NIR was decreased in the re-anastomosed bronchus compared to central bronchus region (83.73 ± 10.92 vs 55.15 ± 17.56; P = .0029). CONCLUSIONS Although both bronchus stumps and anastomosis show an intraoperative reduction of tissue perfusion, there is no difference of tissue hemoglobin level in bronchus anastomosis.
Collapse
Affiliation(s)
- David B Ellebrecht
- Department of Surgery, 9213LungClinic Großhansdorf, Großhansdorf, Germany
| | - Christian Kugler
- Department of Surgery, 9213LungClinic Großhansdorf, Großhansdorf, Germany
| |
Collapse
|
18
|
Avram MF, Lazăr DC, Mariş MI, Olariu S. Artificial intelligence in improving the outcome of surgical treatment in colorectal cancer. Front Oncol 2023; 13:1116761. [PMID: 36733307 PMCID: PMC9886660 DOI: 10.3389/fonc.2023.1116761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 01/03/2023] [Indexed: 01/19/2023] Open
Abstract
Background A considerable number of recent research have used artificial intelligence (AI) in the area of colorectal cancer (CRC). Surgical treatment of CRC still remains the most important curative component. Artificial intelligence in CRC surgery is not nearly as advanced as it is in screening (colonoscopy), diagnosis and prognosis, especially due to the increased complexity and variability of structures and elements in all fields of view, as well as a general shortage of annotated video banks for utilization. Methods A literature search was made and relevant studies were included in the minireview. Results The intraoperative steps which, at this moment, can benefit from AI in CRC are: phase and action recognition, excision plane navigation, endoscopy control, real-time circulation analysis, knot tying, automatic optical biopsy and hyperspectral imaging. This minireview also analyses the current advances in robotic treatment of CRC as well as the present possibility of automated CRC robotic surgery. Conclusions The use of AI in CRC surgery is still at its beginnings. The development of AI models capable of reproducing a colorectal expert surgeon's skill, the creation of large and complex datasets and the standardization of surgical colorectal procedures will contribute to the widespread use of AI in CRC surgical treatment.
Collapse
Affiliation(s)
- Mihaela Flavia Avram
- Department of Surgery X, 1st Surgery Discipline, “Victor Babeş” University of Medicine and Pharmacy Timişoara, Timişoara, Romania,Department of Mathematics, Politehnica University Timisoara, Timişoara, Romania,*Correspondence: Mihaela Flavia Avram,
| | - Daniela Cornelia Lazăr
- Department V of Internal Medicine I, Discipline of Internal Medicine IV, “Victor Babeş” University of Medicine and Pharmacy Timişoara, Timişoara, Romania
| | - Mihaela Ioana Mariş
- Department of Functional Sciences, Division of Physiopathology, “Victor Babes” University of Medicine and Pharmacy Timisoara, Timisoara, Romania,Center for Translational Research and Systems Medicine, “Victor Babes” University of Medicine and Pharmacy Timisoara, Timisoara, Romania
| | - Sorin Olariu
- Department of Surgery X, 1st Surgery Discipline, “Victor Babeş” University of Medicine and Pharmacy Timişoara, Timişoara, Romania
| |
Collapse
|
19
|
Spectral Similarity Measures for In Vivo Human Tissue Discrimination Based on Hyperspectral Imaging. Diagnostics (Basel) 2023; 13:diagnostics13020195. [PMID: 36673005 PMCID: PMC9857871 DOI: 10.3390/diagnostics13020195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 12/08/2022] [Accepted: 12/21/2022] [Indexed: 01/06/2023] Open
Abstract
PROBLEM Similarity measures are widely used as an approved method for spectral discrimination or identification with their applications in different areas of scientific research. Even though a range of works have been presented, only a few showed slightly promising results for human tissue, and these were mostly focused on pathological and non-pathological tissue classification. METHODS In this work, several spectral similarity measures on hyperspectral (HS) images of in vivo human tissue were evaluated for tissue discrimination purposes. Moreover, we introduced two new hybrid spectral measures, called SID-JM-TAN(SAM) and SID-JM-TAN(SCA). We analyzed spectral signatures obtained from 13 different human tissue types and two different materials (gauze, instruments), collected from HS images of 100 patients during surgeries. RESULTS The quantitative results showed the reliable performance of the different similarity measures and the proposed hybrid measures for tissue discrimination purposes. The latter produced higher discrimination values, up to 6.7 times more than the classical spectral similarity measures. Moreover, an application of the similarity measures was presented to support the annotations of the HS images. We showed that the automatic checking of tissue-annotated thyroid and colon tissues was successful in 73% and 60% of the total spectra, respectively. The hybrid measures showed the highest performance. Furthermore, the automatic labeling of wrongly annotated tissues was similar for all measures, with an accuracy of up to 90%. CONCLUSION In future work, the proposed spectral similarity measures will be integrated with tools to support physicians in annotations and tissue labeling of HS images.
Collapse
|
20
|
Tomanic T, Rogelj L, Stergar J, Markelc B, Bozic T, Brezar SK, Sersa G, Milanic M. Estimating quantitative physiological and morphological tissue parameters of murine tumor models using hyperspectral imaging and optical profilometry. JOURNAL OF BIOPHOTONICS 2023; 16:e202200181. [PMID: 36054067 DOI: 10.1002/jbio.202200181] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/18/2022] [Revised: 07/27/2022] [Accepted: 08/05/2022] [Indexed: 06/15/2023]
Abstract
Understanding tumors and their microenvironment are essential for successful and accurate disease diagnosis. Tissue physiology and morphology are altered in tumors compared to healthy tissues, and there is a need to monitor tumors and their surrounding tissues, including blood vessels, non-invasively. This preliminary study utilizes a multimodal optical imaging system combining hyperspectral imaging (HSI) and three-dimensional (3D) optical profilometry (OP) to capture hyperspectral images and surface shapes of subcutaneously grown murine tumor models. Hyperspectral images are corrected with 3D OP data and analyzed using the inverse-adding doubling (IAD) method to extract tissue properties such as melanin volume fraction and oxygenation. Blood vessels are segmented using the B-COSFIRE algorithm from oxygenation maps. From 3D OP data, tumor volumes are calculated and compared to manual measurements using a vernier caliper. Results show that tumors can be distinguished from healthy tissue based on most extracted tissue parameters ( p < 0.05 ). Furthermore, blood oxygenation is 50% higher within the blood vessels than in the surrounding tissue, and tumor volumes calculated using 3D OP agree within 26% with manual measurements using a vernier caliper. Results suggest that combining HSI and OP could provide relevant quantitative information about tumors and improve the disease diagnosis.
Collapse
Affiliation(s)
- Tadej Tomanic
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
| | - Luka Rogelj
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
| | - Jost Stergar
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
- Jozef Stefan Institute, Ljubljana, Slovenia
| | - Bostjan Markelc
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Health Sciences, University of Ljubljana, Ljubljana, Slovenia
| | - Tim Bozic
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia
| | - Simona Kranjc Brezar
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia
| | - Gregor Sersa
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Health Sciences, University of Ljubljana, Ljubljana, Slovenia
| | - Matija Milanic
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
- Jozef Stefan Institute, Ljubljana, Slovenia
| |
Collapse
|
21
|
Preoperative Immunocite-Derived Ratios Predict Surgical Complications Better when Artificial Neural Networks Are Used for Analysis-A Pilot Comparative Study. J Pers Med 2023; 13:jpm13010101. [PMID: 36675762 PMCID: PMC9861480 DOI: 10.3390/jpm13010101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 12/24/2022] [Accepted: 12/27/2022] [Indexed: 01/04/2023] Open
Abstract
We aimed to comparatively assess the prognostic preoperative value of the main peripheral blood components and their ratios-the systemic immune-inflammation index (SII), neutrophil-to-lymphocyte ratio (NLR), lymphocyte-to-monocyte ratio (LMR), and platelet-to-lymphocyte ratio (PLR)-to the use of artificial-neural-network analysis in determining undesired postoperative outcomes in colorectal cancer patients. Our retrospective study included 281 patients undergoing elective radical surgery for colorectal cancer in the last seven years. The preoperative values of SII, NLR, LMR, and PLR were analyzed in relation to postoperative complications, with a special emphasis on their ability to accurately predict the occurrence of anastomotic leak. A feed-forward fully connected multilayer perceptron network (MLP) was trained and tested alongside conventional statistical tools to assess the predictive value of the abovementioned blood markers in terms of sensitivity and specificity. Statistically significant differences and moderate correlation levels were observed for SII and NLR in predicting the anastomotic leak rate and degree of postoperative complications. No correlations were found between the LMR and PLR or the abovementioned outcomes. The MLP network analysis showed superior prediction value in terms of both sensitivity (0.78 ± 0.07; 0.74 ± 0.04; 0.71 ± 0.13) and specificity (0.81 ± 0.11; 0.69 ± 0.03; 0.9 ± 0.04) for all the given tasks. Preoperative SII and NLR appear to be modest prognostic factors for anastomotic leakage and overall morbidity. Using an artificial neural network offers superior prognostic results in the preoperative risk assessment for overall morbidity and anastomotic leak rate.
Collapse
|
22
|
Wang W, Zhang X, Wang SH, Zhang YD. Covid-19 Diagnosis by WE-SAJ. SYSTEMS SCIENCE & CONTROL ENGINEERING 2022; 10:325-335. [PMID: 36568847 PMCID: PMC7613983 DOI: 10.1080/21642583.2022.2045645] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/16/2021] [Accepted: 02/19/2022] [Indexed: 05/31/2023]
Abstract
With a global COVID-19 pandemic, the number of confirmed patients increases rapidly, leaving the world with very few medical resources. Therefore, the fast diagnosis and monitoring of COVID-19 are one of the world's most critical challenges today. Artificial intelligence-based CT image classification models can quickly and accurately distinguish infected patients from healthy populations. Our research proposes a deep learning model (WE-SAJ) using wavelet entropy for feature extraction, two-layer FNNs for classification and the adaptive Jaya algorithm as a training algorithm. It achieves superior performance compared to the Jaya-based model. The model has a sensitivity of 85.47±1.84, specificity of 87.23±1.67 precision of 87.03±1.34, an accuracy of 86.35±0.70, and F1 score of 86.23±0.77, Matthews correlation coefficient of 72.75±1.38, and Fowlkes-Mallows Index of 86.24±0.76. Our experiments demonstrate the potential of artificial intelligence techniques for COVID-19 diagnosis and the effectiveness of the Self-adaptive Jaya algorithm compared to the Jaya algorithm for medical image classification tasks.
Collapse
Affiliation(s)
- Wei Wang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, LE1 7RH, UK
| | - Xin Zhang
- Department of Medical Imaging, The Fourth People's Hospital of Huai'an, Huai'an, Jiangsu Province, 223002, China
| | - Shui-Hua Wang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, LE1 7RH, UK
| | - Yu-Dong Zhang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, LE1 7RH, UK
| |
Collapse
|
23
|
Cui R, Yu H, Xu T, Xing X, Cao X, Yan K, Chen J. Deep Learning in Medical Hyperspectral Images: A Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22249790. [PMID: 36560157 PMCID: PMC9784550 DOI: 10.3390/s22249790] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 12/11/2022] [Accepted: 12/12/2022] [Indexed: 06/13/2023]
Abstract
With the continuous progress of development, deep learning has made good progress in the analysis and recognition of images, which has also triggered some researchers to explore the area of combining deep learning with hyperspectral medical images and achieve some progress. This paper introduces the principles and techniques of hyperspectral imaging systems, summarizes the common medical hyperspectral imaging systems, and summarizes the progress of some emerging spectral imaging systems through analyzing the literature. In particular, this article introduces the more frequently used medical hyperspectral images and the pre-processing techniques of the spectra, and in other sections, it discusses the main developments of medical hyperspectral combined with deep learning for disease diagnosis. On the basis of the previous review, tne limited factors in the study on the application of deep learning to hyperspectral medical images are outlined, promising research directions are summarized, and the future research prospects are provided for subsequent scholars.
Collapse
Affiliation(s)
- Rong Cui
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - He Yu
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
- Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun 130022, China
| | - Tingfa Xu
- Image Engineering & Video Technology Lab, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Institute of Technology Chongqing Innovation Center, Chongqing 401120, China
| | - Xiaoxue Xing
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
- Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun 130022, China
| | - Xiaorui Cao
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - Kang Yan
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - Jiexi Chen
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| |
Collapse
|
24
|
Imaging perfusion changes in oncological clinical applications by hyperspectral imaging: a literature review. Radiol Oncol 2022; 56:420-429. [PMID: 36503709 PMCID: PMC9784371 DOI: 10.2478/raon-2022-0051] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Accepted: 11/02/2022] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Hyperspectral imaging (HSI) is a promising imaging modality that uses visible light to obtain information about blood flow. It has the distinct advantage of being noncontact, nonionizing, and noninvasive without the need for a contrast agent. Among the many applications of HSI in the medical field are the detection of various types of tumors and the evaluation of their blood flow, as well as the healing processes of grafts and wounds. Since tumor perfusion is one of the critical factors in oncology, we assessed the value of HSI in quantifying perfusion changes during interventions in clinical oncology through a systematic review of the literature. MATERIALS AND METHODS The PubMed and Web of Science electronic databases were searched using the terms "hyperspectral imaging perfusion cancer" and "hyperspectral imaging resection cancer". The inclusion criterion was the use of HSI in clinical oncology, meaning that all animal, phantom, ex vivo, experimental, research and development, and purely methodological studies were excluded. RESULTS Twenty articles met the inclusion criteria. The anatomic locations of the neoplasms in the selected articles were as follows: kidneys (1 article), breasts (2 articles), eye (1 article), brain (4 articles), entire gastrointestinal (GI) tract (1 article), upper GI tract (5 articles), and lower GI tract (6 articles). CONCLUSIONS HSI is a potentially attractive imaging modality for clinical application in oncology, with assessment of mastectomy skin flap perfusion after reconstructive breast surgery and anastomotic perfusion during reconstruction of gastrointenstinal conduit as the most promising at present.
Collapse
|
25
|
Tharwat M, Sakr NA, El-Sappagh S, Soliman H, Kwak KS, Elmogy M. Colon Cancer Diagnosis Based on Machine Learning and Deep Learning: Modalities and Analysis Techniques. SENSORS (BASEL, SWITZERLAND) 2022; 22:9250. [PMID: 36501951 PMCID: PMC9739266 DOI: 10.3390/s22239250] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Accepted: 11/24/2022] [Indexed: 06/17/2023]
Abstract
The treatment and diagnosis of colon cancer are considered to be social and economic challenges due to the high mortality rates. Every year, around the world, almost half a million people contract cancer, including colon cancer. Determining the grade of colon cancer mainly depends on analyzing the gland's structure by tissue region, which has led to the existence of various tests for screening that can be utilized to investigate polyp images and colorectal cancer. This article presents a comprehensive survey on the diagnosis of colon cancer. This covers many aspects related to colon cancer, such as its symptoms and grades as well as the available imaging modalities (particularly, histopathology images used for analysis) in addition to common diagnosis systems. Furthermore, the most widely used datasets and performance evaluation metrics are discussed. We provide a comprehensive review of the current studies on colon cancer, classified into deep-learning (DL) and machine-learning (ML) techniques, and we identify their main strengths and limitations. These techniques provide extensive support for identifying the early stages of cancer that lead to early treatment of the disease and produce a lower mortality rate compared with the rate produced after symptoms develop. In addition, these methods can help to prevent colorectal cancer from progressing through the removal of pre-malignant polyps, which can be achieved using screening tests to make the disease easier to diagnose. Finally, the existing challenges and future research directions that open the way for future work in this field are presented.
Collapse
Affiliation(s)
- Mai Tharwat
- Information Technology Department, Faculty of Computers and Information, Mansoura University, Mansoura 35516, Egypt
| | - Nehal A. Sakr
- Information Technology Department, Faculty of Computers and Information, Mansoura University, Mansoura 35516, Egypt
| | - Shaker El-Sappagh
- Information Systems Department, Faculty of Computers and Artificial Intelligence, Benha University, Benha 13512, Egypt
- Faculty of Computer Science and Engineering, Galala University, Suez 435611, Egypt
| | - Hassan Soliman
- Information Technology Department, Faculty of Computers and Information, Mansoura University, Mansoura 35516, Egypt
| | - Kyung-Sup Kwak
- Department of Information and Communication Engineering, Inha University, Incheon 22212, Republic of Korea
| | - Mohammed Elmogy
- Information Technology Department, Faculty of Computers and Information, Mansoura University, Mansoura 35516, Egypt
| |
Collapse
|
26
|
Premachandran S, Haldavnekar R, Das S, Venkatakrishnan K, Tan B. DEEP Surveillance of Brain Cancer Using Self-Functionalized 3D Nanoprobes for Noninvasive Liquid Biopsy. ACS NANO 2022; 16:17948-17964. [PMID: 36112671 DOI: 10.1021/acsnano.2c04187] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Brain cancers, one of the most fatal malignancies, require accurate diagnosis for guided therapeutic intervention. However, conventional methods for brain cancer prognosis (imaging and tissue biopsy) face challenges due to the complex nature and inaccessible anatomy of the brain. Therefore, deep analysis of brain cancer is necessary to (i) detect the presence of a malignant tumor, (ii) identify primary or secondary origin, and (iii) find where the tumor is housed. In order to provide a diagnostic technique with such exhaustive information here, we attempted a liquid biopsy-based deep surveillance of brain cancer using a very minimal amount of blood serum (5 μL) in real time. We hypothesize that holistic analysis of serum can act as a reliable source for deep brain cancer surveillance. To identify minute amounts of tumor-derived material in circulation, we synthesized an ultrasensitive 3D nanosensor, adopted SERS as a diagnostic methodology, and undertook a DEEP neural network-based brain cancer surveillance. Detection of primary and secondary tumor achieved 100% accuracy. Prediction of intracranial tumor location achieved 96% accuracy. This modality of using patient sera for deep surveillance is a promising noninvasive liquid biopsy tool with the potential to complement current brain cancer diagnostic methodologies.
Collapse
Affiliation(s)
- Srilakshmi Premachandran
- Institute for Biomedical Engineering, Science and Technology (I BEST), Partnership between Toronto Metropolitan University (formerly Ryerson University) and St. Michael's Hospital, Toronto, Ontario M5B 1W8, Canada
- Ultrashort Laser Nanomanufacturing Research Facility, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
- Nano Characterization Laboratory, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
- Nano-Bio Interface facility, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
| | - Rupa Haldavnekar
- Institute for Biomedical Engineering, Science and Technology (I BEST), Partnership between Toronto Metropolitan University (formerly Ryerson University) and St. Michael's Hospital, Toronto, Ontario M5B 1W8, Canada
- Ultrashort Laser Nanomanufacturing Research Facility, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
- Nano Characterization Laboratory, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
- Nano-Bio Interface facility, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
| | - Sunit Das
- Scientist, St. Michael's Hospital, Toronto, Ontario M5B 1W8, Canada
- Institute of Medical Sciences, Neurosurgery, University of Toronto, Toronto, Ontario M5S 1A8, Canada
| | - Krishnan Venkatakrishnan
- Keenan Research Center for Biomedical Science, Unity Health Toronto, Toronto, Ontario M5B 1W8, Canada
- Institute for Biomedical Engineering, Science and Technology (I BEST), Partnership between Toronto Metropolitan University (formerly Ryerson University) and St. Michael's Hospital, Toronto, Ontario M5B 1W8, Canada
- Ultrashort Laser Nanomanufacturing Research Facility, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
- Nano-Bio Interface facility, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
| | - Bo Tan
- Keenan Research Center for Biomedical Science, Unity Health Toronto, Toronto, Ontario M5B 1W8, Canada
- Institute for Biomedical Engineering, Science and Technology (I BEST), Partnership between Toronto Metropolitan University (formerly Ryerson University) and St. Michael's Hospital, Toronto, Ontario M5B 1W8, Canada
- Nano Characterization Laboratory, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
- Nano-Bio Interface facility, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, Ontario M5B 2K3, Canada
| |
Collapse
|
27
|
Martinez-Vega B, Tkachenko M, Matkabi M, Ortega S, Fabelo H, Balea-Fernandez F, La Salvia M, Torti E, Leporati F, Callico GM, Chalopin C. Evaluation of Preprocessing Methods on Independent Medical Hyperspectral Databases to Improve Analysis. SENSORS (BASEL, SWITZERLAND) 2022; 22:8917. [PMID: 36433516 PMCID: PMC9693077 DOI: 10.3390/s22228917] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 11/10/2022] [Accepted: 11/15/2022] [Indexed: 06/16/2023]
Abstract
Currently, one of the most common causes of death worldwide is cancer. The development of innovative methods to support the early and accurate detection of cancers is required to increase the recovery rate of patients. Several studies have shown that medical Hyperspectral Imaging (HSI) combined with artificial intelligence algorithms is a powerful tool for cancer detection. Various preprocessing methods are commonly applied to hyperspectral data to improve the performance of the algorithms. However, there is currently no standard for these methods, and no studies have compared them so far in the medical field. In this work, we evaluated different combinations of preprocessing steps, including spatial and spectral smoothing, Min-Max scaling, Standard Normal Variate normalization, and a median spatial smoothing technique, with the goal of improving tumor detection in three different HSI databases concerning colorectal, esophagogastric, and brain cancers. Two machine learning and deep learning models were used to perform the pixel-wise classification. The results showed that the choice of preprocessing method affects the performance of tumor identification. The method that showed slightly better results with respect to identifing colorectal tumors was Median Filter preprocessing (0.94 of area under the curve). On the other hand, esophagogastric and brain tumors were more accurately identified using Min-Max scaling preprocessing (0.93 and 0.92 of area under the curve, respectively). However, it is observed that the Median Filter method smooths sharp spectral features, resulting in high variability in the classification performance. Therefore, based on these results, obtained with different databases acquired by different HSI instrumentation, the most relevant preprocessing technique identified in this work is Min-Max scaling.
Collapse
Affiliation(s)
- Beatriz Martinez-Vega
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
| | - Mariia Tkachenko
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany
- Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI), University of Leipzig, 04105 Leipzig, Germany
| | - Marianne Matkabi
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany
- Department of Electrical Engineering, Mechanical Engineering and Industrial Engineering, Anhalt University of Applied Science Anhalt, 06366 Köthen, Germany
| | - Samuel Ortega
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
- Nofima, Norwegian Institute of Food Fisheries and Aquaculture Research, NO-9291 Tromsø, Norway
| | - Himar Fabelo
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
- Fundacion Canaria Instituto de Investigación Sanitaria de Canarias (FIISC), 35019 Las Palmas de Gran Canaria, Spain
| | - Francisco Balea-Fernandez
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
- Department of Psychology, Sociology and Social Work, University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
| | - Marco La Salvia
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, I-27100 Pavia, Italy
| | - Emanuele Torti
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, I-27100 Pavia, Italy
| | - Francesco Leporati
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, I-27100 Pavia, Italy
| | - Gustavo M. Callico
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
| | - Claire Chalopin
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany
| |
Collapse
|
28
|
Zhang L, Huang D, Chen X, Zhu L, Chen X, Xie Z, Huang G, Gao J, Shi W, Cui G. Visible near-infrared hyperspectral imaging and supervised classification for the detection of small intestinal necrosis tissue in vivo. BIOMEDICAL OPTICS EXPRESS 2022; 13:6061-6080. [PMID: 36733734 PMCID: PMC9872898 DOI: 10.1364/boe.470202] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 09/23/2022] [Accepted: 09/26/2022] [Indexed: 06/18/2023]
Abstract
Complete recognition of necrotic areas during small bowel tissue resection remains challenging due to the lack of optimal intraoperative aid identification techniques. This research utilizes hyperspectral imaging techniques to automatically distinguish normal and necrotic areas of small intestinal tissue. Sample data were obtained from the animal model of small intestinal tissue of eight Japanese large-eared white rabbits developed by experienced physicians. A spectral library of normal and necrotic regions of small intestinal tissue was created and processed using six different supervised classification algorithms. The results show that hyperspectral imaging combined with supervised classification algorithms can be a suitable technique to automatically distinguish between normal and necrotic areas of small intestinal tissue. This new technique could aid physicians in objectively identify normal and necrotic areas of small intestinal tissue.
Collapse
Affiliation(s)
- LeChao Zhang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - DanFei Huang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - XiaoJing Chen
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - LiBin Zhu
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - XiaoQing Chen
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - ZhongHao Xie
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuangZao Huang
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - JunZhao Gao
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - Wen Shi
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuiHua Cui
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| |
Collapse
|
29
|
Barberio M, Lapergola A, Benedicenti S, Mita M, Barbieri V, Rubichi F, Altamura A, Giaracuni G, Tamburini E, Diana M, Pizzicannella M, Viola MG. Intraoperative bowel perfusion quantification with hyperspectral imaging: a guidance tool for precision colorectal surgery. Surg Endosc 2022; 36:8520-8532. [PMID: 35836033 DOI: 10.1007/s00464-022-09407-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Accepted: 06/19/2022] [Indexed: 02/07/2023]
Abstract
BACKGROUND Poor anastomotic perfusion can cause anastomotic leaks (AL). Hyperspectral imaging (HSI), previously validated experimentally, provides accurate, real-time, contrast-free intestinal perfusion quantification. Clinical experience with HSI is limited. In this study, HSI was used to evaluate bowel perfusion intraoperatively. METHODS Fifty-two patients undergoing elective colorectal surgeries for neoplasia (n = 40) or diverticular disease (n = 12), were enrolled. Intestinal perfusion was assessed with HSI (TIVITA®, Diaspective Vision, Am Salzhaff, Germany). This device generates a perfusion heat map reflecting the tissue oxygen saturation (StO2) amount. Prior to anastomose creation, the clinical transection line (CTL) was highlighted on the proximal bowel and imaged with HSI. Upon StO2 heat map evaluation, the hyperspectral transection line (HTL) was identified. In case of CTL/HTL discrepancy > 5 mm, the bowel was always resected at the HTL. HSI outcomes were compared to the clinical ones. RESULTS AL occurred in one patient who underwent neoadjuvant radiochemotherapy and ultralow anterior resection for rectal cancer. HSI assessment was feasible in all patients, and StO2-values were significantly higher at proximal segments than distal ones. Twenty-six patients showed CTL/HTL discrepancy, and these patients had a lower mean StO2 (54.55 ± 21.30%) than patients without discrepancy (65.10 ± 21.30%, p = 0.000). Patients undergoing neoadjuvant radiochemotherapy showed a lower StO2 (51.41 ± 23.41%) than non-neoadjuvated patients (60.51 ± 24.98%, p = 0.010). CONCLUSION HSI is useful in detecting intraoperatively marginally perfused segments, for which the clinical appreciation is unreliable. Intestinal vascular supply is lower in patients undergoing neoadjuvant radiochemotherapy, and this novel finding together with the clinical impact of HSI perfusion quantification deserves further investigation in larger trials.
Collapse
Affiliation(s)
- Manuel Barberio
- Department of Surgery, Ospedale Card. G. Panico, Tricase, Italy.
- Department of Research, Research Institute Against Digestive Cancer (IRCAD), 1, place de l'Hôpital, 67091, Strasbourg, France.
| | - Alfonso Lapergola
- Department of Visceral and Digestive, Nouvel Hôpital Civil (NHC), Strasbourg, France
| | | | | | | | | | - Amedeo Altamura
- Department of Surgery, Ospedale Card. G. Panico, Tricase, Italy
| | | | | | - Michele Diana
- Department of Research, Research Institute Against Digestive Cancer (IRCAD), 1, place de l'Hôpital, 67091, Strasbourg, France
| | | | | |
Collapse
|
30
|
Collins T, Bencteux V, Benedicenti S, Moretti V, Mita MT, Barbieri V, Rubichi F, Altamura A, Giaracuni G, Marescaux J, Hostettler A, Diana M, Viola MG, Barberio M. Automatic optical biopsy for colorectal cancer using hyperspectral imaging and artificial neural networks. Surg Endosc 2022; 36:8549-8559. [PMID: 36008640 DOI: 10.1007/s00464-022-09524-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Accepted: 07/31/2022] [Indexed: 02/07/2023]
Abstract
BACKGROUND Intraoperative identification of cancerous tissue is fundamental during oncological surgical or endoscopic procedures. This relies on visual assessment supported by histopathological evaluation, implying a longer operative time. Hyperspectral imaging (HSI), a contrast-free and contactless imaging technology, provides spatially resolved spectroscopic analysis, with the potential to differentiate tissue at a cellular level. However, HSI produces "big data", which is impossible to directly interpret by clinicians. We hypothesize that advanced machine learning algorithms (convolutional neural networks-CNNs) can accurately detect colorectal cancer in HSI data. METHODS In 34 patients undergoing colorectal resections for cancer, immediately after extraction, the specimen was opened, the tumor-bearing section was exposed and imaged using HSI. Cancer and normal mucosa were categorized from histopathology. A state-of-the-art CNN was developed to automatically detect regions of colorectal cancer in a hyperspectral image. Accuracy was validated with three levels of cross-validation (twofold, fivefold, and 15-fold). RESULTS 32 patients had colorectal adenocarcinomas confirmed by histopathology (9 left, 11 right, 4 transverse colon, and 9 rectum). 6 patients had a local initial stage (T1-2) and 26 had a local advanced stage (T3-4). The cancer detection performance of the CNN using 15-fold cross-validation showed high sensitivity and specificity (87% and 90%, respectively) and a ROC-AUC score of 0.95 (considered outstanding). In the T1-2 group, the sensitivity and specificity were 89% and 90%, respectively, and in the T3-4 group, the sensitivity and specificity were 81% and 93%, respectively. CONCLUSIONS Automatic colorectal cancer detection on fresh specimens using HSI, using a properly trained CNN is feasible and accurate, even with small datasets, regardless of the local tumor extension. In the near future, this approach may become a useful intraoperative tool during oncological endoscopic and surgical procedures, and may result in precise and non-destructive optical biopsies to support objective and consistent tumor-free resection margins.
Collapse
Affiliation(s)
- Toby Collins
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France.
- Research Institute Against Digestive Cancer (IRCAD Africa), Kigali, Rwanda.
| | - Valentin Bencteux
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- ICUBE Laboratory, Photonics Instrumentation for Health, Strasbourg, France
| | | | | | | | | | | | - Amedeo Altamura
- Department of Surgery, Ospedale Card. G. Panico, Tricase, Italy
| | | | - Jacques Marescaux
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- Research Institute Against Digestive Cancer (IRCAD Africa), Kigali, Rwanda
| | - Alex Hostettler
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- Research Institute Against Digestive Cancer (IRCAD Africa), Kigali, Rwanda
| | - Michele Diana
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- ICUBE Laboratory, Photonics Instrumentation for Health, Strasbourg, France
| | | | - Manuel Barberio
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- Department of Surgery, Ospedale Card. G. Panico, Tricase, Italy
| |
Collapse
|
31
|
Okamoto N, Rodríguez-Luna MR, Bencteux V, Al-Taher M, Cinelli L, Felli E, Urade T, Nkusi R, Mutter D, Marescaux J, Hostettler A, Collins T, Diana M. Computer-Assisted Differentiation between Colon-Mesocolon and Retroperitoneum Using Hyperspectral Imaging (HSI) Technology. Diagnostics (Basel) 2022; 12:diagnostics12092225. [PMID: 36140626 PMCID: PMC9497769 DOI: 10.3390/diagnostics12092225] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 09/10/2022] [Accepted: 09/12/2022] [Indexed: 12/01/2022] Open
Abstract
Complete mesocolic excision (CME), which involves the adequate resection of the tumor-bearing colonic segment with “en bloc” removal of its mesocolon along embryological fascial planes is associated with superior oncological outcomes. However, CME presents a higher complication rate compared to non-CME resections due to a higher risk of vascular injury. Hyperspectral imaging (HSI) is a contrast-free optical imaging technology, which facilitates the quantitative imaging of physiological tissue parameters and the visualization of anatomical structures. This study evaluates the accuracy of HSI combined with deep learning (DL) to differentiate the colon and its mesenteric tissue from retroperitoneal tissue. In an animal study including 20 pig models, intraoperative hyperspectral images of the sigmoid colon, sigmoid mesentery, and retroperitoneum were recorded. A convolutional neural network (CNN) was trained to distinguish the two tissue classes using HSI data, validated with a leave-one-out cross-validation process. The overall recognition sensitivity of the tissues to be preserved (retroperitoneum) and the tissues to be resected (colon and mesentery) was 79.0 ± 21.0% and 86.0 ± 16.0%, respectively. Automatic classification based on HSI and CNNs is a promising tool to automatically, non-invasively, and objectively differentiate the colon and its mesentery from retroperitoneal tissue.
Collapse
Affiliation(s)
- Nariaki Okamoto
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- ICube Laboratory, Photonics Instrumentation for Health, 67081 Strasbourg, France
- Correspondence:
| | - María Rita Rodríguez-Luna
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- ICube Laboratory, Photonics Instrumentation for Health, 67081 Strasbourg, France
| | - Valentin Bencteux
- ICube Laboratory, Photonics Instrumentation for Health, 67081 Strasbourg, France
| | - Mahdi Al-Taher
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Department of Surgery, Maastricht University Medical Center, 6229 ER Maastricht, The Netherlands
| | - Lorenzo Cinelli
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Department of Gastrointestinal Surgery, San Raffaele Hospital IRCCS, 20132 Milan, Italy
| | - Eric Felli
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
| | - Takeshi Urade
- Department of Surgery, Division of Hepato-Biliary-Pancreatic Surgery, Kobe University Graduate School of Medicine, Kobe 6500017, Japan
| | - Richard Nkusi
- Research Institute against Digestive Cancer (IRCAD), Kigali, Rwanda
| | - Didier Mutter
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Department of Digestive and Endocrine Surgery, Nouvel Hôpital Civil, University of Strasbourg, 67091 Strasbourg, France
- IHU-Strasbourg—Institut de Chirurgie Guidée par L’image, 67091 Strasbourg, France
| | - Jacques Marescaux
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
| | - Alexandre Hostettler
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Research Institute against Digestive Cancer (IRCAD), Kigali, Rwanda
| | - Toby Collins
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Research Institute against Digestive Cancer (IRCAD), Kigali, Rwanda
| | - Michele Diana
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- ICube Laboratory, Photonics Instrumentation for Health, 67081 Strasbourg, France
| |
Collapse
|
32
|
Chalopin C, Nickel F, Pfahl A, Köhler H, Maktabi M, Thieme R, Sucher R, Jansen-Winkeln B, Studier-Fischer A, Seidlitz S, Maier-Hein L, Neumuth T, Melzer A, Müller-Stich BP, Gockel I. [Artificial intelligence and hyperspectral imaging for image-guided assistance in minimally invasive surgery]. CHIRURGIE (HEIDELBERG, GERMANY) 2022; 93:940-947. [PMID: 35798904 DOI: 10.1007/s00104-022-01677-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 06/08/2022] [Indexed: 01/18/2023]
Abstract
BACKGROUND Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article. OBJECTIVE What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery? MATERIAL AND METHODS Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors. RESULTS In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91. DISCUSSION This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland.
| | - Felix Nickel
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - René Thieme
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Robert Sucher
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Boris Jansen-Winkeln
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
- Abteilung für Allgemein‑, Viszeral- und Onkologische Chirurgie, Klinikum St. Georg Leipzig, Leipzig, Deutschland
| | - Alexander Studier-Fischer
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Beat Peter Müller-Stich
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Ines Gockel
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| |
Collapse
|
33
|
Raita-Hakola AM, Annala L, Lindholm V, Trops R, Näsilä A, Saari H, Ranki A, Pölönen I. FPI Based Hyperspectral Imager for the Complex Surfaces—Calibration, Illumination and Applications. SENSORS 2022; 22:s22093420. [PMID: 35591109 PMCID: PMC9103796 DOI: 10.3390/s22093420] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 04/13/2022] [Accepted: 04/23/2022] [Indexed: 01/27/2023]
Abstract
Hyperspectral imaging (HSI) applications for biomedical imaging and dermatological applications have been recently under research interest. Medical HSI applications are non-invasive methods with high spatial and spectral resolution. HS imaging can be used to delineate malignant tumours, detect invasions, and classify lesion types. Typical challenges of these applications relate to complex skin surfaces, leaving some skin areas unreachable. In this study, we introduce a novel spectral imaging concept and conduct a clinical pre-test, the findings of which can be used to develop the concept towards a clinical application. The SICSURFIS spectral imager concept combines a piezo-actuated Fabry–Pérot interferometer (FPI) based hyperspectral imager, a specially designed LED module and several sizes of stray light protection cones for reaching and adapting to the complex skin surfaces. The imager is designed for the needs of photometric stereo imaging for providing the skin surface models (3D) for each captured wavelength. The captured HS images contained 33 selected wavelengths (ranging from 477 nm to 891 nm), which were captured simultaneously with accordingly selected LEDs and three specific angles of light. The pre-test results show that the data collected with the new SICSURFIS imager enable the use of the spectral and spatial domains with surface model information. The imager can reach complex skin surfaces. Healthy skin, basal cell carcinomas and intradermal nevi lesions were classified and delineated pixel-wise with promising results, but further studies are needed. The results were obtained with a convolutional neural network.
Collapse
Affiliation(s)
- Anna-Maria Raita-Hakola
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
- Correspondence:
| | - Leevi Annala
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
| | - Vivian Lindholm
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (V.L.); (A.R.)
| | - Roberts Trops
- VTT Technical Research Centre of Finland Ltd., 02150 Espoo, Finland; (R.T.); (A.N.); (H.S.)
| | - Antti Näsilä
- VTT Technical Research Centre of Finland Ltd., 02150 Espoo, Finland; (R.T.); (A.N.); (H.S.)
| | - Heikki Saari
- VTT Technical Research Centre of Finland Ltd., 02150 Espoo, Finland; (R.T.); (A.N.); (H.S.)
| | - Annamari Ranki
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (V.L.); (A.R.)
| | - Ilkka Pölönen
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
| |
Collapse
|
34
|
Ghosh NK, Kumar A. Colorectal cancer: Artificial intelligence and its role in surgical decision making. Artif Intell Gastroenterol 2022; 3:36-45. [DOI: 10.35712/aig.v3.i2.36] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/30/2021] [Revised: 02/02/2022] [Accepted: 04/26/2022] [Indexed: 02/06/2023] Open
Abstract
Despite several advances in the oncological management of colorectal cancer (CRC), there still remains a lacuna in the treatment strategy, which differs from center to center and on the philosophy of the treating clinician that is not without bias. Personalized treatment is essential for the treatment of CRC to achieve better long-term outcomes and to reduce morbidity. Surgery has an important role to play in the treatment. Surgical treatment of CRC is decided based on clinical parameters and investigations and hence likely to have judgmental errors. Artificial intelligence has been reported to be useful in the surveillance, diagnosis, treatment, and follow-up with accuracy in several malignancies. However, it is still evolving and yet to be established in surgical decision making in CRC. It is not only useful preoperatively but also intraoperatively. Artificial intelligence helps to rectify the human surgical decision when clinical data and radiological and laboratory parameters are fed into the computer and may guide correct surgical treatment.
Collapse
Affiliation(s)
- Nalini Kanta Ghosh
- Department of Surgical Gastroenterology, Sanjay Gandhi Post Graduate Institute of Medical Sciences, Lucknow 226014, UP, India
| | - Ashok Kumar
- Department of Surgical Gastroenterology, Sanjay Gandhi Post Graduate Institute of Medical Sciences, Lucknow 226014, UP, India
| |
Collapse
|
35
|
Zenteno O, Trinh DH, Treuillet S, Lucas Y, Bazin T, Lamarque D, Daul C. Optical biopsy mapping on endoscopic image mosaics with a marker-free probe. Comput Biol Med 2022; 143:105234. [PMID: 35093845 DOI: 10.1016/j.compbiomed.2022.105234] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2021] [Revised: 12/25/2021] [Accepted: 01/10/2022] [Indexed: 12/24/2022]
Abstract
Gastric cancer is the second leading cause of cancer-related deaths worldwide. Early diagnosis significantly increases the chances of survival; therefore, improved assisted exploration and screening techniques are necessary. Previously, we made use of an augmented multi-spectral endoscope by inserting an optical probe into the instrumentation channel. However, the limited field of view and the lack of markings left by optical biopsies on the tissue complicate the navigation and revisit of the suspect areas probed in-vivo. In this contribution two innovative tools are introduced to significantly increase the traceability and monitoring of patients in clinical practice: (i) video mosaicing to build a more comprehensive and panoramic view of large gastric areas; (ii) optical biopsy targeting and registration with the endoscopic images. The proposed optical flow-based mosaicing technique selects images that minimize texture discontinuities and is robust despite the lack of texture and illumination variations. The optical biopsy targeting is based on automatic tracking of a free-marker probe in the endoscopic view using deep learning to dynamically estimate its pose during exploration. The accuracy of pose estimation is sufficient to ensure a precise overlapping of the standard white-light color image and the hyperspectral probe image, assuming that the small target area of the organ is almost flat. This allows the mapping of all spatio-temporally tracked biopsy sites onto the panoramic mosaic. Experimental validations are carried out from videos acquired on patients in hospital. The proposed technique is purely software-based and therefore easily integrable into clinical practice. It is also generic and compatible to any imaging modality that connects to a fiberscope.
Collapse
Affiliation(s)
- Omar Zenteno
- Laboratoire PRISME, Université d'Orléans, Orléans, France
| | - Dinh-Hoan Trinh
- CRAN, UMR 7039 CNRS and Université de Lorraine, Vandœuvre-lès-Nancy, France
| | | | - Yves Lucas
- Laboratoire PRISME, Université d'Orléans, Orléans, France
| | - Thomas Bazin
- Service d'Hépato-gastroentérologie et oncologie digestive, Hôpital Ambroise Paré, Boulogne-Billancourt, France
| | - Dominique Lamarque
- Service d'Hépato-gastroentérologie et oncologie digestive, Hôpital Ambroise Paré, Boulogne-Billancourt, France
| | - Christian Daul
- CRAN, UMR 7039 CNRS and Université de Lorraine, Vandœuvre-lès-Nancy, France.
| |
Collapse
|
36
|
Intraoperative Anwendung künstlicher Intelligenz und neuer hyperspektraler Bildgebungsverfahren in der kolorektalen Chirurgie. COLOPROCTOLOGY 2022. [DOI: 10.1007/s00053-022-00592-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
37
|
Barberio M, Benedicenti S, Pizzicannella M, Felli E, Collins T, Jansen-Winkeln B, Marescaux J, Viola MG, Diana M. Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons. Diagnostics (Basel) 2021; 11:diagnostics11112066. [PMID: 34829413 PMCID: PMC8624094 DOI: 10.3390/diagnostics11112066] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2021] [Revised: 11/03/2021] [Accepted: 11/05/2021] [Indexed: 12/12/2022] Open
Abstract
Hyperspectral imaging (HSI) is a novel optical imaging modality, which has recently found diverse applications in the medical field. HSI is a hybrid imaging modality, combining a digital photographic camera with a spectrographic unit, and it allows for a contactless and non-destructive biochemical analysis of living tissue. HSI provides quantitative and qualitative information of the tissue composition at molecular level in a contrast-free manner, hence making it possible to objectively discriminate between different tissue types and between healthy and pathological tissue. Over the last two decades, HSI has been increasingly used in the medical field, and only recently it has found an application in the operating room. In the last few years, several research groups have used this imaging modality as an intraoperative guidance tool within different surgical disciplines. Despite its great potential, HSI still remains far from being routinely used in the daily surgical practice, since it is still largely unknown to most of the surgical community. The aim of this study is to provide clinical surgeons with an overview of the capabilities, current limitations, and future directions of HSI for intraoperative guidance.
Collapse
Affiliation(s)
- Manuel Barberio
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (T.C.); (J.M.); (M.D.)
- General Surgery Department, Ospedale Card. G. Panico, 73039 Tricase, Italy; (S.B.); (M.P.); (M.G.V.)
- Correspondence:
| | - Sara Benedicenti
- General Surgery Department, Ospedale Card. G. Panico, 73039 Tricase, Italy; (S.B.); (M.P.); (M.G.V.)
| | - Margherita Pizzicannella
- General Surgery Department, Ospedale Card. G. Panico, 73039 Tricase, Italy; (S.B.); (M.P.); (M.G.V.)
| | - Eric Felli
- Department of Visceral Surgery and Medicine, Inselspital, Bern University Hospital, University of Bern, 3008 Bern, Switzerland;
- Department for BioMedical Research, Visceral Surgery and Medicine, University of Bern, 3008 Bern, Switzerland
| | - Toby Collins
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (T.C.); (J.M.); (M.D.)
| | | | - Jacques Marescaux
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (T.C.); (J.M.); (M.D.)
| | - Massimo Giuseppe Viola
- General Surgery Department, Ospedale Card. G. Panico, 73039 Tricase, Italy; (S.B.); (M.P.); (M.G.V.)
| | - Michele Diana
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (T.C.); (J.M.); (M.D.)
- ICube Laboratory, Photonics Instrumentation for Health, University of Strasbourg, 67400 Strasbourg, France
| |
Collapse
|
38
|
Maktabi M, Tkachenko M, Kohler H, Schierle K, Gockel I, Jansen-Winkeln B, Chalopin C. Using physiological parameters measured by hyperspectral imaging to detect colorectal cancer. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3865-3868. [PMID: 34892077 DOI: 10.1109/embc46164.2021.9630160] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/19/2023]
Abstract
The accurate detection of malignant tissue during colorectal surgery impacts operation outcome. The non-invasive spectral imaging combined with machine learning (ML) methods showed to be promising for tumor identification. However, large spectral range implies large computing time. To reduce the number of features, ML methods (e.g. logistic regression and convolutional neuronal network CNN) were evaluated based on four physiological tissue parameters to automatically classify cancer and healthy mucosa in resected colon tissue. A ROC AUC of 0.81 was achieved with the CNN. This study shows that the use of only specific wavelengths bands can detect cancer.Clinical Relevance- These outcomes support the possibility to automatically classify colon tumor based on physiological parameters calculated using only specific wavelength bands. Hence, future image-guided colorectal surgeries can be performed with real-time multispectral imaging.
Collapse
|
39
|
Zhuang H, Zhang J, Liao F. A systematic review on application of deep learning in digestive system image processing. THE VISUAL COMPUTER 2021; 39:2207-2222. [PMID: 34744231 PMCID: PMC8557108 DOI: 10.1007/s00371-021-02322-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 09/30/2021] [Indexed: 05/07/2023]
Abstract
With the advent of the big data era, the application of artificial intelligence represented by deep learning in medicine has become a hot topic. In gastroenterology, deep learning has accomplished remarkable accomplishments in endoscopy, imageology, and pathology. Artificial intelligence has been applied to benign gastrointestinal tract lesions, early cancer, tumors, inflammatory bowel diseases, livers, pancreas, and other diseases. Computer-aided diagnosis significantly improve diagnostic accuracy and reduce physicians' workload and provide a shred of evidence for clinical diagnosis and treatment. In the near future, artificial intelligence will have high application value in the field of medicine. This paper mainly summarizes the latest research on artificial intelligence in diagnosing and treating digestive system diseases and discussing artificial intelligence's future in digestive system diseases. We sincerely hope that our work can become a stepping stone for gastroenterologists and computer experts in artificial intelligence research and facilitate the application and development of computer-aided image processing technology in gastroenterology.
Collapse
Affiliation(s)
- Huangming Zhuang
- Gastroenterology Department, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| | - Jixiang Zhang
- Gastroenterology Department, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| | - Fei Liao
- Gastroenterology Department, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| |
Collapse
|
40
|
[Epigastric pain in "gastric tumors" : The hummingbird among the differential diagnoses]. Chirurg 2021; 93:395-398. [PMID: 34665283 PMCID: PMC8960647 DOI: 10.1007/s00104-021-01522-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/03/2021] [Indexed: 10/27/2022]
|
41
|
Collins T, Maktabi M, Barberio M, Bencteux V, Jansen-Winkeln B, Chalopin C, Marescaux J, Hostettler A, Diana M, Gockel I. Automatic Recognition of Colon and Esophagogastric Cancer with Machine Learning and Hyperspectral Imaging. Diagnostics (Basel) 2021; 11:diagnostics11101810. [PMID: 34679508 PMCID: PMC8535008 DOI: 10.3390/diagnostics11101810] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Revised: 09/18/2021] [Accepted: 09/23/2021] [Indexed: 01/23/2023] Open
Abstract
There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models.
Collapse
Affiliation(s)
- Toby Collins
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
- Correspondence:
| | - Marianne Maktabi
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (M.M.); (C.C.)
| | - Manuel Barberio
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
- General Surgery Department, Card. G. Panico, 73039 Tricase, Italy
| | - Valentin Bencteux
- ICUBE Laboratory, Photonics Instrumentation for Health, University of Strasbourg, 67400 Strasbourg, France;
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (B.J.-W.); (I.G.)
| | - Claire Chalopin
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (M.M.); (C.C.)
| | - Jacques Marescaux
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
| | - Alexandre Hostettler
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
| | - Michele Diana
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
- ICUBE Laboratory, Photonics Instrumentation for Health, University of Strasbourg, 67400 Strasbourg, France;
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, 67091 Strasbourg, France
- INSERM, Institute of Viral and Liver Disease, 67091 Strasbourg, France
- Mitochondrion, Oxidative Stress and Muscle Protection (MSP)-EA 3072, Institute of Physiology, Faculty of Medicine, University of Strasbourg, 67085 Strasbourg, France
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (B.J.-W.); (I.G.)
| |
Collapse
|
42
|
Barberio M, Collins T, Bencteux V, Nkusi R, Felli E, Viola MG, Marescaux J, Hostettler A, Diana M. Deep Learning Analysis of In Vivo Hyperspectral Images for Automated Intraoperative Nerve Detection. Diagnostics (Basel) 2021; 11:1508. [PMID: 34441442 PMCID: PMC8391550 DOI: 10.3390/diagnostics11081508] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Revised: 07/27/2021] [Accepted: 08/09/2021] [Indexed: 12/16/2022] Open
Abstract
Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.
Collapse
Affiliation(s)
- Manuel Barberio
- Department of Research, Institute of Image-Guided Surgery, IHU-Strasbourg, 67091 Strasbourg, France; (V.B.); (E.F.)
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
- Department of Surgery, Ospedale Card. G. Panico, 73039 Tricase, Italy;
| | - Toby Collins
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
| | - Valentin Bencteux
- Department of Research, Institute of Image-Guided Surgery, IHU-Strasbourg, 67091 Strasbourg, France; (V.B.); (E.F.)
| | - Richard Nkusi
- Department of Research, Research Institute against Digestive Cancer, IRCAD Africa, Kigali 2 KN 30 ST, Rwanda;
| | - Eric Felli
- Department of Research, Institute of Image-Guided Surgery, IHU-Strasbourg, 67091 Strasbourg, France; (V.B.); (E.F.)
| | | | - Jacques Marescaux
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
| | - Alexandre Hostettler
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
| | - Michele Diana
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
- ICUBE Laboratory, Photonics Instrumentation for Health, 67412 Strasbourg, France
| |
Collapse
|
43
|
Tu T, Wei X, Yang Y, Zhang N, Li W, Tu X, Li W. Deep learning-based framework for the distinction of membranous nephropathy: a new approach through hyperspectral imagery. BMC Nephrol 2021; 22:231. [PMID: 34147076 PMCID: PMC8214276 DOI: 10.1186/s12882-021-02421-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2020] [Accepted: 06/03/2021] [Indexed: 11/10/2022] Open
Abstract
Background Common subtypes seen in Chinese patients with membranous nephropathy (MN) include idiopathic membranous nephropathy (IMN) and hepatitis B virus-related membranous nephropathy (HBV-MN). However, the morphologic differences are not visible under the light microscope in certain renal biopsy tissues. Methods We propose here a deep learning-based framework for processing hyperspectral images of renal biopsy tissue to define the difference between IMN and HBV-MN based on the component of their immune complex deposition. Results The proposed framework can achieve an overall accuracy of 95.04% in classification, which also leads to better performance than support vector machine (SVM)-based algorithms. Conclusion IMN and HBV-MN can be correctly separated via the deep learning framework using hyperspectral imagery. Our results suggest the potential of the deep learning algorithm as a new method to aid in the diagnosis of MN.
Collapse
Affiliation(s)
- Tianqi Tu
- Peking University China-Japan Friendship School of Clinical Medicine, Beijing, China.,Department of Nephrology, China-Japan Friendship Hospital, Beijing, China
| | - Xueling Wei
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
| | - Yue Yang
- Department of Nephrology, China-Japan Friendship Hospital, Beijing, China
| | - Nianrong Zhang
- Department of Nephrology, China-Japan Friendship Hospital, Beijing, China
| | - Wei Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China.
| | - Xiaowen Tu
- Department of Nephrology, PLA Rocket Force Characteristic Medical Center, Beijing, China.
| | - Wenge Li
- Department of Nephrology, China-Japan Friendship Hospital, Beijing, China.
| |
Collapse
|
44
|
Jansen-Winkeln B, Mehdorn M, Lange U, Köhler H, Chalopin C, Gockel I. Precision Surgery In Rectal Resection With Hyperspectral and Fluorescence Imaging And Pelvic Intraoperative Neuromonitoring (With Video). Surg Technol Int 2021; 38:154-158. [PMID: 34081769 DOI: 10.52198/21.sti.38.cr1383] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Oncologic visceral surgery has recently been revolutionized by robotics, artificial intelligence (AI), sparing of functionally important structures and innovative intraoperative imaging tools. These techniques enable new dimensions of precision surgery and oncology. Currently, data-driven, cognitive operating rooms are standing at the forefront of the latest technical and didactic developments in abdominal surgery. Rectal low anterior resection with total mesorectal excision (TME) for lower- and middle-third rectal cancer is a challenging operation due to the narrow pelvis and the tender guiding structures. Thus, new approaches have been needed to simplify the procedure and to upgrade the results. The combination of robotics with pelvic intraoperative neuromonitoring (pIONM) and new possibilities of visualization, such as multi- and hyperspectral imaging (MSI / HSI) or fluorescence imaging (FI) with indocyanine green (ICG) is a forward-looking modality to enhance surgical precision and reduce postoperative complications while improving oncologic and functional outcomes with a better quality of life. The aim of our video-paper is to show how to achieve maximum precision by combining robotic surgery with pelvic intraoperative neuromonitoring and new imaging devices for rectal cancer.
Collapse
Affiliation(s)
- Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| | - Mathias Mehdorn
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| | - Undine Lange
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), University of Leipzig, Leipzig, Germany
| | - Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), University of Leipzig, Leipzig, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| |
Collapse
|