1
|
Jian M, Chen H, Zhang Z, Yang N, Zhang H, Ma L, Xu W, Zhi H. A Lung Nodule Dataset with Histopathology-based Cancer Type Annotation. Sci Data 2024; 11:824. [PMID: 39068171 PMCID: PMC11283520 DOI: 10.1038/s41597-024-03658-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Accepted: 07/17/2024] [Indexed: 07/30/2024] Open
Abstract
Recently, Computer-Aided Diagnosis (CAD) systems have emerged as indispensable tools in clinical diagnostic workflows, significantly alleviating the burden on radiologists. Nevertheless, despite their integration into clinical settings, CAD systems encounter limitations. Specifically, while CAD systems can achieve high performance in the detection of lung nodules, they face challenges in accurately predicting multiple cancer types. This limitation can be attributed to the scarcity of publicly available datasets annotated with expert-level cancer type information. This research aims to bridge this gap by providing publicly accessible datasets and reliable tools for medical diagnosis, facilitating a finer categorization of different types of lung diseases so as to offer precise treatment recommendations. To achieve this objective, we curated a diverse dataset of lung Computed Tomography (CT) images, comprising 330 annotated nodules (nodules are labeled as bounding boxes) from 95 distinct patients. The quality of the dataset was evaluated using a variety of classical classification and detection models, and these promising results demonstrate that the dataset has a feasible application and further facilitate intelligent auxiliary diagnosis.
Collapse
Affiliation(s)
- Muwei Jian
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China.
- School of Information Science and Technology, Linyi University, Linyi, China.
| | - Hongyu Chen
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Zaiyong Zhang
- Thoracic Surgery Department of Linyi Central Hospital, Linyi, China
| | - Nan Yang
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Haorang Zhang
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China
| | - Lifu Ma
- Personnel Department of Linyi Central Hospital, Linyi, China
| | - Wenjing Xu
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Huixiang Zhi
- School of Information Science and Technology, Linyi University, Linyi, China
| |
Collapse
|
2
|
Kohnke J, Pattberg K, Nensa F, Kuhlmann H, Brenner T, Schmidt K, Hosch R, Espeter F. A proof of concept for microcirculation monitoring using machine learning based hyperspectral imaging in critically ill patients: a monocentric observational study. Crit Care 2024; 28:230. [PMID: 38987802 PMCID: PMC11238485 DOI: 10.1186/s13054-024-05023-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2024] [Accepted: 07/06/2024] [Indexed: 07/12/2024] Open
Abstract
BACKGROUND Impaired microcirculation is a cornerstone of sepsis development and leads to reduced tissue oxygenation, influenced by fluid and catecholamine administration during treatment. Hyperspectral imaging (HSI) is a non-invasive bedside technology for visualizing physicochemical tissue characteristics. Machine learning (ML) for skin HSI might offer an automated approach for bedside microcirculation assessment, providing an individualized tissue fingerprint of critically ill patients in intensive care. The study aimed to determine if machine learning could be utilized to automatically identify regions of interest (ROIs) in the hand, thereby distinguishing between healthy individuals and critically ill patients with sepsis using HSI. METHODS HSI raw data from 75 critically ill sepsis patients and from 30 healthy controls were recorded using TIVITA® Tissue System and analyzed using an automated ML approach. Additionally, patients were divided into two groups based on their SOFA scores for further subanalysis: less severely ill (SOFA ≤ 5) and severely ill (SOFA > 5). The analysis of the HSI raw data was fully-automated using MediaPipe for ROI detection (palm and fingertips) and feature extraction. HSI Features were statistically analyzed to highlight relevant wavelength combinations using Mann-Whitney-U test and Benjamini, Krieger, and Yekutieli (BKY) correction. In addition, Random Forest models were trained using bootstrapping, and feature importances were determined to gain insights regarding the wavelength importance for a model decision. RESULTS An automated pipeline for generating ROIs and HSI feature extraction was successfully established. HSI raw data analysis accurately distinguished healthy controls from sepsis patients. Wavelengths at the fingertips differed in the ranges of 575-695 nm and 840-1000 nm. For the palm, significant differences were observed in the range of 925-1000 nm. Feature importance plots indicated relevant information in the same wavelength ranges. Combining palm and fingertip analysis provided the highest reliability, with an AUC of 0.92 to distinguish between sepsis patients and healthy controls. CONCLUSION Based on this proof of concept, the integration of automated and standardized ROIs along with automated skin HSI analyzes, was able to differentiate between healthy individuals and patients with sepsis. This approach offers a reliable and objective assessment of skin microcirculation, facilitating the rapid identification of critically ill patients.
Collapse
Affiliation(s)
- Judith Kohnke
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
- Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen, Essen, Germany
| | - Kevin Pattberg
- Department of Anesthesiology and Intensive Care Medicine, University Hospital Essen, University Duisburg-Essen, Essen, Germany
| | - Felix Nensa
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
- Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen, Essen, Germany
| | - Henning Kuhlmann
- Department of Anesthesiology and Intensive Care Medicine, University Hospital Essen, University Duisburg-Essen, Essen, Germany
| | - Thorsten Brenner
- Department of Anesthesiology and Intensive Care Medicine, University Hospital Essen, University Duisburg-Essen, Essen, Germany
| | - Karsten Schmidt
- Department of Anesthesiology and Intensive Care Medicine, University Hospital Essen, University Duisburg-Essen, Essen, Germany
| | - René Hosch
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
- Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen, Essen, Germany
| | - Florian Espeter
- Department of Anesthesiology and Intensive Care Medicine, University Hospital Essen, University Duisburg-Essen, Essen, Germany.
| |
Collapse
|
3
|
Bannone E, Collins T, Esposito A, Cinelli L, De Pastena M, Pessaux P, Felli E, Andreotti E, Okamoto N, Barberio M, Felli E, Montorsi RM, Ingaglio N, Rodríguez-Luna MR, Nkusi R, Marescaux J, Hostettler A, Salvia R, Diana M. Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial. Surg Endosc 2024; 38:3758-3772. [PMID: 38789623 DOI: 10.1007/s00464-024-10880-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Accepted: 04/23/2024] [Indexed: 05/26/2024]
Abstract
BACKGROUND Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting. METHODS Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized. RESULTS A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%. CONCLUSION To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.
Collapse
Affiliation(s)
- Elisa Bannone
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France.
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy.
| | - Toby Collins
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | - Alessandro Esposito
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Lorenzo Cinelli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of Gastrointestinal Surgery, San Raffaele Hospital IRCCS, Milan, Italy
| | - Matteo De Pastena
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Patrick Pessaux
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, Strasbourg, France
- Institut of Viral and Liver Disease, Inserm U1110, University of Strasbourg, Strasbourg, France
| | - Emanuele Felli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, Strasbourg, France
- Institut of Viral and Liver Disease, Inserm U1110, University of Strasbourg, Strasbourg, France
| | - Elena Andreotti
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Nariaki Okamoto
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
| | - Manuel Barberio
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- General Surgery Department, Ospedale Cardinale G. Panico, Tricase, Italy
| | - Eric Felli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of Visceral Surgery and Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Roberto Maria Montorsi
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Naomi Ingaglio
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - María Rita Rodríguez-Luna
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
| | - Richard Nkusi
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | - Jacque Marescaux
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | | | - Roberto Salvia
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Michele Diana
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
- Department of Surgery, University Hospital of Geneva, Geneva, Switzerland
| |
Collapse
|
4
|
Bhargava A, Sachdeva A, Sharma K, Alsharif MH, Uthansakul P, Uthansakul M. Hyperspectral imaging and its applications: A review. Heliyon 2024; 10:e33208. [PMID: 39021975 PMCID: PMC11253060 DOI: 10.1016/j.heliyon.2024.e33208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Revised: 06/13/2024] [Accepted: 06/17/2024] [Indexed: 07/20/2024] Open
Abstract
Hyperspectral imaging has emerged as an effective powerful tool in plentiful military, environmental, and civil applications over the last three decades. The modern remote sensing approaches are adequate for covering huge earth surfaces with phenomenal temporal, spectral, and spatial resolutions. These features make HSI more effective in various applications of remote sensing depending upon the physical estimation of identical material identification and manifold composite surfaces having accomplished spectral resolutions. Recently, HSI has attained immense significance in the research on safety and quality assessment of food, medical analysis, and agriculture applications. This review focuses on HSI fundamentals and its applications like safety and quality assessment of food, medical analysis, agriculture, water resources, plant stress identification, weed & crop discrimination, and flood management. Various investigators have promising solutions for automatic systems depending upon HSI. Future research may use this review as a baseline and future advancement analysis.
Collapse
Affiliation(s)
| | - Ashish Sachdeva
- Chitkara University Institute of Engineering and Technology, Chitkara University, Punjab, India
| | - Kulbhushan Sharma
- VLSI Centre of Excellence, Chitkara University Institute of Engineering and Technology, Chitkara University, Punjab, India
| | - Mohammed H. Alsharif
- Department of Electrical Engineering, College of Electronics and Information Engineering, Sejong University, Seoul, 05006, South Korea
| | - Peerapong Uthansakul
- School of Telecommunication Engineering, Suranaree University of Technology, Nakhon Ratchasima, 30000, Thailand
| | - Monthippa Uthansakul
- School of Telecommunication Engineering, Suranaree University of Technology, Nakhon Ratchasima, 30000, Thailand
| |
Collapse
|
5
|
Qasim AB, Motta A, Studier-Fischer A, Sellner J, Ayala L, Hübner M, Bressan M, Özdemir B, Kowalewski KF, Nickel F, Seidlitz S, Maier-Hein L. Test-time augmentation with synthetic data addresses distribution shifts in spectral imaging. Int J Comput Assist Radiol Surg 2024; 19:1021-1031. [PMID: 38483702 PMCID: PMC11178652 DOI: 10.1007/s11548-024-03085-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Accepted: 02/22/2024] [Indexed: 06/15/2024]
Abstract
PURPOSE Surgical scene segmentation is crucial for providing context-aware surgical assistance. Recent studies highlight the significant advantages of hyperspectral imaging (HSI) over traditional RGB data in enhancing segmentation performance. Nevertheless, the current hyperspectral imaging (HSI) datasets remain limited and do not capture the full range of tissue variations encountered clinically. METHODS Based on a total of 615 hyperspectral images from a total of 16 pigs, featuring porcine organs in different perfusion states, we carry out an exploration of distribution shifts in spectral imaging caused by perfusion alterations. We further introduce a novel strategy to mitigate such distribution shifts, utilizing synthetic data for test-time augmentation. RESULTS The effect of perfusion changes on state-of-the-art (SOA) segmentation networks depended on the organ and the specific perfusion alteration induced. In the case of the kidney, we observed a performance decline of up to 93% when applying a state-of-the-art (SOA) network under ischemic conditions. Our method improved on the state-of-the-art (SOA) by up to 4.6 times. CONCLUSION Given its potential wide-ranging relevance to diverse pathologies, our approach may serve as a pivotal tool to enhance neural network generalization within the realm of spectral imaging.
Collapse
Affiliation(s)
- Ahmad Bin Qasim
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany.
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany.
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany.
| | - Alessandro Motta
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Studier-Fischer
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Jan Sellner
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, A Partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - Leonardo Ayala
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Marco Hübner
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - Marc Bressan
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Berkin Özdemir
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Karl Friedrich Kowalewski
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
- Department of Urology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Felix Nickel
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, A Partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, A Partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
| |
Collapse
|
6
|
Zhang Y, Dong J. SAHIS-Net: a spectral attention and feature enhancement network for microscopic hyperspectral cholangiocarcinoma image segmentation. BIOMEDICAL OPTICS EXPRESS 2024; 15:3147-3162. [PMID: 38855697 PMCID: PMC11161366 DOI: 10.1364/boe.519090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 04/03/2024] [Accepted: 04/09/2024] [Indexed: 06/11/2024]
Abstract
Cholangiocarcinoma (CCA) poses a significant clinical challenge due to its aggressive nature and poor prognosis. While traditional diagnosis relies on color-based histopathology, hyperspectral imaging (HSI) offers rich, high-dimensional data holding potential for more accurate diagnosis. However, extracting meaningful insights from this data remains challenging. This work investigates the application of deep learning for CCA segmentation in microscopic HSI images, and introduces two novel neural networks: (1) Histogram Matching U-Net (HM-UNet) for efficient image pre-processing, and (2) Spectral Attention based Hyperspectral Image Segmentation Net (SAHIS-Net) for CCA segmentation. SAHIS-Net integrates a novel Spectral Attention (SA) module for adaptively weighing spectral information, an improved attention-aware feature enhancement (AFE) mechanism for better providing the model with more discriminative features, and a multi-loss training strategy for effective early stage feature extraction. We compare SAHIS-Net against several general and CCA-specific models, demonstrating its superior performance in segmenting CCA regions. These results highlight the potential of our approach for segmenting medical HSI images.
Collapse
Affiliation(s)
- Yunchu Zhang
- School of Future Science and Engineering, Soochow University, Suzhou 215222, Jiangsu, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Suzhou 215163, Jiangsu, China
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, Jiangsu, China
| | - Jianfei Dong
- School of Future Science and Engineering, Soochow University, Suzhou 215222, Jiangsu, China
| |
Collapse
|
7
|
Egen L, Demmel GS, Grilli M, Studier-Fischer A, Nickel F, Haney CM, Mühlbauer J, Hartung FO, Menold HS, Piazza P, Rivas JG, Checcucci E, Puliatti S, Belenchon IR, Taratkin M, Rodler S, Cacciamani G, Michel MS, Kowalewski KF. Biophotonics-Intraoperative Guidance During Partial Nephrectomy: A Systematic Review and Meta-analysis. Eur Urol Focus 2024; 10:248-258. [PMID: 38278713 DOI: 10.1016/j.euf.2024.01.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Revised: 12/11/2023] [Accepted: 01/15/2024] [Indexed: 01/28/2024]
Abstract
CONTEXT Partial nephrectomy (PN) with intraoperative guidance by biophotonics has the potential to improve surgical outcomes due to higher precision. However, its value remains unclear since high-level evidence is lacking. OBJECTIVE To provide a comprehensive analysis of biophotonic techniques used for intraoperative real-time assistance during PN. EVIDENCE ACQUISITION We performed a comprehensive database search based on the PICO criteria, including studies published before October 2022. Two independent reviewers screened the titles and abstracts followed by full-text screening of eligible studies. For a quantitative analysis, a meta-analysis was conducted. EVIDENCE SYNTHESIS In total, 35 studies were identified for the qualitative analysis, including 27 studies on near-infrared fluorescence (NIRF) imaging using indocyanine green, four studies on hyperspectral imaging, two studies on folate-targeted molecular imaging, and one study each on optical coherence tomography and 5-aminolevulinic acid. The meta-analysis investigated seven studies on selective arterial clamping using NIRF. There was a significantly shorter warm ischemia time in the NIRF-PN group (mean difference [MD]: -2.9; 95% confidence interval [CI]: -5.6, -0.1; p = 0.04). No differences were noted regarding transfusions (odds ratio [OR]: 0.5; 95% CI: 0.2, 1.7; p = 0.27), positive surgical margins (OR: 0.7; 95% CI: 0.2, 2.0; p = 0.46), or major complications (OR: 0.4; 95% CI: 0.1, 1.2; p = 0.08). In the NIRF-PN group, functional results were favorable at short-term follow-up (MD of glomerular filtration rate decline: 7.6; 95% CI: 4.6, 10.5; p < 0.01), but leveled off at long-term follow-up (MD: 7.0; 95% CI: -2.8, 16.9; p = 0.16). Remarkably, these findings were not confirmed by the included randomized controlled trial. CONCLUSIONS Biophotonics comprises a heterogeneous group of imaging modalities that serve intraoperative decision-making and guidance. Implementation into clinical practice and cost effectiveness are the limitations that should be addressed by future research. PATIENT SUMMARY We reviewed the application of biophotonics during partial removal of the kidney in patients with kidney cancer. Our results suggest that these techniques support the surgeon in successfully performing the challenging steps of the procedure.
Collapse
Affiliation(s)
- Luisa Egen
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany.
| | - Greta S Demmel
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Maurizio Grilli
- Library of the Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Alexander Studier-Fischer
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Felix Nickel
- Department of General, Visceral, and Thoracic Surgery, University Hospital Hamburg-Eppendorf, Hamburg, Germany
| | - Caelan M Haney
- Department of Urology, University Hospital Leipzig, Leipzig, Germany
| | - Julia Mühlbauer
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Friedrich O Hartung
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Hanna S Menold
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Pietro Piazza
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Division of Urology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Juan Gomez Rivas
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Department of Urology, Hospital Clinico San Carlos, Madrid, Spain
| | - Enrico Checcucci
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Department of Surgery, FPO-IRCCS Candiolo Cancer Institute, Turin, Italy
| | - Stefano Puliatti
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Department of Urology, University of Modena, and Reggio Emilia, Modena, Italy
| | - Ines Rivero Belenchon
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Urology and Nephrology Department, Virgen del Rocío University Hospital, Seville, Spain
| | - Mark Taratkin
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands
| | - Severin Rodler
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Department of Urology, University Hospital LMU Munich, Munich, Germany
| | - Giovanni Cacciamani
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; USC Institute of Urology, University of Southern California, Los Angeles, CA, USA
| | - Maurice S Michel
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Karl-Friedrich Kowalewski
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany; Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands
| |
Collapse
|
8
|
Ilgen A, Köhler H, Pfahl A, Stelzner S, Mehdorn M, Jansen-Winkeln B, Gockel I, Moulla Y. Intraoperative Laparoscopic Hyperspectral Imaging during Esophagectomy-A Pilot Study Evaluating Esophagogastric Perfusion at the Anastomotic Sites. Bioengineering (Basel) 2024; 11:69. [PMID: 38247946 PMCID: PMC10812999 DOI: 10.3390/bioengineering11010069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2023] [Revised: 12/01/2023] [Accepted: 01/04/2024] [Indexed: 01/23/2024] Open
Abstract
Hyperspectral imaging (HSI) is a non-invasive and contactless technique that enables the real-time acquisition of comprehensive information on tissue within the surgical field. In this pilot study, we investigated whether a new HSI system for minimally-invasive surgery, TIVITA® Mini (HSI-MIS), provides reliable insights into tissue perfusion of the proximal and distal esophagogastric anastomotic sites during 21 laparoscopic/thoracoscopic or robotic Ivor Lewis esophagectomies of patients with cancer to minimize the risk of dreaded anastomotic insufficiency. In this pioneering investigation, physiological tissue parameters were derived from HSI measurements of the proximal site of the anastomosis (esophageal stump) and the distal site of the anastomosis (tip of the gastric conduit) during the thoracic phase of the procedure. Tissue oxygenation (StO2), Near Infrared Perfusion Index (NIR-PI), and Tissue Water Index (TWI) showed similar median values at both anastomotic sites. Significant differences were observed only for NIR-PI (median: 76.5 vs. 63.9; p = 0.012) at the distal site (gastric conduit) compared to our previous study using an HSI system for open surgery. For all 21 patients, reliable and informative measurements were attainable, confirming the feasibility of HSI-MIS to assess anastomotic viability. Further studies on the added benefit of this new technique aiming to reduce anastomotic insufficiency are warranted.
Collapse
Affiliation(s)
- Annalena Ilgen
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103 Leipzig, Germany; (H.K.); (A.P.)
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103 Leipzig, Germany; (H.K.); (A.P.)
| | - Sigmar Stelzner
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| | - Matthias Mehdorn
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| | - Boris Jansen-Winkeln
- Department of General, Visceral, Thoracic and Vascular Surgery, Klinikum St. Georg, Delitzscher Str. 141, 04129 Leipzig, Germany;
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| |
Collapse
|
9
|
Pruitt K, Rathgeb A, Gahan JC, Johnson BA, Strand DW, Fei B. A dual-camera hyperspectral laparoscopic imaging system. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12831:1283107. [PMID: 38708175 PMCID: PMC11069412 DOI: 10.1117/12.3005893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2024]
Abstract
Minimally invasive surgery (MIS) has expanded broadly in the field of abdominal and pelvic surgery. However, there are still prevalent issues surrounding intracorporeal surgery, such as iatrogenic injury, anastomotic leakage, or the presence of positive tumor margins after resection. Current approaches to address these issues and advance laparoscopic imaging techniques often involve fluorescence imaging agents, such as indocyanine green (ICG), to improve visualization, but these have drawbacks. Hyperspectral imaging (HSI) is an emerging optical imaging modality that takes advantage of spectral characteristics of different tissues. Various applications include tissue classification and digital pathology. In this study, we developed a dual-camera system for high-speed hyperspectral imaging. This includes the development of a custom application interface and corresponding hardware setup. Characterization of the system was performed, including spectral accuracy and spatial resolution, showing little sacrifice in speed for the approximate doubling of the covered spectral range, with our system acquiring 29 spectral images from 460-850 nm. Reference color tiles with various reflectance profiles were imaged and a RMSE of 3.56 ± 1.36% was achieved. Sub-millimeter resolution was shown at 7 cm working distance for both hyperspectral cameras. Finally, we image ex vivo tissues, including porcine stomach, liver, intestine, and kidney with our system and use a high-resolution, radiometrically calibrated spectrometer for comparison and evaluation of spectral fidelity. The dual-camera hyperspectral laparoscopic imaging system can have immediate applications in various surgeries.
Collapse
Affiliation(s)
- Kelden Pruitt
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Armand Rathgeb
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Jeffrey C. Gahan
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Brett A. Johnson
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Douglas W. Strand
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
10
|
Studier-Fischer A, Schwab FM, Rees M, Seidlitz S, Sellner J, Özdemir B, Ayala L, Odenthal J, Knoedler S, Kowalewski KF, Haney CM, Dietrich M, Salg GA, Kenngott HG, Müller-Stich BP, Maier-Hein L, Nickel F. ICG-augmented hyperspectral imaging for visualization of intestinal perfusion compared to conventional ICG fluorescence imaging: an experimental study. Int J Surg 2023; 109:3883-3895. [PMID: 38258996 PMCID: PMC10720797 DOI: 10.1097/js9.0000000000000706] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Accepted: 08/13/2023] [Indexed: 01/24/2024]
Abstract
BACKGROUND Small bowel malperfusion (SBM) can cause high morbidity and severe surgical consequences. However, there is no standardized objective measuring tool for the quantification of SBM. Indocyanine green (ICG) imaging can be used for visualization, but lacks standardization and objectivity. Hyperspectral imaging (HSI) as a newly emerging technology in medicine might present advantages over conventional ICG fluorescence or in combination with it. METHODS HSI baseline data from physiological small bowel, avascular small bowel and small bowel after intravenous application of ICG was recorded in a total number of 54 in-vivo pig models. Visualizations of avascular small bowel after mesotomy were compared between HSI only (1), ICG-augmented HSI (IA-HSI) (2), clinical evaluation through the eyes of the surgeon (3) and conventional ICG imaging (4). The primary research focus was the localization of resection borders as suggested by each of the four methods. Distances between these borders were measured and histological samples were obtained from the regions in between in order to quantify necrotic changes 6 h after mesotomy for every region. RESULTS StO2 images (1) were capable of visualizing areas of physiological perfusion and areas of clearly impaired perfusion. However, exact borders where physiological perfusion started to decrease could not be clearly identified. Instead, IA-HSI (2) suggested a sharp-resection line where StO2 values started to decrease. Clinical evaluation (3) suggested a resection line 23 mm (±7 mm) and conventional ICG imaging (4) even suggested a resection line 53 mm (±13 mm) closer towards the malperfused region. Histopathological evaluation of the region that was sufficiently perfused only according to conventional ICG (R3) already revealed a significant increase in pre-necrotic changes in 27% (±9%) of surface area. Therefore, conventional ICG seems less sensitive than IA-HSI with regards to detection of insufficient tissue perfusion. CONCLUSIONS In this experimental animal study, IA-HSI (2) was superior for the visualization of segmental SBM compared to conventional HSI imaging (1), clinical evaluation (3) or conventional ICG imaging (4) regarding histopathological safety. ICG application caused visual artifacts in the StO2 values of the HSI camera as values significantly increase. This is caused by optical properties of systemic ICG and does not resemble a true increase in oxygenation levels. However, this empirical finding can be used to visualize segmental SBM utilizing ICG as contrast agent in an approach for IA-HSI. Clinical applicability and relevance will have to be explored in clinical trials. LEVEL OF EVIDENCE Not applicable. Translational animal science. Original article.
Collapse
Affiliation(s)
| | | | - Maike Rees
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)
- Faculty of Mathematics and Computer Science
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)
- Faculty of Mathematics and Computer Science
- HIDSS4Health—Helmholtz Information and Data Science School for Health, Karlsruhe
- National Center for Tumor Diseases (NCT) Heidelberg, a partnership between DKFZ and Heidelberg University Hospital, Heidelberg
| | - Jan Sellner
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)
- Faculty of Mathematics and Computer Science
- HIDSS4Health—Helmholtz Information and Data Science School for Health, Karlsruhe
| | - Berkin Özdemir
- Departments ofGeneral, Visceral, and Transplantation Surgery
| | - Leonardo Ayala
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)
- Medical Faculty, Heidelberg University
| | - Jan Odenthal
- Departments ofGeneral, Visceral, and Transplantation Surgery
| | - Samuel Knoedler
- Departments ofGeneral, Visceral, and Transplantation Surgery
- Division of Plastic Surgery, Department of Surgery, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
| | | | | | | | | | | | | | - Lena Maier-Hein
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)
- Faculty of Mathematics and Computer Science
- HIDSS4Health—Helmholtz Information and Data Science School for Health, Karlsruhe
- National Center for Tumor Diseases (NCT) Heidelberg, a partnership between DKFZ and Heidelberg University Hospital, Heidelberg
| | - Felix Nickel
- Departments ofGeneral, Visceral, and Transplantation Surgery
- HIDSS4Health—Helmholtz Information and Data Science School for Health, Karlsruhe
- Department of General, Visceral, and Thoracic Surgery, University Hospital Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
11
|
Chalopin C, Pfahl A, Köhler H, Knospe L, Maktabi M, Unger M, Jansen-Winkeln B, Thieme R, Moulla Y, Mehdorn M, Sucher R, Neumuth T, Gockel I, Melzer A. Alternative intraoperative optical imaging modalities for fluorescence angiography in gastrointestinal surgery: spectral imaging and imaging photoplethysmography. MINIM INVASIV THER 2023; 32:222-232. [PMID: 36622288 DOI: 10.1080/13645706.2022.2164469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 11/29/2022] [Indexed: 01/10/2023]
Abstract
INTRODUCTION Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA. MATERIAL AND METHODS The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works. RESULTS The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method. CONCLUSIONS Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Luise Knospe
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Department of Electrical, Mechanical and Industrial Engineering, Anhalt University of Applied Science, Köthen (Anhalt), Germany
| | - Michael Unger
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
- Department of General, Visceral and Oncological Surgery, St. Georg Hospital, Leipzig, Germany
| | - René Thieme
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Matthias Mehdorn
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Robert Sucher
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Institute of Medical Science and Technology (IMSAT), University of Dundee, Dundee, UK
| |
Collapse
|
12
|
Zhang C, Zhang Z, Yu D, Cheng Q, Shan S, Li M, Mou L, Yang X, Ma X. Unsupervised band selection of medical hyperspectral images guided by data gravitation and weak correlation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107721. [PMID: 37506601 DOI: 10.1016/j.cmpb.2023.107721] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 07/06/2023] [Accepted: 07/12/2023] [Indexed: 07/30/2023]
Abstract
BACKGROUND AND OBJECTIVE Medical hyperspectral images (MHSIs) are used for a contact-free examination of patients without harmful radiation. However, high-dimensionality images contain large amounts of data that are sparsely distributed in a high-dimensional space, which leads to the "curse of dimensionality" (called Hughes' phenomenon) and increases the complexity and cost of data processing and storage. Hence, there is a need for spectral dimensionality reduction before the clinical application of MHSIs. Some dimensionality-reducing strategies have been proposed; however, they distort the data within MHSIs. METHODS To compress dimensionality without destroying the original data structure, we propose a method that involves data gravitation and weak correlation-based ranking (DGWCR) for removing bands of noise from MHSIs while clustering signal-containing bands. Band clustering is done by using the connection centre evolution (CCE) algorithm and selecting the most representative bands in each cluster based on the composite force. The bands within the clusters are ranked using the new entropy-containing matrix, and a global ranking of bands is obtained by applying an S-shaped strategy. The source code is available at https://www.github.com/zhangchenglong1116/DGWCR. RESULTS Upon feeding the reduced-dimensional images into various classifiers, the experimental results demonstrated that the small number of bands selected by the proposed DGWCR consistently achieved higher classification accuracy than the original data. Unlike other reference methods (e.g. the latest deep-learning-based strategies), DGWCR chooses the spectral bands with the least redundancy and greatest discrimination. CONCLUSION In this study, we present a method for efficient band selection for MHSIs that alleviates the "curse of dimensionality". Experiments were validated with three MHSIs in the human brain, and they outperformed several other band selection methods, demonstrating the clinical potential of DGWCR.
Collapse
Affiliation(s)
- Chenglong Zhang
- School of Control Science and Engineering, Shandong University, Jinan 250061, China
| | - Zhimin Zhang
- School of Control Science and Engineering, Shandong University, Jinan 250061, China
| | - Dexin Yu
- Radiology Department, Qilu Hospital of Shandong University, Jinan 250000, China
| | - Qiyuan Cheng
- Medical Engineering Department, Shandong Provincial Hospital affiliated to Shandong First Medical University, Jinan 250021, China
| | - Shihao Shan
- School of Control Science and Engineering, Shandong University, Jinan 250061, China
| | - Mengjiao Li
- School of Control Science and Engineering, Shandong University, Jinan 250061, China
| | - Lichao Mou
- Chair of Data Science in Earth Observation, Technical University of Munich (TUM), Munich, 80333, Germany
| | - Xiaoli Yang
- School of Control Science and Engineering, Shandong University, Jinan 250061, China; Weifang Xinli Superconducting Magnet Technology Co., Ltd, Weifang 261005, China.
| | - Xiaopeng Ma
- School of Control Science and Engineering, Shandong University, Jinan 250061, China.
| |
Collapse
|
13
|
MacCormac O, Noonan P, Janatka M, Horgan CC, Bahl A, Qiu J, Elliot M, Trotouin T, Jacobs J, Patel S, Bergholt MS, Ashkan K, Ourselin S, Ebner M, Vercauteren T, Shapey J. Lightfield hyperspectral imaging in neuro-oncology surgery: an IDEAL 0 and 1 study. Front Neurosci 2023; 17:1239764. [PMID: 37790587 PMCID: PMC10544348 DOI: 10.3389/fnins.2023.1239764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Accepted: 08/31/2023] [Indexed: 10/05/2023] Open
Abstract
Introduction Hyperspectral imaging (HSI) has shown promise in the field of intra-operative imaging and tissue differentiation as it carries the capability to provide real-time information invisible to the naked eye whilst remaining label free. Previous iterations of intra-operative HSI systems have shown limitations, either due to carrying a large footprint limiting ease of use within the confines of a neurosurgical theater environment, having a slow image acquisition time, or by compromising spatial/spectral resolution in favor of improvements to the surgical workflow. Lightfield hyperspectral imaging is a novel technique that has the potential to facilitate video rate image acquisition whilst maintaining a high spectral resolution. Our pre-clinical and first-in-human studies (IDEAL 0 and 1, respectively) demonstrate the necessary steps leading to the first in-vivo use of a real-time lightfield hyperspectral system in neuro-oncology surgery. Methods A lightfield hyperspectral camera (Cubert Ultris ×50) was integrated in a bespoke imaging system setup so that it could be safely adopted into the open neurosurgical workflow whilst maintaining sterility. Our system allowed the surgeon to capture in-vivo hyperspectral data (155 bands, 350-1,000 nm) at 1.5 Hz. Following successful implementation in a pre-clinical setup (IDEAL 0), our system was evaluated during brain tumor surgery in a single patient to remove a posterior fossa meningioma (IDEAL 1). Feedback from the theater team was analyzed and incorporated in a follow-up design aimed at implementing an IDEAL 2a study. Results Focusing on our IDEAL 1 study results, hyperspectral information was acquired from the cerebellum and associated meningioma with minimal disruption to the neurosurgical workflow. To the best of our knowledge, this is the first demonstration of HSI acquisition with 100+ spectral bands at a frame rate over 1Hz in surgery. Discussion This work demonstrated that a lightfield hyperspectral imaging system not only meets the design criteria and specifications outlined in an IDEAL-0 (pre-clinical) study, but also that it can translate into clinical practice as illustrated by a successful first in human study (IDEAL 1). This opens doors for further development and optimisation, given the increasing evidence that hyperspectral imaging can provide live, wide-field, and label-free intra-operative imaging and tissue differentiation.
Collapse
Affiliation(s)
- Oscar MacCormac
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
| | - Philip Noonan
- Hypervision Surgical Limited, London, United Kingdom
| | - Mirek Janatka
- Hypervision Surgical Limited, London, United Kingdom
| | | | - Anisha Bahl
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
| | - Jianrong Qiu
- School of Craniofacial and Regenerative Biology, King's College London, London, United Kingdom
| | - Matthew Elliot
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
| | - Théo Trotouin
- Hypervision Surgical Limited, London, United Kingdom
| | - Jaco Jacobs
- Hypervision Surgical Limited, London, United Kingdom
| | - Sabina Patel
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
| | - Mads S. Bergholt
- School of Craniofacial and Regenerative Biology, King's College London, London, United Kingdom
| | - Keyoumars Ashkan
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
| | - Sebastien Ourselin
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Hypervision Surgical Limited, London, United Kingdom
| | - Michael Ebner
- Hypervision Surgical Limited, London, United Kingdom
| | - Tom Vercauteren
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Hypervision Surgical Limited, London, United Kingdom
| | - Jonathan Shapey
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
- Hypervision Surgical Limited, London, United Kingdom
| |
Collapse
|
14
|
Tan D, Mohd Nasir NF, Abdul Manan H, Yahya N. Prediction of toxicity outcomes following radiotherapy using deep learning-based models: A systematic review. Cancer Radiother 2023; 27:398-406. [PMID: 37482464 DOI: 10.1016/j.canrad.2023.05.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Revised: 05/16/2023] [Accepted: 05/19/2023] [Indexed: 07/25/2023]
Abstract
PURPOSE This study aims to perform a comprehensive systematic review of deep learning (DL) models in predicting RT-induced toxicity. MATERIALS AND METHODS A literature review was performed following the PRISMA guidelines. Systematic searches were performed in PubMed, Scopus, Cochrane and Embase databases from the earliest record up to September 2022. Related studies on deep learning models for radiotherapy toxicity prediction were selected based on predefined PICOS criteria. RESULTS Fourteen studies of radiotherapy-treated patients on different types of cancer [prostate (n=2), HNC (n=4), liver (n=2), lung (n=4), cervical (n=1), and oesophagus (n=1)] were eligible for inclusion in the systematic review. Information regarding patient characteristics and model development was summarized. Several approaches, such as ensemble learning, data augmentation, and transfer learning, that were utilized by selected studies were discussed. CONCLUSION Deep learning techniques are able to produce a consistent performance for toxicity prediction. Future research using large and diverse datasets and standardization of the study methodologies are required to improve the consistency of the research output.
Collapse
Affiliation(s)
- D Tan
- Centre of Diagnostic, Therapeutic and Investigative Sciences (CODTIS). Faculty of Health Sciences, Universiti Kebangsaan Malaysia, Jalan Raja Muda Aziz, Kuala Lumpur 50300 Malaysia
| | - N F Mohd Nasir
- Centre of Diagnostic, Therapeutic and Investigative Sciences (CODTIS). Faculty of Health Sciences, Universiti Kebangsaan Malaysia, Jalan Raja Muda Aziz, Kuala Lumpur 50300 Malaysia
| | - H Abdul Manan
- Functional Image Processing Laboratory, Department of Radiology, Universiti Kebangsaan Malaysia Medical Centre, Cheras, Kuala Lumpur 56000, Malaysia
| | - N Yahya
- Centre of Diagnostic, Therapeutic and Investigative Sciences (CODTIS). Faculty of Health Sciences, Universiti Kebangsaan Malaysia, Jalan Raja Muda Aziz, Kuala Lumpur 50300 Malaysia.
| |
Collapse
|
15
|
Studier-Fischer A, Seidlitz S, Sellner J, Bressan M, Özdemir B, Ayala L, Odenthal J, Knoedler S, Kowalewski KF, Haney CM, Salg G, Dietrich M, Kenngott H, Gockel I, Hackert T, Müller-Stich BP, Maier-Hein L, Nickel F. HeiPorSPECTRAL - the Heidelberg Porcine HyperSPECTRAL Imaging Dataset of 20 Physiological Organs. Sci Data 2023; 10:414. [PMID: 37355750 PMCID: PMC10290660 DOI: 10.1038/s41597-023-02315-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 06/15/2023] [Indexed: 06/26/2023] Open
Abstract
Hyperspectral Imaging (HSI) is a relatively new medical imaging modality that exploits an area of diagnostic potential formerly untouched. Although exploratory translational and clinical studies exist, no surgical HSI datasets are openly accessible to the general scientific community. To address this bottleneck, this publication releases HeiPorSPECTRAL ( https://www.heiporspectral.org ; https://doi.org/10.5281/zenodo.7737674 ), the first annotated high-quality standardized surgical HSI dataset. It comprises 5,758 spectral images acquired with the TIVITA® Tissue and annotated with 20 physiological porcine organs from 8 pigs per organ distributed over a total number of 11 pigs. Each HSI image features a resolution of 480 × 640 pixels acquired over the 500-1000 nm wavelength range. The acquisition protocol has been designed such that the variability of organ spectra as a function of several parameters including the camera angle and the individual can be assessed. A comprehensive technical validation confirmed both the quality of the raw data and the annotations. We envision potential reuse within this dataset, but also its reuse as baseline data for future research questions outside this dataset. Measurement(s) Spectral Reflectance Technology Type(s) Hyperspectral Imaging Sample Characteristic - Organism Sus scrofa.
Collapse
Affiliation(s)
- Alexander Studier-Fischer
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe, Heidelberg, Germany
- National Center for Tumor Diseases (NCT) Heidelberg, a partnership between DKFZ and Heidelberg University Hospital, Heidelberg, Germany
| | - Jan Sellner
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe, Heidelberg, Germany
| | - Marc Bressan
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Berkin Özdemir
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Leonardo Ayala
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
| | - Jan Odenthal
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Samuel Knoedler
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
- Division of Plastic Surgery, Department of Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Karl-Friedrich Kowalewski
- Department of Urology, Medical Faculty of Mannheim at the University of Heidelberg, Mannheim, Germany
| | - Caelan Max Haney
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Gabriel Salg
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Maximilian Dietrich
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - Hannes Kenngott
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, Leipzig University Hospital, Leipzig, Germany
| | - Thilo Hackert
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
- Department of General, Visceral, and Thoracic Surgery, University Hospital Hamburg-Eppendorf, Hamburg, Germany
| | - Beat Peter Müller-Stich
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe, Heidelberg, Germany
- National Center for Tumor Diseases (NCT) Heidelberg, a partnership between DKFZ and Heidelberg University Hospital, Heidelberg, Germany
| | - Felix Nickel
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany.
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe, Heidelberg, Germany.
- Department of General, Visceral, and Thoracic Surgery, University Hospital Hamburg-Eppendorf, Hamburg, Germany.
| |
Collapse
|
16
|
Nickel F, Studier-Fischer A, Özdemir B, Odenthal J, Müller LR, Knoedler S, Kowalewski KF, Camplisson I, Allers MM, Dietrich M, Schmidt K, Salg GA, Kenngott HG, Billeter AT, Gockel I, Sagiv C, Hadar OE, Gildenblat J, Ayala L, Seidlitz S, Maier-Hein L, Müller-Stich BP. Optimization of anastomotic technique and gastric conduit perfusion with hyperspectral imaging and machine learning in an experimental model for minimally invasive esophagectomy. EUROPEAN JOURNAL OF SURGICAL ONCOLOGY 2023:S0748-7983(23)00444-4. [PMID: 37105869 DOI: 10.1016/j.ejso.2023.04.007] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2022] [Revised: 03/26/2023] [Accepted: 04/12/2023] [Indexed: 04/29/2023]
Abstract
INTRODUCTION Esophagectomy is the mainstay of esophageal cancer treatment, but anastomotic insufficiency related morbidity and mortality remain challenging for patient outcome. Therefore, the objective of this work was to optimize anastomotic technique and gastric conduit perfusion with hyperspectral imaging (HSI) for total minimally invasive esophagectomy (MIE) with linear stapled anastomosis. MATERIAL AND METHODS A live porcine model (n = 58) for MIE was used with gastric conduit formation and simulation of linear stapled side-to-side esophagogastrostomy. Four main experimental groups differed in stapling length (3 vs. 6 cm) and simulation of anastomotic position on the conduit (cranial vs. caudal). Tissue oxygenation around the anastomotic simulation site was evaluated using HSI and was validated with histopathology. RESULTS The tissue oxygenation (ΔStO2) after the anastomotic simulation remained constant only for the short stapler in caudal position (-0.4 ± 4.4%, n.s.) while it was impaired markedly in the other groups (short-cranial: -15.6 ± 11.5%, p = 0.0002; long-cranial: -20.4 ± 7.6%, p = 0.0126; long-caudal: -16.1 ± 9.4%, p < 0.0001). Tissue samples from avascular stomach as measured by HSI showed correspondent eosinophilic pre-necrotic changes in 35.7 ± 9.7% of the surface area. CONCLUSION Tissue oxygenation at the site of anastomotic simulation of the gastric conduit during MIE is influenced by stapling technique. Optimal oxygenation was achieved with a short stapler (3 cm) and sufficient distance of the simulated anastomosis to the cranial end of the gastric conduit. HSI tissue deoxygenation corresponded to histopathologic necrotic tissue changes. The experimental model with HSI and ML allow for systematic optimization of gastric conduit perfusion and anastomotic technique while clinical translation will have to be proven.
Collapse
Affiliation(s)
- F Nickel
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany; HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg and Karlsruhe, Germany
| | - A Studier-Fischer
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany; School of Medicine, Heidelberg University, Heidelberg, Germany
| | - B Özdemir
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - J Odenthal
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - L R Müller
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg and Karlsruhe, Germany; Division of Computer Assisted Medical Interventions, German Cancer Research Center (DKFZ), Heidelberg, Germany; Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - S Knoedler
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - K F Kowalewski
- Department of Urology, Medical Faculty of Mannheim at the University of Heidelberg, Mannheim, Germany
| | - I Camplisson
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, USA
| | - M M Allers
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - M Dietrich
- Department of Anaesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - K Schmidt
- Department of Anaesthesiology and Intensive Care Medicine, Essen University Hospital, Essen, Germany
| | - G A Salg
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - H G Kenngott
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - A T Billeter
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - I Gockel
- Department of Visceral, Transplantation, Thoracic and Vascular Surgery, Leipzig University Hospital, Leipzig, Germany
| | - C Sagiv
- DeePathology Ltd., Ra'anana, Israel
| | | | | | - L Ayala
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg and Karlsruhe, Germany; Division of Computer Assisted Medical Interventions, German Cancer Research Center (DKFZ), Heidelberg, Germany; Medical Faculty, Heidelberg University, Heidelberg, Germany
| | - S Seidlitz
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg and Karlsruhe, Germany; Division of Computer Assisted Medical Interventions, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - L Maier-Hein
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg and Karlsruhe, Germany; Division of Computer Assisted Medical Interventions, German Cancer Research Center (DKFZ), Heidelberg, Germany; Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany; Medical Faculty, Heidelberg University, Heidelberg, Germany
| | - B P Müller-Stich
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany; HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg and Karlsruhe, Germany.
| |
Collapse
|
17
|
Ayala L, Adler TJ, Seidlitz S, Wirkert S, Engels C, Seitel A, Sellner J, Aksenov A, Bodenbach M, Bader P, Baron S, Vemuri A, Wiesenfarth M, Schreck N, Mindroc D, Tizabi M, Pirmann S, Everitt B, Kopp-Schneider A, Teber D, Maier-Hein L. Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery. SCIENCE ADVANCES 2023; 9:eadd6778. [PMID: 36897951 PMCID: PMC10005169 DOI: 10.1126/sciadv.add6778] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Accepted: 02/03/2023] [Indexed: 06/18/2023]
Abstract
Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.
Collapse
Affiliation(s)
- Leonardo Ayala
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
| | - Tim J. Adler
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
| | - Sebastian Wirkert
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Alexander Seitel
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jan Sellner
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
| | | | | | - Pia Bader
- Städtisches Klinikum Karlsruhe, Karlsruhe, Germany
| | | | - Anant Vemuri
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Manuel Wiesenfarth
- Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Nicholas Schreck
- Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Diana Mindroc
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Minu Tizabi
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sebastian Pirmann
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Brittaney Everitt
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Dogu Teber
- Städtisches Klinikum Karlsruhe, Karlsruhe, Germany
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
| |
Collapse
|
18
|
Garcia Peraza Herrera LC, Horgan C, Ourselin S, Ebner M, Vercauteren T. Hyperspectral image segmentation: a preliminary study on the Oral and Dental Spectral Image Database (ODSI-DB). COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2023. [DOI: 10.1080/21681163.2022.2160377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Affiliation(s)
| | - Conor Horgan
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | | | - Michael Ebner
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | - Tom Vercauteren
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| |
Collapse
|
19
|
In vivo evaluation of a hyperspectral imaging system for minimally invasive surgery (HSI-MIS). Surg Endosc 2023; 37:3691-3700. [PMID: 36645484 PMCID: PMC10156625 DOI: 10.1007/s00464-023-09874-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2022] [Accepted: 01/06/2023] [Indexed: 01/17/2023]
Abstract
BACKGROUND Hyperspectral Imaging (HSI) is a reliable and safe imaging method for taking intraoperative perfusion measurements. This is the first study translating intraoperative HSI to an in vivo laparoscopic setting using a CE-certified HSI-system for minimally invasive surgery (HSI-MIS). We aim to compare it to an established HSI-system for open surgery (HSI-Open). METHODS Intraoperative HSI was done using the HSI-MIS and HSI-Open at the Region of Interest (ROI). 19 patients undergoing gastrointestinal resections were analyzed in this study. The HSI-MIS-acquired images were aligned with those from the HSI-Open, and spectra and parameter images were compared pixel-wise. We calculated the Mean Absolute Error (MAE) for Tissue Oxygen Saturation (StO2), Near-Infrared Perfusion Index (NIR-PI), Tissue Water Index (TWI), and Organ Hemoglobin Index (OHI), as well as the Root Mean Squared Error (RMSE) over the whole spectrum. Our analysis of parameters was optimized using partial least squares (PLS) regression. Two experienced surgeons carried out an additional color-change analysis, comparing the ROI images and deciding whether they provided the same (acceptable) or different visual information (rejected). RESULTS HSI and subsequent image registration was possible in 19 patients. MAE results for the original calculation were StO2 orig. 17.2% (± 7.7%), NIR-PIorig. 16.0 (± 9.5), TWIorig. 18.1 (± 7.9), OHIorig. 14.4 (± 4.5). For the PLS calculation, they were StO2 PLS 12.6% (± 5.2%), NIR-PIPLS 10.3 (± 6.0), TWIPLS 10.6 (± 5.1), and OHIPLS 11.6 (± 3.0). The RMSE between both systems was 0.14 (± 0.06). In the color-change analysis; both surgeons accepted more images generated using the PLS method. CONCLUSION Intraoperative HSI-MIS is a new technology and holds great potential for future applications in surgery. Parameter deviations are attributable to technical differences and can be reduced by applying improved calculation methods. This study is an important step toward the clinical implementation of HSI for minimally invasive surgery.
Collapse
|
20
|
Knoedler S, Hoch CC, Huelsboemer L, Knoedler L, Stögner VA, Pomahac B, Kauke-Navarro M, Colen D. Postoperative free flap monitoring in reconstructive surgery-man or machine? Front Surg 2023; 10:1130566. [PMID: 36911625 PMCID: PMC9992807 DOI: 10.3389/fsurg.2023.1130566] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 02/07/2023] [Indexed: 02/24/2023] Open
Abstract
Free tissue transfer is widely used for the reconstruction of complex tissue defects. The survival of free flaps depends on the patency and integrity of the microvascular anastomosis. Accordingly, the early detection of vascular comprise and prompt intervention are indispensable to increase flap survival rates. Such monitoring strategies are commonly integrated into the perioperative algorithm, with clinical examination still being considered the gold standard for routine free flap monitoring. Despite its widespread acceptance as state of the art, the clinical examination also has its pitfalls, such as the limited applicability in buried flaps and the risk of poor interrater agreement due to inconsistent flap (failure) appearances. To compensate for these shortcomings, a plethora of alternative monitoring tools have been proposed in recent years, each of them with inherent strengths and limitations. Given the ongoing demographic change, the number of older patients requiring free flap reconstruction, e.g., after cancer resection, is rising. Yet, age-related morphologic changes may complicate the free flap evaluation in elderly patients and delay the prompt detection of clinical signs of flap compromise. In this review, we provide an overview of currently available and employed methods for free flap monitoring, with a special focus on elderly patients and how senescence may impact standard free flap monitoring strategies.
Collapse
Affiliation(s)
- Samuel Knoedler
- Department of Plastic, Hand and Reconstructive Surgery, University Hospital Regensburg, Regensburg, Germany
- Department of Surgery, Division of Plastic Surgery, Yale School of Medicine, Yale New Haven Hospital,New Haven, CT, United States
- Correspondence: Samuel Knoedler Martin Kauke-Navarro
| | - Cosima C. Hoch
- Department of Otolaryngology, Head and Neck Surgery, Rechts der Isar Hospital, Technical University Munich, Munich, Germany
| | - Lioba Huelsboemer
- Department of Surgery, Division of Plastic Surgery, Yale School of Medicine, Yale New Haven Hospital,New Haven, CT, United States
| | - Leonard Knoedler
- Department of Surgery, Division of Plastic Surgery, Yale School of Medicine, Yale New Haven Hospital,New Haven, CT, United States
| | - Viola A. Stögner
- Department of Surgery, Division of Plastic Surgery, Yale School of Medicine, Yale New Haven Hospital,New Haven, CT, United States
| | - Bohdan Pomahac
- Department of Surgery, Division of Plastic Surgery, Yale School of Medicine, Yale New Haven Hospital,New Haven, CT, United States
| | - Martin Kauke-Navarro
- Department of Surgery, Division of Plastic Surgery, Yale School of Medicine, Yale New Haven Hospital,New Haven, CT, United States
- Correspondence: Samuel Knoedler Martin Kauke-Navarro
| | - David Colen
- Department of Surgery, Division of Plastic Surgery, Yale School of Medicine, Yale New Haven Hospital,New Haven, CT, United States
| |
Collapse
|
21
|
Cui R, Yu H, Xu T, Xing X, Cao X, Yan K, Chen J. Deep Learning in Medical Hyperspectral Images: A Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22249790. [PMID: 36560157 PMCID: PMC9784550 DOI: 10.3390/s22249790] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 12/11/2022] [Accepted: 12/12/2022] [Indexed: 06/13/2023]
Abstract
With the continuous progress of development, deep learning has made good progress in the analysis and recognition of images, which has also triggered some researchers to explore the area of combining deep learning with hyperspectral medical images and achieve some progress. This paper introduces the principles and techniques of hyperspectral imaging systems, summarizes the common medical hyperspectral imaging systems, and summarizes the progress of some emerging spectral imaging systems through analyzing the literature. In particular, this article introduces the more frequently used medical hyperspectral images and the pre-processing techniques of the spectra, and in other sections, it discusses the main developments of medical hyperspectral combined with deep learning for disease diagnosis. On the basis of the previous review, tne limited factors in the study on the application of deep learning to hyperspectral medical images are outlined, promising research directions are summarized, and the future research prospects are provided for subsequent scholars.
Collapse
Affiliation(s)
- Rong Cui
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - He Yu
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
- Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun 130022, China
| | - Tingfa Xu
- Image Engineering & Video Technology Lab, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Institute of Technology Chongqing Innovation Center, Chongqing 401120, China
| | - Xiaoxue Xing
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
- Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun 130022, China
| | - Xiaorui Cao
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - Kang Yan
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - Jiexi Chen
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| |
Collapse
|
22
|
Chalopin C, Nickel F, Pfahl A, Köhler H, Maktabi M, Thieme R, Sucher R, Jansen-Winkeln B, Studier-Fischer A, Seidlitz S, Maier-Hein L, Neumuth T, Melzer A, Müller-Stich BP, Gockel I. [Artificial intelligence and hyperspectral imaging for image-guided assistance in minimally invasive surgery]. CHIRURGIE (HEIDELBERG, GERMANY) 2022; 93:940-947. [PMID: 35798904 DOI: 10.1007/s00104-022-01677-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 06/08/2022] [Indexed: 01/18/2023]
Abstract
BACKGROUND Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article. OBJECTIVE What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery? MATERIAL AND METHODS Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors. RESULTS In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91. DISCUSSION This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland.
| | - Felix Nickel
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - René Thieme
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Robert Sucher
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Boris Jansen-Winkeln
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
- Abteilung für Allgemein‑, Viszeral- und Onkologische Chirurgie, Klinikum St. Georg Leipzig, Leipzig, Deutschland
| | - Alexander Studier-Fischer
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Beat Peter Müller-Stich
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Ines Gockel
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| |
Collapse
|