1
|
Livecchi TT, Jacques SL, Subhash HM, Pierce MC. Hyperspectral imaging with deep learning for quantification of tissue hemoglobin, melanin, and scattering. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:093507. [PMID: 39247058 PMCID: PMC11378079 DOI: 10.1117/1.jbo.29.9.093507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/04/2024] [Revised: 08/20/2024] [Accepted: 08/20/2024] [Indexed: 09/10/2024]
Abstract
Significance Hyperspectral cameras capture spectral information at each pixel in an image. Acquired spectra can be analyzed to estimate quantities of absorbing and scattering components, but the use of traditional fitting algorithms over megapixel images can be computationally intensive. Deep learning algorithms can be trained to rapidly analyze spectral data and can potentially process hyperspectral camera data in real time. Aim A hyperspectral camera was used to capture 1216 × 1936 pixel wide-field reflectance images of in vivo human tissue at 205 wavelength bands from 420 to 830 nm. Approach The optical properties of oxyhemoglobin, deoxyhemoglobin, melanin, and scattering were used with multi-layer Monte Carlo models to generate simulated diffuse reflectance spectra for 24,000 random combinations of physiologically relevant tissue components. These spectra were then used to train an artificial neural network (ANN) to predict tissue component concentrations from an input reflectance spectrum. Results The ANN achieved low root mean square errors in a test set of 6000 independent simulated diffuse reflectance spectra while calculating concentration values more than 4000× faster than a conventional iterative least squares approach. Conclusions In vivo finger occlusion and gingival abrasion studies demonstrate the ability of this approach to rapidly generate high-resolution images of tissue component concentrations from a hyperspectral dataset acquired from human subjects.
Collapse
Affiliation(s)
- Thomas T Livecchi
- Rutgers, The State University of New Jersey, Department of Biomedical Engineering, Piscataway, New Jersey, United States
- Colgate-Palmolive Company, Global Technology and Design Center, Piscataway, New Jersey, United States
| | - Steven L Jacques
- University of Washington, Department of Bioengineering, Seattle, Washington, United States
| | - Hrebesh M Subhash
- Colgate-Palmolive Company, Global Technology and Design Center, Piscataway, New Jersey, United States
| | - Mark C Pierce
- Rutgers, The State University of New Jersey, Department of Biomedical Engineering, Piscataway, New Jersey, United States
| |
Collapse
|
2
|
Kim HH, Song IS, Cha RJ. Advancing DIEP Flap Monitoring with Optical Imaging Techniques: A Narrative Review. SENSORS (BASEL, SWITZERLAND) 2024; 24:4457. [PMID: 39065854 PMCID: PMC11280549 DOI: 10.3390/s24144457] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/17/2024] [Revised: 07/05/2024] [Accepted: 07/08/2024] [Indexed: 07/28/2024]
Abstract
OBJECTIVES This review aims to explore recent advancements in optical imaging techniques for monitoring the viability of Deep Inferior Epigastric Perforator (DIEP) flap reconstruction. The objectives include highlighting the principles, applications, and clinical utility of optical imaging modalities such as near-infrared spectroscopy (NIRS), indocyanine green (ICG) fluorescence angiography, laser speckle contrast imaging (LSCI), hyperspectral imaging (HSI), dynamic infrared thermography (DIRT), and short-wave infrared thermography (SWIR) in assessing tissue perfusion and oxygenation. Additionally, this review aims to discuss the potential of these techniques in enhancing surgical outcomes by enabling timely intervention in cases of compromised flap perfusion. MATERIALS AND METHODS A comprehensive literature review was conducted to identify studies focusing on optical imaging techniques for monitoring DIEP flap viability. We searched PubMed, MEDLINE, and relevant databases, including Google Scholar, Web of Science, Scopus, PsycINFO, IEEE Xplore, and ProQuest Dissertations & Theses, among others, using specific keywords related to optical imaging, DIEP flap reconstruction, tissue perfusion, and surgical outcomes. This extensive search ensured we gathered comprehensive data for our analysis. Articles discussing the principles, applications, and clinical use of NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR in DIEP flap monitoring were selected for inclusion. Data regarding the techniques' effectiveness, advantages, limitations, and potential impact on surgical decision-making were extracted and synthesized. RESULTS Optical imaging modalities, including NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR offer a non- or minimal-invasive, real-time assessment of tissue perfusion and oxygenation in DIEP flap reconstruction. These techniques provide objective and quantitative data, enabling surgeons to monitor flap viability accurately. Studies have demonstrated the effectiveness of optical imaging in detecting compromised perfusion and facilitating timely intervention, thereby reducing the risk of flap complications such as partial or total loss. Furthermore, optical imaging modalities have shown promise in improving surgical outcomes by guiding intraoperative decision-making and optimizing patient care. CONCLUSIONS Recent advancements in optical imaging techniques present valuable tools for monitoring the viability of DIEP flap reconstruction. NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR offer a non- or minimal-invasive, real-time assessment of tissue perfusion and oxygenation, enabling accurate evaluation of flap viability. These modalities have the potential to enhance surgical outcomes by facilitating timely intervention in cases of compromised perfusion, thereby reducing the risk of flap complications. Incorporating optical imaging into clinical practice can provide surgeons with objective and quantitative data, assisting in informed decision-making for optimal patient care in DIEP flap reconstruction surgeries.
Collapse
Affiliation(s)
- Hailey Hwiram Kim
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC 20010, USA; (H.H.K.); (R.J.C.)
| | - In-Seok Song
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC 20010, USA; (H.H.K.); (R.J.C.)
- Department of Oral & Maxillofacial Surgery, Korea University Anam Hospital, 73 Goryeodae-ro, Seongbuk-gu, Seoul 02841, Republic of Korea
| | - Richard Jaepyeong Cha
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC 20010, USA; (H.H.K.); (R.J.C.)
- Department of Pediatrics, The George Washington University School of Medicine and Health Sciences, Washington, DC 20052, USA
| |
Collapse
|
3
|
Bannone E, Collins T, Esposito A, Cinelli L, De Pastena M, Pessaux P, Felli E, Andreotti E, Okamoto N, Barberio M, Felli E, Montorsi RM, Ingaglio N, Rodríguez-Luna MR, Nkusi R, Marescaux J, Hostettler A, Salvia R, Diana M. Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial. Surg Endosc 2024; 38:3758-3772. [PMID: 38789623 DOI: 10.1007/s00464-024-10880-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Accepted: 04/23/2024] [Indexed: 05/26/2024]
Abstract
BACKGROUND Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting. METHODS Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized. RESULTS A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%. CONCLUSION To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.
Collapse
Affiliation(s)
- Elisa Bannone
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France.
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy.
| | - Toby Collins
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | - Alessandro Esposito
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Lorenzo Cinelli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of Gastrointestinal Surgery, San Raffaele Hospital IRCCS, Milan, Italy
| | - Matteo De Pastena
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Patrick Pessaux
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, Strasbourg, France
- Institut of Viral and Liver Disease, Inserm U1110, University of Strasbourg, Strasbourg, France
| | - Emanuele Felli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, Strasbourg, France
- Institut of Viral and Liver Disease, Inserm U1110, University of Strasbourg, Strasbourg, France
| | - Elena Andreotti
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Nariaki Okamoto
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
| | - Manuel Barberio
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- General Surgery Department, Ospedale Cardinale G. Panico, Tricase, Italy
| | - Eric Felli
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Department of Visceral Surgery and Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Roberto Maria Montorsi
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Naomi Ingaglio
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - María Rita Rodríguez-Luna
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
| | - Richard Nkusi
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | - Jacque Marescaux
- Research Institute Against Digestive Cancer (IRCAD), 67000, Strasbourg, France
| | | | - Roberto Salvia
- Department of General and Pancreatic Surgery, The Pancreas Institute, University of Verona Hospital Trust, P.Le Scuro 10, 37134, Verona, Italy
| | - Michele Diana
- Photonics Instrumentation for Health, iCube Laboratory, University of Strasbourg, Strasbourg, France
- Department of Surgery, University Hospital of Geneva, Geneva, Switzerland
| |
Collapse
|
4
|
Ortega S, Quintana-Quintana L, Leon R, Fabelo H, Plaza MDLL, Camacho R, Callico GM. Histological Hyperspectral Glioblastoma Dataset (HistologyHSI-GB). Sci Data 2024; 11:681. [PMID: 38914542 PMCID: PMC11196658 DOI: 10.1038/s41597-024-03510-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Accepted: 06/10/2024] [Indexed: 06/26/2024] Open
Abstract
Hyperspectral (HS) imaging (HSI) technology combines the main features of two existing technologies: imaging and spectroscopy. This allows to analyse simultaneously the morphological and chemical attributes of the objects captured by a HS camera. In recent years, the use of HSI provides valuable insights into the interaction between light and biological tissues, and makes it possible to detect patterns, cells, or biomarkers, thus, being able to identify diseases. This work presents the HistologyHSI-GB dataset, which contains 469 HS images from 13 patients diagnosed with brain tumours, specifically glioblastoma. The slides were stained with haematoxylin and eosin (H&E) and captured using a microscope at 20× power magnification. Skilled histopathologists diagnosed the slides and provided image-level annotations. The dataset was acquired using custom HSI instrumentation, consisting of a microscope equipped with an HS camera covering the spectral range from 400 to 1000 nm.
Collapse
Affiliation(s)
- Samuel Ortega
- Seafood Industry Department, Norwegian Institute of Food, Fisheries and Aquaculture Research (Nofima), Tromsø, Norway.
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain.
- Department of Mathematics and Statistics, UiT The Arctic University of Norway, Tromsø, Norway.
| | - Laura Quintana-Quintana
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Raquel Leon
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Himar Fabelo
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
- Fundación Canaria Instituto de Investigación Sanitaria de Canarias (FIISC), Las Palmas de Gran Canaria, Spain
- Research Unit, Hospital Universitario de Gran Canaria Doctor Negrín, Las Palmas de Gran Canaria, Spain
| | - María de la Luz Plaza
- Department of Pathological Anatomy, Hospital Universitario de Gran Canaria Doctor Negrín, Las Palmas de Gran Canaria, Spain
| | - Rafael Camacho
- Department of Pathological Anatomy, Hospital Universitario de Gran Canaria Doctor Negrín, Las Palmas de Gran Canaria, Spain
| | - Gustavo M Callico
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| |
Collapse
|
5
|
Burström G, Amini M, El-Hajj VG, Arfan A, Gharios M, Buwaider A, Losch MS, Manni F, Edström E, Elmi-Terander A. Optical Methods for Brain Tumor Detection: A Systematic Review. J Clin Med 2024; 13:2676. [PMID: 38731204 PMCID: PMC11084501 DOI: 10.3390/jcm13092676] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Revised: 04/28/2024] [Accepted: 04/30/2024] [Indexed: 05/13/2024] Open
Abstract
Background: In brain tumor surgery, maximal tumor resection is typically desired. This is complicated by infiltrative tumor cells which cannot be visually distinguished from healthy brain tissue. Optical methods are an emerging field that can potentially revolutionize brain tumor surgery through intraoperative differentiation between healthy and tumor tissues. Methods: This study aimed to systematically explore and summarize the existing literature on the use of Raman Spectroscopy (RS), Hyperspectral Imaging (HSI), Optical Coherence Tomography (OCT), and Diffuse Reflectance Spectroscopy (DRS) for brain tumor detection. MEDLINE, Embase, and Web of Science were searched for studies evaluating the accuracy of these systems for brain tumor detection. Outcome measures included accuracy, sensitivity, and specificity. Results: In total, 44 studies were included, covering a range of tumor types and technologies. Accuracy metrics in the studies ranged between 54 and 100% for RS, 69 and 99% for HSI, 82 and 99% for OCT, and 42 and 100% for DRS. Conclusions: This review provides insightful evidence on the use of optical methods in distinguishing tumor from healthy brain tissue.
Collapse
Affiliation(s)
- Gustav Burström
- Department of Clinical Neuroscience, Karolinska Institute, 171 77 Stockholm, Sweden; (G.B.); (M.A.); (A.A.); (M.G.); (A.B.); (E.E.)
| | - Misha Amini
- Department of Clinical Neuroscience, Karolinska Institute, 171 77 Stockholm, Sweden; (G.B.); (M.A.); (A.A.); (M.G.); (A.B.); (E.E.)
| | - Victor Gabriel El-Hajj
- Department of Clinical Neuroscience, Karolinska Institute, 171 77 Stockholm, Sweden; (G.B.); (M.A.); (A.A.); (M.G.); (A.B.); (E.E.)
| | - Arooj Arfan
- Department of Clinical Neuroscience, Karolinska Institute, 171 77 Stockholm, Sweden; (G.B.); (M.A.); (A.A.); (M.G.); (A.B.); (E.E.)
| | - Maria Gharios
- Department of Clinical Neuroscience, Karolinska Institute, 171 77 Stockholm, Sweden; (G.B.); (M.A.); (A.A.); (M.G.); (A.B.); (E.E.)
| | - Ali Buwaider
- Department of Clinical Neuroscience, Karolinska Institute, 171 77 Stockholm, Sweden; (G.B.); (M.A.); (A.A.); (M.G.); (A.B.); (E.E.)
| | - Merle S. Losch
- Department of Biomechanical Engineering, Faculty of Mechanical Engineering, Delft University of Technology, 2627 Delft, The Netherlands
| | - Francesca Manni
- Department of Electrical Engineering, Eindhoven University of Technology (TU/e), 5612 Eindhoven, The Netherlands;
| | - Erik Edström
- Department of Clinical Neuroscience, Karolinska Institute, 171 77 Stockholm, Sweden; (G.B.); (M.A.); (A.A.); (M.G.); (A.B.); (E.E.)
- Capio Spine Center Stockholm, Löwenströmska Hospital, 194 80 Upplands-Väsby, Sweden
- Department of Medical Sciences, Örebro University, 701 85 Örebro, Sweden
| | - Adrian Elmi-Terander
- Department of Clinical Neuroscience, Karolinska Institute, 171 77 Stockholm, Sweden; (G.B.); (M.A.); (A.A.); (M.G.); (A.B.); (E.E.)
- Capio Spine Center Stockholm, Löwenströmska Hospital, 194 80 Upplands-Väsby, Sweden
- Department of Medical Sciences, Örebro University, 701 85 Örebro, Sweden
- Department of Surgical Sciences, Uppsala University, 751 35 Uppsala, Sweden
| |
Collapse
|
6
|
Egen L, Demmel GS, Grilli M, Studier-Fischer A, Nickel F, Haney CM, Mühlbauer J, Hartung FO, Menold HS, Piazza P, Rivas JG, Checcucci E, Puliatti S, Belenchon IR, Taratkin M, Rodler S, Cacciamani G, Michel MS, Kowalewski KF. Biophotonics-Intraoperative Guidance During Partial Nephrectomy: A Systematic Review and Meta-analysis. Eur Urol Focus 2024; 10:248-258. [PMID: 38278713 DOI: 10.1016/j.euf.2024.01.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Revised: 12/11/2023] [Accepted: 01/15/2024] [Indexed: 01/28/2024]
Abstract
CONTEXT Partial nephrectomy (PN) with intraoperative guidance by biophotonics has the potential to improve surgical outcomes due to higher precision. However, its value remains unclear since high-level evidence is lacking. OBJECTIVE To provide a comprehensive analysis of biophotonic techniques used for intraoperative real-time assistance during PN. EVIDENCE ACQUISITION We performed a comprehensive database search based on the PICO criteria, including studies published before October 2022. Two independent reviewers screened the titles and abstracts followed by full-text screening of eligible studies. For a quantitative analysis, a meta-analysis was conducted. EVIDENCE SYNTHESIS In total, 35 studies were identified for the qualitative analysis, including 27 studies on near-infrared fluorescence (NIRF) imaging using indocyanine green, four studies on hyperspectral imaging, two studies on folate-targeted molecular imaging, and one study each on optical coherence tomography and 5-aminolevulinic acid. The meta-analysis investigated seven studies on selective arterial clamping using NIRF. There was a significantly shorter warm ischemia time in the NIRF-PN group (mean difference [MD]: -2.9; 95% confidence interval [CI]: -5.6, -0.1; p = 0.04). No differences were noted regarding transfusions (odds ratio [OR]: 0.5; 95% CI: 0.2, 1.7; p = 0.27), positive surgical margins (OR: 0.7; 95% CI: 0.2, 2.0; p = 0.46), or major complications (OR: 0.4; 95% CI: 0.1, 1.2; p = 0.08). In the NIRF-PN group, functional results were favorable at short-term follow-up (MD of glomerular filtration rate decline: 7.6; 95% CI: 4.6, 10.5; p < 0.01), but leveled off at long-term follow-up (MD: 7.0; 95% CI: -2.8, 16.9; p = 0.16). Remarkably, these findings were not confirmed by the included randomized controlled trial. CONCLUSIONS Biophotonics comprises a heterogeneous group of imaging modalities that serve intraoperative decision-making and guidance. Implementation into clinical practice and cost effectiveness are the limitations that should be addressed by future research. PATIENT SUMMARY We reviewed the application of biophotonics during partial removal of the kidney in patients with kidney cancer. Our results suggest that these techniques support the surgeon in successfully performing the challenging steps of the procedure.
Collapse
Affiliation(s)
- Luisa Egen
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany.
| | - Greta S Demmel
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Maurizio Grilli
- Library of the Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Alexander Studier-Fischer
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Felix Nickel
- Department of General, Visceral, and Thoracic Surgery, University Hospital Hamburg-Eppendorf, Hamburg, Germany
| | - Caelan M Haney
- Department of Urology, University Hospital Leipzig, Leipzig, Germany
| | - Julia Mühlbauer
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Friedrich O Hartung
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Hanna S Menold
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Pietro Piazza
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Division of Urology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Juan Gomez Rivas
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Department of Urology, Hospital Clinico San Carlos, Madrid, Spain
| | - Enrico Checcucci
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Department of Surgery, FPO-IRCCS Candiolo Cancer Institute, Turin, Italy
| | - Stefano Puliatti
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Department of Urology, University of Modena, and Reggio Emilia, Modena, Italy
| | - Ines Rivero Belenchon
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Urology and Nephrology Department, Virgen del Rocío University Hospital, Seville, Spain
| | - Mark Taratkin
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands
| | - Severin Rodler
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; Department of Urology, University Hospital LMU Munich, Munich, Germany
| | - Giovanni Cacciamani
- Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands; USC Institute of Urology, University of Southern California, Los Angeles, CA, USA
| | - Maurice S Michel
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany
| | - Karl-Friedrich Kowalewski
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim at Heidelberg University, Mannheim, Germany; Association of Urology Young Academic Urologist-Urotechnology Working Party, Arnhem, The Netherlands
| |
Collapse
|
7
|
Perkov S, Vorobev V, Kurochkin MA, Gorodkov S, Gorin D. Rapid low-cost hyperspectral imaging system for quantitative assessment of infantile hemangioma. JOURNAL OF BIOPHOTONICS 2024; 17:e202300375. [PMID: 38009761 DOI: 10.1002/jbio.202300375] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 10/29/2023] [Accepted: 11/05/2023] [Indexed: 11/29/2023]
Abstract
Hemangioma, the predominant benign tumor occurring in infancy, exhibits a wide range of prognoses and associated outcomes. The accurate determination of prognosis through noninvasive imaging modalities holds essential importance in enabling effective personalized treatment strategies and minimizing unnecessary surgical interventions for individual patients. The present study focuses on advancing the personalized prognosis of hemangioma by leveraging noninvasive optical sensing technologies by the development of a novel rapid hyperspectral sensor (image collection in 5 s, lateral resolution of 10 μm) that is capable of quantifying hemoglobin oxygenation and vascularization dynamics during the course of tumor evolution. We have developed a quantitative parameter for hemangioma assessment, that demonstrated agreement with the clinician's conclusion in 90% among all cases during clinical studies on six patients, who visited clinician from two to four times. The presented methodology has potential to be implemented as a supportive tool for accurate hemangioma diagnostics in clinics.
Collapse
Affiliation(s)
- Sergei Perkov
- Center for Photonic Science and Engineering, Skolkovo Institute of Science and Technology, Moscow, Russia
| | - Viktor Vorobev
- Center for Photonic Science and Engineering, Skolkovo Institute of Science and Technology, Moscow, Russia
| | - Maxim A Kurochkin
- Center for Photonic Science and Engineering, Skolkovo Institute of Science and Technology, Moscow, Russia
| | - Sergey Gorodkov
- Center for Photonic Science and Engineering, Skolkovo Institute of Science and Technology, Moscow, Russia
- Faculty of Pediatrics, Saratov State Medical University, Saratov, Russia
| | - Dmitry Gorin
- Center for Photonic Science and Engineering, Skolkovo Institute of Science and Technology, Moscow, Russia
| |
Collapse
|
8
|
De Winne J, Strumane A, Babin D, Luthman S, Luong H, Philips W. Multispectral indices for real-time and non-invasive tissue ischemia monitoring using snapshot cameras. BIOMEDICAL OPTICS EXPRESS 2024; 15:641-655. [PMID: 38404312 PMCID: PMC10890856 DOI: 10.1364/boe.506084] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Revised: 11/14/2023] [Accepted: 11/14/2023] [Indexed: 02/27/2024]
Abstract
An adequate supply of oxygen-rich blood is vital to maintain cell homeostasis, cellular metabolism, and overall tissue health. While classical methods of measuring tissue ischemia are often invasive, localized and require skin contact or contrast agents, spectral imaging shows promise as a non-invasive, wide field, and contrast-free approach. We evaluate three novel reflectance-based spectral indices from the 460 - 840 nm spectral range. With the aim of enabling real time visualization of tissue ischemia, information is extracted from only 2-3 spectral bands. Video-rate spectral data was acquired from arm occlusion experiments in 27 healthy volunteers. The performance of the indices was evaluated against binary Support Vector Machine (SVM) classification of healthy versus ischemic skin tissue, two other indices from literature, and tissue oxygenation estimated using spectral unmixing. Robustness was tested by evaluating these under various lighting conditions and on both the dorsal and palmar sides of the hand. A novel index with real-time capabilities using reflectance information only from 547 nm and 556 nm achieves an average classification accuracy of 88.48, compared to 92.65 using an SVM trained on all available wavelengths. Furthermore, the index has a higher accuracy compared to reference methods and its time dynamics compare well against the expected clinical responses. This holds promise for robust real-time detection of tissue ischemia, possibly contributing to improved patient care and clinical outcomes.
Collapse
Affiliation(s)
- Jens De Winne
- Department of Telecommunications and Information Processing (TELIN) - PI Research Group, Ghent University-imec, 9000 Ghent, Belgium
- Interuniversity Micro-Electronics Center (IMEC) vzw, 3000 Leuven, Belgium
| | - Anoek Strumane
- Department of Telecommunications and Information Processing (TELIN) - PI Research Group, Ghent University-imec, 9000 Ghent, Belgium
| | - Danilo Babin
- Department of Telecommunications and Information Processing (TELIN) - PI Research Group, Ghent University-imec, 9000 Ghent, Belgium
| | - Siri Luthman
- Interuniversity Micro-Electronics Center (IMEC) vzw, 3000 Leuven, Belgium
| | - Hiep Luong
- Department of Telecommunications and Information Processing (TELIN) - PI Research Group, Ghent University-imec, 9000 Ghent, Belgium
| | - Wilfried Philips
- Department of Telecommunications and Information Processing (TELIN) - PI Research Group, Ghent University-imec, 9000 Ghent, Belgium
| |
Collapse
|
9
|
Chalopin C, Pfahl A, Köhler H, Knospe L, Maktabi M, Unger M, Jansen-Winkeln B, Thieme R, Moulla Y, Mehdorn M, Sucher R, Neumuth T, Gockel I, Melzer A. Alternative intraoperative optical imaging modalities for fluorescence angiography in gastrointestinal surgery: spectral imaging and imaging photoplethysmography. MINIM INVASIV THER 2023; 32:222-232. [PMID: 36622288 DOI: 10.1080/13645706.2022.2164469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 11/29/2022] [Indexed: 01/10/2023]
Abstract
INTRODUCTION Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA. MATERIAL AND METHODS The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works. RESULTS The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method. CONCLUSIONS Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Luise Knospe
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Department of Electrical, Mechanical and Industrial Engineering, Anhalt University of Applied Science, Köthen (Anhalt), Germany
| | - Michael Unger
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
- Department of General, Visceral and Oncological Surgery, St. Georg Hospital, Leipzig, Germany
| | - René Thieme
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Matthias Mehdorn
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Robert Sucher
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Institute of Medical Science and Technology (IMSAT), University of Dundee, Dundee, UK
| |
Collapse
|
10
|
MacCormac O, Noonan P, Janatka M, Horgan CC, Bahl A, Qiu J, Elliot M, Trotouin T, Jacobs J, Patel S, Bergholt MS, Ashkan K, Ourselin S, Ebner M, Vercauteren T, Shapey J. Lightfield hyperspectral imaging in neuro-oncology surgery: an IDEAL 0 and 1 study. Front Neurosci 2023; 17:1239764. [PMID: 37790587 PMCID: PMC10544348 DOI: 10.3389/fnins.2023.1239764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Accepted: 08/31/2023] [Indexed: 10/05/2023] Open
Abstract
Introduction Hyperspectral imaging (HSI) has shown promise in the field of intra-operative imaging and tissue differentiation as it carries the capability to provide real-time information invisible to the naked eye whilst remaining label free. Previous iterations of intra-operative HSI systems have shown limitations, either due to carrying a large footprint limiting ease of use within the confines of a neurosurgical theater environment, having a slow image acquisition time, or by compromising spatial/spectral resolution in favor of improvements to the surgical workflow. Lightfield hyperspectral imaging is a novel technique that has the potential to facilitate video rate image acquisition whilst maintaining a high spectral resolution. Our pre-clinical and first-in-human studies (IDEAL 0 and 1, respectively) demonstrate the necessary steps leading to the first in-vivo use of a real-time lightfield hyperspectral system in neuro-oncology surgery. Methods A lightfield hyperspectral camera (Cubert Ultris ×50) was integrated in a bespoke imaging system setup so that it could be safely adopted into the open neurosurgical workflow whilst maintaining sterility. Our system allowed the surgeon to capture in-vivo hyperspectral data (155 bands, 350-1,000 nm) at 1.5 Hz. Following successful implementation in a pre-clinical setup (IDEAL 0), our system was evaluated during brain tumor surgery in a single patient to remove a posterior fossa meningioma (IDEAL 1). Feedback from the theater team was analyzed and incorporated in a follow-up design aimed at implementing an IDEAL 2a study. Results Focusing on our IDEAL 1 study results, hyperspectral information was acquired from the cerebellum and associated meningioma with minimal disruption to the neurosurgical workflow. To the best of our knowledge, this is the first demonstration of HSI acquisition with 100+ spectral bands at a frame rate over 1Hz in surgery. Discussion This work demonstrated that a lightfield hyperspectral imaging system not only meets the design criteria and specifications outlined in an IDEAL-0 (pre-clinical) study, but also that it can translate into clinical practice as illustrated by a successful first in human study (IDEAL 1). This opens doors for further development and optimisation, given the increasing evidence that hyperspectral imaging can provide live, wide-field, and label-free intra-operative imaging and tissue differentiation.
Collapse
Affiliation(s)
- Oscar MacCormac
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
| | - Philip Noonan
- Hypervision Surgical Limited, London, United Kingdom
| | - Mirek Janatka
- Hypervision Surgical Limited, London, United Kingdom
| | | | - Anisha Bahl
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
| | - Jianrong Qiu
- School of Craniofacial and Regenerative Biology, King's College London, London, United Kingdom
| | - Matthew Elliot
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
| | - Théo Trotouin
- Hypervision Surgical Limited, London, United Kingdom
| | - Jaco Jacobs
- Hypervision Surgical Limited, London, United Kingdom
| | - Sabina Patel
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
| | - Mads S. Bergholt
- School of Craniofacial and Regenerative Biology, King's College London, London, United Kingdom
| | - Keyoumars Ashkan
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
| | - Sebastien Ourselin
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Hypervision Surgical Limited, London, United Kingdom
| | - Michael Ebner
- Hypervision Surgical Limited, London, United Kingdom
| | - Tom Vercauteren
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Hypervision Surgical Limited, London, United Kingdom
| | - Jonathan Shapey
- School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
- Department of Neurosurgery, King's College Hospital, London, United Kingdom
- Hypervision Surgical Limited, London, United Kingdom
| |
Collapse
|
11
|
Khan DZ, Hanrahan JG, Baldeweg SE, Dorward NL, Stoyanov D, Marcus HJ. Current and Future Advances in Surgical Therapy for Pituitary Adenoma. Endocr Rev 2023; 44:947-959. [PMID: 37207359 PMCID: PMC10502574 DOI: 10.1210/endrev/bnad014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 03/14/2023] [Accepted: 05/17/2023] [Indexed: 05/21/2023]
Abstract
The vital physiological role of the pituitary gland, alongside its proximity to critical neurovascular structures, means that pituitary adenomas can cause significant morbidity or mortality. While enormous advancements have been made in the surgical care of pituitary adenomas, numerous challenges remain, such as treatment failure and recurrence. To meet these clinical challenges, there has been an enormous expansion of novel medical technologies (eg, endoscopy, advanced imaging, artificial intelligence). These innovations have the potential to benefit each step of the patient's journey, and ultimately, drive improved outcomes. Earlier and more accurate diagnosis addresses this in part. Analysis of novel patient data sets, such as automated facial analysis or natural language processing of medical records holds potential in achieving an earlier diagnosis. After diagnosis, treatment decision-making and planning will benefit from radiomics and multimodal machine learning models. Surgical safety and effectiveness will be transformed by smart simulation methods for trainees. Next-generation imaging techniques and augmented reality will enhance surgical planning and intraoperative navigation. Similarly, surgical abilities will be augmented by the future operative armamentarium, including advanced optical devices, smart instruments, and surgical robotics. Intraoperative support to surgical team members will benefit from a data science approach, utilizing machine learning analysis of operative videos to improve patient safety and orientate team members to a common workflow. Postoperatively, neural networks leveraging multimodal datasets will allow early detection of individuals at risk of complications and assist in the prediction of treatment failure, thus supporting patient-specific discharge and monitoring protocols. While these advancements in pituitary surgery hold promise to enhance the quality of care, clinicians must be the gatekeepers of the translation of such technologies, ensuring systematic assessment of risk and benefit prior to clinical implementation. In doing so, the synergy between these innovations can be leveraged to drive improved outcomes for patients of the future.
Collapse
Affiliation(s)
- Danyal Z Khan
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London WC1N 3BG, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London W1W 7TY, UK
| | - John G Hanrahan
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London WC1N 3BG, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London W1W 7TY, UK
| | - Stephanie E Baldeweg
- Department of Diabetes & Endocrinology, University College London Hospitals NHS Foundation Trust, London NW1 2BU, UK
- Centre for Obesity and Metabolism, Department of Experimental and Translational Medicine, Division of Medicine, University College London, London WC1E 6BT, UK
| | - Neil L Dorward
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London WC1N 3BG, UK
| | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London W1W 7TY, UK
- Digital Surgery Ltd, Medtronic, London WD18 8WW, UK
| | - Hani J Marcus
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London WC1N 3BG, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London W1W 7TY, UK
| |
Collapse
|
12
|
Renna MS, Grzeda MT, Bailey J, Hainsworth A, Ourselin S, Ebner M, Vercauteren T, Schizas A, Shapey J. Intraoperative bowel perfusion assessment methods and their effects on anastomotic leak rates: meta-analysis. Br J Surg 2023; 110:1131-1142. [PMID: 37253021 PMCID: PMC10416696 DOI: 10.1093/bjs/znad154] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 03/24/2023] [Accepted: 04/29/2023] [Indexed: 06/01/2023]
Abstract
BACKGROUND Anastomotic leak is one of the most feared complications of colorectal surgery, and probably linked to poor blood supply to the anastomotic site. Several technologies have been described for intraoperative assessment of bowel perfusion. This systematic review and meta-analysis aimed to evaluate the most frequently used bowel perfusion assessment modalities in elective colorectal procedures, and to assess their associated risk of anastomotic leak. Technologies included indocyanine green fluorescence angiography, diffuse reflectance spectroscopy, laser speckle contrast imaging, and hyperspectral imaging. METHODS The review was preregistered with PROSPERO (CRD42021297299). A comprehensive literature search was performed using Embase, MEDLINE, Cochrane Library, Scopus, and Web of Science. The final search was undertaken on 29 July 2022. Data were extracted by two reviewers and the MINORS criteria were applied to assess the risk of bias. RESULTS Some 66 eligible studies involving 11 560 participants were included. Indocyanine green fluorescence angiography was most used with 10 789 participants, followed by diffuse reflectance spectroscopy with 321, hyperspectral imaging with 265, and laser speckle contrast imaging with 185. In the meta-analysis, the total pooled effect of an intervention on anastomotic leak was 0.05 (95 per cent c.i. 0.04 to 0.07) in comparison with 0.10 (0.08 to 0.12) without. Use of indocyanine green fluorescence angiography, hyperspectral imaging, or laser speckle contrast imaging was associated with a significant reduction in anastomotic leak. CONCLUSION Bowel perfusion assessment reduced the incidence of anastomotic leak, with intraoperative indocyanine green fluorescence angiography, hyperspectral imaging, and laser speckle contrast imaging all demonstrating comparable results.
Collapse
Affiliation(s)
- Maxwell S Renna
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
- Department of General Surgery, Guy’s and St Thomas’ NHS Foundation Trust, London, UK
| | - Mariusz T Grzeda
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
| | - James Bailey
- Department of General Surgery, University of Nottingham, Nottingham, UK
| | - Alison Hainsworth
- Department of General Surgery, Guy’s and St Thomas’ NHS Foundation Trust, London, UK
| | - Sebastien Ourselin
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | | | - Tom Vercauteren
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | - Alexis Schizas
- Department of General Surgery, Guy’s and St Thomas’ NHS Foundation Trust, London, UK
| | - Jonathan Shapey
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
- Department of Neurosurgery, King’s College Hospital, London, UK
| |
Collapse
|
13
|
Bahl A, Horgan CC, Janatka M, MacCormac OJ, Noonan P, Xie Y, Qiu J, Cavalcanti N, Fürnstahl P, Ebner M, Bergholt MS, Shapey J, Vercauteren T. Synthetic white balancing for intra-operative hyperspectral imaging. J Med Imaging (Bellingham) 2023; 10:046001. [PMID: 37492187 PMCID: PMC10363486 DOI: 10.1117/1.jmi.10.4.046001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 06/23/2023] [Accepted: 07/05/2023] [Indexed: 07/27/2023] Open
Abstract
Purpose Hyperspectral imaging shows promise for surgical applications to non-invasively provide spatially resolved, spectral information. For calibration purposes, a white reference image of a highly reflective Lambertian surface should be obtained under the same imaging conditions. Standard white references are not sterilizable and so are unsuitable for surgical environments. We demonstrate the necessity for in situ white references and address this by proposing a novel, sterile, synthetic reference construction algorithm. Approach The use of references obtained at different distances and lighting conditions to the subject were examined. Spectral and color reconstructions were compared with standard measurements qualitatively and quantitatively, using Δ E and normalized RMSE, respectively. The algorithm forms a composite image from a video of a standard sterile ruler, whose imperfect reflectivity is compensated for. The reference is modeled as the product of independent spatial and spectral components, and a scalar factor accounting for gain, exposure, and light intensity. Evaluation of synthetic references against ideal but non-sterile references is performed using the same metrics alongside pixel-by-pixel errors. Finally, intraoperative integration is assessed though cadaveric experiments. Results Improper white balancing leads to increases in all quantitative and qualitative errors. Synthetic references achieve median pixel-by-pixel errors lower than 6.5% and produce similar reconstructions and errors to an ideal reference. The algorithm integrated well into surgical workflow, achieving median pixel-by-pixel errors of 4.77% while maintaining good spectral and color reconstruction. Conclusions We demonstrate the importance of in situ white referencing and present a novel synthetic referencing algorithm. This algorithm is suitable for surgery while maintaining the quality of classical data reconstruction.
Collapse
Affiliation(s)
- Anisha Bahl
- King’s College London, School of Biomedical Engineering and Imaging Sciences, London, United Kingdom
| | - Conor C. Horgan
- King’s College London, School of Biomedical Engineering and Imaging Sciences, London, United Kingdom
- Hypervision Surgical Ltd., London, United Kingdom
| | | | - Oscar J. MacCormac
- King’s College London, School of Biomedical Engineering and Imaging Sciences, London, United Kingdom
- King’s College Hospital, Denmark Hill, London, United Kingdom
| | | | - Yijing Xie
- King’s College London, School of Biomedical Engineering and Imaging Sciences, London, United Kingdom
| | - Jianrong Qiu
- King’s College London, School of Biomedical Engineering and Imaging Sciences, London, United Kingdom
- King’s College London, Centre for Craniofacial and Regenerative Biology, London, United Kingdom
| | | | | | | | - Mads S. Bergholt
- King’s College London, Centre for Craniofacial and Regenerative Biology, London, United Kingdom
| | - Jonathan Shapey
- King’s College London, School of Biomedical Engineering and Imaging Sciences, London, United Kingdom
- Hypervision Surgical Ltd., London, United Kingdom
- King’s College Hospital, Denmark Hill, London, United Kingdom
| | - Tom Vercauteren
- King’s College London, School of Biomedical Engineering and Imaging Sciences, London, United Kingdom
- Hypervision Surgical Ltd., London, United Kingdom
| |
Collapse
|
14
|
Li P, Asad M, Horgan C, MacCormac O, Shapey J, Vercauteren T. Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: application to surgical imaging. Int J Comput Assist Radiol Surg 2023; 18:981-988. [PMID: 36961613 PMCID: PMC10284955 DOI: 10.1007/s11548-023-02865-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Accepted: 03/03/2023] [Indexed: 03/25/2023]
Abstract
PURPOSE Hyperspectral imaging has the potential to improve intraoperative decision making if tissue characterisation is performed in real-time and with high-resolution. Hyperspectral snapshot mosaic sensors offer a promising approach due to their fast acquisition speed and compact size. However, a demosaicking algorithm is required to fully recover the spatial and spectral information of the snapshot images. Most state-of-the-art demosaicking algorithms require ground-truth training data with paired snapshot and high-resolution hyperspectral images, but such imagery pairs with the exact same scene are physically impossible to acquire in intraoperative settings. In this work, we present a fully unsupervised hyperspectral image demosaicking algorithm which only requires exemplar snapshot images for training purposes. METHODS We regard hyperspectral demosaicking as an ill-posed linear inverse problem which we solve using a deep neural network. We take advantage of the spectral correlation occurring in natural scenes to design a novel inter spectral band regularisation term based on spatial gradient consistency. By combining our proposed term with standard regularisation techniques and exploiting a standard data fidelity term, we obtain an unsupervised loss function for training deep neural networks, which allows us to achieve real-time hyperspectral image demosaicking. RESULTS Quantitative results on hyperspetral image datasets show that our unsupervised demosaicking approach can achieve similar performance to its supervised counter-part, and significantly outperform linear demosaicking. A qualitative user study on real snapshot hyperspectral surgical images confirms the results from the quantitative analysis. CONCLUSION Our results suggest that the proposed unsupervised algorithm can achieve promising hyperspectral demosaicking in real-time thus advancing the suitability of the modality for intraoperative use.
Collapse
Affiliation(s)
- Peichao Li
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.
| | - Muhammad Asad
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
| | - Conor Horgan
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
| | - Oscar MacCormac
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
- Department of Neurosurgery, King's College Hospital NHS Foundation Trust, London, UK
| | - Jonathan Shapey
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
- Department of Neurosurgery, King's College Hospital NHS Foundation Trust, London, UK
| | - Tom Vercauteren
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
| |
Collapse
|
15
|
Ayala L, Adler TJ, Seidlitz S, Wirkert S, Engels C, Seitel A, Sellner J, Aksenov A, Bodenbach M, Bader P, Baron S, Vemuri A, Wiesenfarth M, Schreck N, Mindroc D, Tizabi M, Pirmann S, Everitt B, Kopp-Schneider A, Teber D, Maier-Hein L. Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery. SCIENCE ADVANCES 2023; 9:eadd6778. [PMID: 36897951 PMCID: PMC10005169 DOI: 10.1126/sciadv.add6778] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Accepted: 02/03/2023] [Indexed: 06/18/2023]
Abstract
Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.
Collapse
Affiliation(s)
- Leonardo Ayala
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
| | - Tim J. Adler
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
| | - Sebastian Wirkert
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Alexander Seitel
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jan Sellner
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
| | | | | | - Pia Bader
- Städtisches Klinikum Karlsruhe, Karlsruhe, Germany
| | | | - Anant Vemuri
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Manuel Wiesenfarth
- Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Nicholas Schreck
- Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Diana Mindroc
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Minu Tizabi
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sebastian Pirmann
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Brittaney Everitt
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Dogu Teber
- Städtisches Klinikum Karlsruhe, Karlsruhe, Germany
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
| |
Collapse
|
16
|
Aloupogianni E, Ishikawa M, Ichimura T, Hamada M, Murakami T, Sasaki A, Nakamura K, Kobayashi N, Obi T. Effects of dimension reduction of hyperspectral images in skin gross pathology. Skin Res Technol 2023; 29:e13270. [PMID: 36823506 PMCID: PMC10155843 DOI: 10.1111/srt.13270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 12/17/2022] [Indexed: 02/05/2023]
Abstract
BACKGROUND Hyperspectral imaging (HSI) is an emerging modality for the gross pathology of the skin. Spectral signatures of HSI could discriminate malignant from benign tissue. Because of inherent redundancies in HSI and in order to facilitate the use of deep-learning models, dimension reduction is a common preprocessing step. The effects of dimension reduction choice, training scope, and number of retained dimensions have not been evaluated on skin HSI for segmentation tasks. MATERIALS AND METHODS An in-house dataset of HSI signatures from pigmented skin lesions was prepared and labeled with histology. Eleven different dimension reduction methods were used as preprocessing for tumor margin detection with support vector machines. Cluster-wise principal component analysis (ClusterPCA), a new variant of PCA, was proposed. The scope of application for dimension reduction was also investigated. RESULTS The components produced by ClusterPCA show good agreement with the expected optical properties of skin chromophores. Random forest importance performed best during classification. However, all methods suffered from low sensitivity and generalization. CONCLUSION Investigation of more complex reduction and segmentation schemes with emphasis on the nature of HSI and optical properties of the skin is necessary. Insights on dimension reduction for skin tissue could facilitate the development of HSI-based systems for cancer margin detection at gross level.
Collapse
Affiliation(s)
- Eleni Aloupogianni
- Department of Information and Communications EngineeringTokyo Institute of TechnologyYokohamaJapan
| | - Masahiro Ishikawa
- Faculty of Health and Medical CareSaitama Medical University Hidaka CampusHidakaJapan
| | - Takaya Ichimura
- Department of PathologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Mei Hamada
- Department of PathologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Takuo Murakami
- Department of DermatologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Atsushi Sasaki
- Department of PathologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Koichiro Nakamura
- Department of DermatologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Naoki Kobayashi
- Department of Information and Communications EngineeringTokyo Institute of TechnologyYokohamaJapan
| | - Takashi Obi
- Department of Information and Communications EngineeringTokyo Institute of TechnologyYokohamaJapan
- Institute of Innovative Research, Tokyo Institute of TechnologyTokyoJapan
| |
Collapse
|
17
|
Garcia Peraza Herrera LC, Horgan C, Ourselin S, Ebner M, Vercauteren T. Hyperspectral image segmentation: a preliminary study on the Oral and Dental Spectral Image Database (ODSI-DB). COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2023. [DOI: 10.1080/21681163.2022.2160377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Affiliation(s)
| | - Conor Horgan
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | | | - Michael Ebner
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | - Tom Vercauteren
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| |
Collapse
|
18
|
Liu GS, Shenson JA, Farrell JE, Blevins NH. Signal to noise ratio quantifies the contribution of spectral channels to classification of human head and neck tissues ex vivo using deep learning and multispectral imaging. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:016004. [PMID: 36726664 PMCID: PMC9884103 DOI: 10.1117/1.jbo.28.1.016004] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Accepted: 01/06/2023] [Indexed: 05/09/2023]
Abstract
SIGNIFICANCE Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems. AIM Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI. APPROACH MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods. RESULTS Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( p < 0.001 ). Removing spectral channels with SNR > 20 reduced tissue classification accuracy. CONCLUSIONS The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.
Collapse
Affiliation(s)
- George S. Liu
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
| | - Jared A. Shenson
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
| | - Joyce E. Farrell
- Stanford University, Department of Electrical Engineering, Stanford, California, United States
| | - Nikolas H. Blevins
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
- Address all correspondence to Nikolas H. Blevins,
| |
Collapse
|
19
|
Aloupogianni E, Ichimura T, Hamada M, Ishikawa M, Murakami T, Sasaki A, Nakamura K, Kobayashi N, Obi T. Hyperspectral imaging for tumor segmentation on pigmented skin lesions. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:106007. [PMID: 36316301 PMCID: PMC9619132 DOI: 10.1117/1.jbo.27.10.106007] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 10/10/2022] [Indexed: 06/16/2023]
Abstract
SIGNIFICANCE Malignant skin tumors, which include melanoma and nonmelanoma skin cancers, are the most prevalent type of malignant tumor. Gross pathology of pigmented skin lesions (PSL) remains manual, time-consuming, and heavily dependent on the expertise of the medical personnel. Hyperspectral imaging (HSI) can assist in the detection of tumors and evaluate the status of tumor margins by their spectral signatures. AIM Tumor segmentation of medical HSI data is a research field. The goal of this study is to propose a framework for HSI-based tumor segmentation of PSL. APPROACH An HSI dataset of 28 PSL was prepared. Two frameworks for data preprocessing and tumor segmentation were proposed. Models based on machine learning and deep learning were used at the core of each framework. RESULTS Cross-validation performance showed that pixel-wise processing achieves higher segmentation performance, in terms of the Jaccard coefficient. Simultaneous use of spatio-spectral features produced more comprehensive tumor masks. A three-dimensional Xception-based network achieved performance similar to state-of-the-art networks while allowing for more detailed detection of the tumor border. CONCLUSIONS Good performance was achieved for melanocytic lesions, but margins were difficult to detect in some cases of basal cell carcinoma. The frameworks proposed in this study could be further improved for robustness against different pathologies and detailed delineation of tissue margins to facilitate computer-assisted diagnosis during gross pathology.
Collapse
Affiliation(s)
- Eleni Aloupogianni
- Tokyo Institute of Technology, Department of Information and Communications Engineering, Meguro, Japan
| | - Takaya Ichimura
- Saitama Medical University Moroyama Campus, Department of Pathology, Faculty of Medicine, Iruma, Japan
| | - Mei Hamada
- Saitama Medical University Moroyama Campus, Department of Pathology, Faculty of Medicine, Iruma, Japan
| | - Masahiro Ishikawa
- Saitama Medical University Hidaka Campus, Faculty of Health and Medical Care, Hidaka, Japan
| | - Takuo Murakami
- Saitama Medical University Moroyama Campus, Department of Dermatology, Faculty of Medicine, Iruma, Japan
| | - Atsushi Sasaki
- Saitama Medical University Moroyama Campus, Department of Pathology, Faculty of Medicine, Iruma, Japan
| | - Koichiro Nakamura
- Saitama Medical University Moroyama Campus, Department of Dermatology, Faculty of Medicine, Iruma, Japan
| | - Naoki Kobayashi
- Saitama Medical University Hidaka Campus, Faculty of Health and Medical Care, Hidaka, Japan
| | - Takashi Obi
- Tokyo Institute of Technology, Department of Information and Communications Engineering, Meguro, Japan
- Tokyo Institute of Technology, Institute of Innovative Research, Yokohama, Japan
| |
Collapse
|
20
|
Köhler H, Pfahl A, Moulla Y, Thomaßen MT, Maktabi M, Gockel I, Neumuth T, Melzer A, Chalopin C. Comparison of image registration methods for combining laparoscopic video and spectral image data. Sci Rep 2022; 12:16459. [PMID: 36180520 PMCID: PMC9525266 DOI: 10.1038/s41598-022-20816-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 09/19/2022] [Indexed: 11/09/2022] Open
Abstract
Laparoscopic procedures can be assisted by intraoperative modalities, such as quantitative perfusion imaging based on fluorescence or hyperspectral data. If these modalities are not available at video frame rate, fast image registration is needed for the visualization in augmented reality. Three feature-based algorithms and one pre-trained deep homography neural network (DH-NN) were tested for single and multi-homography estimation. Fine-tuning was used to bridge the domain gap of the DH-NN for non-rigid registration of laparoscopic images. The methods were validated on two datasets: an open-source record of 750 manually annotated laparoscopic images, presented in this work, and in-vivo data from a novel laparoscopic hyperspectral imaging system. All feature-based single homography methods outperformed the fine-tuned DH-NN in terms of reprojection error, Structural Similarity Index Measure, and processing time. The feature detector and descriptor ORB1000 enabled video-rate registration of laparoscopic images on standard hardware with submillimeter accuracy.
Collapse
Affiliation(s)
- Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, 04103, Leipzig, Germany.
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, 04103, Leipzig, Germany
| | - Yusef Moulla
- Department of Visceral, Thoracic, Transplant, and Vascular Surgery, University Hospital of Leipzig, 04103, Leipzig, Germany
| | - Madeleine T Thomaßen
- Department of Visceral, Thoracic, Transplant, and Vascular Surgery, University Hospital of Leipzig, 04103, Leipzig, Germany
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, 04103, Leipzig, Germany
| | - Ines Gockel
- Department of Visceral, Thoracic, Transplant, and Vascular Surgery, University Hospital of Leipzig, 04103, Leipzig, Germany
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, 04103, Leipzig, Germany
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, 04103, Leipzig, Germany
| | - Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, 04103, Leipzig, Germany
| |
Collapse
|
21
|
Lew B, George M, Blair S, Zhu Z, Liang Z, Ludwig J, Kim CY, Kim KK, Gruev V, Choi H. Protease-activated indocyanine green nanoprobes for intraoperative NIR fluorescence imaging of primary tumors. NANOSCALE ADVANCES 2022; 4:4041-4050. [PMID: 36285222 PMCID: PMC9514568 DOI: 10.1039/d2na00276k] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Accepted: 06/25/2022] [Indexed: 05/17/2023]
Abstract
Tumor-targeted fluorescent probes in the near-infrared spectrum can provide invaluable information about the location and extent of primary and metastatic tumors during intraoperative procedures to ensure no residual tumors are left in the patient's body. Even though the first fluorescence-guided surgery was performed more than 50 years ago, it is still not accepted as a standard of care in part due to the lack of efficient and non-toxic targeted probes approved by regulatory agencies around the world. Herein, we report protease-activated cationic gelatin nanoparticles encapsulating indocyanine green (ICG) for the detection of primary breast tumors in murine models with high tumor-to-background ratios. Upon intravenous administration, these nanoprobes remain optically silent due to the energy resonance transfer among the bound ICG molecules. As the nanoprobes extravasate and are exposed to the acidic tumor microenvironment, their positive surface charges increase, facilitating cellular uptake. The internalized nanoprobes are activated upon proteolytic degradation of gelatin to allow high contrast between the tumor and normal tissue. Since both gelatin and ICG are FDA-approved for intravenous administration, this activatable nanoprobe can lead to quick clinical adoption and improve the treatment of patients undergoing image-guided cancer surgery.
Collapse
Affiliation(s)
- Benjamin Lew
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
| | - Mebin George
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
| | - Steven Blair
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
| | - Zhongmin Zhu
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
| | - Zuodong Liang
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
| | - Jamie Ludwig
- Division of Animal Resources, University of Illinois Urbana IL 61801 USA
| | - Celeste Y Kim
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
| | - Kyekyoon Kevin Kim
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
- Department of Bioengineering, University of Illinois Urbana IL 61801 USA
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana IL 61801 USA
| | - Viktor Gruev
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana IL 61801 USA
- Carle Illinois College of Medicine, University of Illinois Urbana IL 61801 USA
| | - Hyungsoo Choi
- Department of Electrical and Computer Engineering, University of Illinois Urbana IL 61801 USA
| |
Collapse
|
22
|
Massalimova A, Timmermans M, Esfandiari H, Carrillo F, Laux CJ, Farshad M, Denis K, Fürnstahl P. Intraoperative tissue classification methods in orthopedic and neurological surgeries: A systematic review. Front Surg 2022; 9:952539. [PMID: 35990097 PMCID: PMC9381957 DOI: 10.3389/fsurg.2022.952539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 07/19/2022] [Indexed: 11/13/2022] Open
Abstract
Accurate tissue differentiation during orthopedic and neurological surgeries is critical, given that such surgeries involve operations on or in the vicinity of vital neurovascular structures and erroneous surgical maneuvers can lead to surgical complications. By now, the number of emerging technologies tackling the problem of intraoperative tissue classification methods is increasing. Therefore, this systematic review paper intends to give a general overview of existing technologies. The review was done based on the PRISMA principle and two databases: PubMed and IEEE Xplore. The screening process resulted in 60 full-text papers. The general characteristics of the methodology from extracted papers included data processing pipeline, machine learning methods if applicable, types of tissues that can be identified with them, phantom used to conduct the experiment, and evaluation results. This paper can be useful in identifying the problems in the current status of the state-of-the-art intraoperative tissue classification methods and designing new enhanced techniques.
Collapse
Affiliation(s)
- Aidana Massalimova
- Research in Orthopedic Computer Science (ROCS), Balgrist Campus, University of Zurich, Zurich, Switzerland
- Correspondence: Aidana Massalimova
| | - Maikel Timmermans
- KU Leuven, Campus Group T, BioMechanics (BMe), Smart Instrumentation Group, Leuven, Belgium
| | - Hooman Esfandiari
- Research in Orthopedic Computer Science (ROCS), Balgrist Campus, University of Zurich, Zurich, Switzerland
| | - Fabio Carrillo
- Research in Orthopedic Computer Science (ROCS), Balgrist Campus, University of Zurich, Zurich, Switzerland
| | - Christoph J. Laux
- Department of Orthopaedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Mazda Farshad
- Department of Orthopaedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Kathleen Denis
- KU Leuven, Campus Group T, BioMechanics (BMe), Smart Instrumentation Group, Leuven, Belgium
| | - Philipp Fürnstahl
- Research in Orthopedic Computer Science (ROCS), Balgrist Campus, University of Zurich, Zurich, Switzerland
| |
Collapse
|
23
|
Chalopin C, Nickel F, Pfahl A, Köhler H, Maktabi M, Thieme R, Sucher R, Jansen-Winkeln B, Studier-Fischer A, Seidlitz S, Maier-Hein L, Neumuth T, Melzer A, Müller-Stich BP, Gockel I. [Artificial intelligence and hyperspectral imaging for image-guided assistance in minimally invasive surgery]. CHIRURGIE (HEIDELBERG, GERMANY) 2022; 93:940-947. [PMID: 35798904 DOI: 10.1007/s00104-022-01677-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 06/08/2022] [Indexed: 01/18/2023]
Abstract
BACKGROUND Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article. OBJECTIVE What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery? MATERIAL AND METHODS Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors. RESULTS In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91. DISCUSSION This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland.
| | - Felix Nickel
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - René Thieme
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Robert Sucher
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Boris Jansen-Winkeln
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
- Abteilung für Allgemein‑, Viszeral- und Onkologische Chirurgie, Klinikum St. Georg Leipzig, Leipzig, Deutschland
| | - Alexander Studier-Fischer
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Beat Peter Müller-Stich
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Ines Gockel
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| |
Collapse
|
24
|
Aloupogianni E, Ishikawa M, Kobayashi N, Obi T. Hyperspectral and multispectral image processing for gross-level tumor detection in skin lesions: a systematic review. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-220029VR. [PMID: 35676751 PMCID: PMC9174598 DOI: 10.1117/1.jbo.27.6.060901] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 05/23/2022] [Indexed: 05/11/2023]
Abstract
SIGNIFICANCE Skin cancer is one of the most prevalent cancers worldwide. In the advent of medical digitization and telepathology, hyper/multispectral imaging (HMSI) allows for noninvasive, nonionizing tissue evaluation at a macroscopic level. AIM We aim to summarize proposed frameworks and recent trends in HMSI-based classification and segmentation of gross-level skin tissue. APPROACH A systematic review was performed, targeting HMSI-based systems for the classification and segmentation of skin lesions during gross pathology, including melanoma, pigmented lesions, and bruises. The review adhered to the 2020 Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. For eligible reports published from 2010 to 2020, trends in HMSI acquisition, preprocessing, and analysis were identified. RESULTS HMSI-based frameworks for skin tissue classification and segmentation vary greatly. Most reports implemented simple image processing or machine learning, due to small training datasets. Methodologies were evaluated on heavily curated datasets, with the majority targeting melanoma detection. The choice of preprocessing scheme influenced the performance of the system. Some form of dimension reduction is commonly applied to avoid redundancies that are inherent in HMSI systems. CONCLUSIONS To use HMSI for tumor margin detection in practice, the focus of system evaluation should shift toward the explainability and robustness of the decision-making process.
Collapse
Affiliation(s)
- Eleni Aloupogianni
- Tokyo Institute of Technology, Department of Information and Communication Engineering, Tokyo, Japan
- Address all correspondence to Eleni Aloupogianni,
| | - Masahiro Ishikawa
- Saitama Medical University, Faculty of Health and Medical Care, Saitama, Japan
| | - Naoki Kobayashi
- Saitama Medical University, Faculty of Health and Medical Care, Saitama, Japan
| | - Takashi Obi
- Tokyo Institute of Technology, Department of Information and Communication Engineering, Tokyo, Japan
- Institute of Innovative Research, Tokyo Institute of Technology, Yokohama, Japan
| |
Collapse
|
25
|
Shapey J, Xie Y, Nabavi E, Ebner M, Saeed SR, Kitchen N, Dorward N, Grieve J, McEvoy AW, Miserocchi A, Grover P, Bradford R, Lim YM, Ourselin S, Brandner S, Jaunmuktane Z, Vercauteren T. Optical properties of human brain and tumour tissue: An ex vivo study spanning the visible range to beyond the second near-infrared window. JOURNAL OF BIOPHOTONICS 2022; 15:e202100072. [PMID: 35048541 DOI: 10.1002/jbio.202100072] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Revised: 12/15/2021] [Accepted: 12/16/2021] [Indexed: 06/14/2023]
Abstract
Neuro-oncology surgery would benefit from detailed intraoperative tissue characterization provided by noncontact, contrast-agent-free, noninvasive optical imaging methods. In-depth knowledge of target tissue optical properties across a wide-wavelength spectrum could inform the design of optical imaging and computational methods to enable robust tissue analysis during surgery. We adapted a dual-beam integrating sphere to analyse small tissue samples and investigated ex vivo optical properties of five types of human brain tumour (meningioma, pituitary adenoma, schwannoma, low- and high-grade glioma) and nine different types of healthy brain tissue across a wavelength spectrum of 400 to 1800 nm. Fresh and frozen tissue samples were analysed. All tissue types demonstrated similar absorption spectra, but the reduced scattering coefficients of tumours show visible differences in the obtained optical spectrum compared to those of surrounding normal tissue. These results underline the potential of optical imaging technologies for intraoperative tissue characterization.
Collapse
Affiliation(s)
- Jonathan Shapey
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Yijing Xie
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Elham Nabavi
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Michael Ebner
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Shakeel R Saeed
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
- The Ear Institute, University College London, London, UK
- The Royal National Throat, Nose and Ear Hospital, London, UK
| | - Neil Kitchen
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
| | - Neil Dorward
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
| | - Joan Grieve
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
| | - Andrew W McEvoy
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
| | - Anna Miserocchi
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
| | - Patrick Grover
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
| | - Robert Bradford
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
| | - Yau-Mun Lim
- Division of Neuropathology, UCL Queen Square Institute of Neurology, and The National Hospital for Neurology and Neurosurgery, University College Hospitals NHS Foundation Trust, London, UK
| | - Sebastien Ourselin
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Sebastian Brandner
- Division of Neuropathology, UCL Queen Square Institute of Neurology, and The National Hospital for Neurology and Neurosurgery, University College Hospitals NHS Foundation Trust, London, UK
| | - Zane Jaunmuktane
- Division of Neuropathology, UCL Queen Square Institute of Neurology, and The National Hospital for Neurology and Neurosurgery, University College Hospitals NHS Foundation Trust, London, UK
| | - Tom Vercauteren
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| |
Collapse
|
26
|
J Waterhouse D, Stoyanov D. Optimized spectral filter design enables more accurate estimation of oxygen saturation in spectral imaging. BIOMEDICAL OPTICS EXPRESS 2022; 13:2156-2173. [PMID: 35519287 PMCID: PMC9045927 DOI: 10.1364/boe.446975] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/28/2021] [Revised: 01/18/2022] [Accepted: 01/23/2022] [Indexed: 06/14/2023]
Abstract
Oxygen saturation (SO2) in tissue is a crucially important physiological parameter with ubiquitous clinical utility in diagnosis, treatment, and monitoring, as well as widespread use as an invaluable preclinical research tool. Multispectral imaging can be used to visualize SO2 non-invasively, non-destructively and without contact in real-time using narrow spectral filter sets, but typically, these spectral filter sets are poorly suited to a specific clinical task, application, or tissue type. In this work, we demonstrate the merit of optimizing spectral filter sets for more accurate estimation of SO2. Using tissue modelling and simulated multispectral imaging, we demonstrate filter optimization reduces the root-mean-square-error (RMSE) in estimating SO2 by up to 37% compared with evenly spaced filters. Moreover, we demonstrate up to a 79% decrease in RMSE for optimized filter sets compared with filter sets chosen to minimize mutual information. Wider adoption of this approach will result in more effective multispectral imaging systems that can address specific clinical needs and consequently, more widespread adoption of multispectral imaging technologies in disease diagnosis and treatment.
Collapse
Affiliation(s)
- Dale J Waterhouse
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, Department of Medical Physics and Biomedical Engineering, University College London, UK
| | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, Department of Medical Physics and Biomedical Engineering, University College London, UK
| |
Collapse
|
27
|
Lindholm V, Raita-Hakola AM, Annala L, Salmivuori M, Jeskanen L, Saari H, Koskenmies S, Pitkänen S, Pölönen I, Isoherranen K, Ranki A. Differentiating Malignant from Benign Pigmented or Non-Pigmented Skin Tumours-A Pilot Study on 3D Hyperspectral Imaging of Complex Skin Surfaces and Convolutional Neural Networks. J Clin Med 2022; 11:1914. [PMID: 35407522 PMCID: PMC8999463 DOI: 10.3390/jcm11071914] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2022] [Accepted: 03/28/2022] [Indexed: 02/08/2023] Open
Abstract
Several optical imaging techniques have been developed to ease the burden of skin cancer disease on our health care system. Hyperspectral images can be used to identify biological tissues by their diffuse reflected spectra. In this second part of a three-phase pilot study, we used a novel hand-held SICSURFIS Spectral Imager with an adaptable field of view and target-wise selectable wavelength channels to provide detailed spectral and spatial data for lesions on complex surfaces. The hyperspectral images (33 wavelengths, 477-891 nm) provided photometric data through individually controlled illumination modules, enabling convolutional networks to utilise spectral, spatial, and skin-surface models for the analyses. In total, 42 lesions were studied: 7 melanomas, 13 pigmented and 7 intradermal nevi, 10 basal cell carcinomas, and 5 squamous cell carcinomas. All lesions were excised for histological analyses. A pixel-wise analysis provided map-like images and classified pigmented lesions with a sensitivity of 87% and a specificity of 93%, and 79% and 91%, respectively, for non-pigmented lesions. A majority voting analysis, which provided the most probable lesion diagnosis, diagnosed 41 of 42 lesions correctly. This pilot study indicates that our non-invasive hyperspectral imaging system, which involves shape and depth data analysed by convolutional neural networks, is feasible for differentiating between malignant and benign pigmented and non-pigmented skin tumours, even on complex skin surfaces.
Collapse
Affiliation(s)
- Vivian Lindholm
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (M.S.); (L.J.); (S.K.); (S.P.); (K.I.); (A.R.)
| | - Anna-Maria Raita-Hakola
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
| | - Leevi Annala
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
| | - Mari Salmivuori
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (M.S.); (L.J.); (S.K.); (S.P.); (K.I.); (A.R.)
| | - Leila Jeskanen
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (M.S.); (L.J.); (S.K.); (S.P.); (K.I.); (A.R.)
| | - Heikki Saari
- VTT Technical Research Centre of Finland, 02150 Espoo, Finland;
| | - Sari Koskenmies
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (M.S.); (L.J.); (S.K.); (S.P.); (K.I.); (A.R.)
| | - Sari Pitkänen
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (M.S.); (L.J.); (S.K.); (S.P.); (K.I.); (A.R.)
| | - Ilkka Pölönen
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
| | - Kirsi Isoherranen
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (M.S.); (L.J.); (S.K.); (S.P.); (K.I.); (A.R.)
| | - Annamari Ranki
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (M.S.); (L.J.); (S.K.); (S.P.); (K.I.); (A.R.)
| |
Collapse
|
28
|
Hu X, Liu H, Qiu C, Liu D. Inspection of Line Defects in Transition Metal Dichalcogenides Using a Microscopic Hyperspectral Imaging Technique. J Phys Chem Lett 2022; 13:2226-2230. [PMID: 35238568 DOI: 10.1021/acs.jpclett.1c03968] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The line defects of two-dimensional (2D) transition metal dichalcogenides (TMDs) play a vital role in determining their device performance. In this work, a microscopic hyperspectral imaging technique based on differential reflectance was introduced for the online inspection of line defects in TMDs. Upon comparison of the measurement results of imaging and spectra, the relationship between optical contrast and differential reflectance spectra was established. A light selection method was proposed to optimize the optical contrast of line defects. Via application of an image processing algorithm, an automatic detection of the line defects with a classification accuracy of 95% was achieved for WS2, MoS2, and MoSe2. This work not only provides a microscopic hyperspectral imaging technique for detecting 2D material defects but also introduces a versatile design strategy for developing an advanced machine vision spectroscopic system.
Collapse
Affiliation(s)
- Xiangmin Hu
- State Key Laboratory of Tribology, Tsinghua University, Beijing 100084, China
| | - Huixian Liu
- State Key Laboratory of Tribology, Tsinghua University, Beijing 100084, China
| | - Cuicui Qiu
- State Key Laboratory of Tribology, Tsinghua University, Beijing 100084, China
| | - Dameng Liu
- State Key Laboratory of Tribology, Tsinghua University, Beijing 100084, China
| |
Collapse
|
29
|
Sucher R, Scheuermann U, Rademacher S, Lederer A, Sucher E, Hau HM, Brandacher G, Schneeberger S, Gockel I, Seehofer D. Intraoperative reperfusion assessment of human pancreas allografts using hyperspectral imaging (HSI). Hepatobiliary Surg Nutr 2022; 11:67-77. [PMID: 35284501 PMCID: PMC8847868 DOI: 10.21037/hbsn-20-744] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/09/2020] [Accepted: 12/24/2020] [Indexed: 12/11/2022]
Abstract
Background The most common causes of early graft loss in pancreas transplantation are insufficient blood supply and leakage of the intestinal anastomosis. Therefore, it is critical to monitor graft perfusion and oxygenation during the early post-transplant period. The goal of our pilot study was to evaluate the utility of hyperspectral imaging (HSI) in monitoring the microcirculation of the graft and adequate perfusion of the intestinal anastomosis during pancreatic allotransplantation. Methods We imaged pancreatic grafts and intestinal anastomosis in real-time in three consecutive, simultaneous pancreas-kidney transplantations using the TIVITA® HSI system. Further, the intraoperative oxygen saturation (StO2), tissue perfusion (near-infrared perfusion index, NIR), organ hemoglobin index (OHI), and tissue water index (TWI) were measured 15 minutes after reperfusion by HSI. Results All pancreas grafts showed a high and homogeneous StO2 (92.6%±10.45%). Intraoperative HSI analysis of the intestinal anastomosis displayed significant differences of StO2 (graft duodenum 67.46%±5.60% vs. recipient jejunum: 75.93%±4.71%, P<0.001) and TWI {graft duodenum: 0.63±0.09 [I (Index)] vs. recipient jejunum: 0.72±0.09 [I], P<0.001}. NIR and OHI did not display remarkable differences {NIR duodenum: 0.68±0.06 [I] vs. NIR jejunum: 0.69±0.04 [I], P=0.747; OHI duodenum: 0.70±0.12 [I] vs. OHI jejunum: 0.68±0.13 [I], P=0.449}. All 3 patients had an uneventful postoperative course with one displaying a Banff 1a rejection which was responsive to steroid treatment. Conclusions Our study shows that contact-free HSI has potential utility as a novel tool for real-time monitoring of human pancreatic grafts after reperfusion, which could improve the outcome of pancreas transplantation. Further investigations are required to determine the predictive value of intraoperative HSI imaging.
Collapse
Affiliation(s)
- Robert Sucher
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital Leipzig, Leipzig, Germany
| | - Uwe Scheuermann
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital Leipzig, Leipzig, Germany
| | - Sebastian Rademacher
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital Leipzig, Leipzig, Germany
| | - Andri Lederer
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital Leipzig, Leipzig, Germany
| | - Elisabeth Sucher
- Division of Hepatology, Clinic and Polyclinic for Gastroenterology, Hepatology, Infectiology, and Pneumology, University Hospital Leipzig, Leipzig, Germany
| | - Hans-Michael Hau
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital Leipzig, Leipzig, Germany.,Department of Visceral, Transplantation, Vascular and Thoracic Surgery, University Hospital of Dresden, Dresden, Germany
| | - Gerald Brandacher
- Department of Plastic and Reconstructive Surgery, Vascularized Composite Allotransplantation (VCA) Laboratory, Johns Hopkins University, Baltimore, MD, USA
| | - Stefan Schneeberger
- Department of Visceral, Transplant and Thoracic Surgery, Innsbruck Medical University, Innsbruck, Austria
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital Leipzig, Leipzig, Germany
| | - Daniel Seehofer
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital Leipzig, Leipzig, Germany
| |
Collapse
|
30
|
Multisensor perfusion assessment cohort study: Preliminary evidence toward a standardized assessment of indocyanine green fluorescence in colorectal surgery. Surgery 2022; 172:69-73. [DOI: 10.1016/j.surg.2021.12.021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Revised: 12/14/2021] [Accepted: 12/19/2021] [Indexed: 12/22/2022]
|
31
|
Modir N, Shahedi M, Dormer J, Ma L, Ghaderi M, Sirsi S, Cheng YSL, Fei B. LED-based Hyperspectral Endoscopic Imaging. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 11954:1195408. [PMID: 36794092 PMCID: PMC9928531 DOI: 10.1117/12.2609023] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Hyperspectral endoscopy can offer multiple advantages as compared to conventional endoscopy. Our goal is to design and develop a real-time hyperspectral endoscopic imaging system for the diagnosis of gastrointestinal (GI) tract cancers using a micro-LED array as an in-situ illumination source. The wavelengths of the system range from ultraviolet to visible and near infrared. To evaluate the use of the LED array for hyperspectral imaging, we designed a prototype system and conducted ex vivo experiments using normal and cancerous tissues of mice, chicken, and sheep. We compared the results of our LED-based approach with our reference hyperspectral camera system. The results confirm the similarity between the LED-based hyperspectral imaging system and the reference HSI camera. Our LED-based hyperspectral imaging system can be used not only as an endoscope but also as a laparoscopic or handheld devices for cancer detection and surgery.
Collapse
Affiliation(s)
- Naeeme Modir
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Maysam Shahedi
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - James Dormer
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Mohammadaref Ghaderi
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Shashank Sirsi
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Yi-Shing Lisa Cheng
- Department of Diagnostic Sciences, College of Dentistry, Texas A&M University, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
32
|
Li P, Ebner M, Noonan P, Horgan C, Bahl A, Ourselin S, Shapey J, Vercauteren T. Deep learning approach for hyperspectral image demosaicking, spectral correction and high-resolution RGB reconstruction. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING. IMAGING & VISUALIZATION 2021; 10:409-417. [PMID: 38013723 PMCID: PMC10461732 DOI: 10.1080/21681163.2021.1997646] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Accepted: 10/20/2021] [Indexed: 10/05/2023]
Abstract
Hyperspectral imaging is one of the most promising techniques for intraoperative tissue characterisation. Snapshot mosaic cameras, which can capture hyperspectral data in a single exposure, have the potential to make a real-time hyperspectral imaging system for surgical decision-making possible. However, optimal exploitation of the captured data requires solving an ill-posed demosaicking problem and applying additional spectral corrections. In this work, we propose a supervised learning-based image demosaicking algorithm for snapshot hyperspectral images. Due to the lack of publicly available medical images acquired with snapshot mosaic cameras, a synthetic image generation approach is proposed to simulate snapshot images from existing medical image datasets captured by high-resolution, but slow, hyperspectral imaging devices. Image reconstruction is achieved using convolutional neural networks for hyperspectral image super-resolution, followed by spectral correction using a sensor-specific calibration matrix. The results are evaluated both quantitatively and qualitatively, showing clear improvements in image quality compared to a baseline demosaicking method using linear interpolation. Moreover, the fast processing time of 45 ms of our algorithm to obtain super-resolved RGB or oxygenation saturation maps per image for a state-of-the-art snapshot mosaic camera demonstrates the potential for its seamless integration into real-time surgical hyperspectral imaging applications.
Collapse
Affiliation(s)
- Peichao Li
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK
| | - Michael Ebner
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | - Philip Noonan
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK
| | - Conor Horgan
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | - Anisha Bahl
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK
| | - Sébastien Ourselin
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | - Jonathan Shapey
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
- Department of Neurosurgery, King’s College Hospital NHS Foundation Trust, London, UK
| | - Tom Vercauteren
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| |
Collapse
|
33
|
Barberio M, Benedicenti S, Pizzicannella M, Felli E, Collins T, Jansen-Winkeln B, Marescaux J, Viola MG, Diana M. Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons. Diagnostics (Basel) 2021; 11:diagnostics11112066. [PMID: 34829413 PMCID: PMC8624094 DOI: 10.3390/diagnostics11112066] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2021] [Revised: 11/03/2021] [Accepted: 11/05/2021] [Indexed: 12/12/2022] Open
Abstract
Hyperspectral imaging (HSI) is a novel optical imaging modality, which has recently found diverse applications in the medical field. HSI is a hybrid imaging modality, combining a digital photographic camera with a spectrographic unit, and it allows for a contactless and non-destructive biochemical analysis of living tissue. HSI provides quantitative and qualitative information of the tissue composition at molecular level in a contrast-free manner, hence making it possible to objectively discriminate between different tissue types and between healthy and pathological tissue. Over the last two decades, HSI has been increasingly used in the medical field, and only recently it has found an application in the operating room. In the last few years, several research groups have used this imaging modality as an intraoperative guidance tool within different surgical disciplines. Despite its great potential, HSI still remains far from being routinely used in the daily surgical practice, since it is still largely unknown to most of the surgical community. The aim of this study is to provide clinical surgeons with an overview of the capabilities, current limitations, and future directions of HSI for intraoperative guidance.
Collapse
Affiliation(s)
- Manuel Barberio
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (T.C.); (J.M.); (M.D.)
- General Surgery Department, Ospedale Card. G. Panico, 73039 Tricase, Italy; (S.B.); (M.P.); (M.G.V.)
- Correspondence:
| | - Sara Benedicenti
- General Surgery Department, Ospedale Card. G. Panico, 73039 Tricase, Italy; (S.B.); (M.P.); (M.G.V.)
| | - Margherita Pizzicannella
- General Surgery Department, Ospedale Card. G. Panico, 73039 Tricase, Italy; (S.B.); (M.P.); (M.G.V.)
| | - Eric Felli
- Department of Visceral Surgery and Medicine, Inselspital, Bern University Hospital, University of Bern, 3008 Bern, Switzerland;
- Department for BioMedical Research, Visceral Surgery and Medicine, University of Bern, 3008 Bern, Switzerland
| | - Toby Collins
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (T.C.); (J.M.); (M.D.)
| | | | - Jacques Marescaux
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (T.C.); (J.M.); (M.D.)
| | - Massimo Giuseppe Viola
- General Surgery Department, Ospedale Card. G. Panico, 73039 Tricase, Italy; (S.B.); (M.P.); (M.G.V.)
| | - Michele Diana
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (T.C.); (J.M.); (M.D.)
- ICube Laboratory, Photonics Instrumentation for Health, University of Strasbourg, 67400 Strasbourg, France
| |
Collapse
|
34
|
Maktabi M, Tkachenko M, Kohler H, Schierle K, Gockel I, Jansen-Winkeln B, Chalopin C. Using physiological parameters measured by hyperspectral imaging to detect colorectal cancer. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3865-3868. [PMID: 34892077 DOI: 10.1109/embc46164.2021.9630160] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/19/2023]
Abstract
The accurate detection of malignant tissue during colorectal surgery impacts operation outcome. The non-invasive spectral imaging combined with machine learning (ML) methods showed to be promising for tumor identification. However, large spectral range implies large computing time. To reduce the number of features, ML methods (e.g. logistic regression and convolutional neuronal network CNN) were evaluated based on four physiological tissue parameters to automatically classify cancer and healthy mucosa in resected colon tissue. A ROC AUC of 0.81 was achieved with the CNN. This study shows that the use of only specific wavelengths bands can detect cancer.Clinical Relevance- These outcomes support the possibility to automatically classify colon tumor based on physiological parameters calculated using only specific wavelength bands. Hence, future image-guided colorectal surgeries can be performed with real-time multispectral imaging.
Collapse
|
35
|
Waterhouse DJ, Bano S, Januszewicz W, Stoyanov D, Fitzgerald RC, di Pietro M, Bohndiek SE. First-in-human pilot study of snapshot multispectral endoscopy for early detection of Barrett's-related neoplasia. JOURNAL OF BIOMEDICAL OPTICS 2021; 26:JBO-210159R. [PMID: 34628734 PMCID: PMC8501416 DOI: 10.1117/1.jbo.26.10.106002] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2021] [Accepted: 09/02/2021] [Indexed: 05/04/2023]
Abstract
SIGNIFICANCE The early detection of dysplasia in patients with Barrett's esophagus could improve outcomes by enabling curative intervention; however, dysplasia is often inconspicuous using conventional white-light endoscopy. AIM We sought to determine whether multispectral imaging (MSI) could be applied in endoscopy to improve detection of dysplasia in the upper gastrointestinal (GI) tract. APPROACH We used a commercial fiberscope to relay imaging data from within the upper GI tract to a snapshot MSI camera capable of collecting data from nine spectral bands. The system was deployed in a pilot clinical study of 20 patients (ClinicalTrials.gov NCT03388047) to capture 727 in vivo image cubes matched with gold-standard diagnosis from histopathology. We compared the performance of seven learning-based methods for data classification, including linear discriminant analysis, k-nearest neighbor classification, and a neural network. RESULTS Validation of our approach using a Macbeth color chart achieved an image-based classification accuracy of 96.5%. Although our patient cohort showed significant intra- and interpatient variance, we were able to resolve disease-specific contributions to the recorded MSI data. In classification, a combined principal component analysis and k-nearest-neighbor approach performed best, achieving accuracies of 95.8%, 90.7%, and 76.1%, respectively, for squamous, non-dysplastic Barrett's esophagus and neoplasia based on majority decisions per-image. CONCLUSIONS MSI shows promise for disease classification in Barrett's esophagus and merits further investigation as a tool in high-definition "chip-on-tip" endoscopes.
Collapse
Affiliation(s)
- Dale J. Waterhouse
- University of Cambridge, Department of Physics and CRUK Cambridge Institute, Cambridge, United Kingdom
- University College London, Wellcome/EPSRC Centre for Interventional and Surgical Sciences, London, United Kingdom
| | - Sophia Bano
- University College London, Wellcome/EPSRC Centre for Interventional and Surgical Sciences, London, United Kingdom
| | - Wladyslaw Januszewicz
- Medical Centre for Postgraduate Education, Department of Gastroenterology, Hepatology and Clinical Oncology, Warsaw, Poland
| | - Dan Stoyanov
- University College London, Wellcome/EPSRC Centre for Interventional and Surgical Sciences, London, United Kingdom
| | - Rebecca C. Fitzgerald
- University of Cambridge, MRC Cancer Unit, Hutchison/MRC Research Centre, Cambridge, United Kingdom
| | - Massimiliano di Pietro
- University of Cambridge, MRC Cancer Unit, Hutchison/MRC Research Centre, Cambridge, United Kingdom
| | - Sarah E. Bohndiek
- University of Cambridge, Department of Physics and CRUK Cambridge Institute, Cambridge, United Kingdom
| |
Collapse
|
36
|
Fodor M, Hofmann J, Lanser L, Otarashvili G, Pühringer M, Hautz T, Sucher R, Schneeberger S. Hyperspectral Imaging and Machine Perfusion in Solid Organ Transplantation: Clinical Potentials of Combining Two Novel Technologies. J Clin Med 2021; 10:jcm10173838. [PMID: 34501286 PMCID: PMC8432211 DOI: 10.3390/jcm10173838] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Revised: 08/24/2021] [Accepted: 08/25/2021] [Indexed: 12/16/2022] Open
Abstract
Organ transplantation survival rates have continued to improve over the last decades, mostly due to reduction of mortality early after transplantation. The advancement of the field is facilitating a liberalization of the access to organ transplantation with more patients with higher risk profile being added to the waiting list. At the same time, the persisting organ shortage fosters strategies to rescue organs of marginal donors. In this regard, hypothermic and normothermic machine perfusion are recognized as one of the most important developments in the modern era. Owing to these developments, novel non-invasive tools for the assessment of organ quality are on the horizon. Hyperspectral imaging represents a potentially suitable method capable of evaluating tissue morphology and organ perfusion prior to transplantation. Considering the changing environment, we here discuss the hypothetical combination of organ machine perfusion and hyperspectral imaging as a prospective feasibility concept in organ transplantation.
Collapse
Affiliation(s)
- Margot Fodor
- Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, 6020 Innsbruck, Austria; (M.F.); (J.H.); (G.O.); (M.P.); (T.H.)
- OrganLife, Organ Regeneration Center of Excellence, 6020 Innsbruck, Austria
| | - Julia Hofmann
- Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, 6020 Innsbruck, Austria; (M.F.); (J.H.); (G.O.); (M.P.); (T.H.)
- OrganLife, Organ Regeneration Center of Excellence, 6020 Innsbruck, Austria
| | - Lukas Lanser
- Department of Internal Medicine II, Innsbruck Medical University, 6020 Innsbruck, Austria;
| | - Giorgi Otarashvili
- Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, 6020 Innsbruck, Austria; (M.F.); (J.H.); (G.O.); (M.P.); (T.H.)
- OrganLife, Organ Regeneration Center of Excellence, 6020 Innsbruck, Austria
| | - Marlene Pühringer
- Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, 6020 Innsbruck, Austria; (M.F.); (J.H.); (G.O.); (M.P.); (T.H.)
- OrganLife, Organ Regeneration Center of Excellence, 6020 Innsbruck, Austria
| | - Theresa Hautz
- Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, 6020 Innsbruck, Austria; (M.F.); (J.H.); (G.O.); (M.P.); (T.H.)
- OrganLife, Organ Regeneration Center of Excellence, 6020 Innsbruck, Austria
| | - Robert Sucher
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, Leipzig University Clinic, 04103 Leipzig, Germany;
| | - Stefan Schneeberger
- Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, 6020 Innsbruck, Austria; (M.F.); (J.H.); (G.O.); (M.P.); (T.H.)
- OrganLife, Organ Regeneration Center of Excellence, 6020 Innsbruck, Austria
- Correspondence: ; Tel.: +43-512-504-22600
| |
Collapse
|
37
|
Chiba T, Murata M, Kawano T, Hashizume M, Akahoshi T. Reflectance spectra analysis for mucous assessment. World J Gastrointest Oncol 2021; 13:822-834. [PMID: 34457188 PMCID: PMC8371524 DOI: 10.4251/wjgo.v13.i8.822] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Revised: 04/26/2021] [Accepted: 07/09/2021] [Indexed: 02/06/2023] Open
Abstract
This review report represents an overview of research and development on medical hyperspectral imaging technology and its applications. Spectral imaging technology is attracting attention as a new imaging modality for medical applications, especially in disease diagnosis and image-guided surgery. Considering the recent advances in imaging, this technology provides an opportunity for two-dimensional mapping of oxygen saturation (SatO2) of blood with high accuracy, spatial spectral imaging, and its analysis and provides detection and diagnostic information about the tissue physiology and morphology. Multispectral imaging also provides information about tissue oxygenation, perfusion, and potential function during surgery. Analytical algorithm has been examined, and indication of accurate map of relative hemoglobin concentration and SatO2 can be indicated with preferable resolution and frame rate. This technology is expected to provide promising biomedical information in practical use. Several studies suggested that blood flow and SatO2 are associated with gastrointestinal disorders, particularly malignant tumor conditions. The use and analysis of spectroscopic images are expected to potentially play a role in the detection and diagnosis of these diseases.
Collapse
Affiliation(s)
- Toru Chiba
- Pentax_LifeCare, HOYA Corporation, Akishima-shi 196-0012, Tokyo, Japan
| | - Masaharu Murata
- Center for Advanced Medical Innovation, Kyushu University, Fukuoka-shi 812-8582, Fukuoka, Japan
| | - Takahito Kawano
- Center for Advanced Medical Innovation, Kyushu University, Fukuoka-shi 812-8582, Fukuoka, Japan
| | - Makoto Hashizume
- Center for Advanced Medical Innovation, Kyushu University, Fukuoka-shi 812-8582, Fukuoka, Japan
| | - Tomohiko Akahoshi
- Department of Disaster and Emergency Medicine, Graduate School of Medical Sciences, Kyushu University, Fukuoka_shi 812-8582, Fukuoka, Japan
| |
Collapse
|
38
|
Michael Ebner, Nabavi E, Shapey J, Xie Y, Liebmann F, Spirig JM, Hoch A, Farshad M, Saeed SR, Bradford R, Yardley I, Ourselin S, Edwards AD, Führnstahl P, Vercauteren T. Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation. JOURNAL OF PHYSICS D: APPLIED PHYSICS 2021; 54:294003. [PMID: 34024940 PMCID: PMC8132621 DOI: 10.1088/1361-6463/abfbf6] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/29/2020] [Revised: 03/30/2021] [Accepted: 04/27/2021] [Indexed: 10/05/2023]
Abstract
Despite advances in intraoperative surgical imaging, reliable discrimination of critical tissue during surgery remains challenging. As a result, decisions with potentially life-changing consequences for patients are still based on the surgeon's subjective visual assessment. Hyperspectral imaging (HSI) provides a promising solution for objective intraoperative tissue characterisation, with the advantages of being non-contact, non-ionising and non-invasive. However, while its potential to aid surgical decision-making has been investigated for a range of applications, to date no real-time intraoperative HSI (iHSI) system has been presented that follows critical design considerations to ensure a satisfactory integration into the surgical workflow. By establishing functional and technical requirements of an intraoperative system for surgery, we present an iHSI system design that allows for real-time wide-field HSI and responsive surgical guidance in a highly constrained operating theatre. Two systems exploiting state-of-the-art industrial HSI cameras, respectively using linescan and snapshot imaging technology, were designed and investigated by performing assessments against established design criteria and ex vivo tissue experiments. Finally, we report the use of our real-time iHSI system in a clinical feasibility case study as part of a spinal fusion surgery. Our results demonstrate seamless integration into existing surgical workflows.
Collapse
Affiliation(s)
- Michael Ebner
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, United Kingdom
| | - Eli Nabavi
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, United Kingdom
| | - Jonathan Shapey
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, United Kingdom
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, UCL, London, United Kingdom
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, United Kingdom
| | - Yijing Xie
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, United Kingdom
| | - Florentin Liebmann
- Research in Orthopedic Computer Science (ROCS), Balgrist University Hospital, University of Zurich, Balgrist CAMPUS, Zurich, Switzerland
- Laboratory for Orthopaedic Biomechanics, ETH Zurich, Zurich, Switzerland
| | - José Miguel Spirig
- Department of Orthopaedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Armando Hoch
- Department of Orthopaedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Mazda Farshad
- Department of Orthopaedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Shakeel R Saeed
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, United Kingdom
- The Ear Institute, UCL, London, United Kingdom
- The Royal National Throat, Nose and Ear Hospital, London, United Kingdom
| | - Robert Bradford
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, United Kingdom
| | - Iain Yardley
- Department of Paediatric Surgery, Evelina London Children’s Hospital, London, United Kingdom
| | - Sébastien Ourselin
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, United Kingdom
| | - A David Edwards
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, United Kingdom
- Department of Paediatric Surgery, Evelina London Children’s Hospital, London, United Kingdom
| | - Philipp Führnstahl
- Research in Orthopedic Computer Science (ROCS), Balgrist University Hospital, University of Zurich, Balgrist CAMPUS, Zurich, Switzerland
| | - Tom Vercauteren
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, United Kingdom
| |
Collapse
|
39
|
Dietrich M, Marx S, von der Forst M, Bruckner T, Schmitt FCF, Fiedler MO, Nickel F, Studier-Fischer A, Müller-Stich BP, Hackert T, Brenner T, Weigand MA, Uhle F, Schmidt K. Bedside hyperspectral imaging indicates a microcirculatory sepsis pattern - an observational study. Microvasc Res 2021; 136:104164. [PMID: 33831406 DOI: 10.1016/j.mvr.2021.104164] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 03/28/2021] [Accepted: 03/28/2021] [Indexed: 12/17/2022]
Abstract
INTRODUCTION Microcirculatory alterations are key mechanisms in sepsis pathophysiology leading to tissue hypoxia, edema formation, and organ dysfunction. Hyperspectral imaging (HSI) is an emerging imaging technology that uses tissue-light interactions to evaluate biochemical tissue characteristics including tissue oxygenation, hemoglobin content and water content. Currently, clinical data for HSI technologies in critical ill patients are still limited. METHODS AND ANALYSIS TIVITA® Tissue System was used to measure Tissue oxygenation (StO2), Tissue Hemoglobin Index (THI), Near Infrared Perfusion Index (NPI) and Tissue Water Index (TWI) in 25 healthy volunteers and 25 septic patients. HSI measurement sites were the palm, the fingertip, and a suprapatellar knee area. Septic patients were evaluated on admission to the ICU (E), 6 h afterwards (E+6) and three times a day (t3-t9) within a total observation period of 72 h. Primary outcome was the correlation of HSI results with daily SOFA-scores. RESULTS Serial HSI at the three measurement sites in healthy volunteers showed a low mean variance expressing high retest reliability. HSI at E demonstrated significantly lower StO2 and NPI as well as higher TWI at the palm and fingertip in septic patients compared to healthy volunteers. StO2 and TWI showed corresponding results at the suprapatellar knee area. In septic patients, palm and fingertip THI identified survivors (E-t4) and revealed predictivity for 28-day mortality (E). Fingertip StO2 and THI correlated to SOFA-score on day 2. TWI was consistently increased in relation to the TWI range of healthy controls during the observation time. Palm TWI correlated positively with SOFA scores on day 3. DISCUSSION HSI results in septic patients point to a distinctive microcirculatory pattern indicative of reduced skin oxygenation and perfusion quality combined with increased blood pooling and tissue water content. THI might possess risk-stratification properties and TWI could allow tissue edema evaluation in critically ill patients. CONCLUSION HSI technologies could open new perspectives in microcirculatory monitoring by visualizing oxygenation and perfusion quality combined with tissue water content in critically ill patients - a prerequisite for future tissue perfusion guided therapy concepts in intensive care medicine.
Collapse
Affiliation(s)
- M Dietrich
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - S Marx
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - M von der Forst
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - T Bruckner
- Institute of Medical Biometry and Informatics, University of Heidelberg, Heidelberg, Germany
| | - F C F Schmitt
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - M O Fiedler
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - F Nickel
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - A Studier-Fischer
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - B P Müller-Stich
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - T Hackert
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - T Brenner
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany; Department of Anesthesiology and Intensive Care Medicine, University Hospital Essen, University Duisburg-Essen, Essen, Germany
| | - M A Weigand
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - F Uhle
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - K Schmidt
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany; Department of Anesthesiology and Intensive Care Medicine, University Hospital Essen, University Duisburg-Essen, Essen, Germany.
| |
Collapse
|
40
|
Modir N, Shahedi M, Dormer J, Fei B. Development of a real-time spectral imaging system using in-site micro-LED-based illumination and high-speed micro-camera for endoscopic applications. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11654:1165417. [PMID: 35784009 PMCID: PMC9248909 DOI: 10.1117/12.2579097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
We designed a compact, real-time LED-based endoscopic imaging system for the detection of various diseases including cancer. In gastrointestinal applications, conventional endoscopy cannot reliably differentiate tumor from normal tissue. Current hyperspectral imaging systems are too slow to be used for real-time endoscopic applications. We are investigating real-time spectral imaging for different tissue types. Our objective is to develop a catheter for real-time hyperspectral gastrointestinal endoscopy. The endoscope uses multiple wavelengths within UV, visible, and IR light spectra generated by a micro-LED array. We capture images with a monochrome micro camera, which is cost-effective and smaller than the current hyperspectral imagers. A wireless transceiver sends the captured images to a workstation for further processing, such as tumor detection. The spatial resolution of the system is defined by camera resolution and the distance to the object, while the number of LEDs in the multi-wavelength light source determines the spectral resolution. To investigate the properties and the limitations of our high-speed spectral imaging approach, we designed a prototype system. We conducted two experiments to measure the optimal forward voltages and lighting duration of the LEDs. These factors affect the maximum feasible imaging rate and resolution. The lighting duration of each LED can be shorter than 10 ms while producing an image with a high signal-to-noise ratio and no illumination interference. These results support the idea of using a high-speed camera and an LED-array for real-time hyperspectral endoscopic imaging.
Collapse
Affiliation(s)
- Naeeme Modir
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas, Richardson, TX 75080, USA
| | - Maysam Shahedi
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas, Richardson, TX 75080, USA
| | - James Dormer
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas, Richardson, TX 75080, USA
| | - Baowei Fei
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas, Richardson, TX 75080, USA
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX 75235, USA
| |
Collapse
|
41
|
Schulz T, Leuschner S, Siemers F, Marotz J, Houschyar K, Corterier CC. Assessing flap perfusion after free tissue transfer using hyperspectral imaging (HSI). EUROPEAN JOURNAL OF PLASTIC SURGERY 2021. [DOI: 10.1007/s00238-021-01784-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
42
|
Gerstner AOH, Laffers W. [Integrity of swallowing apparatus-past, present, and future]. HNO 2021; 69:185-191. [PMID: 33438078 DOI: 10.1007/s00106-020-00990-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/03/2020] [Indexed: 11/28/2022]
Abstract
BACKGROUND Swallowing is one of the most complex movement patterns. The integrity of the epithelial lining is essential. OBJECTIVE Which surgical techniques were developed at the beginnings of modern surgery and what methods are now available to maintain/reconstitute the integrity of the swallowing apparatus? MATERIALS AND METHODS This study comprises a selective literature search in early operation manuals and online archives, with incorporation of the authors' own experience. RESULTS Up until the 1950s, only very limited techniques were available to reconstruct the epithelial lining. Microvascular reanastomosed grafts were the game changer for reconstructive surgery, enabling reconstitution of the swallowing apparatus in primary surgery but also in challenging secondary interventions after insufficient or complicated primary therapy. CONCLUSION The need for anatomical and functional rehabilitation by reconstructive surgery is as pertinent as ever. Particularly in the oncological context, improved early detection and novel local treatment modalities could minimize treatment-associated damage to swallowing.
Collapse
Affiliation(s)
- A O H Gerstner
- Hals‑, Nasen-, Ohrenklinik, Klinikum Braunschweig, Holwedestraße16, 38118, Braunschweig, Deutschland.
| | - W Laffers
- Klinik für Hals-Nasen-Ohrenheilkunde, Evangelisches Krankenhaus, Carl-von-Ossietzky-Universität, Steinweg 13-17, 26122, Oldenburg, Deutschland.
| |
Collapse
|
43
|
Bjorgan A, Pukstad BS, Randeberg LL. Hyperspectral characterization of re-epithelialization in an in vitro wound model. JOURNAL OF BIOPHOTONICS 2020; 13:e202000108. [PMID: 32558341 DOI: 10.1002/jbio.202000108] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2020] [Revised: 05/27/2020] [Accepted: 06/11/2020] [Indexed: 06/11/2023]
Abstract
In vitro wound models are useful for research on wound re-epithelialization. Hyperspectral imaging represents a non-destructive alternative to histology analysis for detection of re-epithelialization. This study aims to characterize the main optical behavior of a wound model in order to enable development of detection algorithms. K-Means clustering and agglomerative analysis were used to group spatial regions based on the spectral behavior, and an inverse photon transport model was used to explain differences in optical properties. Six samples of the wound model were prepared from human tissue and followed over 22 days. Re-epithelialization occurred at a mean rate of 0.24 mm2 /day after day 8 to 10. Suppression of wound spectral features was the main feature characterizing re-epithelialized and intact tissue. Modeling the photon transport through a diffuse layer placed on top of wound tissue properties reproduced the spectral behavior. The missing top layer represented by wounds is thus optically detectable using hyperspectral imaging.
Collapse
Affiliation(s)
- Asgeir Bjorgan
- Department of Electronic Systems, NTNU Norwegian University of Science and Technology, Trondheim, Norway
| | - Brita S Pukstad
- Department of Clinical and Molecular Medicine, NTNU Norwegian University of Science and Technology, Trondheim, Norway
- Department of Dermatology, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Lise L Randeberg
- Department of Electronic Systems, NTNU Norwegian University of Science and Technology, Trondheim, Norway
| |
Collapse
|
44
|
Bjorgan A, Randeberg LL. Exploiting scale-invariance: a top layer targeted inverse model for hyperspectral images of wounds. BIOMEDICAL OPTICS EXPRESS 2020; 11:5070-5091. [PMID: 33014601 PMCID: PMC7510863 DOI: 10.1364/boe.399636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/11/2020] [Revised: 07/15/2020] [Accepted: 07/28/2020] [Indexed: 05/10/2023]
Abstract
Detection of re-epithelialization in wound healing is important, but challenging. Hyperspectral imaging can be used for non-destructive characterization, but efficient techniques are needed to extract and interpret the information. An inverse photon transport model suitable for characterization of re-epithelialization is validated and explored in this study. It exploits scale-invariance to enable fitting of the epidermal skin layer only. Monte Carlo simulations indicate that the fitted layer transmittance and reflectance spectra are unique, and that there exists an infinite number of coupled parameter solutions. The method is used to explain the optical behavior of and detect re-epithelialization in an in vitro wound model.
Collapse
|
45
|
Shenson JA, Liu GS, Farrell J, Blevins NH. Multispectral Imaging for Automated Tissue Identification of Normal Human Surgical Specimens. Otolaryngol Head Neck Surg 2020; 164:328-335. [PMID: 32838646 DOI: 10.1177/0194599820941013] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
OBJECTIVE Safe surgery requires the accurate discrimination of tissue intraoperatively. We assess the feasibility of using multispectral imaging and deep learning to enhance surgical vision by automated identification of normal human head and neck tissues. STUDY DESIGN Construction and feasibility testing of novel multispectral imaging system for surgery. SETTING Academic university hospital. SUBJECTS AND METHODS Multispectral images of fresh-preserved human cadaveric tissues were captured with our adapted digital operating microscope. Eleven tissue types were sampled, each sequentially exposed to 6 lighting conditions. Two convolutional neural network machine learning models were developed to classify tissues based on multispectral and white-light color images (ARRInet-M and ARRInet-W, respectively). Blinded otolaryngology residents were asked to identify tissue specimens from white-light color images, and their performance was compared with that of the ARRInet models. RESULTS A novel multispectral imaging system was developed with minimal adaptation to an existing digital operating microscope. With 81.8% accuracy in tissue identification of full-size images, the multispectral ARRInet-M classifier outperformed the white-light-only ARRInet-W model (45.5%) and surgical residents (69.7%). Challenges with discrimination occurred with parotid vs fat and blood vessels vs nerve. CONCLUSIONS A deep learning model using multispectral imaging outperformed a similar model and surgical residents using traditional white-light imaging at the task of classifying normal human head and neck tissue ex vivo. These results suggest that multispectral imaging can enhance surgical vision and augment surgeons' ability to identify tissues during a procedure.
Collapse
Affiliation(s)
- Jared A Shenson
- Department of Otolaryngology-Head and Neck Surgery, Stanford University, Stanford, California, USA
| | - George S Liu
- Department of Otolaryngology-Head and Neck Surgery, Stanford University, Stanford, California, USA
| | - Joyce Farrell
- Department of Electrical Engineering, Stanford University, Stanford, California, USA
| | - Nikolas H Blevins
- Department of Otolaryngology-Head and Neck Surgery, Stanford University, Stanford, California, USA
| |
Collapse
|
46
|
Schulz T, Marotz J, Stukenberg A, Reumuth G, Houschyar KS, Siemers F. [Hyperspectral imaging for postoperative flap monitoring of pedicled flaps]. HANDCHIR MIKROCHIR P 2020; 52:316-324. [PMID: 32823364 DOI: 10.1055/a-1167-3089] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022] Open
Abstract
BACKGROUND Since pedicle flaps were first described by the Indian physician Sushruta Samhita in the 6th century B. C., they have become an integral part of reconstructive surgery. As more and more research has been conducted into the underlying physical principles, flap monitoring has developed steadily in the last few decades. Hyperspectral Imaging (HSI) is a new quantitative measuring method for assessing the perfusion of the underlying tissue. OBJECTIVE This study aims to evaluate HSI as a monitoring method for pedicle flaps. PATIENTS AND METHODS In 16 patients who had undergone reconstructive surgery, oxygen saturation, haemoglobin and water concentration of the locoregional flap, necrotic flap areas as well as intact skin were measured on postoperative days 1 to 7. Subsequently, the data were statistically described and graphically illustrated. RESULTS HSI revealed increased haemoglobin concentration and decreased oxygen and water concentration in necrotic flap areas compared with the monitor island and healthy skin. The distribution of the values collected from the vital skin areas and the monitor island was almost identical. CONCLUSION HSI allows for safe, immediate, non-contact measurement of tissue perfusion of transferred tissue areas in patients after pedicle flap surgery. The use of HSI may improve postoperative flap monitoring.
Collapse
Affiliation(s)
- Torsten Schulz
- BG Klinikum Bergmannstrost Halle Handchirurgie, Plastische Chirurgie, Brandverletztenzentrum
| | - Jörg Marotz
- BG Klinikum Bergmannstrost Halle Handchirurgie, Plastische Chirurgie, Brandverletztenzentrum
| | - Anna Stukenberg
- BG Klinikum Bergmannstrost Halle Handchirurgie, Plastische Chirurgie, Brandverletztenzentrum
| | - Georg Reumuth
- BG Klinikum Bergmannstrost Halle Handchirurgie, Plastische Chirurgie, Brandverletztenzentrum
| | | | - Frank Siemers
- BG Klinikum Bergmannstrost Halle Handchirurgie, Plastische Chirurgie, Brandverletztenzentrum
| |
Collapse
|
47
|
Lemmens S, Van Eijgen J, Van Keer K, Jacob J, Moylett S, De Groef L, Vancraenendonck T, De Boever P, Stalmans I. Hyperspectral Imaging and the Retina: Worth the Wave? Transl Vis Sci Technol 2020; 9:9. [PMID: 32879765 PMCID: PMC7442879 DOI: 10.1167/tvst.9.9.9] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2019] [Accepted: 06/23/2020] [Indexed: 02/07/2023] Open
Abstract
Purpose Hyperspectral imaging is gaining attention in the biomedical field because it generates additional spectral information to study physiological and clinical processes. Several technologies have been described; however an independent, systematic literature overview is lacking, especially in the field of ophthalmology. This investigation is the first to systematically overview scientific literature specifically regarding retinal hyperspectral imaging. Methods A systematic literature review was conducted, in accordance with PRISMA Statement 2009 criteria, in four bibliographic databases: Medline, Embase, Cochrane Database of Systematic Reviews, and Web of Science. Results Fifty-six articles were found that meet the review criteria. A range of techniques was reported: Fourier analysis, liquid crystal tunable filters, tunable laser sources, dual-slit monochromators, dispersive prisms and gratings, computed tomography, fiber optics, and Fabry-Perrot cavity filter covered complementary metal oxide semiconductor. We present a narrative synthesis and summary tables of findings of the included articles, because methodologic heterogeneity and diverse research topics prevented a meta-analysis being conducted. Conclusions Application in ophthalmology is still in its infancy. Most previous experiments have been performed in the field of retinal oximetry, providing valuable information in the diagnosis and monitoring of various ocular diseases. To date, none of these applications have graduated to clinical practice owing to the lack of sufficiently large validation studies. Translational Relevance Given the promising results that smaller studies show for hyperspectral imaging (e.g., in Alzheimer's disease), advanced research in larger validation studies is warranted to determine its true clinical potential.
Collapse
Affiliation(s)
- Sophie Lemmens
- University Hospitals UZ Leuven, Department of Ophthalmology, Leuven, Belgium
- KU Leuven, Biomedical Sciences Group, Department of Neurosciences, Research Group Ophthalmology, Leuven, Belgium
- VITO (Flemish Institute for Technological Research), Health Unit, Boeretang, Belgium
| | - Jan Van Eijgen
- University Hospitals UZ Leuven, Department of Ophthalmology, Leuven, Belgium
- KU Leuven, Biomedical Sciences Group, Department of Neurosciences, Research Group Ophthalmology, Leuven, Belgium
- VITO (Flemish Institute for Technological Research), Health Unit, Boeretang, Belgium
| | - Karel Van Keer
- University Hospitals UZ Leuven, Department of Ophthalmology, Leuven, Belgium
- KU Leuven, Biomedical Sciences Group, Department of Neurosciences, Research Group Ophthalmology, Leuven, Belgium
| | - Julie Jacob
- University Hospitals UZ Leuven, Department of Ophthalmology, Leuven, Belgium
- KU Leuven, Biomedical Sciences Group, Department of Neurosciences, Research Group Ophthalmology, Leuven, Belgium
| | - Sinéad Moylett
- Department of Psychiatry, University of Cambridge School of Clinical Medicine, Cambridge Biomedical Campus, Cambridge, UK
| | - Lies De Groef
- Neural Circuit Development and Regeneration Research Group, Department of Biology, KU Leuven, Leuven, Belgium
| | - Toon Vancraenendonck
- VITO (Flemish Institute for Technological Research), Health Unit, Boeretang, Belgium
| | - Patrick De Boever
- VITO (Flemish Institute for Technological Research), Health Unit, Boeretang, Belgium
- Hasselt University, Centre of Environmental Sciences, Agoralaan, Belgium
| | - Ingeborg Stalmans
- University Hospitals UZ Leuven, Department of Ophthalmology, Leuven, Belgium
- KU Leuven, Biomedical Sciences Group, Department of Neurosciences, Research Group Ophthalmology, Leuven, Belgium
| |
Collapse
|
48
|
Köhler H, Kulcke A, Maktabi M, Moulla Y, Jansen-Winkeln B, Barberio M, Diana M, Gockel I, Neumuth T, Chalopin C. Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range. JOURNAL OF BIOMEDICAL OPTICS 2020; 25:JBO-200121RR. [PMID: 32860357 PMCID: PMC7453262 DOI: 10.1117/1.jbo.25.8.086004] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Accepted: 08/12/2020] [Indexed: 05/07/2023]
Abstract
SIGNIFICANCE Hyperspectral imaging (HSI) can support intraoperative perfusion assessment, the identification of tissue structures, and the detection of cancerous lesions. The practical use of HSI for minimal-invasive surgery is currently limited, for example, due to long acquisition times, missing video, or large set-ups. AIM An HSI laparoscope is described and evaluated to address the requirements for clinical use and high-resolution spectral imaging. APPROACH Reflectance measurements with reference objects and resected human tissue from 500 to 1000 nm are performed to show the consistency with an approved medical HSI device for open surgery. Varying object distances are investigated, and the signal-to-noise ratio (SNR) is determined for different light sources. RESULTS The handheld design enables real-time processing and visualization of HSI data during acquisition within 4.6 s. A color video is provided simultaneously and can be augmented with spectral information from push-broom imaging. The reflectance data from the HSI system for open surgery at 50 cm and the HSI laparoscope are consistent for object distances up to 10 cm. A standard rigid laparoscope in combination with a customized LED light source resulted in a mean SNR of 30 to 43 dB (500 to 950 nm). CONCLUSIONS Compact and rapid HSI with a high spatial- and spectral-resolution is feasible in clinical practice. Our work may support future studies on minimally invasive HSI to reduce intra- and postoperative complications.
Collapse
Affiliation(s)
- Hannes Köhler
- University of Leipzig, Innovation Center Computer Assisted Surgery, Leipzig, Germany
- Diaspective Vision GmbH, Am Salzhaff, Germany
- Address all correspondence to Hannes Köhler, E-mail: Hannes.
| | - Axel Kulcke
- Diaspective Vision GmbH, Am Salzhaff, Germany
| | - Marianne Maktabi
- University of Leipzig, Innovation Center Computer Assisted Surgery, Leipzig, Germany
| | - Yusef Moulla
- University Hospital of Leipzig, Department of Visceral, Thoracic, Transplant, and Vascular Surgery, Leipzig, Germany
| | - Boris Jansen-Winkeln
- University Hospital of Leipzig, Department of Visceral, Thoracic, Transplant, and Vascular Surgery, Leipzig, Germany
| | - Manuel Barberio
- IHU-Strasbourg Institute of Image-Guided Surgery, Strasbourg, France
| | - Michele Diana
- IHU-Strasbourg Institute of Image-Guided Surgery, Strasbourg, France
| | - Ines Gockel
- University Hospital of Leipzig, Department of Visceral, Thoracic, Transplant, and Vascular Surgery, Leipzig, Germany
| | - Thomas Neumuth
- University of Leipzig, Innovation Center Computer Assisted Surgery, Leipzig, Germany
| | - Claire Chalopin
- University of Leipzig, Innovation Center Computer Assisted Surgery, Leipzig, Germany
| |
Collapse
|
49
|
Mühle R, Ernst H, Sobottka SB, Morgenstern U. Workflow and hardware for intraoperative hyperspectral data acquisition in neurosurgery. BIOMED ENG-BIOMED TE 2020; 66:/j/bmte.ahead-of-print/bmt-2019-0333/bmt-2019-0333.xml. [PMID: 32706748 DOI: 10.1515/bmt-2019-0333] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2019] [Accepted: 05/27/2020] [Indexed: 01/18/2023]
Abstract
To prevent further brain tumour growth, malignant tissue should be removed as completely as possible in neurosurgical operations. Therefore, differentiation between tumour and brain tissue as well as detecting functional areas is very important. Hyperspectral imaging (HSI) can be used to get spatial information about brain tissue types and characteristics in a quasi-continuous reflection spectrum. In this paper, workflow and some aspects of an adapted hardware system for intraoperative hyperspectral data acquisition in neurosurgery are discussed. By comparing an intraoperative with a laboratory setup, the influences of the surgical microscope are made visible through the differences in illumination and a pixel- and wavelength-specific signal-to-noise ratio (SNR) calculation. Due to the significant differences in shape and wavelength-dependent intensity of light sources, it can be shown which kind of illumination is most suitable for the setups. Spectra between 550 and 1,000 nm are characterized of at least 40 dB SNR in laboratory and 25 dB in intraoperative setup in an area of the image relevant for evaluation. A first validation of the intraoperative hyperspectral imaging hardware setup shows that all system parts and intraoperatively recorded data can be evaluated. Exemplarily, a classification map was generated that allows visualization of measured properties of raw data. The results reveal that it is possible and beneficial to use HSI for wavelength-related intraoperative data acquisition in neurosurgery. There are still technical facts to optimize for raw data detection prior to adapting image processing algorithms to specify tissue quality and function.
Collapse
Affiliation(s)
- Richard Mühle
- Faculty of Electrical and Computer Engineering, Institute of Biomedical Engineering, Technische Universität Dresden, 01062Dresden, Germany
- Department of Neurosurgery, Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, 01307Dresden, Germany
| | - Hannes Ernst
- Faculty of Electrical and Computer Engineering, Institute of Biomedical Engineering, Technische Universität Dresden, 01062Dresden, Germany
| | - Stephan B Sobottka
- Department of Neurosurgery, Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, 01307Dresden, Germany
| | - Ute Morgenstern
- Faculty of Electrical and Computer Engineering, Institute of Biomedical Engineering, Technische Universität Dresden, 01062Dresden, Germany
| |
Collapse
|
50
|
Clancy NT, Jones G, Maier-Hein L, Elson DS, Stoyanov D. Surgical spectral imaging. Med Image Anal 2020; 63:101699. [PMID: 32375102 PMCID: PMC7903143 DOI: 10.1016/j.media.2020.101699] [Citation(s) in RCA: 62] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2019] [Revised: 03/30/2020] [Accepted: 04/06/2020] [Indexed: 12/24/2022]
Abstract
Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013-2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation.
Collapse
Affiliation(s)
- Neil T Clancy
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, United Kingdom; Centre for Medical Image Computing (CMIC), Department of Medical Physics and Biomedical Engineering, University College London, United Kingdom.
| | - Geoffrey Jones
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, United Kingdom; Centre for Medical Image Computing (CMIC), Department of Computer Science, University College London, United Kingdom
| | | | - Daniel S Elson
- Hamlyn Centre for Robotic Surgery, Institute of Global Health Innovation, Imperial College London, United Kingdom; Department of Surgery and Cancer, Imperial College London, United Kingdom
| | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, United Kingdom; Centre for Medical Image Computing (CMIC), Department of Computer Science, University College London, United Kingdom
| |
Collapse
|