1
|
Li Q, Zhang X, Zhang J, Huang H, Li L, Guo C, Li W, Guo Y. Deep learning-based hyperspectral technique identifies metastatic lymph nodes in oral squamous cell carcinoma-A pilot study. Oral Dis 2024. [PMID: 39005220 DOI: 10.1111/odi.15067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Revised: 05/31/2024] [Accepted: 07/02/2024] [Indexed: 07/16/2024]
Abstract
AIMS To establish a system based on hyperspectral imaging and deep learning for the detection of cancer cells in metastatic lymph nodes. MAIN METHODS The continuous sections of metastatic lymph nodes from 45 oral squamous cell carcinoma (OSCC) patients were collected. An improved ResUNet algorithm was established for deep learning to analyze the spectral curve differences between cancer cells and lymphocytes, and that between tumor tissue and normal tissue. KEY FINDINGS It was found that cancer cells, lymphocytes, and erythrocytes in the metastatic lymph nodes could be distinguished basing hyperspectral image, with overall accuracy (OA) as 87.30% and average accuracy (AA) as 85.46%. Cancerous area could be recognized by hyperspectral image and deep learning, and the average intersection over union (IOU) and accuracy were 0.6253 and 0.7692, respectively. SIGNIFICANCE This study indicated that deep learning-based hyperspectral techniques can identify tumor tissue in OSCC metastatic lymph nodes, achieving high accuracy of pathological diagnosis, high work efficiency, and reducing work burden. But these are preliminary results limited to a small sample.
Collapse
Affiliation(s)
- Qingxiang Li
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Xueyu Zhang
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Jianyun Zhang
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
- Department of Oral Pathology, Peking University School and Hospital of Stomatology, Beijing, China
| | - Hongyuan Huang
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Liangliang Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Chuanbin Guo
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Wei Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Yuxing Guo
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| |
Collapse
|
2
|
Takamatsu T, Fukushima R, Sato K, Umezawa M, Yokota H, Soga K, Hernandez-Guedes A, Callico GM, Takemura H. Development of a visible to 1600 nm hyperspectral imaging rigid-scope system using supercontinuum light and an acousto-optic tunable filter. OPTICS EXPRESS 2024; 32:16090-16102. [PMID: 38859246 DOI: 10.1364/oe.515747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Accepted: 03/22/2024] [Indexed: 06/12/2024]
Abstract
In this study, we developed a rigid-scope system that can perform hyperspectral imaging (HSI) between visible and 1600 nm wavelengths using a supercontinuum light source and an acousto-optic tunable filter to emit specific wavelengths. The system optical performance was verified, and the classification ability was investigated. Consequently, it was demonstrated that HSI (490-1600 nm) could be performed. In addition, seven different targets could be classified by the neural network with an accuracy of 99.6%, recall of 93.7%, and specificity of 99.1% when the wavelength range of over 1000 nm (OTN) was extracted from HSI data as train data.
Collapse
|
3
|
Baffa MDFO, Zezell DM, Bachmann L, Pereira TM, Deserno TM, Felipe JC. Deep neural networks can differentiate thyroid pathologies on infrared hyperspectral images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 247:108100. [PMID: 38442622 DOI: 10.1016/j.cmpb.2024.108100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 02/12/2024] [Accepted: 02/23/2024] [Indexed: 03/07/2024]
Abstract
BACKGROUND AND OBJECTIVE The thyroid is a gland responsible for producing important body hormones. Several pathologies can affect this gland, such as thyroiditis, hypothyroidism, and thyroid cancer. The visual histological analysis of thyroid specimens is a valuable process that enables pathologists to detect diseases with high efficiency, providing the patient with a better prognosis. Existing computer vision systems developed to aid in the analysis of histological samples have limitations in distinguishing pathologies with similar characteristics or samples containing multiple diseases. To overcome this challenge, hyperspectral images are being studied to represent biological samples based on their molecular interaction with light. METHODS In this study, we address the acquisition of infrared absorbance spectra from each voxel of histological specimens. This data is then used for the development of a multiclass fully-connected neural network model that discriminates spectral patterns, enabling the classification of voxels as healthy, cancerous, or goiter. RESULTS Through experiments using the k-fold cross-validation protocol, we obtained an average accuracy of 93.66 %, a sensitivity of 93.47 %, and a specificity of 96.93 %. Our results demonstrate the feasibility of using infrared hyperspectral imaging to characterize healthy tissue and thyroid pathologies using absorbance measurements. The proposed deep learning model has the potential to improve diagnostic efficiency and enhance patient outcomes.
Collapse
Affiliation(s)
| | | | - Luciano Bachmann
- Department of Physics, University of São Paulo, Ribeirão Preto, SP, Brazil
| | - Thiago Martini Pereira
- Department of Science and Technology, Federal University of São Paulo, São José dos Campos, SP, Brazil
| | - Thomas Martin Deserno
- Peter L. Reichertz Institute for Medical Informatics, Technische Universität Braunschweig, Braunschweig, Germany
| | - Joaquim Cezar Felipe
- Department of Computing and Mathematics, University of São Paulo, Ribeirão Preto, SP, Brazil
| |
Collapse
|
4
|
Wise PA, Studier-Fischer A, Nickel F, Hackert T. [Status Quo of Surgical Navigation]. Zentralbl Chir 2023. [PMID: 38056501 DOI: 10.1055/a-2211-4898] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/08/2023]
Abstract
Surgical navigation, also referred to as computer-assisted or image-guided surgery, is a technique that employs a variety of methods - such as 3D imaging, tracking systems, specialised software, and robotics to support surgeons during surgical interventions. These emerging technologies aim not only to enhance the accuracy and precision of surgical procedures, but also to enable less invasive approaches, with the objective of reducing complications and improving operative outcomes for patients. By harnessing the integration of emerging digital technologies, surgical navigation holds the promise of assisting complex procedures across various medical disciplines. In recent years, the field of surgical navigation has witnessed significant advances. Abdominal surgical navigation, particularly endoscopy, laparoscopic, and robot-assisted surgery, is currently undergoing a phase of rapid evolution. Emphases include image-guided navigation, instrument tracking, and the potential integration of augmented and mixed reality (AR, MR). This article will comprehensively delve into the latest developments in surgical navigation, spanning state-of-the-art intraoperative technologies like hyperspectral and fluorescent imaging, to the integration of preoperative radiological imaging within the intraoperative setting.
Collapse
Affiliation(s)
- Philipp Anthony Wise
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Alexander Studier-Fischer
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Felix Nickel
- Klinik für Allgemein-, Viszeral- und Thoraxchirurgie, Universitätsklinikum Hamburg-Eppendorf, Hamburg, Deutschland
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Thilo Hackert
- Klinik für Allgemein-, Viszeral- und Thoraxchirurgie, Universitätsklinikum Hamburg-Eppendorf, Hamburg, Deutschland
| |
Collapse
|
5
|
Zhang W, Suo J, Dong K, Li L, Yuan X, Pei C, Dai Q. Handheld snapshot multi-spectral camera at tens-of-megapixel resolution. Nat Commun 2023; 14:5043. [PMID: 37598234 PMCID: PMC10439928 DOI: 10.1038/s41467-023-40739-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Accepted: 08/07/2023] [Indexed: 08/21/2023] Open
Abstract
Multi-spectral imaging is a fundamental tool characterizing the constituent energy of scene radiation. However, current multi-spectral video cameras cannot scale up beyond megapixel resolution due to optical constraints and the complexity of the reconstruction algorithms. To circumvent the above issues, we propose a tens-of-megapixel handheld multi-spectral videography approach (THETA), with a proof-of-concept camera achieving 65-megapixel videography of 12 wavebands within visible light range. The high performance is brought by multiple designs: We propose an imaging scheme to fabricate a thin mask for encoding spatio-spectral data using a conventional film camera. Afterwards, a fiber optic plate is introduced for building a compact prototype supporting pixel-wise encoding with a large space-bandwidth product. Finally, a deep-network-based algorithm is adopted for large-scale multi-spectral data decoding, with the coding pattern specially designed to facilitate efficient coarse-to-fine model training. Experimentally, we demonstrate THETA's advantageous and wide applications in outdoor imaging of large macroscopic scenes.
Collapse
Affiliation(s)
- Weihang Zhang
- Department of Automation, Tsinghua University, Beijing, 100084, China
| | - Jinli Suo
- Department of Automation, Tsinghua University, Beijing, 100084, China.
- Institute of Brain and Cognitive Sciences, Tsinghua University, Beijing, 10008, China.
- Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China.
| | - Kaiming Dong
- Department of Automation, Tsinghua University, Beijing, 100084, China
| | - Lianglong Li
- Department of Automation, Tsinghua University, Beijing, 100084, China
| | - Xin Yuan
- WestLake University, Hangzhou, 310030, Zhejiang, China
| | | | - Qionghai Dai
- Department of Automation, Tsinghua University, Beijing, 100084, China.
- Institute of Brain and Cognitive Sciences, Tsinghua University, Beijing, 10008, China.
| |
Collapse
|
6
|
Chen R, Luo T, Nie J, Chu Y. Blood cancer diagnosis using hyperspectral imaging combined with the forward searching method and machine learning. ANALYTICAL METHODS : ADVANCING METHODS AND APPLICATIONS 2023; 15:3885-3892. [PMID: 37503555 DOI: 10.1039/d3ay00787a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
Hyperspectral imaging (HSI), a widely used biosensing technique, has been applied to tumor detection. Rapid, accurate, and low-cost detection of blood cancer using hyperspectral technology remains a challenge. We developed a new method to discriminate blood cancer using hyperspectral imaging (HSI) and the forward searching method (FSM). Four commonly used classification models are applied for four types of blood cancer spectra recognition. The support vector machine (SVM) model with the highest recognition accuracy (94.5%) combined with HSI achieves high-precision tumor identification. For higher recognition accuracy and lower hardware barriers, based on the selection probabilities of spectral lines calculated by a multi-objective atomic orbital search method, the FSM is proposed for HSI feature selection. With the proposed method, the wavelength band range of the spectrum is reduced by at least 50%. Compared with the traditional dimensionality reduction methods, the FSM can obtain a higher accuracy rate with lower hardware requirements. These results show that our proposed method can achieve non-invasive rapid screening of blood cancers with lower hardware requirements. Therefore, HSI assisted with FSM and SVM hybrid models can be a powerful and promising tool for blood cancer detection.
Collapse
Affiliation(s)
- Riheng Chen
- Hunan Provincial Key Laboratory of Grids Operation and Control on Multi-Power Sources Area, Shaoyang University, Shaoyang, Hunan, 422000, China.
| | - Ting Luo
- Hunan Provincial Key Laboratory of Grids Operation and Control on Multi-Power Sources Area, Shaoyang University, Shaoyang, Hunan, 422000, China.
| | - Junfei Nie
- Hunan Provincial Key Laboratory of Grids Operation and Control on Multi-Power Sources Area, Shaoyang University, Shaoyang, Hunan, 422000, China.
| | - Yanwu Chu
- Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, Sichuan, 610209, China.
| |
Collapse
|
7
|
Sijilmassi O, López Alonso JM, Del Río Sevilla A, Barrio Asensio MDC. Multispectral Imaging Method for Rapid Identification and Analysis of Paraffin-Embedded Pathological Tissues. J Digit Imaging 2023; 36:1663-1674. [PMID: 37072579 PMCID: PMC10406798 DOI: 10.1007/s10278-023-00826-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Revised: 03/28/2023] [Accepted: 03/30/2023] [Indexed: 04/20/2023] Open
Abstract
The study of the interaction between light and biological tissue is of great help in the identification of diseases as well as structural alterations in tissues. In the present study, we have developed a tissue diagnostic technique by using multispectral imaging in the visible spectrum combined with principal component analysis (PCA). We used information from the propagation of light through paraffin-embedded tissues to assess differences in the eye tissues of control mouse embryos compared to mouse embryos whose mothers were deprived of folic acid (FA), a crucial vitamin necessary for the growth and development of the fetus. After acquiring the endmembers from the multispectral images, spectral unmixing was used to identify the abundances of those endmembers in each pixel. For each acquired image, the final analysis was performed by performing a pixel-by-pixel and wavelength-by-wavelength absorbance calculation. Non-negative least squares (NNLS) were used in this research. The abundance maps obtained for the first endmember revealed vascular alterations (vitreous and choroid) in the embryos with maternal FA deficiency. However, the abundance maps obtained for the third endmember showed alterations in the texture of some tissues such as the lens and retina. Results indicated that multispectral imaging applied to paraffin-embedded tissues enhanced tissue visualization. Using this method, first, it can be seen tissue damage location and then decide what kind of biological techniques to apply.
Collapse
Affiliation(s)
- Ouafa Sijilmassi
- Faculty of Optics and Optometry, Anatomy and Embryology Department, Universidad Complutense de Madrid, Madrid, Spain.
- Optics Department, Faculty of Optics and Optometry, Universidad Complutense De Madrid, Madrid, Spain.
| | - José-Manuel López Alonso
- Optics Department, Faculty of Optics and Optometry, Universidad Complutense De Madrid, Madrid, Spain
| | - Aurora Del Río Sevilla
- Faculty of Optics and Optometry, Anatomy and Embryology Department, Universidad Complutense de Madrid, Madrid, Spain
| | | |
Collapse
|
8
|
Studier-Fischer A, Seidlitz S, Sellner J, Bressan M, Özdemir B, Ayala L, Odenthal J, Knoedler S, Kowalewski KF, Haney CM, Salg G, Dietrich M, Kenngott H, Gockel I, Hackert T, Müller-Stich BP, Maier-Hein L, Nickel F. HeiPorSPECTRAL - the Heidelberg Porcine HyperSPECTRAL Imaging Dataset of 20 Physiological Organs. Sci Data 2023; 10:414. [PMID: 37355750 PMCID: PMC10290660 DOI: 10.1038/s41597-023-02315-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 06/15/2023] [Indexed: 06/26/2023] Open
Abstract
Hyperspectral Imaging (HSI) is a relatively new medical imaging modality that exploits an area of diagnostic potential formerly untouched. Although exploratory translational and clinical studies exist, no surgical HSI datasets are openly accessible to the general scientific community. To address this bottleneck, this publication releases HeiPorSPECTRAL ( https://www.heiporspectral.org ; https://doi.org/10.5281/zenodo.7737674 ), the first annotated high-quality standardized surgical HSI dataset. It comprises 5,758 spectral images acquired with the TIVITA® Tissue and annotated with 20 physiological porcine organs from 8 pigs per organ distributed over a total number of 11 pigs. Each HSI image features a resolution of 480 × 640 pixels acquired over the 500-1000 nm wavelength range. The acquisition protocol has been designed such that the variability of organ spectra as a function of several parameters including the camera angle and the individual can be assessed. A comprehensive technical validation confirmed both the quality of the raw data and the annotations. We envision potential reuse within this dataset, but also its reuse as baseline data for future research questions outside this dataset. Measurement(s) Spectral Reflectance Technology Type(s) Hyperspectral Imaging Sample Characteristic - Organism Sus scrofa.
Collapse
Affiliation(s)
- Alexander Studier-Fischer
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe, Heidelberg, Germany
- National Center for Tumor Diseases (NCT) Heidelberg, a partnership between DKFZ and Heidelberg University Hospital, Heidelberg, Germany
| | - Jan Sellner
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe, Heidelberg, Germany
| | - Marc Bressan
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Berkin Özdemir
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Leonardo Ayala
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
| | - Jan Odenthal
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Samuel Knoedler
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
- Division of Plastic Surgery, Department of Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Karl-Friedrich Kowalewski
- Department of Urology, Medical Faculty of Mannheim at the University of Heidelberg, Mannheim, Germany
| | - Caelan Max Haney
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Gabriel Salg
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Maximilian Dietrich
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - Hannes Kenngott
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, Leipzig University Hospital, Leipzig, Germany
| | - Thilo Hackert
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
- Department of General, Visceral, and Thoracic Surgery, University Hospital Hamburg-Eppendorf, Hamburg, Germany
| | - Beat Peter Müller-Stich
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe, Heidelberg, Germany
- National Center for Tumor Diseases (NCT) Heidelberg, a partnership between DKFZ and Heidelberg University Hospital, Heidelberg, Germany
| | - Felix Nickel
- Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany.
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe, Heidelberg, Germany.
- Department of General, Visceral, and Thoracic Surgery, University Hospital Hamburg-Eppendorf, Hamburg, Germany.
| |
Collapse
|
9
|
Collins T, Bencteux V, Benedicenti S, Moretti V, Mita MT, Barbieri V, Rubichi F, Altamura A, Giaracuni G, Marescaux J, Hostettler A, Diana M, Viola MG, Barberio M. Automatic optical biopsy for colorectal cancer using hyperspectral imaging and artificial neural networks. Surg Endosc 2022; 36:8549-8559. [PMID: 36008640 DOI: 10.1007/s00464-022-09524-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Accepted: 07/31/2022] [Indexed: 02/07/2023]
Abstract
BACKGROUND Intraoperative identification of cancerous tissue is fundamental during oncological surgical or endoscopic procedures. This relies on visual assessment supported by histopathological evaluation, implying a longer operative time. Hyperspectral imaging (HSI), a contrast-free and contactless imaging technology, provides spatially resolved spectroscopic analysis, with the potential to differentiate tissue at a cellular level. However, HSI produces "big data", which is impossible to directly interpret by clinicians. We hypothesize that advanced machine learning algorithms (convolutional neural networks-CNNs) can accurately detect colorectal cancer in HSI data. METHODS In 34 patients undergoing colorectal resections for cancer, immediately after extraction, the specimen was opened, the tumor-bearing section was exposed and imaged using HSI. Cancer and normal mucosa were categorized from histopathology. A state-of-the-art CNN was developed to automatically detect regions of colorectal cancer in a hyperspectral image. Accuracy was validated with three levels of cross-validation (twofold, fivefold, and 15-fold). RESULTS 32 patients had colorectal adenocarcinomas confirmed by histopathology (9 left, 11 right, 4 transverse colon, and 9 rectum). 6 patients had a local initial stage (T1-2) and 26 had a local advanced stage (T3-4). The cancer detection performance of the CNN using 15-fold cross-validation showed high sensitivity and specificity (87% and 90%, respectively) and a ROC-AUC score of 0.95 (considered outstanding). In the T1-2 group, the sensitivity and specificity were 89% and 90%, respectively, and in the T3-4 group, the sensitivity and specificity were 81% and 93%, respectively. CONCLUSIONS Automatic colorectal cancer detection on fresh specimens using HSI, using a properly trained CNN is feasible and accurate, even with small datasets, regardless of the local tumor extension. In the near future, this approach may become a useful intraoperative tool during oncological endoscopic and surgical procedures, and may result in precise and non-destructive optical biopsies to support objective and consistent tumor-free resection margins.
Collapse
Affiliation(s)
- Toby Collins
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France.
- Research Institute Against Digestive Cancer (IRCAD Africa), Kigali, Rwanda.
| | - Valentin Bencteux
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- ICUBE Laboratory, Photonics Instrumentation for Health, Strasbourg, France
| | | | | | | | | | | | - Amedeo Altamura
- Department of Surgery, Ospedale Card. G. Panico, Tricase, Italy
| | | | - Jacques Marescaux
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- Research Institute Against Digestive Cancer (IRCAD Africa), Kigali, Rwanda
| | - Alex Hostettler
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- Research Institute Against Digestive Cancer (IRCAD Africa), Kigali, Rwanda
| | - Michele Diana
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- ICUBE Laboratory, Photonics Instrumentation for Health, Strasbourg, France
| | | | - Manuel Barberio
- Research Institute Against Digestive Cancer (IRCAD France), Strasbourg, France
- Department of Surgery, Ospedale Card. G. Panico, Tricase, Italy
| |
Collapse
|
10
|
Hao S, Xiong Y, Zheng M, Wang J. Bimodal polarization-sensitive imaging and diffuse reflectance spectroscopy facilitated rapid augmented reality for tissue stratification. JOURNAL OF BIOPHOTONICS 2022; 15:e202200101. [PMID: 35852139 DOI: 10.1002/jbio.202200101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Revised: 06/29/2022] [Accepted: 07/01/2022] [Indexed: 06/15/2023]
Abstract
We reported the development of a bimodal large field of view (~95 cm2 ) polarization-sensitive (PS) imaging and point-wise diffuse reflectance (DR) spectroscopy platform synergising the advantages of the two modalities. PS-DR spectroscopy allows simultaneous label-free biochemical and structural analysis of the investigated tissue. For real applications, the PS imaging first grossly stratified tissue into different categories, followed by high information dimension PS-DR spectroscopy ascertaining ambiguous tissue regions identified by the PS imaging. Tested on a tissue phantom consisting of porcine esophagus muscularis propria and submucosa, it has demonstrated the system developed could complete tissue interrogation within 0.6 minutes and with separation accuracy of 95.87%. Further test on tissue phantom consisting of natural and crushed bovine tendon mimicking normal and cancerous breast tissues validated the potential of system developed to effectively and rapidly investigate large tissue area, facilitating augmented tissue boundary demarcation.
Collapse
Affiliation(s)
- Shicheng Hao
- Key Laboratory of Photoelectronic Imaging Technology and System of Ministry of Education of China, School of Optics and Photonics, Beijing Institute of Technology, Beijing, China
| | - Ying Xiong
- Key Laboratory of Photoelectronic Imaging Technology and System of Ministry of Education of China, School of Optics and Photonics, Beijing Institute of Technology, Beijing, China
| | - Meng Zheng
- Key Laboratory of Photoelectronic Imaging Technology and System of Ministry of Education of China, School of Optics and Photonics, Beijing Institute of Technology, Beijing, China
| | - Jianfeng Wang
- Key Laboratory of Photoelectronic Imaging Technology and System of Ministry of Education of China, School of Optics and Photonics, Beijing Institute of Technology, Beijing, China
- Institute of Engineering Medicine, Beijing Institute of Technology, Beijing, China
| |
Collapse
|
11
|
Huang SY, Mukundan A, Tsao YM, Kim Y, Lin FC, Wang HC. Recent Advances in Counterfeit Art, Document, Photo, Hologram, and Currency Detection Using Hyperspectral Imaging. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22197308. [PMID: 36236407 PMCID: PMC9571956 DOI: 10.3390/s22197308] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 09/15/2022] [Accepted: 09/23/2022] [Indexed: 05/08/2023]
Abstract
Forgery and tampering continue to provide unnecessary economic burdens. Although new anti-forgery and counterfeiting technologies arise, they inadvertently lead to the sophistication of forgery techniques over time, to a point where detection is no longer viable without technological aid. Among the various optical techniques, one of the recently used techniques to detect counterfeit products is HSI, which captures a range of electromagnetic data. To aid in the further exploration and eventual application of the technique, this study categorizes and summarizes existing related studies on hyperspectral imaging and creates a mini meta-analysis of this stream of literature. The literature review has been classified based on the product HSI has used in counterfeit documents, photos, holograms, artwork, and currency detection.
Collapse
Affiliation(s)
- Shuan-Yu Huang
- Department of Optometry, Central Taiwan University of Science and Technology, No. 666, Buzih Road, Beitun District, Taichung City 406053, Taiwan
| | - Arvind Mukundan
- Department of Mechanical Engineering, Advanced Institute of Manufacturing with High Tech Innovations (AIM-HI), Center for Innovative Research on Aging Society (CIRAS), National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Yu-Ming Tsao
- Department of Mechanical Engineering, Advanced Institute of Manufacturing with High Tech Innovations (AIM-HI), Center for Innovative Research on Aging Society (CIRAS), National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Youngjo Kim
- Department of Mechanical Engineering, Far Eastern University, P. Paredes St., Sampaloc, Manila 1015, Philippines
| | - Fen-Chi Lin
- Department of Ophthalmology, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st Rd., Lingya District, Kaohsiung City 80284, Taiwan
- Correspondence: (F.-C.L.); (H.-C.W.)
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, Advanced Institute of Manufacturing with High Tech Innovations (AIM-HI), Center for Innovative Research on Aging Society (CIRAS), National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
- Correspondence: (F.-C.L.); (H.-C.W.)
| |
Collapse
|
12
|
Okamoto N, Rodríguez-Luna MR, Bencteux V, Al-Taher M, Cinelli L, Felli E, Urade T, Nkusi R, Mutter D, Marescaux J, Hostettler A, Collins T, Diana M. Computer-Assisted Differentiation between Colon-Mesocolon and Retroperitoneum Using Hyperspectral Imaging (HSI) Technology. Diagnostics (Basel) 2022; 12:diagnostics12092225. [PMID: 36140626 PMCID: PMC9497769 DOI: 10.3390/diagnostics12092225] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 09/10/2022] [Accepted: 09/12/2022] [Indexed: 12/01/2022] Open
Abstract
Complete mesocolic excision (CME), which involves the adequate resection of the tumor-bearing colonic segment with “en bloc” removal of its mesocolon along embryological fascial planes is associated with superior oncological outcomes. However, CME presents a higher complication rate compared to non-CME resections due to a higher risk of vascular injury. Hyperspectral imaging (HSI) is a contrast-free optical imaging technology, which facilitates the quantitative imaging of physiological tissue parameters and the visualization of anatomical structures. This study evaluates the accuracy of HSI combined with deep learning (DL) to differentiate the colon and its mesenteric tissue from retroperitoneal tissue. In an animal study including 20 pig models, intraoperative hyperspectral images of the sigmoid colon, sigmoid mesentery, and retroperitoneum were recorded. A convolutional neural network (CNN) was trained to distinguish the two tissue classes using HSI data, validated with a leave-one-out cross-validation process. The overall recognition sensitivity of the tissues to be preserved (retroperitoneum) and the tissues to be resected (colon and mesentery) was 79.0 ± 21.0% and 86.0 ± 16.0%, respectively. Automatic classification based on HSI and CNNs is a promising tool to automatically, non-invasively, and objectively differentiate the colon and its mesentery from retroperitoneal tissue.
Collapse
Affiliation(s)
- Nariaki Okamoto
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- ICube Laboratory, Photonics Instrumentation for Health, 67081 Strasbourg, France
- Correspondence:
| | - María Rita Rodríguez-Luna
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- ICube Laboratory, Photonics Instrumentation for Health, 67081 Strasbourg, France
| | - Valentin Bencteux
- ICube Laboratory, Photonics Instrumentation for Health, 67081 Strasbourg, France
| | - Mahdi Al-Taher
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Department of Surgery, Maastricht University Medical Center, 6229 ER Maastricht, The Netherlands
| | - Lorenzo Cinelli
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Department of Gastrointestinal Surgery, San Raffaele Hospital IRCCS, 20132 Milan, Italy
| | - Eric Felli
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
| | - Takeshi Urade
- Department of Surgery, Division of Hepato-Biliary-Pancreatic Surgery, Kobe University Graduate School of Medicine, Kobe 6500017, Japan
| | - Richard Nkusi
- Research Institute against Digestive Cancer (IRCAD), Kigali, Rwanda
| | - Didier Mutter
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Department of Digestive and Endocrine Surgery, Nouvel Hôpital Civil, University of Strasbourg, 67091 Strasbourg, France
- IHU-Strasbourg—Institut de Chirurgie Guidée par L’image, 67091 Strasbourg, France
| | - Jacques Marescaux
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
| | - Alexandre Hostettler
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Research Institute against Digestive Cancer (IRCAD), Kigali, Rwanda
| | - Toby Collins
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- Research Institute against Digestive Cancer (IRCAD), Kigali, Rwanda
| | - Michele Diana
- Research Institute against Digestive Cancer (IRCAD), 67091 Strasbourg, France
- ICube Laboratory, Photonics Instrumentation for Health, 67081 Strasbourg, France
| |
Collapse
|
13
|
Tumor cell identification and classification in esophageal adenocarcinoma specimens by hyperspectral imaging. Sci Rep 2022; 12:4508. [PMID: 35296685 PMCID: PMC8927097 DOI: 10.1038/s41598-022-07524-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Accepted: 02/17/2022] [Indexed: 12/24/2022] Open
Abstract
Esophageal cancer is the sixth leading cause of cancer-related death worldwide. Histopathological confirmation is a key step in tumor diagnosis. Therefore, simplification in decision-making by discrimination between malignant and non-malignant cells of histological specimens can be provided by combination of new imaging technology and artificial intelligence (AI). In this work, hyperspectral imaging (HSI) data from 95 patients were used to classify three different histopathological features (squamous epithelium cells, esophageal adenocarcinoma (EAC) cells, and tumor stroma cells), based on a multi-layer perceptron with two hidden layers. We achieved an accuracy of 78% for EAC and stroma cells, and 80% for squamous epithelium. HSI combined with machine learning algorithms is a promising and innovative technique, which allows image acquisition beyond Red–Green–Blue (RGB) images. Further method validation and standardization will be necessary, before automated tumor cell identification algorithms can be used in daily clinical practice.
Collapse
|
14
|
Border Line Definition Using Hyperspectral Imaging in Colorectal Resections. Cancers (Basel) 2022; 14:cancers14051188. [PMID: 35267496 PMCID: PMC8909141 DOI: 10.3390/cancers14051188] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Revised: 02/14/2022] [Accepted: 02/16/2022] [Indexed: 02/04/2023] Open
Abstract
Simple Summary Good oxygenation of both bowel ends is an important prerequisite to promote anastomotic healing after colorectal resections. Bowel oxygenation is usually assessed clinically. Hyperspectral imaging is a contactless and contrast-free tool that allows quantifying tissue oxygen intraoperatively. In this study, the results of 105 colorectal resections with hyperspectral imaging are reported. Abstract Background: A perfusion deficit is a well-defined and intraoperatively influenceable cause of anastomotic leak (AL). Current intraoperative perfusion assessment methods do not provide objective and quantitative results. In this study, the ability of hyperspectral imaging (HSI) to quantify tissue oxygenation intraoperatively was assessed. Methods: 115 patients undergoing colorectal resections were included in the final analysis. Before anastomotic formation, the bowel was extracted and the resection line was outlined and imaged using a compact HSI camera, in order to provide instantaneously quantitative perfusion assessment. Results: In 105 patients, a clear demarcation line was visible with HSI one minute after marginal artery transection, reaching a plateau after 3 min. In 58 (55.2%) patients, the clinically determined transection line matched with HSI. In 23 (21.9%) patients, the clinically established resection margin was entirely within the less perfused area. In 24 patients (22.8%), the HSI transection line had an irregular course and crossed the clinically established resection line. In four cases, HSI disclosed a clinically undetected lesion of the marginal artery. Conclusions: Intraoperative HSI is safe, well reproducible, and does not disrupt the surgical workflow. It also quantifies bowel surface perfusion. HSI might become an intraoperative guidance tool, potentially preventing postoperative complications.
Collapse
|
15
|
Image Correction and In Situ Spectral Calibration for Low-Cost, Smartphone Hyperspectral Imaging. REMOTE SENSING 2022. [DOI: 10.3390/rs14051152] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
Developments in the portability of low-cost hyperspectral imaging instruments translate to significant benefits to agricultural industries and environmental monitoring applications. These advances can be further explicated by removing the need for complex post-processing and calibration. We propose a method for substantially increasing the utility of portable hyperspectral imaging. Vertical and horizontal spatial distortions introduced into images by ‘operator shake’ are corrected by an in-scene reference card with two spatial references. In situ light-source-independent spectral calibration is performed. This is achieved by a comparison of the ground-truth spectral reflectance of an in-scene red–green–blue target to the uncalibrated output of the hyperspectral data. Finally, bias introduced into the hyperspectral images due to the non-flat spectral output of the illumination is removed. This allows for low-skilled operation of a truly handheld, low-cost hyperspectral imager for agriculture, environmental monitoring, or other visible hyperspectral imaging applications.
Collapse
|
16
|
Knospe L, Gockel I, Jansen-Winkeln B, Thieme R, Niebisch S, Moulla Y, Stelzner S, Lyros O, Diana M, Marescaux J, Chalopin C, Köhler H, Pfahl A, Maktabi M, Park JH, Yang HK. New Intraoperative Imaging Tools and Image-Guided Surgery in Gastric Cancer Surgery. Diagnostics (Basel) 2022; 12:diagnostics12020507. [PMID: 35204597 PMCID: PMC8871069 DOI: 10.3390/diagnostics12020507] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Revised: 02/06/2022] [Accepted: 02/10/2022] [Indexed: 02/05/2023] Open
Abstract
Innovations and new advancements in intraoperative real-time imaging have gained significant importance in the field of gastric cancer surgery in the recent past. Currently, the most promising procedures include indocyanine green fluorescence imaging (ICG-FI) and hyperspectral imaging or multispectral imaging (HSI, MSI). ICG-FI is utilized in a broad range of clinical applications, e.g., assessment of perfusion or lymphatic drainage, and additional implementations are currently investigated. HSI is still in the experimental phase and its value and clinical relevance require further evaluation, but initial studies have shown a successful application in perfusion assessment, and prospects concerning non-invasive tissue and tumor classification are promising. The application of machine learning and artificial intelligence technologies might enable an automatic evaluation of the acquired image data in the future. Both methods facilitate the accurate visualization of tissue characteristics that are initially indistinguishable for the human eye. By aiding surgeons in optimizing the surgical procedure, image-guided surgery can contribute to the oncologic safety and reduction of complications in gastric cancer surgery and recent advances hold promise for the application of HSI in intraoperative tissue diagnostics.
Collapse
Affiliation(s)
- Luise Knospe
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
- Correspondence:
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
- Department of General, Visceral and Oncological Surgery, St. Georg Hospital, 04129 Leipzig, Germany
| | - René Thieme
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Stefan Niebisch
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Sigmar Stelzner
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Orestis Lyros
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Michele Diana
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.D.); (J.M.)
- ICUBE Laboratory, Photonics Instrumentation for Health, University of Strasbourg, 67400 Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, 67091 Strasbourg, France
| | - Jacques Marescaux
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.D.); (J.M.)
| | - Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, 04103 Leipzig, Germany; (C.C.); (H.K.); (A.P.); (M.M.)
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, 04103 Leipzig, Germany; (C.C.); (H.K.); (A.P.); (M.M.)
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, 04103 Leipzig, Germany; (C.C.); (H.K.); (A.P.); (M.M.)
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, 04103 Leipzig, Germany; (C.C.); (H.K.); (A.P.); (M.M.)
| | - Ji-Hyeon Park
- Department of Surgery, Seoul National University Hospital, Seoul 03080, Korea; (J.-H.P.); (H.-K.Y.)
| | - Han-Kwang Yang
- Department of Surgery, Seoul National University Hospital, Seoul 03080, Korea; (J.-H.P.); (H.-K.Y.)
| |
Collapse
|
17
|
Alabi RO, Almangush A, Elmusrati M, Mäkitie AA. Deep Machine Learning for Oral Cancer: From Precise Diagnosis to Precision Medicine. FRONTIERS IN ORAL HEALTH 2022; 2:794248. [PMID: 35088057 PMCID: PMC8786902 DOI: 10.3389/froh.2021.794248] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Accepted: 12/13/2021] [Indexed: 12/21/2022] Open
Abstract
Oral squamous cell carcinoma (OSCC) is one of the most prevalent cancers worldwide and its incidence is on the rise in many populations. The high incidence rate, late diagnosis, and improper treatment planning still form a significant concern. Diagnosis at an early-stage is important for better prognosis, treatment, and survival. Despite the recent improvement in the understanding of the molecular mechanisms, late diagnosis and approach toward precision medicine for OSCC patients remain a challenge. To enhance precision medicine, deep machine learning technique has been touted to enhance early detection, and consequently to reduce cancer-specific mortality and morbidity. This technique has been reported to have made a significant progress in data extraction and analysis of vital information in medical imaging in recent years. Therefore, it has the potential to assist in the early-stage detection of oral squamous cell carcinoma. Furthermore, automated image analysis can assist pathologists and clinicians to make an informed decision regarding cancer patients. This article discusses the technical knowledge and algorithms of deep learning for OSCC. It examines the application of deep learning technology in cancer detection, image classification, segmentation and synthesis, and treatment planning. Finally, we discuss how this technique can assist in precision medicine and the future perspective of deep learning technology in oral squamous cell carcinoma.
Collapse
Affiliation(s)
- Rasheed Omobolaji Alabi
- Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Department of Industrial Digitalization, School of Technology and Innovations, University of Vaasa, Vaasa, Finland
| | - Alhadi Almangush
- Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Department of Pathology, University of Helsinki, Helsinki, Finland
- Institute of Biomedicine, Pathology, University of Turku, Turku, Finland
| | - Mohammed Elmusrati
- Department of Industrial Digitalization, School of Technology and Innovations, University of Vaasa, Vaasa, Finland
| | - Antti A. Mäkitie
- Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Department of Otorhinolaryngology – Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
- Division of Ear, Nose and Throat Diseases, Department of Clinical Sciences, Intervention and Technology, Karolinska Institute and Karolinska University Hospital, Stockholm, Sweden
| |
Collapse
|
18
|
Bassler MC, Stefanakis M, Sequeira I, Ostertag E, Wagner A, Bartsch JW, Roeßler M, Mandic R, Reddmann EF, Lorenz A, Rebner K, Brecht M. Comparison of Whiskbroom and Pushbroom darkfield elastic light scattering spectroscopic imaging for head and neck cancer identification in a mouse model. Anal Bioanal Chem 2021; 413:7363-7383. [PMID: 34799750 PMCID: PMC8626402 DOI: 10.1007/s00216-021-03726-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2021] [Revised: 08/30/2021] [Accepted: 10/08/2021] [Indexed: 12/20/2022]
Abstract
The early detection of head and neck cancer is a prolonged challenging task. It requires a precise and accurate identification of tissue alterations as well as a distinct discrimination of cancerous from healthy tissue areas. A novel approach for this purpose uses microspectroscopic techniques with special focus on hyperspectral imaging (HSI) methods. Our proof-of-principle study presents the implementation and application of darkfield elastic light scattering spectroscopy (DF ELSS) as a non-destructive, high-resolution, and fast imaging modality to distinguish lingual healthy from altered tissue regions in a mouse model. The main aspect of our study deals with the comparison of two varying HSI detection principles, which are a point-by-point and line scanning imaging, and whether one might be more appropriate in differentiating several tissue types. Statistical models are formed by deploying a principal component analysis (PCA) with the Bayesian discriminant analysis (DA) on the elastic light scattering (ELS) spectra. Overall accuracy, sensitivity, and precision values of 98% are achieved for both models whereas the overall specificity results in 99%. An additional classification of model-unknown ELS spectra is performed. The predictions are verified with histopathological evaluations of identical HE-stained tissue areas to prove the model’s capability of tissue distinction. In the context of our proof-of-principle study, we assess the Pushbroom PCA-DA model to be more suitable for tissue type differentiations and thus tissue classification. In addition to the HE-examination in head and neck cancer diagnosis, the usage of HSI-based statistical models might be conceivable in a daily clinical routine.
Collapse
Affiliation(s)
- Miriam C Bassler
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
- Institute of Physical and Theoretical Chemistry, University of Tübingen, Auf der Morgenstelle 18, 72076, Tübingen, Germany
| | - Mona Stefanakis
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
- Institute of Physical and Theoretical Chemistry, University of Tübingen, Auf der Morgenstelle 18, 72076, Tübingen, Germany
| | - Inês Sequeira
- Institute of Dentistry, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
| | - Edwin Ostertag
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
| | - Alexandra Wagner
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
- Institute of Physical and Theoretical Chemistry, University of Tübingen, Auf der Morgenstelle 18, 72076, Tübingen, Germany
| | - Jörg W Bartsch
- Department of Neurosurgery, Philipps University Marburg, Baldingerstraße, 35033, Marburg, Germany
| | - Marion Roeßler
- Department of Pathology, Philipps University Marburg, Baldingerstraße, 35033, Marburg, Germany
| | - Robert Mandic
- Department of Otorhinolaryngology, Philipps University Marburg, Baldingerstraße, 35033, Marburg, Germany
| | - Eike F Reddmann
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
| | - Anita Lorenz
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
| | - Karsten Rebner
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
| | - Marc Brecht
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany.
- Institute of Physical and Theoretical Chemistry, University of Tübingen, Auf der Morgenstelle 18, 72076, Tübingen, Germany.
| |
Collapse
|
19
|
|
20
|
Cihan M, Ceylan M, Soylu H, Konak M. Fast Evaluation of Unhealthy and Healthy Neonates Using Hyperspectral Features on 700-850 Nm Wavelengths, ROI Extraction, and 3D-CNN. Ing Rech Biomed 2021. [DOI: 10.1016/j.irbm.2021.06.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
21
|
Ren R, Luo H, Su C, Yao Y, Liao W. Machine learning in dental, oral and craniofacial imaging: a review of recent progress. PeerJ 2021; 9:e11451. [PMID: 34046262 PMCID: PMC8136280 DOI: 10.7717/peerj.11451] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2020] [Accepted: 04/22/2021] [Indexed: 02/05/2023] Open
Abstract
Artificial intelligence has been emerging as an increasingly important aspect of our daily lives and is widely applied in medical science. One major application of artificial intelligence in medical science is medical imaging. As a major component of artificial intelligence, many machine learning models are applied in medical diagnosis and treatment with the advancement of technology and medical imaging facilities. The popularity of convolutional neural network in dental, oral and craniofacial imaging is heightening, as it has been continually applied to a broader spectrum of scientific studies. Our manuscript reviews the fundamental principles and rationales behind machine learning, and summarizes its research progress and its recent applications specifically in dental, oral and craniofacial imaging. It also reviews the problems that remain to be resolved and evaluates the prospect of the future development of this field of scientific study.
Collapse
Affiliation(s)
- Ruiyang Ren
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, West China School of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Haozhe Luo
- School of Computer Science, Sichuan University, Chengdu, Sichuan, China
| | - Chongying Su
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, West China School of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Yang Yao
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Implantology, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Wen Liao
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
- Department of Orthodontics, Osaka Dental University, Hirakata, Osaka, Japan
| |
Collapse
|
22
|
Takamatsu T, Kitagawa Y, Akimoto K, Iwanami R, Endo Y, Takashima K, Okubo K, Umezawa M, Kuwata T, Sato D, Kadota T, Mitsui T, Ikematsu H, Yokota H, Soga K, Takemura H. Over 1000 nm Near-Infrared Multispectral Imaging System for Laparoscopic In Vivo Imaging. SENSORS (BASEL, SWITZERLAND) 2021; 21:2649. [PMID: 33918935 PMCID: PMC8069262 DOI: 10.3390/s21082649] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/25/2021] [Revised: 04/01/2021] [Accepted: 04/06/2021] [Indexed: 01/17/2023]
Abstract
In this study, a laparoscopic imaging device and a light source able to select wavelengths by bandpass filters were developed to perform multispectral imaging (MSI) using over 1000 nm near-infrared (OTN-NIR) on regions under a laparoscope. Subsequently, MSI (wavelengths: 1000-1400 nm) was performed using the built device on nine live mice before and after tumor implantation. The normal and tumor pixels captured within the mice were used as teaching data sets, and the tumor-implanted mice data were classified using a neural network applied following a leave-one-out cross-validation procedure. The system provided a specificity of 89.5%, a sensitivity of 53.5%, and an accuracy of 87.8% for subcutaneous tumor discrimination. Aggregated true-positive (TP) pixels were confirmed in all tumor-implanted mice, which indicated that the laparoscopic OTN-NIR MSI could potentially be applied in vivo for classifying target lesions such as cancer in deep tissues.
Collapse
Affiliation(s)
- Toshihiro Takamatsu
- Exploratory Oncology Research & Clinical Trial Center, National Cancer Center, Kashiwa, Chiba 277-8577, Japan;
- Research Institute for Biomedical Sciences, Tokyo University of Science, Noda, Chiba 278-8510, Japan; (K.S.); (H.T.)
| | - Yuichi Kitagawa
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika, Tokyo 162-8601, Japan; (Y.K.); (R.I.); (K.O.); (M.U.)
| | - Kohei Akimoto
- Department of Mechanical Engineering, Tokyo University of Science, Noda, Chiba 278-8510, Japan; (K.A.); (Y.E.)
| | - Ren Iwanami
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika, Tokyo 162-8601, Japan; (Y.K.); (R.I.); (K.O.); (M.U.)
| | - Yuto Endo
- Department of Mechanical Engineering, Tokyo University of Science, Noda, Chiba 278-8510, Japan; (K.A.); (Y.E.)
| | - Kenji Takashima
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba 277-8577, Japan; (K.T.); (D.S.); (T.K.); (T.M.)
| | - Kyohei Okubo
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika, Tokyo 162-8601, Japan; (Y.K.); (R.I.); (K.O.); (M.U.)
| | - Masakazu Umezawa
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika, Tokyo 162-8601, Japan; (Y.K.); (R.I.); (K.O.); (M.U.)
| | - Takeshi Kuwata
- Department of Pathology and Clinical Laboratories, National Cancer Center Hospital East, Kashiwa, Chiba 277-8577, Japan;
| | - Daiki Sato
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba 277-8577, Japan; (K.T.); (D.S.); (T.K.); (T.M.)
| | - Tomohiro Kadota
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba 277-8577, Japan; (K.T.); (D.S.); (T.K.); (T.M.)
| | - Tomohiro Mitsui
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba 277-8577, Japan; (K.T.); (D.S.); (T.K.); (T.M.)
| | - Hiroaki Ikematsu
- Exploratory Oncology Research & Clinical Trial Center, National Cancer Center, Kashiwa, Chiba 277-8577, Japan;
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba 277-8577, Japan; (K.T.); (D.S.); (T.K.); (T.M.)
| | - Hideo Yokota
- RIKEN Center for Advanced Photonics, Wako, Saitama 351-0198, Japan;
| | - Kohei Soga
- Research Institute for Biomedical Sciences, Tokyo University of Science, Noda, Chiba 278-8510, Japan; (K.S.); (H.T.)
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika, Tokyo 162-8601, Japan; (Y.K.); (R.I.); (K.O.); (M.U.)
| | - Hiroshi Takemura
- Research Institute for Biomedical Sciences, Tokyo University of Science, Noda, Chiba 278-8510, Japan; (K.S.); (H.T.)
- Department of Mechanical Engineering, Tokyo University of Science, Noda, Chiba 278-8510, Japan; (K.A.); (Y.E.)
| |
Collapse
|
23
|
Trajanovski S, Shan C, Weijtmans PJC, de Koning SGB, Ruers TJM. Tongue Tumor Detection in Hyperspectral Images Using Deep Learning Semantic Segmentation. IEEE Trans Biomed Eng 2021; 68:1330-1340. [PMID: 32976092 DOI: 10.1109/tbme.2020.3026683] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
OBJECTIVE The utilization of hyperspectral imaging (HSI) in real-time tumor segmentation during a surgery have recently received much attention, but it remains a very challenging task. METHODS In this work, we propose semantic segmentation methods, and compare them with other relevant deep learning algorithms for tongue tumor segmentation. To the best of our knowledge, this is the first work using deep learning semantic segmentation for tumor detection in HSI data using channel selection, and accounting for more spatial tissue context, and global comparison between the prediction map, and the annotation per sample. Results, and Conclusion: On a clinical data set with tongue squamous cell carcinoma, our best method obtains very strong results of average dice coefficient, and area under the ROC-curve of [Formula: see text], and [Formula: see text], respectively on the original spatial image size. The results show that a very good performance can be achieved even with a limited amount of data. We demonstrate that important information regarding tumor decision is encoded in various channels, but some channel selection, and filtering is beneficial over the full spectra. Moreover, we use both visual (VIS), and near-infrared (NIR) spectrum, rather than commonly used only VIS spectrum; although VIS spectrum is generally of higher significance, we demonstrate NIR spectrum is crucial for tumor capturing in some cases. SIGNIFICANCE The HSI technology augmented with accurate deep learning algorithms has a huge potential to be a promising alternative to digital pathology or a doctors' supportive tool in real-time surgeries.
Collapse
|
24
|
Aboughaleb IH, Matboli M, Shawky SM, El-Sharkawy YH. Integration of transcriptomes analysis with spectral signature of total RNA for generation of affordable remote sensing of Hepatocellular carcinoma in serum clinical specimens. Heliyon 2021; 7:e06388. [PMID: 33748469 PMCID: PMC7972971 DOI: 10.1016/j.heliyon.2021.e06388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2020] [Revised: 01/08/2021] [Accepted: 02/25/2021] [Indexed: 12/24/2022] Open
Abstract
Hepatocellular carcinoma (HCC) is a major global health problem with about 841,000 new cases and 782,000 deaths annually, due to lacking early biomarker/s, and centralized diagnosis. Transcriptomes research despite its infancy has proved excellence in its implementation in identifying a coherent specific cancer RNAs differential expression. However, results are sometimes overlapped by other cancer types which negatively affecting specificity, plus the high cost of the equipment used. Hyperspectral imaging (HSI) is an advanced tool with unique, spectroscopic features, is an emerging tool that has widely been used in cancer detection. Herein, a pilot study has been performed for HCC diagnosis, by exploiting HIS properties and the analysis of the transcriptome for the development of non-invasive remote HCC sensing. HSI data cube images of the sera extracted total RNA have been analyzed in HCC, normal subject, liver benign tumor, and chronic HCV with cirrhotic/non-cirrhotic liver groups. Data analyses have revealed a specific spectral signature for all groups and can be easily discriminated; at the computed optimum wavelength. Moreover, we have developed a simple setup based on a commercial laser pointer for sample illumination and a Smartphone CCD camera, with HSI consistent data output. We hypothesized that RNA differential expression and its spatial organization/folding are the key players in the obtained spectral signatures. To the best of our knowledge, we are the first to use HSI for sensing cancer based on total RNA in serum, using a Smartphone CCD camera/laser pointer. The proposed biosensor is simple, rapid (2 min), and affordable with specificity and sensitivity of more than 98% and high accuracy.
Collapse
Affiliation(s)
| | - Marwa Matboli
- Medical Biochemistry and Molecular Biology Department, Faculty of Medicine, Ain Shams University, Cairo, Egypt
| | - Sherif M. Shawky
- Center of Genomics, Helmy Medical Institute, Zewail City of Science and Technology, Ahmed Zewail Road, October Gardens, 6th of October City, 12578 Giza, Egypt
- Misr University for Science and Technology, Faculty of Pharmacy, Biochemistry Department, Al-Motamayez District. P.O.BOX: 77, 6thOctober City, Giza, Egypt
| | | |
Collapse
|
25
|
Jansen-Winkeln B, Barberio M, Chalopin C, Schierle K, Diana M, Köhler H, Gockel I, Maktabi M. Feedforward Artificial Neural Network-Based Colorectal Cancer Detection Using Hyperspectral Imaging: A Step towards Automatic Optical Biopsy. Cancers (Basel) 2021; 13:cancers13050967. [PMID: 33669082 PMCID: PMC7956537 DOI: 10.3390/cancers13050967] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Revised: 02/18/2021] [Accepted: 02/20/2021] [Indexed: 12/14/2022] Open
Abstract
Simple Summary Detection of colorectal carcinoma is performed visually by investigators and is confirmed pathologically. With hyperspectral imaging, an expanded spectral range of optical information is now available for analysis. The acquired recordings were analyzed with a neural network, and it was possible to differentiate tumor from healthy mucosa in colorectal carcinoma by automatic classification with high reliability. Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. This is a step towards optical biopsy. Abstract Currently, colorectal cancer (CRC) is mainly identified via a visual assessment during colonoscopy, increasingly used artificial intelligence algorithms, or surgery. Subsequently, CRC is confirmed through a histopathological examination by a pathologist. Hyperspectral imaging (HSI), a non-invasive optical imaging technology, has shown promising results in the medical field. In the current study, we combined HSI with several artificial intelligence algorithms to discriminate CRC. Between July 2019 and May 2020, 54 consecutive patients undergoing colorectal resections for CRC were included. The tumor was imaged from the mucosal side with a hyperspectral camera. The image annotations were classified into three groups (cancer, CA; adenomatous margin around the central tumor, AD; and healthy mucosa, HM). Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. Hyperspectral imaging combined with automatic classification can be used to differentiate between CRC and healthy mucosa. Additionally, the biological changes induced by chemotherapy to the tissue are detectable with HSI.
Collapse
Affiliation(s)
- Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (M.B.); (I.G.)
- Correspondence: ; Tel.: +49-341-9717211; Fax: +49-341-9728167
| | - Manuel Barberio
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (M.B.); (I.G.)
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France;
- Department of General Surgery, Hospital Card. G. Panico, 73039 Tricase, Italy
| | - Claire Chalopin
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (C.C.); (H.K.); (M.M.)
| | - Katrin Schierle
- Institute of Pathology, University Hospital Leipzig, 04103 Leipzig, Germany;
| | - Michele Diana
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France;
| | - Hannes Köhler
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (C.C.); (H.K.); (M.M.)
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (M.B.); (I.G.)
| | - Marianne Maktabi
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (C.C.); (H.K.); (M.M.)
| |
Collapse
|
26
|
Okubo K, Kitagawa Y, Hosokawa N, Umezawa M, Kamimura M, Kamiya T, Ohtani N, Soga K. Visualization of quantitative lipid distribution in mouse liver through near-infrared hyperspectral imaging. BIOMEDICAL OPTICS EXPRESS 2021; 12:823-835. [PMID: 33680544 PMCID: PMC7901335 DOI: 10.1364/boe.413712] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Revised: 12/21/2020] [Accepted: 01/03/2021] [Indexed: 06/12/2023]
Abstract
Lipid distribution in the liver provides crucial information for diagnosing the severity of fatty liver and fatty liver-associated liver cancer. Therefore, a noninvasive, label-free, and quantitative modality is eagerly anticipated. We report near-infrared hyperspectral imaging for the quantitative visualization of lipid content in mouse liver based on partial least square regression (PLSR) and support vector regression (SVR). Analysis results indicate that SVR with standard normal variate pretreatment outperforms PLSR by achieving better root mean square error (15.3 mg/g) and higher determination coefficient (0.97). The quantitative mapping of lipid content in the mouse liver is realized using SVR.
Collapse
Affiliation(s)
- Kyohei Okubo
- Department of Materials Science and Technology, Faculty of Industrial Science and Technology, Tokyo University of Science, 6-3-1 Niijuku, Katsushika-ku, Tokyo 125-8585, Japan
| | - Yuichi Kitagawa
- Department of Materials Science and Technology, Faculty of Industrial Science and Technology, Tokyo University of Science, 6-3-1 Niijuku, Katsushika-ku, Tokyo 125-8585, Japan
| | - Naoki Hosokawa
- Department of Materials Science and Technology, Faculty of Industrial Science and Technology, Tokyo University of Science, 6-3-1 Niijuku, Katsushika-ku, Tokyo 125-8585, Japan
| | - Masakazu Umezawa
- Department of Materials Science and Technology, Faculty of Industrial Science and Technology, Tokyo University of Science, 6-3-1 Niijuku, Katsushika-ku, Tokyo 125-8585, Japan
| | - Masao Kamimura
- Department of Materials Science and Technology, Faculty of Industrial Science and Technology, Tokyo University of Science, 6-3-1 Niijuku, Katsushika-ku, Tokyo 125-8585, Japan
| | - Tomonori Kamiya
- Department of Pathophysiology, Graduate School of Medicine, Osaka City University, 1-4-3 Asahi-machi, Abeno-ku, Osaka 545-8585, Japan
| | - Naoko Ohtani
- Department of Pathophysiology, Graduate School of Medicine, Osaka City University, 1-4-3 Asahi-machi, Abeno-ku, Osaka 545-8585, Japan
| | - Kohei Soga
- Department of Materials Science and Technology, Faculty of Industrial Science and Technology, Tokyo University of Science, 6-3-1 Niijuku, Katsushika-ku, Tokyo 125-8585, Japan
| |
Collapse
|
27
|
Intraoperative Perfusion Assessment in Enhanced Reality Using Quantitative Optical Imaging: An Experimental Study in a Pancreatic Partial Ischemia Model. Diagnostics (Basel) 2021; 11:diagnostics11010093. [PMID: 33430038 PMCID: PMC7826658 DOI: 10.3390/diagnostics11010093] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2020] [Revised: 01/04/2021] [Accepted: 01/04/2021] [Indexed: 02/06/2023] Open
Abstract
To reduce the risk of pancreatic fistula after pancreatectomy, a satisfactory blood flow at the pancreatic stump is considered crucial. Our group has developed and validated a real-time computational imaging analysis of tissue perfusion, using fluorescence imaging, the fluorescence-based enhanced reality (FLER). Hyperspectral imaging (HSI) is another emerging technology, which provides tissue-specific spectral signatures, allowing for perfusion quantification. Both imaging modalities were employed to estimate perfusion in a porcine model of partial pancreatic ischemia. Perfusion quantification was assessed using the metrics of both imaging modalities (slope of the time to reach maximum fluorescence intensity and tissue oxygen saturation (StO2), for FLER and HSI, respectively). We found that the HSI-StO2 and the FLER slope were statistically correlated using the Spearman analysis (R = 0.697; p = 0.013). Local capillary lactate values were statistically correlated to the HSI-StO2 and to the FLER slope (R = −0.88; p < 0.001 and R = −0.608; p = 0.0074). HSI-based and FLER-based lactate prediction models had statistically similar predictive abilities (p = 0.112). Both modalities are promising to assess real-time pancreatic perfusion. Clinical translation in human pancreatic surgery is currently underway.
Collapse
|
28
|
Browning CM, Deal J, Mayes S, Arshad A, Rich TC, Leavesley SJ. Excitation-scanning hyperspectral video endoscopy: enhancing the light at the end of the tunnel. BIOMEDICAL OPTICS EXPRESS 2021; 12:247-271. [PMID: 33520384 PMCID: PMC7818959 DOI: 10.1364/boe.411640] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/02/2020] [Revised: 11/24/2020] [Accepted: 11/27/2020] [Indexed: 06/12/2023]
Abstract
Colorectal cancer is the 3rd leading cancer for incidence and mortality rates. Positive treatment outcomes have been associated with early detection; however, early stage lesions have limited contrast to surrounding mucosa. A potential technology to enhance early stagise detection is hyperspectral imaging (HSI). While HSI technologies have been previously utilized to detect colorectal cancer ex vivo or post-operation, they have been difficult to employ in real-time endoscopy scenarios. Here, we describe an LED-based multifurcated light guide and spectral light source that can provide illumination for spectral imaging at frame rates necessary for video-rate endoscopy. We also present an updated light source optical ray-tracing model that resulted in further optimization and provided a ∼10X light transmission increase compared to the initial prototype. Future work will iterate simulation and benchtop testing of the hyperspectral endoscopic system to achieve the goal of video-rate spectral endoscopy.
Collapse
Affiliation(s)
- Craig M. Browning
- Chemical and Biomolecular Engineering, University of South Alabama, AL 36688, USA
- Systems Engineering, University of South Alabama, AL 36688, USA
| | - Joshua Deal
- Pharmacology, University of South Alabama, AL 36688, USA
- Center for Lung Biology, University of South Alabama, AL 36688, USA
| | - Sam Mayes
- Chemical and Biomolecular Engineering, University of South Alabama, AL 36688, USA
- Systems Engineering, University of South Alabama, AL 36688, USA
| | - Arslan Arshad
- Chemical and Biomolecular Engineering, University of South Alabama, AL 36688, USA
| | - Thomas C. Rich
- Pharmacology, University of South Alabama, AL 36688, USA
- Center for Lung Biology, University of South Alabama, AL 36688, USA
| | - Silas J. Leavesley
- Chemical and Biomolecular Engineering, University of South Alabama, AL 36688, USA
- Pharmacology, University of South Alabama, AL 36688, USA
- Center for Lung Biology, University of South Alabama, AL 36688, USA
| |
Collapse
|
29
|
Sato D, Takamatsu T, Umezawa M, Kitagawa Y, Maeda K, Hosokawa N, Okubo K, Kamimura M, Kadota T, Akimoto T, Kinoshita T, Yano T, Kuwata T, Ikematsu H, Takemura H, Yokota H, Soga K. Distinction of surgically resected gastrointestinal stromal tumor by near-infrared hyperspectral imaging. Sci Rep 2020; 10:21852. [PMID: 33318595 PMCID: PMC7736345 DOI: 10.1038/s41598-020-79021-7] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2020] [Accepted: 12/02/2020] [Indexed: 12/16/2022] Open
Abstract
The diagnosis of gastrointestinal stromal tumor (GIST) using conventional endoscopy is difficult because submucosal tumor (SMT) lesions like GIST are covered by a mucosal layer. Near-infrared hyperspectral imaging (NIR-HSI) can obtain optical information from deep inside tissues. However, far less progress has been made in the development of techniques for distinguishing deep lesions like GIST. This study aimed to investigate whether NIR-HSI is suitable for distinguishing deep SMT lesions. In this study, 12 gastric GIST lesions were surgically resected and imaged with an NIR hyperspectral camera from the aspect of the mucosal surface. Thus, the images were obtained ex-vivo. The site of the GIST was defined by a pathologist using the NIR image to prepare training data for normal and GIST regions. A machine learning algorithm, support vector machine, was then used to predict normal and GIST regions. Results were displayed using color-coded regions. Although 7 specimens had a mucosal layer (thickness 0.4-2.5 mm) covering the GIST lesion, NIR-HSI analysis by machine learning showed normal and GIST regions as color-coded areas. The specificity, sensitivity, and accuracy of the results were 73.0%, 91.3%, and 86.1%, respectively. The study suggests that NIR-HSI analysis may potentially help distinguish deep lesions.
Collapse
Affiliation(s)
- Daiki Sato
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
- Course of Advanced Clinical Research of Cancer, Juntendo University Graduate School of Medicine, Bunkyo-ku, Tokyo, Japan
| | - Toshihiro Takamatsu
- Exploratory Oncology Research & Clinical Trial Center, National Cancer Center, Kashiwa, Chiba, Japan.
- Research Institute for Biomedical Sciences, Tokyo University of Science, Noda, Chiba, Japan.
| | - Masakazu Umezawa
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika-ku, Tokyo, Japan
| | - Yuichi Kitagawa
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika-ku, Tokyo, Japan
| | - Kosuke Maeda
- Department of Mechanical Engineering, Tokyo University of Science, Noda, Chiba, Japan
| | - Naoki Hosokawa
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika-ku, Tokyo, Japan
| | - Kyohei Okubo
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika-ku, Tokyo, Japan
- Imaging Frontier Center, Tokyo University of Science, Noda, Chiba, Japan
| | - Masao Kamimura
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika-ku, Tokyo, Japan
- Imaging Frontier Center, Tokyo University of Science, Noda, Chiba, Japan
| | - Tomohiro Kadota
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Tetsuo Akimoto
- Division of Radiation Oncology, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
- Course of Advanced Clinical Research of Cancer, Juntendo University Graduate School of Medicine, Bunkyo-ku, Tokyo, Japan
| | - Takahiro Kinoshita
- Department of Gastric Surgery, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Tomonori Yano
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Takeshi Kuwata
- Department of Pathology and Clinical Laboratories, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Hiroaki Ikematsu
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
- Exploratory Oncology Research & Clinical Trial Center, National Cancer Center, Kashiwa, Chiba, Japan
| | - Hiroshi Takemura
- Research Institute for Biomedical Sciences, Tokyo University of Science, Noda, Chiba, Japan
- Department of Mechanical Engineering, Tokyo University of Science, Noda, Chiba, Japan
| | - Hideo Yokota
- RIKEN Center for Advanced Photonics, Wako, Saitama, Japan
| | - Kohei Soga
- Research Institute for Biomedical Sciences, Tokyo University of Science, Noda, Chiba, Japan
- Department of Materials Science and Technology, Tokyo University of Science, Katsushika-ku, Tokyo, Japan
- Imaging Frontier Center, Tokyo University of Science, Noda, Chiba, Japan
| |
Collapse
|
30
|
Aref MH, Aboughaleb IH, El-Sharkawy YH. Custom optical imaging system for ex-vivo breast cancer detection based on spectral signature. Surg Oncol 2020; 35:547-555. [PMID: 33212419 DOI: 10.1016/j.suronc.2020.10.019] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2020] [Revised: 10/19/2020] [Accepted: 10/27/2020] [Indexed: 01/28/2023]
Abstract
BACKGROUND AND PURPOSE Breast cancer is a popular well-known tumor in women globally and the subsequent driving reason for malignancy death. The purpose of the present study is to develop Low cost, commercial, and affordable system that discriminates malignant from normal breast tissues by exploiting the unique properties of Hyperspectral (HS) Imaging. MATERIALS AND METHODS The difference in the optical properties of the investigated breast tissues gives various reactions to light transmission, absorption, and especially the reflection over the spectral range. A custom optical imaging system (COIS) was designed to assess variable responses to monochromatic LEDs (415, 565, 660 nm) to highlight the differences in the reflectance properties of malignant/normal tissue. Statistical analysis was computed for determining the ideal wavelength to differentiate between normal and malignant regions. The experiment was repeated using the same LEDs, and low-cost CCD camera to examine the capability of such a system to discriminate between normal and malignant tissue. RESULTS Spectral images obtained by Hyperspectral camera, have been analyzed to reveal the difference of reflectance malignant and normal breast tissue. Superficial spectral reflection image with blue LED (415 nm) showed high variance (10.11). However, a more-depth reflection image with red LED (660 nm) showed low variance (4.44). So the optimum contrast image was produced by combining the three spectral information images from blue, green, and red LED. The COIS using a commercial CCD camera was in agreement with the HS camera. CONCLUSIONS The novel COIS of the commercial Low-cost CCD Camera is reliable and can be used with endoscopy technique as an assistant tool for surgical doctor to make decision and assess the resection edges in real time during surgery.
Collapse
Affiliation(s)
- Mohamed Hisham Aref
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt.
| | - Ibrahim H Aboughaleb
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt
| | - Yasser H El-Sharkawy
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt
| |
Collapse
|
31
|
El-Sharkawy YH, Elbasuney S. Laser induced fluorescence with 2-D Hilbert transform edge detection algorithm and 3D fluorescence images for white spot early recognition. SPECTROCHIMICA ACTA. PART A, MOLECULAR AND BIOMOLECULAR SPECTROSCOPY 2020; 240:118616. [PMID: 32604049 DOI: 10.1016/j.saa.2020.118616] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2020] [Revised: 06/12/2020] [Accepted: 06/14/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVE There is a must for non-invasive caries detection method to supplement traditional visual assessment by the clinician before lesion progression reaches a stage requiring invasive therapy. MATERIAL AND METHODS In this paper, the nature of tissue response to light interaction has been used for early diagnosis, using laser induced fluorescence spectroscopy. Human tooth sample was illuminated with He-Ne laser sources (633 nm) with energy 5 mW. The reflected and emitted spectra of investigated samples were collected using hyperspectral camera to develop multispectral images. The variation of reflected and emitted energy as function in wavelength was employed to generate characteristic spectrum of each tooth tissue. Human teeth caries lesion releases their excess energy by emitting fluorescence light producing chemical footprint signature for each tooth elements and caries state. RESULTS This non-invasive, non-contact, and non-ionizing hyperspectral imaging system was employed to diagnose and classify different caries types and stages. Reconstructed 3D fluorescence images offered discrimination between enamel and dentin caries at 633 nm illumination spectral line; white spot lesion was clearly detected and recognized at far visible and infrared wavelength ranges. CONCLUSION This study reports on customized optical imaging system that can offer high sensitivity, high resolution. Optimum stimulating wavelength for early caries detection was reported to be 633 nm. This novel approach can offer a full map of caries degree status.
Collapse
Affiliation(s)
- Yasser H El-Sharkawy
- Department of Biomedical Engineering, Military Technical College, Kobry Elkoba, Cairo, Egypt
| | - Sherif Elbasuney
- Nanotechnology Center, Military Technical College, Kobry El-Kobba, Cairo, Egypt.
| |
Collapse
|
32
|
Felli E, Urade T, Al-Taher M, Felli E, Barberio M, Goffin L, Ettorre GM, Marescaux J, Pessaux P, Swanstrom L, Diana M. Demarcation Line Assessment in Anatomical Liver Resection: An Overview. Surg Innov 2020; 27:424-430. [DOI: 10.1177/1553350620953651] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/30/2023]
Abstract
Anatomical liver resection (ALR) is the preferred oncological approach for the treatment of primary liver malignancies, such as hepatocellular carcinoma and intrahepatic cholangiocarcinoma. The demarcation line (DL) is formed by means of selective vascular occlusion and is used by surgeons to guide ALR. Emerging intraoperative technologies are playing a major role to enhance the surgeon’s vision and ensure a precise oncologic surgery. In this article, a brief overview of modalities to assess the DL during ALRs is presented, from the established conventional techniques to future perspectives.
Collapse
Affiliation(s)
- Eric Felli
- IHU-Strasbourg, Institute of Image-Guided Surgery, France
- Institute of Physiology, EA3072 Mitochondria Respiration and Oxidative Stress, University of Strasbourg, France
| | - Takeshi Urade
- IHU-Strasbourg, Institute of Image-Guided Surgery, France
| | - Mahdi Al-Taher
- IHU-Strasbourg, Institute of Image-Guided Surgery, France
| | - Emanuele Felli
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, France
- INSERM U1110, Institute of Viral and Liver Disease, University of Strasbourg, France
| | - Manuel Barberio
- IHU-Strasbourg, Institute of Image-Guided Surgery, France
- Institute of Physiology, EA3072 Mitochondria Respiration and Oxidative Stress, University of Strasbourg, France
| | | | - Giuseppe M. Ettorre
- Department of Transplantation and General Surgery, San Camillo Hospital, Italy
| | - Jacques Marescaux
- IHU-Strasbourg, Institute of Image-Guided Surgery, France
- IRCAD, Research Institute against Digestive Cancer, France
| | - Patrick Pessaux
- IHU-Strasbourg, Institute of Image-Guided Surgery, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, France
- INSERM U1110, Institute of Viral and Liver Disease, University of Strasbourg, France
| | - Lee Swanstrom
- IHU-Strasbourg, Institute of Image-Guided Surgery, France
| | - Michele Diana
- IHU-Strasbourg, Institute of Image-Guided Surgery, France
- Institute of Physiology, EA3072 Mitochondria Respiration and Oxidative Stress, University of Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, France
- IRCAD, Research Institute against Digestive Cancer, France
- ICUBE Laboratory, Photonic Instrumentation for Health, France
| |
Collapse
|
33
|
Parallel Classification Pipelines for Skin Cancer Detection Exploiting Hyperspectral Imaging on Hybrid Systems. ELECTRONICS 2020. [DOI: 10.3390/electronics9091503] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
The early detection of skin cancer is of crucial importance to plan an effective therapy to treat the lesion. In routine medical practice, the diagnosis is based on the visual inspection of the lesion and it relies on the dermatologists’ expertise. After a first examination, the dermatologist may require a biopsy to confirm if the lesion is malignant or not. This methodology suffers from false positives and negatives issues, leading to unnecessary surgical procedures. Hyperspectral imaging is gaining relevance in this medical field since it is a non-invasive and non-ionizing technique, capable of providing higher accuracy than traditional imaging methods. Therefore, the development of an automatic classification system based on hyperspectral images could improve the medical practice to distinguish pigmented skin lesions from malignant, benign, and atypical lesions. Additionally, the system can assist general practitioners in first aid care to prevent noncritical lesions from reaching dermatologists, thereby alleviating the workload of medical specialists. In this paper is presented a parallel pipeline for skin cancer detection that exploits hyperspectral imaging. The computational times of the serial processing have been reduced by adopting multicore and many-core technologies, such as OpenMP and CUDA paradigms. Different parallel approaches have been combined, leading to the development of fifteen classification pipeline versions. Experimental results using in-vivo hyperspectral images show that a hybrid parallel approach is capable of classifying an image of 50 × 50 pixels with 125 bands in less than 1 s.
Collapse
|
34
|
Tissue characterization utilizing hyperspectral imaging for liver thermal ablation. Photodiagnosis Photodyn Ther 2020; 31:101899. [DOI: 10.1016/j.pdpdt.2020.101899] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2020] [Revised: 06/20/2020] [Accepted: 06/22/2020] [Indexed: 02/07/2023]
|
35
|
Mehta N, Sahu SP, Shaik S, Devireddy R, Gartia MR. Dark-field hyperspectral imaging for label free detection of nano-bio-materials. WILEY INTERDISCIPLINARY REVIEWS-NANOMEDICINE AND NANOBIOTECHNOLOGY 2020; 13:e1661. [PMID: 32755036 DOI: 10.1002/wnan.1661] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/11/2019] [Revised: 05/21/2020] [Accepted: 06/19/2020] [Indexed: 12/12/2022]
Abstract
Nanomaterials are playing an increasingly important role in cancer diagnosis and treatment. Nanoparticle (NP)-based technologies have been utilized for targeted drug delivery during chemotherapies, photodynamic therapy, and immunotherapy. Another active area of research is the toxicity studies of these nanomaterials to understand the cellular uptake and transport of these materials in cells, tissues, and environment. Traditional techniques such as transmission electron microscopy, and mass spectrometry to analyze NP-based cellular transport or toxicity effect are expensive, require extensive sample preparation, and are low-throughput. Dark-field hyperspectral imaging (DF-HSI), an integration of spectroscopy and microscopy/imaging, provides the ability to investigate cellular transport of these NPs and to quantify the distribution of them within bio-materials. DF-HSI also offers versatility in non-invasively monitoring microorganisms, single cell, and proteins. DF-HSI is a low-cost, label-free technique that is minimally invasive and is a viable choice for obtaining high-throughput quantitative molecular analyses. Multimodal imaging modalities such as Fourier transform infrared and Raman spectroscopy are also being integrated with HSI systems to enable chemical imaging of the samples. HSI technology is being applied in surgeries to obtain molecular information about the tissues in real-time. This article provides brief overview of fundamental principles of DF-HSI and its application for nanomaterials, protein-detection, single-cell analysis, microbiology, surgical procedures along with technical challenges and future integrative approach with other imaging and measurement modalities. This article is categorized under: Diagnostic Tools > in vitro Nanoparticle-Based Sensing Diagnostic Tools > in vivo Nanodiagnostics and Imaging Implantable Materials and Surgical Technologies > Nanoscale Tools and Techniques in Surgery.
Collapse
Affiliation(s)
- Nishir Mehta
- Department of Mechanical and Industrial Engineering, Louisiana State University, Baton Rouge, Louisiana, USA
| | - Sushant P Sahu
- Department of Mechanical and Industrial Engineering, Louisiana State University, Baton Rouge, Louisiana, USA
| | - Shahensha Shaik
- Department of Mechanical and Industrial Engineering, Louisiana State University, Baton Rouge, Louisiana, USA
| | - Ram Devireddy
- Department of Mechanical and Industrial Engineering, Louisiana State University, Baton Rouge, Louisiana, USA
| | - Manas Ranjan Gartia
- Department of Mechanical and Industrial Engineering, Louisiana State University, Baton Rouge, Louisiana, USA
| |
Collapse
|
36
|
Aboughaleb IH, Aref MH, El-Sharkawy YH. Hyperspectral imaging for diagnosis and detection of ex-vivo breast cancer. Photodiagnosis Photodyn Ther 2020; 31:101922. [PMID: 32726640 DOI: 10.1016/j.pdpdt.2020.101922] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2020] [Revised: 06/09/2020] [Accepted: 07/10/2020] [Indexed: 01/28/2023]
Abstract
BACKGROUND AND PURPOSE Breast cancer is one of the most widely recognized tumors. .Diagnosis made in the early stage of disease may imporve outcomes. The discovery of malignant growth utilizing noninvasive light intrusive methods in lieu of conventional excisional biopsy may assist in achieving this goal. MATERIALS AND METHODS The change of the optical properties of ex-vivo breast tissues provides different responses to light transmission, absorption, and particularly the reflection over the spectrum range. We offer the use of Hyperspectral imaging (HSI) with advanced image processing and pattern recognition in order to analyze HSI data for breast cancer detection. The spectral signatures were mined and evaluated in both malignant and normal tissue. K-mean clustering was designed for classifying hyperspectral data in order to evaluate and detection of cancer tissue. This method was used to detect ex-vivo breast cancer. Spatial spectral images were created to high spot the differences in the reflectance properties of malignant versus normal tissue. RESULTS Trials showed that the superficial spectral reflection images within 500 nm wavelength showed high variance (214.65) between cancerous and normal breast tissues. On the other hand, image within 620 nm wavelength showed low variance (0.0020).However, the superimposed of spectral region 420-620 nm was proposed as the optimum bandwidth. Finally, the proposed HS imaging system was capable to discriminate the tumor region from normal tissue of the ex-vivo breast sample with sensitivity and a specificity of 95 % and 96 %. CONCLUSIONS High sensitivity and specificity were achieved, which proposes potential for HSI as an edge evaluation method to enhance the surgical outcome compared to the presently available techniques in the clinics.
Collapse
Affiliation(s)
- Ibrahim H Aboughaleb
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt.
| | - Mohamed Hisham Aref
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt.
| | - Yasser H El-Sharkawy
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt.
| |
Collapse
|
37
|
Ma L, Halicek M, Fei B. In Vivo Cancer Detection in Animal Model Using Hyperspectral Image Classification with Wavelet Feature Extraction. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11317. [PMID: 32476705 DOI: 10.1117/12.2549397] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Hyperspectral imaging (HSI) is a promising optical imaging technique for cancer detection. However, quantitative methods need to be developed in order to utilize the rich spectral information and subtle spectral variation in such images. In this study, we explore the feasibility of using wavelet-based features from in vivo hyperspectral images for head and neck cancer detection. Hyperspectral reflectance data were collected from 12 mice bearing head and neck cancer. Catenation of 5-level wavelet decomposition outputs of hyperspectral images was used as a feature for tumor discrimination. A support vector machine (SVM) was utilized as the classifier. Seven types of mother wavelets were tested to select the one with the best performance. Classifications with raw reflectance spectra, 1-level wavelet decomposition output, and 2-level wavelet decomposition output, as well as the proposed feature were carried out for comparison. Our results show that the proposed wavelet-based feature yields better classification accuracy, and that using different type and order of mother wavelet achieves different classification results. The wavelet-based classification method provides a new approach for HSI detection of head and neck cancer in the animal model.
Collapse
Affiliation(s)
- Ling Ma
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX 75080.,Tianjin University, State Key Laboratory of Precision Measurement Technology and Instrument, Tianjin, China 300072
| | - Martin Halicek
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX 75080.,Georgia Inst. of Tech. & Emory Univ., Dept. of Biomedical Engineering, Atlanta, GA.,Medical College of Georgia, Augusta University, Augusta, GA
| | - Baowei Fei
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX 75080.,Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX.,Univ. of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
38
|
Ortega S, Halicek M, Fabelo H, Callico GM, Fei B. Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review [Invited]. BIOMEDICAL OPTICS EXPRESS 2020; 11:3195-3233. [PMID: 32637250 PMCID: PMC7315999 DOI: 10.1364/boe.386338] [Citation(s) in RCA: 76] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2019] [Revised: 03/28/2020] [Accepted: 05/08/2020] [Indexed: 05/06/2023]
Abstract
Hyperspectral imaging (HSI) and multispectral imaging (MSI) technologies have the potential to transform the fields of digital and computational pathology. Traditional digitized histopathological slides are imaged with RGB imaging. Utilizing HSI/MSI, spectral information across wavelengths within and beyond the visual range can complement spatial information for the creation of computer-aided diagnostic tools for both stained and unstained histological specimens. In this systematic review, we summarize the methods and uses of HSI/MSI for staining and color correction, immunohistochemistry, autofluorescence, and histopathological diagnostic research. Studies include hematology, breast cancer, head and neck cancer, skin cancer, and diseases of central nervous, gastrointestinal, and genitourinary systems. The use of HSI/MSI suggest an improvement in the detection of diseases and clinical practice compared with traditional RGB analysis, and brings new opportunities in histological analysis of samples, such as digital staining or alleviating the inter-laboratory variability of digitized samples. Nevertheless, the number of studies in this field is currently limited, and more research is needed to confirm the advantages of this technology compared to conventional imagery.
Collapse
Affiliation(s)
- Samuel Ortega
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX 75080, USA
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), Campus de Tafira, 35017, Las Palmas de Gran Canaria, Las Palmas, Spain
- These authors contributed equally to this work
| | - Martin Halicek
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX 75080, USA
- Department of Biomedical Engineering, Georgia Inst. of Tech. and Emory University, Atlanta, GA 30322, USA
- These authors contributed equally to this work
| | - Himar Fabelo
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), Campus de Tafira, 35017, Las Palmas de Gran Canaria, Las Palmas, Spain
| | - Gustavo M Callico
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), Campus de Tafira, 35017, Las Palmas de Gran Canaria, Las Palmas, Spain
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX 75080, USA
- University of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX 75235, USA
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX 75235, USA
| |
Collapse
|
39
|
Liu N, Guo Y, Jiang H, Yi W. Gastric cancer diagnosis using hyperspectral imaging with principal component analysis and spectral angle mapper. JOURNAL OF BIOMEDICAL OPTICS 2020; 25:1-9. [PMID: 32594664 PMCID: PMC7320226 DOI: 10.1117/1.jbo.25.6.066005] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/20/2019] [Accepted: 06/12/2020] [Indexed: 05/27/2023]
Abstract
SIGNIFICANCE Hyperspectral imaging (HSI) is an emerging optical technique that has a double function of spectroscopy and imaging. AIM Near-infrared hyperspectral imaging (NIR-HSI) (900 to 1700 nm) with the help of chemometrics was investigated for gastric cancer diagnosis. APPROACH Mean spectra and standard deviation of normal and cancerous pixels were extracted. Principal component analysis (PCA) was used to compress the dimension of hypercube data and select the optimal wavelengths. Moreover, spectral angle mapper (SAM) was utilized as chemometrics to discriminate gastric cancer from normal. RESULTS Major spectral difference of cancerous and normal gastric tissue was observed around 975, 1215, and 1450 nm by comparison. A total of six wavelengths (i.e., 975, 1075, 1215, 1275, 1390, and 1450 nm) were then selected as optimal wavelengths by PCA. The accuracy using SAM is up to 90% according to hematoxylin-eosin results. CONCLUSIONS These results suggest that NIR-HSI has the potential as a cutting-edge optical diagnostic technique for gastric cancer diagnosis with suitable chemometrics.
Collapse
Affiliation(s)
- Ningliang Liu
- Huazhong Agricultural University, College of Science, Wuhan, China
| | - Yaxiong Guo
- Huazhong Agricultural University, College of Science, Wuhan, China
| | - Houmin Jiang
- People’s Hospital of Huangpi District, Wuhan, China
| | - Weisong Yi
- Huazhong Agricultural University, College of Science, Wuhan, China
| |
Collapse
|
40
|
Hyperspectral enhanced reality (HYPER) for anatomical liver resection. Surg Endosc 2020; 35:1844-1850. [DOI: 10.1007/s00464-020-07586-5] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2020] [Accepted: 04/22/2020] [Indexed: 12/19/2022]
|
41
|
de Lucena DV, da Silva Soares A, Coelho CJ, Wastowski IJ, Filho ARG. Detection of Tumoral Epithelial Lesions Using Hyperspectral Imaging and Deep Learning. LECTURE NOTES IN COMPUTER SCIENCE 2020. [PMCID: PMC7304037 DOI: 10.1007/978-3-030-50420-5_45] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
We propose a new method for the analysis and classification of HSI images. The method uses deep learning to interpret the molecular vibrational behaviour of healthy and tumoral human epithelial tissue, based on data gathered via SWIR (short-wave infrared) spectroscopy. We analyzed samples of Melanoma, Dysplastic Nevus and healthy skin. Preliminary results show that human epithelial tissue is sensitive to SWIR to the point of making possible the differentiation between healthy and tumor tissues. We conclude that HSI-SWIR can be used to build new methods for tumor classification.
Collapse
|
42
|
Huang Q, Li W, Zhang B, Li Q, Tao R, Lovell NH. Blood Cell Classification Based on Hyperspectral Imaging With Modulated Gabor and CNN. IEEE J Biomed Health Inform 2020; 24:160-170. [DOI: 10.1109/jbhi.2019.2905623] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/30/2023]
|
43
|
Martinez B, Leon R, Fabelo H, Ortega S, Piñeiro JF, Szolna A, Hernandez M, Espino C, J. O’Shanahan A, Carrera D, Bisshopp S, Sosa C, Marquez M, Camacho R, Plaza MDLL, Morera J, M. Callico G. Most Relevant Spectral Bands Identification for Brain Cancer Detection Using Hyperspectral Imaging. SENSORS (BASEL, SWITZERLAND) 2019; 19:E5481. [PMID: 31842410 PMCID: PMC6961052 DOI: 10.3390/s19245481] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/25/2019] [Revised: 12/01/2019] [Accepted: 12/10/2019] [Indexed: 02/07/2023]
Abstract
Hyperspectral imaging (HSI) is a non-ionizing and non-contact imaging technique capable of obtaining more information than conventional RGB (red green blue) imaging. In the medical field, HSI has commonly been investigated due to its great potential for diagnostic and surgical guidance purposes. However, the large amount of information provided by HSI normally contains redundant or non-relevant information, and it is extremely important to identify the most relevant wavelengths for a certain application in order to improve the accuracy of the predictions and reduce the execution time of the classification algorithm. Additionally, some wavelengths can contain noise and removing such bands can improve the classification stage. The work presented in this paper aims to identify such relevant spectral ranges in the visual-and-near-infrared (VNIR) region for an accurate detection of brain cancer using in vivo hyperspectral images. A methodology based on optimization algorithms has been proposed for this task, identifying the relevant wavelengths to achieve the best accuracy in the classification results obtained by a supervised classifier (support vector machines), and employing the lowest possible number of spectral bands. The results demonstrate that the proposed methodology based on the genetic algorithm optimization slightly improves the accuracy of the tumor identification in ~5%, using only 48 bands, with respect to the reference results obtained with 128 bands, offering the possibility of developing customized acquisition sensors that could provide real-time HS imaging. The most relevant spectral ranges found comprise between 440.5-465.96 nm, 498.71-509.62 nm, 556.91-575.1 nm, 593.29-615.12 nm, 636.94-666.05 nm, 698.79-731.53 nm and 884.32-902.51 nm.
Collapse
Affiliation(s)
- Beatriz Martinez
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (R.L.); (H.F.); (S.O.); (G.M.C.)
| | - Raquel Leon
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (R.L.); (H.F.); (S.O.); (G.M.C.)
| | - Himar Fabelo
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (R.L.); (H.F.); (S.O.); (G.M.C.)
| | - Samuel Ortega
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (R.L.); (H.F.); (S.O.); (G.M.C.)
| | - Juan F. Piñeiro
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Adam Szolna
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Maria Hernandez
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Carlos Espino
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Aruma J. O’Shanahan
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - David Carrera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Sara Bisshopp
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Coralia Sosa
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Mariano Marquez
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Rafael Camacho
- Department of Pathological Anatomy, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (R.C.); (M.d.l.L.P.)
| | - Maria de la Luz Plaza
- Department of Pathological Anatomy, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (R.C.); (M.d.l.L.P.)
| | - Jesus Morera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, 35010 Barranco de la Ballena s/n, Las Palmas de Gran Canaria, Spain; (J.F.P.); (A.S.); (M.H.); (C.E.); (A.J.O.); (D.C.); (S.B.); (C.S.); (M.M.); (J.M.)
| | - Gustavo M. Callico
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (R.L.); (H.F.); (S.O.); (G.M.C.)
| |
Collapse
|
44
|
High-Level Synthesis of Multiclass SVM Using Code Refactoring to Classify Brain Cancer from Hyperspectral Images. ELECTRONICS 2019. [DOI: 10.3390/electronics8121494] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/28/2023]
Abstract
Currently, high-level synthesis (HLS) methods and tools are a highly relevant area in the strategy of several leading companies in the field of system-on-chips (SoCs) and field programmable gate arrays (FPGAs). HLS facilitates the work of system developers, who benefit from integrated and automated design workflows, considerably reducing the design time. Although many advances have been made in this research field, there are still some uncertainties about the quality and performance of the designs generated with the use of HLS methodologies. In this paper, we propose an optimization of the HLS methodology by code refactoring using Xilinx SDSoCTM (Software-Defined System-On-Chip). Several options were analyzed for each alternative through code refactoring of a multiclass support vector machine (SVM) classifier written in C, using two different Zynq®-7000 SoC devices from Xilinx, the ZC7020 (ZedBoard) and the ZC7045 (ZC706). The classifier was evaluated using a brain cancer database of hyperspectral images. The proposed methodology not only reduces the required resources using less than 20% of the FPGA, but also reduces the power consumption −23% compared to the full implementation. The speedup obtained of 2.86× (ZC7045) is the highest found in the literature for SVM hardware implementations.
Collapse
|
45
|
Ma L, Lu G, Wang D, Qin X, Chen ZG, Fei B. Adaptive deep learning for head and neck cancer detection using hyperspectral imaging. Vis Comput Ind Biomed Art 2019; 2:18. [PMID: 32190408 PMCID: PMC7055573 DOI: 10.1186/s42492-019-0023-8] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2019] [Accepted: 10/09/2019] [Indexed: 12/02/2022] Open
Abstract
It can be challenging to detect tumor margins during surgery for complete resection. The purpose of this work is to develop a novel learning method that learns the difference between the tumor and benign tissue adaptively for cancer detection on hyperspectral images in an animal model. Specifically, an auto-encoder network is trained based on the wavelength bands on hyperspectral images to extract the deep information to create a pixel-wise prediction of cancerous and benign pixel. According to the output hypothesis of each pixel, the misclassified pixels would be reclassified in the right prediction direction based on their adaptive weights. The auto-encoder network is again trained based on these updated pixels. The learner can adaptively improve the ability to identify the cancer and benign tissue by focusing on the misclassified pixels, and thus can improve the detection performance. The adaptive deep learning method highlighting the tumor region proved to be accurate in detecting the tumor boundary on hyperspectral images and achieved a sensitivity of 92.32% and a specificity of 91.31% in our animal experiments. This adaptive learning method on hyperspectral imaging has the potential to provide a noninvasive tool for tumor detection, especially, for the tumor whose margin is indistinct and irregular.
Collapse
Affiliation(s)
- Ling Ma
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA 30322 USA
- College of Software, Nankai University, Tianjin, 300350 People’s Republic of China
| | - Guolan Lu
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA 30322 USA
| | - Dongsheng Wang
- Department of Hematology and Medical Oncology, Emory University, Atlanta, GA 30322 USA
| | - Xulei Qin
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA 30322 USA
| | - Zhuo Georgia Chen
- Department of Hematology and Medical Oncology, Emory University, Atlanta, GA 30322 USA
| | - Baowei Fei
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA 30322 USA
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX 75080 USA
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX 75390 USA
| |
Collapse
|
46
|
Kho E, de Boer LL, Post AL, Van de Vijver KK, Jóźwiak K, Sterenborg HJCM, Ruers TJM. Imaging depth variations in hyperspectral imaging: Development of a method to detect tumor up to the required tumor-free margin width. JOURNAL OF BIOPHOTONICS 2019; 12:e201900086. [PMID: 31290280 DOI: 10.1002/jbio.201900086] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/06/2019] [Revised: 05/20/2019] [Accepted: 07/09/2019] [Indexed: 06/09/2023]
Abstract
Hyperspectral imaging is a promising technique for resection margin assessment during cancer surgery. Thereby, only a specific amount of the tissue below the resection surface, the clinically defined margin width, should be assessed. Since the imaging depth of hyperspectral imaging varies with wavelength and tissue composition, this can have consequences for the clinical use of hyperspectral imaging as margin assessment technique. In this study, a method was developed that allows for hyperspectral analysis of resection margins in breast cancer. This method uses the spectral slope of the diffuse reflectance spectrum at wavelength regions where the imaging depth in tumor and healthy tissue is equal. Thereby, tumor can be discriminated from healthy breast tissue while imaging up to a similar depth as the required tumor-free margin width of 2 mm. Applying this method to hyperspectral images acquired during surgery would allow for robust margin assessment of resected specimens. In this paper, we focused on breast cancer, but the same approach can be applied to develop a method for other types of cancer.
Collapse
Affiliation(s)
- Esther Kho
- Department of Surgery, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Lisanne L de Boer
- Department of Surgery, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Anouk L Post
- Department of Surgery, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, Amsterdam, Netherlands
| | - Koen K Van de Vijver
- Department of Pathology, the Netherlands Cancer Institute, Amsterdam, Netherlands
- Department of Pathology, Ghent University Hospital, Ghent, Belgium
| | - Katarzyna Jóźwiak
- Department of Epidemiology and Biostatistics, the Netherlands Cancer Institute, Amsterdam, Netherlands
| | - Henricus J C M Sterenborg
- Department of Surgery, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, Amsterdam, Netherlands
| | - Theo J M Ruers
- Department of Surgery, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Faculty of Science and Technology, University of Twente, Enschede, Netherlands
| |
Collapse
|
47
|
Zherebtsov E, Dremin V, Popov A, Doronin A, Kurakina D, Kirillin M, Meglinski I, Bykov A. Hyperspectral imaging of human skin aided by artificial neural networks. BIOMEDICAL OPTICS EXPRESS 2019; 10:3545-3559. [PMID: 31467793 PMCID: PMC6706048 DOI: 10.1364/boe.10.003545] [Citation(s) in RCA: 44] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2019] [Revised: 06/06/2019] [Accepted: 06/11/2019] [Indexed: 05/06/2023]
Abstract
We developed a compact, hand-held hyperspectral imaging system for 2D neural network-based visualization of skin chromophores and blood oxygenation. State-of-the-art micro-optic multichannel matrix sensor combined with the tunable Fabry-Perot micro interferometer enables a portable diagnostic device sensitive to the changes of the oxygen saturation as well as the variations of blood volume fraction of human skin. Generalized object-oriented Monte Carlo model is used extensively for the training of an artificial neural network utilized for the hyperspectral image processing. In addition, the results are verified and validated via actual experiments with tissue phantoms and human skin in vivo. The proposed approach enables a tool combining both the speed of an artificial neural network processing and the accuracy and flexibility of advanced Monte Carlo modeling. Finally, the results of the feasibility studies and the experimental tests on biotissue phantoms and healthy volunteers are presented.
Collapse
Affiliation(s)
- Evgeny Zherebtsov
- Opto-Electronics and Measurement Techniques Research Unit, Faculty of Information Technology and Electrical Engineering, University of Oulu, PO Box 4500, 90014 Oulu, Finland
| | - Viktor Dremin
- Opto-Electronics and Measurement Techniques Research Unit, Faculty of Information Technology and Electrical Engineering, University of Oulu, PO Box 4500, 90014 Oulu, Finland
| | - Alexey Popov
- Opto-Electronics and Measurement Techniques Research Unit, Faculty of Information Technology and Electrical Engineering, University of Oulu, PO Box 4500, 90014 Oulu, Finland
| | - Alexander Doronin
- School of Engineering and Computer Science, Victoria University of Wellington, PO Box 600, 6140 Wellington, New Zealand
| | - Daria Kurakina
- Institute of Applied Physics of the Russian Academy of Sciences, 46 Ul’yanov Street, 603950 Nizhny Novgorod, Russia
| | - Mikhail Kirillin
- Institute of Applied Physics of the Russian Academy of Sciences, 46 Ul’yanov Street, 603950 Nizhny Novgorod, Russia
- Lobachevsky State University of Nizhny Novgorod, 23 Gagarin Avenue, 603950 Nizhny Novgorod, Russia
| | - Igor Meglinski
- Opto-Electronics and Measurement Techniques Research Unit, Faculty of Information Technology and Electrical Engineering, University of Oulu, PO Box 4500, 90014 Oulu, Finland
- Institute of Engineering Physics for Biomedicine, National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), 115409 Moscow, Russia
| | - Alexander Bykov
- Opto-Electronics and Measurement Techniques Research Unit, Faculty of Information Technology and Electrical Engineering, University of Oulu, PO Box 4500, 90014 Oulu, Finland
| |
Collapse
|
48
|
Abstract
INTRODUCTION Anastomotic insufficiency (AI) remains the most feared surgical complication in gastrointestinal surgery, which is closely associated with a prolonged inpatient hospital stay and significant postoperative mortality. Hyperspectral imaging (HSI) is a relatively new medical imaging procedure which has proven to be promising in tissue identification as well as in the analysis of tissue oxygenation and water content. Until now, no data exist on the in vivo HSI analysis of gastrointestinal anastomoses. METHODS Intraoperative images were obtained using the TIVITA™ tissue system HSI camera from Diaspective Vision GmbH (Pepelow, Germany). In 47 patients who underwent gastrointestinal surgery with esophageal, gastric, pancreatic, small bowel or colorectal anastomoses, 97 assessable recordings were generated. Parameters obtained at the sites of the anastomoses included tissue oxygenation (StO2), the tissue hemoglobin index (THI), near-infrared (NIR) perfusion index, and tissue water index (TWI). RESULTS Obtaining and analyzing the intraoperative images with this non-invasive imaging system proved practicable and delivered good results on a consistent basis. A NIR gradient along and across the anastomosis was observed and, furthermore, analysis of the tissue water and oxygenation content showed specific changes at the site of anastomosis. CONCLUSION The HSI method provides a non-contact, non-invasive, intraoperative imaging procedure without the use of a contrast medium, which enables a real-time analysis of physiological anastomotic parameters, which may contribute to determine the "ideal" anastomotic region. In light of this, the establishment of this methodology in the field of visceral surgery, enabling the generation of normal or cut off values for different gastrointestinal anastomotic types, is an obvious necessity.
Collapse
|
49
|
Dong X, Xiao X, Pan Y, Wang G, Yu Y. DMD-based hyperspectral imaging system with tunable spatial and spectral resolution. OPTICS EXPRESS 2019; 27:16995-17006. [PMID: 31252917 DOI: 10.1364/oe.27.016995] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2019] [Accepted: 05/20/2019] [Indexed: 05/28/2023]
Abstract
Pushbroom hyperspectral imaging (HSI) has been used in many areas from air to land. However, its inherent operational drawback of the bulky slit leads to a limited field of view (FOV) and high energy consumption. Accordingly, a new and versatile HSI system is proposed by employing a smart digital micromirror device (DMD) to replace the mechanical scanning component. Moreover, tunable spatial and spectral resolution is implemented through adjusting the on-chip scanning linewidth and adopting the pixel fusion method, respectively. Meanwhile, three scanning modes including rough scanning, fine scanning, and regional scanning are achieved. These multiple choices increase the system's flexibility, universality, and intelligence, which is attractive for practically different applications, especially for military and remote sensing fields in need of a large FOV, and medical and food fields in need of tunable resolution for various samples.
Collapse
|
50
|
Halicek M, Fabelo H, Ortega S, Callico GM, Fei B. In-Vivo and Ex-Vivo Tissue Analysis through Hyperspectral Imaging Techniques: Revealing the Invisible Features of Cancer. Cancers (Basel) 2019; 11:E756. [PMID: 31151223 PMCID: PMC6627361 DOI: 10.3390/cancers11060756] [Citation(s) in RCA: 88] [Impact Index Per Article: 17.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2019] [Revised: 05/20/2019] [Accepted: 05/24/2019] [Indexed: 12/27/2022] Open
Abstract
In contrast to conventional optical imaging modalities, hyperspectral imaging (HSI) is able to capture much more information from a certain scene, both within and beyond the visual spectral range (from 400 to 700 nm). This imaging modality is based on the principle that each material provides different responses to light reflection, absorption, and scattering across the electromagnetic spectrum. Due to these properties, it is possible to differentiate and identify the different materials/substances presented in a certain scene by their spectral signature. Over the last two decades, HSI has demonstrated potential to become a powerful tool to study and identify several diseases in the medical field, being a non-contact, non-ionizing, and a label-free imaging modality. In this review, the use of HSI as an imaging tool for the analysis and detection of cancer is presented. The basic concepts related to this technology are detailed. The most relevant, state-of-the-art studies that can be found in the literature using HSI for cancer analysis are presented and summarized, both in-vivo and ex-vivo. Lastly, we discuss the current limitations of this technology in the field of cancer detection, together with some insights into possible future steps in the improvement of this technology.
Collapse
Affiliation(s)
- Martin Halicek
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080, USA.
- Department of Biomedical Engineering, Emory University and The Georgia Institute of Technology, 1841 Clifton Road NE, Atlanta, GA 30329, USA.
| | - Himar Fabelo
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080, USA.
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain.
| | - Samuel Ortega
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain.
| | - Gustavo M Callico
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain.
| | - Baowei Fei
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080, USA.
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center, 5323 Harry Hine Blvd, Dallas, TX 75390, USA.
- Department of Radiology, University of Texas Southwestern Medical Center, 5323 Harry Hine Blvd, Dallas, TX 75390, USA.
| |
Collapse
|