1
|
Liu W, Qu A, Yuan J, Wang L, Chen J, Zhang X, Wang H, Han Z, Li Y. Colorectal cancer histopathology image analysis: A comparative study of prognostic values of automatically extracted morphometric nuclear features in multispectral and red-blue-green imagery. Histol Histopathol 2024; 39:1303-1316. [PMID: 38343355 DOI: 10.14670/hh-18-715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/22/2024]
Abstract
OBJECTIVES Multispectral imaging (MSI) has been utilized to predict the prognosis of colorectal cancer (CRC) patients, however, our understanding of the prognostic value of nuclear morphological parameters of bright-field MSI in CRC is still limited. This study was designed to compare the efficiency of MSI and standard red-green-blue (RGB) images in predicting the prognosis of CRC. METHODS We compared the efficiency of MS and conventional RGB images on the quantitative assessment of hematoxylin-eosin (HE) stained histopathology images. A pipeline was developed using a pixel-wise support vector machine (SVM) classifier for gland-stroma segmentation, and a marker-controlled watershed algorithm was used for nuclei segmentation. The correlation between extracted morphological parameters and the five-year disease-free survival (5-DFS) was analyzed. RESULTS Forty-seven nuclear morphological parameters were extracted in total. Based on Kaplan-Meier analysis, eight features derived from MS images and seven featured derived from RGB images were significantly associated with 5-DFS, respectively. Compared with RGB images, MSI showed higher accuracy, precision, and Dice index in nuclei segmentation. Multivariate analysis indicated that both integrated parameters 1 (factors negatively correlated with CRC prognosis including nuclear number, circularity, eccentricity, major axis length) and 2 (factors positively correlated with CRC prognosis including nuclear average area, area perimeter, total area/total perimeter ratio, average area/perimeter ratio) in MS images were independent prognostic factors of 5-DFS, in contrast with only integrated parameter 1 (P<0.001) in RGB images. More importantly, the quantification of HE-stained MS images displayed higher accuracy in predicting 5-DFS compared with RGB images (76.9% vs 70.9%). CONCLUSIONS Quantitative evaluation of HE-stained MS images could yield more information and better predictive performance for CRC prognosis than conventional RGB images, thereby contributing to precision oncology.
Collapse
Affiliation(s)
- Wenlou Liu
- Department of Oncology, Affiliated Hospital of Xuzhou Medical University, Xuzhou, China
| | - Aiping Qu
- School of Computer, University of South China, Hengyang, China
| | - Jingping Yuan
- Department of Pathology, Renmin Hospital of Wuhan University, Wuhan, China
| | - Linwei Wang
- Department of Oncology, Zhongnan Hospital of Wuhan University, Wuhan, China
| | - Jiamei Chen
- Department of Oncology, Renmin Hospital of Wuhan University, Wuhan, China
| | - Xiuli Zhang
- Department of Radiology, Affiliated Hospital of Xuzhou Medical University, Xuzhou, China
| | - Hongmei Wang
- Department of Oncology, Affiliated Hospital of Xuzhou Medical University, Xuzhou, China
| | - Zhengxiang Han
- Department of Oncology, Affiliated Hospital of Xuzhou Medical University, Xuzhou, China.
| | - Yan Li
- Department of Cancer Surgery, Beijing Tsinghua Changgung Hospital, Tsinghua University, Beijing, China.
| |
Collapse
|
2
|
Thiem DGE, Stephan D, Ziebart A, Ruemmler R, Riedel J, Vinayahalingam S, Al-Nawas B, Blatt S, Kämmerer PW. Effects of volume management on free flap perfusion and metabolism in a large animal model study. Lab Anim (NY) 2024; 53:268-275. [PMID: 39122993 DOI: 10.1038/s41684-024-01410-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 06/27/2024] [Indexed: 08/12/2024]
Abstract
Free flap failure represents a substantial clinical burden. The role of intraoperative volume management remains controversial, with valid studies lacking. Here, using a large animal model, we investigated the influence of volume management on free flap perfusion and metabolism. Autotransfer of a musculocutaneous gracilis flap was performed on 31 German domestic pigs, with arterial anastomosis and catheterization of the pedicle vein for sequential blood sampling. Flap reperfusion was followed by induction of a hemorrhagic shock with maintenance for 30 min and subsequent circulation stabilization with crystalloid solution, crystalloid solution and catecholamine, autotransfusion or colloidal solution. Flap perfusion and oxygenation were periodically assessed using hyperspectral imaging. Flap metabolism was assessed via periodic blood gas analyses. Hyperspectral imaging revealed no difference in either superficial or deep tissue oxygen saturation, tissue hemoglobin or tissue water content between the test groups at any time point. Blood gas analyses showed that lactate levels were significantly increased in the group that received crystalloid solution and catecholamine, after circulatory stabilization and up to 2 h after. We conclude that, in hemorrhagic shock, volume management impacts acid-base balance in free flaps. Crystalloid solutions with norepinephrine increase lactate levels, yet short-term effects on flap perfusion seem minimal, suggesting that vasopressors are not detrimental.
Collapse
Affiliation(s)
- Daniel G E Thiem
- Department of Oral and Maxillofacial Surgery, Facial Plastic Surgery, University Medical Centre of the Johannes Gutenberg-University Mainz, Mainz, Germany.
| | - Daniel Stephan
- Department of Oral and Maxillofacial Surgery, Facial Plastic Surgery, University Medical Centre of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Alexander Ziebart
- Department of Anaesthesiology, University Medical Centre of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Robert Ruemmler
- Department of Anaesthesiology, University Medical Centre of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Julian Riedel
- Department of Anaesthesiology, University Medical Centre of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Shankeeth Vinayahalingam
- Department of Oral and Maxillofacial Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Bilal Al-Nawas
- Department of Oral and Maxillofacial Surgery, Facial Plastic Surgery, University Medical Centre of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Sebastian Blatt
- Department of Oral and Maxillofacial Surgery, Facial Plastic Surgery, University Medical Centre of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Peer W Kämmerer
- Department of Oral and Maxillofacial Surgery, Facial Plastic Surgery, University Medical Centre of the Johannes Gutenberg-University Mainz, Mainz, Germany
| |
Collapse
|
3
|
Leung JH, Karmakar R, Mukundan A, Lin WS, Anwar F, Wang HC. Technological Frontiers in Brain Cancer: A Systematic Review and Meta-Analysis of Hyperspectral Imaging in Computer-Aided Diagnosis Systems. Diagnostics (Basel) 2024; 14:1888. [PMID: 39272675 PMCID: PMC11394276 DOI: 10.3390/diagnostics14171888] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2024] [Revised: 08/19/2024] [Accepted: 08/23/2024] [Indexed: 09/15/2024] Open
Abstract
Brain cancer is a substantial factor in the mortality associated with cancer, presenting difficulties in the timely identification of the disease. The precision of diagnoses is significantly dependent on the proficiency of radiologists and neurologists. Although there is potential for early detection with computer-aided diagnosis (CAD) algorithms, the majority of current research is hindered by its modest sample sizes. This meta-analysis aims to comprehensively assess the diagnostic test accuracy (DTA) of computer-aided design (CAD) models specifically designed for the detection of brain cancer utilizing hyperspectral (HSI) technology. We employ Quadas-2 criteria to choose seven papers and classify the proposed methodologies according to the artificial intelligence method, cancer type, and publication year. In order to evaluate heterogeneity and diagnostic performance, we utilize Deeks' funnel plot, the forest plot, and accuracy charts. The results of our research suggest that there is no notable variation among the investigations. The CAD techniques that have been examined exhibit a notable level of precision in the automated detection of brain cancer. However, the absence of external validation hinders their potential implementation in real-time clinical settings. This highlights the necessity for additional studies in order to authenticate the CAD models for wider clinical applicability.
Collapse
Affiliation(s)
- Joseph-Hang Leung
- Department of Radiology, Ditmanson Medical Foundation Chia-yi Christian Hospital, Chia Yi 60002, Taiwan
| | - Riya Karmakar
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Arvind Mukundan
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Wen-Shou Lin
- Neurology Division, Department of Internal Medicine, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st. Rd., Lingya District, Kaohsiung City 80284, Taiwan
| | - Fathima Anwar
- Faculty of Allied Health Sciences, The University of Lahore, 1-Km Defense Road, Lahore 54590, Punjab, Pakistan
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
- Department of Medical Research, Dalin Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, No. 2, Minsheng Road, Dalin, Chia Yi 62247, Taiwan
- Department of Technology Development, Hitspectra Intelligent Technology Co., Ltd., 8F.11-1, No. 25, Chenggong 2nd Rd., Qianzhen Dist., Kaohsiung City 80661, Taiwan
| |
Collapse
|
4
|
Jong LJS, Post AL, Geldof F, Dashtbozorg B, Ruers TJM, Sterenborg HJCM. Separating Surface Reflectance from Volume Reflectance in Medical Hyperspectral Imaging. Diagnostics (Basel) 2024; 14:1812. [PMID: 39202300 PMCID: PMC11353750 DOI: 10.3390/diagnostics14161812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2024] [Revised: 08/13/2024] [Accepted: 08/16/2024] [Indexed: 09/03/2024] Open
Abstract
Hyperspectral imaging has shown great promise for diagnostic applications, particularly in cancer surgery. However, non-bulk tissue-related spectral variations complicate the data analysis. Common techniques, such as standard normal variate normalization, often lead to a loss of amplitude and scattering information. This study investigates a novel approach to address these spectral variations in hyperspectral images of optical phantoms and excised human breast tissue. Our method separates surface and volume reflectance, hypothesizing that spectral variability arises from significant variations in surface reflectance across pixels. An illumination setup was developed to measure samples with a hyperspectral camera from different axial positions but with identical zenith angles. This configuration, combined with a novel data analysis approach, allows for the estimation and separation of surface reflectance for each direction and volume reflectance across all directions. Validated with optical phantoms, our method achieved an 83% reduction in spectral variability. Its functionality was further demonstrated in excised human breast tissue. Our method effectively addresses variations caused by surface reflectance or glare while conserving surface reflectance information, which may enhance sample analysis and evaluation. It benefits samples with unknown refractive index spectra and can be easily adapted and applied across a wide range of fields where hyperspectral imaging is used.
Collapse
Affiliation(s)
- Lynn-Jade S. Jong
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Anouk L. Post
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| | - Freija Geldof
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Behdad Dashtbozorg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| | - Theo J. M. Ruers
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | | |
Collapse
|
5
|
Goessinger EV, Dittrich PG, Nöcker P, Notni G, Weber S, Cerminara S, Mühleisen B, Navarini AA, Maul LV. Classification of melanocytic lesions using direct illumination multispectral imaging. Sci Rep 2024; 14:19036. [PMID: 39152181 PMCID: PMC11329730 DOI: 10.1038/s41598-024-69773-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2024] [Accepted: 08/08/2024] [Indexed: 08/19/2024] Open
Abstract
With rising melanoma incidence and mortality, early detection and surgical removal of primary lesions is essential. Multispectral imaging is a new, non-invasive technique that can facilitate skin cancer detection by measuring the reflectance spectra of biological tissues. Currently, incident illumination allows little light to be reflected from deeper skin layers due to high surface reflectance. A pilot study was conducted at the University Hospital Basel to evaluate, whether multispectral imaging with direct light coupling could extract more information from deeper skin layers for more accurate dignity classification of melanocytic lesions. 27 suspicious pigmented lesions from 23 patients were included (6 melanomas, 6 dysplastic nevi, 12 melanocytic nevi, 3 other). Lesions were imaged before excision using a prototype snapshot mosaic multispectral camera with incident and direct illumination with subsequent dignity classification by a pre-trained multispectral image analysis model. Using incident light, a sensitivity of 83.3% and a specificity of 58.8% were achieved compared to dignity as determined by histopathological examination. Direct light coupling resulted in a superior sensitivity of 100% and specificity of 82.4%. Convolutional neural network classification of corresponding red, green, and blue lesion images resulted in 16.7% lower sensitivity (83.3%, 5/6 malignant lesions detected) and 20.9% lower specificity (61.5%) compared to direct light coupling with multispectral image classification. Our results show that incorporating direct light multispectral imaging into the melanoma detection process could potentially increase the accuracy of dignity classification. This newly evaluated illumination method could improve multispectral applications in skin cancer detection. Further larger studies are needed to validate the camera prototype.
Collapse
Affiliation(s)
| | - Paul-Gerald Dittrich
- Imaging and Sensing Department, Fraunhofer Institute for Applied Optics and Precision Engineering IOF, Albert-Einstein-Strasse 7, 07745, Jena, Germany.
- SpectroNet c/o Technologie- und Innovationspark Jena GmbH, Jena, Germany.
- Group for Quality Assurance and Industrial Image Processing, Department of Mechanical Engineering, Technische Universität Ilmenau, Ilmenau, Germany.
| | - Philipp Nöcker
- Imaging and Sensing Department, Fraunhofer Institute for Applied Optics and Precision Engineering IOF, Albert-Einstein-Strasse 7, 07745, Jena, Germany
| | - Gunther Notni
- Group for Quality Assurance and Industrial Image Processing, Department of Mechanical Engineering, Technische Universität Ilmenau, Ilmenau, Germany
| | | | - Sara Cerminara
- Department of Dermatology, University Hospital of Basel, Basel, Switzerland
| | - Beda Mühleisen
- Department of Dermatology, University Hospital of Basel, Basel, Switzerland
| | | | - Lara Valeska Maul
- Department of Dermatology, University Hospital of Basel, Basel, Switzerland.
- Department of Dermatology, University Hospital Zurich, Zurich, Switzerland.
- Faculty of Medicine, University of Zurich, Zurich, Switzerland.
- Department of Dermatology, Felix Platter Hospital, University Hospital Basel, Burgfelderstrasse 101, 4055, Basel, Switzerland.
| |
Collapse
|
6
|
Cruz‐Guerrero IA, Campos‐Delgado DU, Mejía‐Rodríguez AR, Leon R, Ortega S, Fabelo H, Camacho R, Plaza MDLL, Callico G. Hybrid brain tumor classification of histopathology hyperspectral images by linear unmixing and an ensemble of deep neural networks. Healthc Technol Lett 2024; 11:240-251. [PMID: 39100499 PMCID: PMC11294933 DOI: 10.1049/htl2.12084] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Revised: 02/09/2024] [Accepted: 03/18/2024] [Indexed: 08/06/2024] Open
Abstract
Hyperspectral imaging has demonstrated its potential to provide correlated spatial and spectral information of a sample by a non-contact and non-invasive technology. In the medical field, especially in histopathology, HSI has been applied for the classification and identification of diseased tissue and for the characterization of its morphological properties. In this work, we propose a hybrid scheme to classify non-tumor and tumor histological brain samples by hyperspectral imaging. The proposed approach is based on the identification of characteristic components in a hyperspectral image by linear unmixing, as a features engineering step, and the subsequent classification by a deep learning approach. For this last step, an ensemble of deep neural networks is evaluated by a cross-validation scheme on an augmented dataset and a transfer learning scheme. The proposed method can classify histological brain samples with an average accuracy of 88%, and reduced variability, computational cost, and inference times, which presents an advantage over methods in the state-of-the-art. Hence, the work demonstrates the potential of hybrid classification methodologies to achieve robust and reliable results by combining linear unmixing for features extraction and deep learning for classification.
Collapse
Affiliation(s)
- Inés A. Cruz‐Guerrero
- Facultad de CienciasUniversidad Autonoma de San Luis Potosí (UASLP)San Luis PotosiMexico
- Department of Biostatistics and Informatics, Colorado School of Public HealthUniversity of Colorado Anschutz Medical CampusColoradoUSA
| | | | | | - Raquel Leon
- Institute for Applied Microelectronics (IUMA)University of Las Palmas de Gran CanariaLas Palmas de Gran CanariaSpain
| | - Samuel Ortega
- Institute for Applied Microelectronics (IUMA)University of Las Palmas de Gran CanariaLas Palmas de Gran CanariaSpain
| | - Himar Fabelo
- Institute for Applied Microelectronics (IUMA)University of Las Palmas de Gran CanariaLas Palmas de Gran CanariaSpain
| | - Rafael Camacho
- Department of Pathological AnatomyUniversity Hospital Doctor Negrin of Gran CanariaLas Palmas de Gran CanariaSpain
| | - Maria de la Luz Plaza
- Department of Pathological AnatomyUniversity Hospital Doctor Negrin of Gran CanariaLas Palmas de Gran CanariaSpain
| | - Gustavo Callico
- Institute for Applied Microelectronics (IUMA)University of Las Palmas de Gran CanariaLas Palmas de Gran CanariaSpain
| |
Collapse
|
7
|
Li Q, Zhang X, Zhang J, Huang H, Li L, Guo C, Li W, Guo Y. Deep learning-based hyperspectral technique identifies metastatic lymph nodes in oral squamous cell carcinoma-A pilot study. Oral Dis 2024. [PMID: 39005220 DOI: 10.1111/odi.15067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Revised: 05/31/2024] [Accepted: 07/02/2024] [Indexed: 07/16/2024]
Abstract
AIMS To establish a system based on hyperspectral imaging and deep learning for the detection of cancer cells in metastatic lymph nodes. MAIN METHODS The continuous sections of metastatic lymph nodes from 45 oral squamous cell carcinoma (OSCC) patients were collected. An improved ResUNet algorithm was established for deep learning to analyze the spectral curve differences between cancer cells and lymphocytes, and that between tumor tissue and normal tissue. KEY FINDINGS It was found that cancer cells, lymphocytes, and erythrocytes in the metastatic lymph nodes could be distinguished basing hyperspectral image, with overall accuracy (OA) as 87.30% and average accuracy (AA) as 85.46%. Cancerous area could be recognized by hyperspectral image and deep learning, and the average intersection over union (IOU) and accuracy were 0.6253 and 0.7692, respectively. SIGNIFICANCE This study indicated that deep learning-based hyperspectral techniques can identify tumor tissue in OSCC metastatic lymph nodes, achieving high accuracy of pathological diagnosis, high work efficiency, and reducing work burden. But these are preliminary results limited to a small sample.
Collapse
Affiliation(s)
- Qingxiang Li
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Xueyu Zhang
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Jianyun Zhang
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
- Department of Oral Pathology, Peking University School and Hospital of Stomatology, Beijing, China
| | - Hongyuan Huang
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Liangliang Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Chuanbin Guo
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Wei Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Yuxing Guo
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| |
Collapse
|
8
|
Morales-Conde S, Navarro-Morales L, Moreno-Suero F, Balla A, Licardie E. Fluorescence and tracers in surgery: the coming future. Cir Esp 2024; 102 Suppl 1:S45-S60. [PMID: 38851317 DOI: 10.1016/j.cireng.2024.05.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2024] [Accepted: 05/23/2024] [Indexed: 06/10/2024]
Abstract
The revolution that we are seeing in the world of surgery will determine the way we understand surgical approaches in coming years. Since the implementation of minimally invasive surgery, innovations have constantly been developed to allow the laparoscopic approach to go further and be applied to more and more procedures. In recent years, we have been in the middle of another revolutionary era, with robotic surgery, the application of artificial intelligence and image-guided surgery. The latter includes 3D reconstructions for surgical planning, virtual reality, holograms or tracer-guided surgery, where ICG-guided fluorescence has provided a different perspective on surgery. ICG has been used to identify anatomical structures, assess tissue perfusion, and identify tumors or tumor lymphatic drainage. But the most important thing is that this technology has come hand in hand with the potential to develop other types of tracers that will facilitate the identification of tumor cells and ureters, as well as different light beams to identify anatomical structures. These will lead to other types of systems to assess tissue perfusion without the use of tracers, such as hyperspectral imaging. Combined with the upcoming introduction of ICG quantification, these developments represent a real revolution in the surgical world. With the imminent implementation of these technological advances, a review of their clinical application in general surgery is timely, and this review serves that aim.
Collapse
Affiliation(s)
- Salvador Morales-Conde
- Servicio de Cirugía General y Digestiva, Hospital Universitario Virgen Macarena, Facultad de Medicina, Universidad de Sevilla, Sevilla, Spain; Servicio de Cirugía General y Digestiva, Hospital Quironsalud Sagrado Corazón, Sevilla, Spain.
| | - Laura Navarro-Morales
- Servicio de Cirugía General y Digestiva, Hospital Quironsalud Sagrado Corazón, Sevilla, Spain.
| | - Francisco Moreno-Suero
- Servicio de Cirugía General y Digestiva, Hospital Quironsalud Sagrado Corazón, Sevilla, Spain.
| | - Andrea Balla
- Servicio de Cirugía General y Digestiva, Hospital Universitario Virgen Macarena, Facultad de Medicina, Universidad de Sevilla, Sevilla, Spain; Servicio de Cirugía General y Digestiva, Hospital Quironsalud Sagrado Corazón, Sevilla, Spain.
| | - Eugenio Licardie
- Servicio de Cirugía General y Digestiva, Hospital Universitario Virgen Macarena, Facultad de Medicina, Universidad de Sevilla, Sevilla, Spain; Servicio de Cirugía General y Digestiva, Hospital Quironsalud Sagrado Corazón, Sevilla, Spain.
| |
Collapse
|
9
|
Pertzborn D, Bali A, Mühlig A, von Eggeling F, Guntinas-Lichius O. Hyperspectral imaging and evaluation of surgical margins: where do we stand? Curr Opin Otolaryngol Head Neck Surg 2024; 32:96-104. [PMID: 38193544 DOI: 10.1097/moo.0000000000000957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2024]
Abstract
PURPOSE OF REVIEW To highlight the recent literature on the use of hyperspectral imaging (HSI) for cancer margin evaluation ex vivo, for head and neck cancer pathology and in vivo during head and neck cancer surgery. RECENT FINDINGS HSI can be used ex vivo on unstained and stained tissue sections to analyze head and neck tissue and tumor cells in combination with machine learning approaches to analyze head and neck cancer cell characteristics and to discriminate the tumor border from normal tissue. Data on in vivo applications during head and neck cancer surgery are preliminary and limited. Even now an accuracy of 80% for tumor versus nonneoplastic tissue classification can be achieved for certain tasks, within the current in vivo settings. SUMMARY Significant progress has been made to introduce HSI for ex vivo head and neck cancer pathology evaluation and for an intraoperative use to define the tumor margins. To optimize the accuracy for in vivo use, larger HSI databases with annotations for head and neck cancer are needed.
Collapse
Affiliation(s)
- David Pertzborn
- Department of Otorhinolaryngology, Jena University Hospital, Jena, Germany
| | | | | | | | | |
Collapse
|
10
|
Myslicka M, Kawala-Sterniuk A, Bryniarska A, Sudol A, Podpora M, Gasz R, Martinek R, Kahankova Vilimkova R, Vilimek D, Pelc M, Mikolajewski D. Review of the application of the most current sophisticated image processing methods for the skin cancer diagnostics purposes. Arch Dermatol Res 2024; 316:99. [PMID: 38446274 DOI: 10.1007/s00403-024-02828-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 12/28/2023] [Accepted: 01/25/2024] [Indexed: 03/07/2024]
Abstract
This paper presents the most current and innovative solutions applying modern digital image processing methods for the purpose of skin cancer diagnostics. Skin cancer is one of the most common types of cancers. It is said that in the USA only, one in five people will develop skin cancer and this trend is constantly increasing. Implementation of new, non-invasive methods plays a crucial role in both identification and prevention of skin cancer occurrence. Early diagnosis and treatment are needed in order to decrease the number of deaths due to this disease. This paper also contains some information regarding the most common skin cancer types, mortality and epidemiological data for Poland, Europe, Canada and the USA. It also covers the most efficient and modern image recognition methods based on the artificial intelligence applied currently for diagnostics purposes. In this work, both professional, sophisticated as well as inexpensive solutions were presented. This paper is a review paper and covers the period of 2017 and 2022 when it comes to solutions and statistics. The authors decided to focus on the latest data, mostly due to the rapid technology development and increased number of new methods, which positively affects diagnosis and prognosis.
Collapse
Affiliation(s)
- Maria Myslicka
- Faculty of Medicine, Wroclaw Medical University, J. Mikulicza-Radeckiego 5, 50-345, Wroclaw, Poland.
| | - Aleksandra Kawala-Sterniuk
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland.
| | - Anna Bryniarska
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Adam Sudol
- Faculty of Natural Sciences and Technology, University of Opole, Dmowskiego 7-9, 45-368, Opole, Poland
| | - Michal Podpora
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Rafal Gasz
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Radek Martinek
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Radana Kahankova Vilimkova
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Dominik Vilimek
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Mariusz Pelc
- Institute of Computer Science, University of Opole, Oleska 48, 45-052, Opole, Poland
- School of Computing and Mathematical Sciences, University of Greenwich, Old Royal Naval College, Park Row, SE10 9LS, London, UK
| | - Dariusz Mikolajewski
- Institute of Computer Science, Kazimierz Wielki University in Bydgoszcz, ul. Kopernika 1, 85-074, Bydgoszcz, Poland
- Neuropsychological Research Unit, 2nd Clinic of the Psychiatry and Psychiatric Rehabilitation, Medical University in Lublin, Gluska 1, 20-439, Lublin, Poland
| |
Collapse
|
11
|
Jong LJS, Appelman JGC, Sterenborg HJCM, Ruers TJM, Dashtbozorg B. Spatial and Spectral Reconstruction of Breast Lumpectomy Hyperspectral Images. SENSORS (BASEL, SWITZERLAND) 2024; 24:1567. [PMID: 38475103 DOI: 10.3390/s24051567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 02/23/2024] [Accepted: 02/27/2024] [Indexed: 03/14/2024]
Abstract
(1) Background: Hyperspectral imaging has emerged as a promising margin assessment technique for breast-conserving surgery. However, to be implicated intraoperatively, it should be both fast and capable of yielding high-quality images to provide accurate guidance and decision-making throughout the surgery. As there exists a trade-off between image quality and data acquisition time, higher resolution images come at the cost of longer acquisition times and vice versa. (2) Methods: Therefore, in this study, we introduce a deep learning spatial-spectral reconstruction framework to obtain a high-resolution hyperspectral image from a low-resolution hyperspectral image combined with a high-resolution RGB image as input. (3) Results: Using the framework, we demonstrate the ability to perform a fast data acquisition during surgery while maintaining a high image quality, even in complex scenarios where challenges arise, such as blur due to motion artifacts, dead pixels on the camera sensor, noise from the sensor's reduced sensitivity at spectral extremities, and specular reflections caused by smooth surface areas of the tissue. (4) Conclusion: This gives the opportunity to facilitate an accurate margin assessment through intraoperative hyperspectral imaging.
Collapse
Affiliation(s)
- Lynn-Jade S Jong
- Image-Guided Surgery, Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Jelmer G C Appelman
- Image-Guided Surgery, Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Faculty of Science, Vrije Universiteit Amsterdam, De Boelelaan 1111, 1081 HV Amsterdam, The Netherlands
| | - Henricus J C M Sterenborg
- Image-Guided Surgery, Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| | - Theo J M Ruers
- Image-Guided Surgery, Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Behdad Dashtbozorg
- Image-Guided Surgery, Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| |
Collapse
|
12
|
Tran MH, Bryarly M, Ma L, Yousuf MS, Price TJ, Fei B. Nerve Detection and Visualization Using Hyperspectral Imaging for Surgical Guidance. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12930:129302A. [PMID: 38707637 PMCID: PMC11070131 DOI: 10.1117/12.3008470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2024]
Abstract
During surgery of delicate regions, differentiation between nerve and surrounding tissue is crucial. Hyperspectral imaging (HSI) techniques can enhance the contrast between types of tissue beyond what the human eye can differentiate. Whereas an RGB image captures 3 bands within the visible light range (e.g., 400 nm to 700 nm), HSI can acquire many bands in wavelength increments that highlight regions of an image across a wavelength spectrum. We developed a workflow to identify nerve tissues from other similar tissues such as fat, bone, and muscle. Our workflow uses spectral angle mapper (SAM) and endmember selection. The method is robust for different types of environment and lighting conditions. We validated our workflow on two samples of human tissues. We used a compact HSI system that can image from 400 to 1700 nm to produce HSI of the samples. On these two samples, we achieved an intersection-over-union (IoU) segmentation score of 84.15% and 76.73%, respectively. We showed that our workflow identifies nerve segments that are not easily seen in RGB images. This method is fast, does not rely on special hardware, and can be applied in real time. The hyperspectral imaging and nerve detection approach may provide a powerful tool for image-guided surgery.
Collapse
Affiliation(s)
- Minh Ha Tran
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Michelle Bryarly
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | | | - Theodore J. Price
- Department of Neuroscience, University of Texas at Dallas, Richardson, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
13
|
Zhang L, Xue J, Xie Y, Huang D, Xie Z, Zhu L, Chen X, Cui G, Ali S, Huang G, Chen X. Automatic detection of ischemic necrotic sites in small intestinal tissue using hyperspectral imaging and transfer learning. JOURNAL OF BIOPHOTONICS 2024; 17:e202300315. [PMID: 38018735 DOI: 10.1002/jbio.202300315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 11/25/2023] [Accepted: 11/27/2023] [Indexed: 11/30/2023]
Abstract
Acquiring large amounts of hyperspectral data of small intestinal tissue with real labels in the clinic is difficult, and the data shows inter-patient variability. Building an automatic identification model using a small dataset presents a crucial challenge in obtaining a strong generalization of the model. This study aimed to explore the performance of hyperspectral imaging and transfer learning techniques in the automatic identification of normal and ischemic necrotic sites in small intestinal tissue. Hyperspectral data of small intestinal tissues were collected from eight white rabbit samples. The transfer component analysis (TCA) method was performed to transfer learning on hyperspectral data between different samples and the variability of data distribution between samples was reduced. The results showed that the TCA transfer learning method improved the accuracy of the classification model with less training data. This study provided a reliable method for single-sample modelling to detect necrotic sites in small intestinal tissue .
Collapse
Affiliation(s)
- Lechao Zhang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, China
| | - Jianxia Xue
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, China
| | - Yi Xie
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, China
| | - Danfei Huang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, China
| | - Zhonghao Xie
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, China
| | - Libin Zhu
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, China
| | - Xiaoqing Chen
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, China
| | - Guihua Cui
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, China
| | - Shujat Ali
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, China
| | - Guangzao Huang
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, China
| | - Xiaojing Chen
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, China
| |
Collapse
|
14
|
Ilgen A, Köhler H, Pfahl A, Stelzner S, Mehdorn M, Jansen-Winkeln B, Gockel I, Moulla Y. Intraoperative Laparoscopic Hyperspectral Imaging during Esophagectomy-A Pilot Study Evaluating Esophagogastric Perfusion at the Anastomotic Sites. Bioengineering (Basel) 2024; 11:69. [PMID: 38247946 PMCID: PMC10812999 DOI: 10.3390/bioengineering11010069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2023] [Revised: 12/01/2023] [Accepted: 01/04/2024] [Indexed: 01/23/2024] Open
Abstract
Hyperspectral imaging (HSI) is a non-invasive and contactless technique that enables the real-time acquisition of comprehensive information on tissue within the surgical field. In this pilot study, we investigated whether a new HSI system for minimally-invasive surgery, TIVITA® Mini (HSI-MIS), provides reliable insights into tissue perfusion of the proximal and distal esophagogastric anastomotic sites during 21 laparoscopic/thoracoscopic or robotic Ivor Lewis esophagectomies of patients with cancer to minimize the risk of dreaded anastomotic insufficiency. In this pioneering investigation, physiological tissue parameters were derived from HSI measurements of the proximal site of the anastomosis (esophageal stump) and the distal site of the anastomosis (tip of the gastric conduit) during the thoracic phase of the procedure. Tissue oxygenation (StO2), Near Infrared Perfusion Index (NIR-PI), and Tissue Water Index (TWI) showed similar median values at both anastomotic sites. Significant differences were observed only for NIR-PI (median: 76.5 vs. 63.9; p = 0.012) at the distal site (gastric conduit) compared to our previous study using an HSI system for open surgery. For all 21 patients, reliable and informative measurements were attainable, confirming the feasibility of HSI-MIS to assess anastomotic viability. Further studies on the added benefit of this new technique aiming to reduce anastomotic insufficiency are warranted.
Collapse
Affiliation(s)
- Annalena Ilgen
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103 Leipzig, Germany; (H.K.); (A.P.)
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103 Leipzig, Germany; (H.K.); (A.P.)
| | - Sigmar Stelzner
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| | - Matthias Mehdorn
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| | - Boris Jansen-Winkeln
- Department of General, Visceral, Thoracic and Vascular Surgery, Klinikum St. Georg, Delitzscher Str. 141, 04129 Leipzig, Germany;
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Liebigstr. 20, 04103 Leipzig, Germany; (A.I.); (S.S.); (I.G.)
| |
Collapse
|
15
|
Poudel P, Frost SM, Eslick S, Sohrabi HR, Taddei K, Martins RN, Hone E. Hyperspectral Retinal Imaging as a Non-Invasive Marker to Determine Brain Amyloid Status. J Alzheimers Dis 2024; 100:S131-S152. [PMID: 39121128 DOI: 10.3233/jad-240631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/11/2024]
Abstract
Background As an extension of the central nervous system (CNS), the retina shares many similarities with the brain and can manifest signs of various neurological diseases, including Alzheimer's disease (AD). Objective To investigate the retinal spectral features and develop a classification model to differentiate individuals with different brain amyloid levels. Methods Sixty-six participants with varying brain amyloid-β protein levels were non-invasively imaged using a hyperspectral retinal camera in the wavelength range of 450-900 nm in 5 nm steps. Multiple retina features from the central and superior views were selected and analyzed to identify their variability among individuals with different brain amyloid loads. Results The retinal reflectance spectra in the 450-585 nm wavelengths exhibited a significant difference in individuals with increasing brain amyloid. The retinal features in the superior view showed higher inter-subject variability. A classification model was trained to differentiate individuals with varying amyloid levels using the spectra of extracted retinal features. The performance of the spectral classification model was dependent upon retinal features and showed 0.758-0.879 accuracy, 0.718-0.909 sensitivity, 0.764-0.912 specificity, and 0.745-0.891 area under curve for the right eye. Conclusions This study highlights the spectral variation of retinal features associated with brain amyloid loads. It also demonstrates the feasibility of the retinal hyperspectral imaging technique as a potential method to identify individuals in the preclinical phase of AD as an inexpensive alternative to brain imaging.
Collapse
Affiliation(s)
- Purna Poudel
- Alzheimer's Research Australia, Ralph and Patricia Sarich Neuroscience Research Institute, Nedlands, WA, Australia
- Centre of Excellence for Alzheimer's Disease Research and Care, School of Medical and Health Sciences, Edith Cowan University, Joondalup, WA, Australia
| | - Shaun M Frost
- Commonwealth Scientific and Industrial Research Organisation (CSIRO), Kensington, WA, Australia
- Australian e-Health Research Centre, Floreat, WA, Australia
| | - Shaun Eslick
- Lifespan Health and Wellbeing Research Centre, Macquarie Medical School, Macquarie University, Macquarie Park, NSW, Australia
| | - Hamid R Sohrabi
- Alzheimer's Research Australia, Ralph and Patricia Sarich Neuroscience Research Institute, Nedlands, WA, Australia
- Centre for Healthy Ageing, Health Futures Institute, Murdoch University, Perth, WA, Australia
| | - Kevin Taddei
- Alzheimer's Research Australia, Ralph and Patricia Sarich Neuroscience Research Institute, Nedlands, WA, Australia
- Centre of Excellence for Alzheimer's Disease Research and Care, School of Medical and Health Sciences, Edith Cowan University, Joondalup, WA, Australia
- Lions Alzheimer's Foundation, Perth, WA, Australia
| | - Ralph N Martins
- Alzheimer's Research Australia, Ralph and Patricia Sarich Neuroscience Research Institute, Nedlands, WA, Australia
- Centre of Excellence for Alzheimer's Disease Research and Care, School of Medical and Health Sciences, Edith Cowan University, Joondalup, WA, Australia
- Lifespan Health and Wellbeing Research Centre, Macquarie Medical School, Macquarie University, Macquarie Park, NSW, Australia
- Lions Alzheimer's Foundation, Perth, WA, Australia
| | - Eugene Hone
- Alzheimer's Research Australia, Ralph and Patricia Sarich Neuroscience Research Institute, Nedlands, WA, Australia
- Centre of Excellence for Alzheimer's Disease Research and Care, School of Medical and Health Sciences, Edith Cowan University, Joondalup, WA, Australia
- Lions Alzheimer's Foundation, Perth, WA, Australia
| |
Collapse
|
16
|
Bassler MC, Knoblich M, Gerhard-Hartmann E, Mukherjee A, Youssef A, Hagen R, Haug L, Goncalves M, Scherzad A, Stöth M, Ostertag E, Steinke M, Brecht M, Hackenberg S, Meyer TJ. Differentiation of Salivary Gland and Salivary Gland Tumor Tissue via Raman Imaging Combined with Multivariate Data Analysis. Diagnostics (Basel) 2023; 14:92. [PMID: 38201401 PMCID: PMC10795677 DOI: 10.3390/diagnostics14010092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 12/10/2023] [Accepted: 12/29/2023] [Indexed: 01/12/2024] Open
Abstract
Salivary gland tumors (SGTs) are a relevant, highly diverse subgroup of head and neck tumors whose entity determination can be difficult. Confocal Raman imaging in combination with multivariate data analysis may possibly support their correct classification. For the analysis of the translational potential of Raman imaging in SGT determination, a multi-stage evaluation process is necessary. By measuring a sample set of Warthin tumor, pleomorphic adenoma and non-tumor salivary gland tissue, Raman data were obtained and a thorough Raman band analysis was performed. This evaluation revealed highly overlapping Raman patterns with only minor spectral differences. Consequently, a principal component analysis (PCA) was calculated and further combined with a discriminant analysis (DA) to enable the best possible distinction. The PCA-DA model was characterized by accuracy, sensitivity, selectivity and precision values above 90% and validated by predicting model-unknown Raman spectra, of which 93% were classified correctly. Thus, we state our PCA-DA to be suitable for parotid tumor and non-salivary salivary gland tissue discrimination and prediction. For evaluation of the translational potential, further validation steps are necessary.
Collapse
Affiliation(s)
- Miriam C. Bassler
- Process Analysis and Technology (PA&T), School of Life Science, Reutlingen University, Alteburgstr. 150, 72762 Reutlingen, Germany; (M.C.B.); (M.K.); (A.M.); (E.O.)
- Institute of Physical and Theoretical Chemistry, Faculty of Science, University of Tübingen, Auf der Morgenstelle 18, 72076 Tübingen, Germany
| | - Mona Knoblich
- Process Analysis and Technology (PA&T), School of Life Science, Reutlingen University, Alteburgstr. 150, 72762 Reutlingen, Germany; (M.C.B.); (M.K.); (A.M.); (E.O.)
- Institute of Physical and Theoretical Chemistry, Faculty of Science, University of Tübingen, Auf der Morgenstelle 18, 72076 Tübingen, Germany
| | - Elena Gerhard-Hartmann
- Institute of Pathology, University of Würzburg, Josef-Schneider-Str. 2, 97080 Würzburg, Germany; (E.G.-H.); (A.Y.); (L.H.)
| | - Ashutosh Mukherjee
- Process Analysis and Technology (PA&T), School of Life Science, Reutlingen University, Alteburgstr. 150, 72762 Reutlingen, Germany; (M.C.B.); (M.K.); (A.M.); (E.O.)
- Institute of Physical and Theoretical Chemistry, Faculty of Science, University of Tübingen, Auf der Morgenstelle 18, 72076 Tübingen, Germany
| | - Almoatazbellah Youssef
- Institute of Pathology, University of Würzburg, Josef-Schneider-Str. 2, 97080 Würzburg, Germany; (E.G.-H.); (A.Y.); (L.H.)
| | - Rudolf Hagen
- Department of Oto-Rhino-Laryngology, Plastic, Aesthetic & Reconstructive Head and Neck Surgery, University Hospital Würzburg, Josef-Schneider-Str. 11, 97080 Würzburg, Germany; (R.H.); (M.G.); (A.S.); (M.S.); (S.H.)
| | - Lukas Haug
- Institute of Pathology, University of Würzburg, Josef-Schneider-Str. 2, 97080 Würzburg, Germany; (E.G.-H.); (A.Y.); (L.H.)
| | - Miguel Goncalves
- Department of Oto-Rhino-Laryngology, Plastic, Aesthetic & Reconstructive Head and Neck Surgery, University Hospital Würzburg, Josef-Schneider-Str. 11, 97080 Würzburg, Germany; (R.H.); (M.G.); (A.S.); (M.S.); (S.H.)
| | - Agmal Scherzad
- Department of Oto-Rhino-Laryngology, Plastic, Aesthetic & Reconstructive Head and Neck Surgery, University Hospital Würzburg, Josef-Schneider-Str. 11, 97080 Würzburg, Germany; (R.H.); (M.G.); (A.S.); (M.S.); (S.H.)
| | - Manuel Stöth
- Department of Oto-Rhino-Laryngology, Plastic, Aesthetic & Reconstructive Head and Neck Surgery, University Hospital Würzburg, Josef-Schneider-Str. 11, 97080 Würzburg, Germany; (R.H.); (M.G.); (A.S.); (M.S.); (S.H.)
| | - Edwin Ostertag
- Process Analysis and Technology (PA&T), School of Life Science, Reutlingen University, Alteburgstr. 150, 72762 Reutlingen, Germany; (M.C.B.); (M.K.); (A.M.); (E.O.)
| | - Maria Steinke
- Chair of Tissue Engineering and Regenerative Medicine, University Hospital Würzburg, Röntgenring 11, 97070 Würzburg, Germany;
- Fraunhofer Institute for Silicate Research ISC, Röntgenring 11, 97070 Würzburg, Germany
| | - Marc Brecht
- Process Analysis and Technology (PA&T), School of Life Science, Reutlingen University, Alteburgstr. 150, 72762 Reutlingen, Germany; (M.C.B.); (M.K.); (A.M.); (E.O.)
- Institute of Physical and Theoretical Chemistry, Faculty of Science, University of Tübingen, Auf der Morgenstelle 18, 72076 Tübingen, Germany
| | - Stephan Hackenberg
- Department of Oto-Rhino-Laryngology, Plastic, Aesthetic & Reconstructive Head and Neck Surgery, University Hospital Würzburg, Josef-Schneider-Str. 11, 97080 Würzburg, Germany; (R.H.); (M.G.); (A.S.); (M.S.); (S.H.)
| | - Till Jasper Meyer
- Department of Oto-Rhino-Laryngology, Plastic, Aesthetic & Reconstructive Head and Neck Surgery, University Hospital Würzburg, Josef-Schneider-Str. 11, 97080 Würzburg, Germany; (R.H.); (M.G.); (A.S.); (M.S.); (S.H.)
| |
Collapse
|
17
|
Leon R, Fabelo H, Ortega S, Cruz-Guerrero IA, Campos-Delgado DU, Szolna A, Piñeiro JF, Espino C, O'Shanahan AJ, Hernandez M, Carrera D, Bisshopp S, Sosa C, Balea-Fernandez FJ, Morera J, Clavo B, Callico GM. Hyperspectral imaging benchmark based on machine learning for intraoperative brain tumour detection. NPJ Precis Oncol 2023; 7:119. [PMID: 37964078 PMCID: PMC10646050 DOI: 10.1038/s41698-023-00475-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Accepted: 10/24/2023] [Indexed: 11/16/2023] Open
Abstract
Brain surgery is one of the most common and effective treatments for brain tumour. However, neurosurgeons face the challenge of determining the boundaries of the tumour to achieve maximum resection, while avoiding damage to normal tissue that may cause neurological sequelae to patients. Hyperspectral (HS) imaging (HSI) has shown remarkable results as a diagnostic tool for tumour detection in different medical applications. In this work, we demonstrate, with a robust k-fold cross-validation approach, that HSI combined with the proposed processing framework is a promising intraoperative tool for in-vivo identification and delineation of brain tumours, including both primary (high-grade and low-grade) and secondary tumours. Analysis of the in-vivo brain database, consisting of 61 HS images from 34 different patients, achieve a highest median macro F1-Score result of 70.2 ± 7.9% on the test set using both spectral and spatial information. Here, we provide a benchmark based on machine learning for further developments in the field of in-vivo brain tumour detection and delineation using hyperspectral imaging to be used as a real-time decision support tool during neurosurgical workflows.
Collapse
Affiliation(s)
- Raquel Leon
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain.
| | - Himar Fabelo
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain.
- Fundación Canaria Instituto de Investigación Sanitaria de Canarias (FIISC), Las Palmas de Gran Canaria, Spain.
| | - Samuel Ortega
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
- Nofima, Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway
| | - Ines A Cruz-Guerrero
- Facultad de Ciencias, Universidad Autónoma de San Luis Potosí, San Luis Potosí, México
- Department of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA
- Department of Pediatric Plastic and Reconstructive Surgery, Children's Hospital Colorado, Aurora, Colorado, USA
| | - Daniel Ulises Campos-Delgado
- Facultad de Ciencias, Universidad Autónoma de San Luis Potosí, San Luis Potosí, México
- Instituto de Investigación en Comunicación Óptica, Universidad Autónoma de San Luis Potosí, San Luis Potosí, México
| | - Adam Szolna
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Juan F Piñeiro
- Instituto de Investigación en Comunicación Óptica, Universidad Autónoma de San Luis Potosí, San Luis Potosí, México
| | - Carlos Espino
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Aruma J O'Shanahan
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Maria Hernandez
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - David Carrera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Sara Bisshopp
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Coralia Sosa
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Francisco J Balea-Fernandez
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
- Department of Psychology, Sociology and Social Work, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Jesus Morera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Bernardino Clavo
- Fundación Canaria Instituto de Investigación Sanitaria de Canarias (FIISC), Las Palmas de Gran Canaria, Spain
- Research Unit, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Gustavo M Callico
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| |
Collapse
|
18
|
Cui R, Wang L, Lin L, Li J, Lu R, Liu S, Liu B, Gu Y, Zhang H, Shang Q, Chen L, Tian D. Deep Learning in Barrett's Esophagus Diagnosis: Current Status and Future Directions. Bioengineering (Basel) 2023; 10:1239. [PMID: 38002363 PMCID: PMC10669008 DOI: 10.3390/bioengineering10111239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 10/13/2023] [Accepted: 10/16/2023] [Indexed: 11/26/2023] Open
Abstract
Barrett's esophagus (BE) represents a pre-malignant condition characterized by abnormal cellular proliferation in the distal esophagus. A timely and accurate diagnosis of BE is imperative to prevent its progression to esophageal adenocarcinoma, a malignancy associated with a significantly reduced survival rate. In this digital age, deep learning (DL) has emerged as a powerful tool for medical image analysis and diagnostic applications, showcasing vast potential across various medical disciplines. In this comprehensive review, we meticulously assess 33 primary studies employing varied DL techniques, predominantly featuring convolutional neural networks (CNNs), for the diagnosis and understanding of BE. Our primary focus revolves around evaluating the current applications of DL in BE diagnosis, encompassing tasks such as image segmentation and classification, as well as their potential impact and implications in real-world clinical settings. While the applications of DL in BE diagnosis exhibit promising results, they are not without challenges, such as dataset issues and the "black box" nature of models. We discuss these challenges in the concluding section. Essentially, while DL holds tremendous potential to revolutionize BE diagnosis, addressing these challenges is paramount to harnessing its full capacity and ensuring its widespread application in clinical practice.
Collapse
Affiliation(s)
- Ruichen Cui
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Lei Wang
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
- West China School of Nursing, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China
| | - Lin Lin
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
- West China School of Nursing, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China
| | - Jie Li
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
- West China School of Nursing, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China
| | - Runda Lu
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Shixiang Liu
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Bowei Liu
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Yimin Gu
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Hanlu Zhang
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Qixin Shang
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Longqi Chen
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Dong Tian
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| |
Collapse
|
19
|
Kifle N, Teti S, Ning B, Donoho DA, Katz I, Keating R, Cha RJ. Pediatric Brain Tissue Segmentation Using a Snapshot Hyperspectral Imaging (sHSI) Camera and Machine Learning Classifier. Bioengineering (Basel) 2023; 10:1190. [PMID: 37892919 PMCID: PMC10603997 DOI: 10.3390/bioengineering10101190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2023] [Revised: 10/06/2023] [Accepted: 10/11/2023] [Indexed: 10/29/2023] Open
Abstract
Pediatric brain tumors are the second most common type of cancer, accounting for one in four childhood cancer types. Brain tumor resection surgery remains the most common treatment option for brain cancer. While assessing tumor margins intraoperatively, surgeons must send tissue samples for biopsy, which can be time-consuming and not always accurate or helpful. Snapshot hyperspectral imaging (sHSI) cameras can capture scenes beyond the human visual spectrum and provide real-time guidance where we aim to segment healthy brain tissues from lesions on pediatric patients undergoing brain tumor resection. With the institutional research board approval, Pro00011028, 139 red-green-blue (RGB), 279 visible, and 85 infrared sHSI data were collected from four subjects with the system integrated into an operating microscope. A random forest classifier was used for data analysis. The RGB, infrared sHSI, and visible sHSI models achieved average intersection of unions (IoUs) of 0.76, 0.59, and 0.57, respectively, while the tumor segmentation achieved a specificity of 0.996, followed by the infrared HSI and visible HSI models at 0.93 and 0.91, respectively. Despite the small dataset considering pediatric cases, our research leveraged sHSI technology and successfully segmented healthy brain tissues from lesions with a high specificity during pediatric brain tumor resection procedures.
Collapse
Affiliation(s)
- Naomi Kifle
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC 20010, USA; (N.K.); (I.K.)
| | - Saige Teti
- Department of Neurosurgery, Children’s National Hospital, Washington, DC 20010, USA; (S.T.); (D.A.D.)
| | - Bo Ning
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC 20010, USA; (N.K.); (I.K.)
| | - Daniel A. Donoho
- Department of Neurosurgery, Children’s National Hospital, Washington, DC 20010, USA; (S.T.); (D.A.D.)
| | - Itai Katz
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC 20010, USA; (N.K.); (I.K.)
| | - Robert Keating
- Department of Neurosurgery, Children’s National Hospital, Washington, DC 20010, USA; (S.T.); (D.A.D.)
| | - Richard Jaepyeong Cha
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC 20010, USA; (N.K.); (I.K.)
- Department of Pediatrics, George Washington University School of Medicine, Washington, DC 20010, USA
| |
Collapse
|
20
|
Chalopin C, Pfahl A, Köhler H, Knospe L, Maktabi M, Unger M, Jansen-Winkeln B, Thieme R, Moulla Y, Mehdorn M, Sucher R, Neumuth T, Gockel I, Melzer A. Alternative intraoperative optical imaging modalities for fluorescence angiography in gastrointestinal surgery: spectral imaging and imaging photoplethysmography. MINIM INVASIV THER 2023; 32:222-232. [PMID: 36622288 DOI: 10.1080/13645706.2022.2164469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 11/29/2022] [Indexed: 01/10/2023]
Abstract
INTRODUCTION Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA. MATERIAL AND METHODS The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works. RESULTS The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method. CONCLUSIONS Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Luise Knospe
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Department of Electrical, Mechanical and Industrial Engineering, Anhalt University of Applied Science, Köthen (Anhalt), Germany
| | - Michael Unger
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
- Department of General, Visceral and Oncological Surgery, St. Georg Hospital, Leipzig, Germany
| | - René Thieme
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Matthias Mehdorn
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Robert Sucher
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Institute of Medical Science and Technology (IMSAT), University of Dundee, Dundee, UK
| |
Collapse
|
21
|
Qin X, Ran T, Chen Y, Zhang Y, Wang D, Zhou C, Zou D. Artificial Intelligence in Endoscopic Ultrasonography-Guided Fine-Needle Aspiration/Biopsy (EUS-FNA/B) for Solid Pancreatic Lesions: Opportunities and Challenges. Diagnostics (Basel) 2023; 13:3054. [PMID: 37835797 PMCID: PMC10572518 DOI: 10.3390/diagnostics13193054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 09/06/2023] [Accepted: 09/06/2023] [Indexed: 10/15/2023] Open
Abstract
Solid pancreatic lesions (SPLs) encompass a variety of benign and malignant diseases and accurate diagnosis is crucial for guiding appropriate treatment decisions. Endoscopic ultrasonography-guided fine-needle aspiration/biopsy (EUS-FNA/B) serves as a front-line diagnostic tool for pancreatic mass lesions and is widely used in clinical practice. Artificial intelligence (AI) is a mathematical technique that automates the learning and recognition of data patterns. Its strong self-learning ability and unbiased nature have led to its gradual adoption in the medical field. In this paper, we describe the fundamentals of AI and provide a summary of reports on AI in EUS-FNA/B to help endoscopists understand and realize its potential in improving pathological diagnosis and guiding targeted EUS-FNA/B. However, AI models have limitations and shortages that need to be addressed before clinical use. Furthermore, as most AI studies are retrospective, large-scale prospective clinical trials are necessary to evaluate their clinical usefulness accurately. Although AI in EUS-FNA/B is still in its infancy, the constant input of clinical data and the advancements in computer technology are expected to make computer-aided diagnosis and treatment more feasible.
Collapse
Affiliation(s)
| | | | | | | | | | - Chunhua Zhou
- Department of Gastroenterology, Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai 200025, China; (X.Q.); (T.R.); (Y.C.); (Y.Z.); (D.W.)
| | - Duowu Zou
- Department of Gastroenterology, Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai 200025, China; (X.Q.); (T.R.); (Y.C.); (Y.Z.); (D.W.)
| |
Collapse
|
22
|
Jiang S, Ma D, Tan X, Yang M, Jiao Q, Xu L. Bibliometric analysis of the current status and trends on medical hyperspectral imaging. Front Med (Lausanne) 2023; 10:1235955. [PMID: 37795419 PMCID: PMC10545955 DOI: 10.3389/fmed.2023.1235955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Accepted: 08/30/2023] [Indexed: 10/06/2023] Open
Abstract
Hyperspectral imaging (HSI) is a promising technology that can provide valuable support for the advancement of the medical field. Bibliometrics can analyze a vast number of publications on both macroscopic and microscopic levels, providing scholars with essential foundations to shape future directions. The purpose of this study is to comprehensively review the existing literature on medical hyperspectral imaging (MHSI). Based on the Web of Science (WOS) database, this study systematically combs through literature using bibliometric methods and visualization software such as VOSviewer and CiteSpace to draw scientific conclusions. The analysis yielded 2,274 articles from 73 countries/regions, involving 7,401 authors, 2,037 institutions, 1,038 journals/conferences, and a total of 7,522 keywords. The field of MHSI is currently in a positive stage of development and has conducted extensive research worldwide. This research encompasses not only HSI technology but also its application to diverse medical research subjects, such as skin, cancer, tumors, etc., covering a wide range of hardware constructions and software algorithms. In addition to advancements in hardware, the future should focus on the development of algorithm standards for specific medical research targets and cultivate medical professionals of managing vast amounts of technical information.
Collapse
Affiliation(s)
| | | | | | | | | | - Liang Xu
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin,China
| |
Collapse
|
23
|
Maurya R, Mishra A, Yadav CS, Upadhyay A, Sharma G, Kumar S, Singh V. A novel tunable metal-clad planar waveguide with 0.62PMN-0.38PT material for detection of cancer cells. JOURNAL OF BIOPHOTONICS 2023; 16:e202300148. [PMID: 37280718 DOI: 10.1002/jbio.202300148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Revised: 06/01/2023] [Accepted: 06/01/2023] [Indexed: 06/08/2023]
Abstract
A dynamically tunable metal clad planar waveguide having 0.62PMN-0.38PT material is simulated and optimized for detection of cancer cells. Angular interrogation of the TE0 mode of waveguide shows that critical angle increases greater than the resonance angle with increasing of cover refractive index, which limits the detection range of waveguide. To overcome this limitation, proposed waveguide applies a potential on the PMN-PT adlayer. Although a sensitivity of 105.42 degree/RIU was achieved at 70 Volts in testing the proposed waveguide, it was found that the optimal performance parameters were obtained at 60 Volts. At this voltage, the waveguide demonstrated detection range 1.3330-1.5030, a detection accuracy 2393.33, and a figure of merit 2243.59 RIU-1 , which enabled the detection of the entire range of the targeted cancer cells. Therefore, it is recommended to apply a potential of 60 Volts to achieve the best performance from the proposed waveguide.
Collapse
Affiliation(s)
- Rajiv Maurya
- Department of Physics, Institute of Science, Banaras Hindu University, Varanasi, India
| | - Ankit Mishra
- Department of Physics, Institute of Science, Banaras Hindu University, Varanasi, India
| | - Chandan Singh Yadav
- Department of Physics, Institute of Science, Banaras Hindu University, Varanasi, India
| | - Abhishek Upadhyay
- Department of Physics, Institute of Science, Banaras Hindu University, Varanasi, India
| | - Gaurav Sharma
- Department of Physics, Institute of Science, Banaras Hindu University, Varanasi, India
| | - Sushil Kumar
- Department of Physics, Sri Shankar College Sasaram, Bihar, India
| | - Vivek Singh
- Department of Physics, Institute of Science, Banaras Hindu University, Varanasi, India
| |
Collapse
|
24
|
Chen R, Luo T, Nie J, Chu Y. Blood cancer diagnosis using hyperspectral imaging combined with the forward searching method and machine learning. ANALYTICAL METHODS : ADVANCING METHODS AND APPLICATIONS 2023; 15:3885-3892. [PMID: 37503555 DOI: 10.1039/d3ay00787a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
Hyperspectral imaging (HSI), a widely used biosensing technique, has been applied to tumor detection. Rapid, accurate, and low-cost detection of blood cancer using hyperspectral technology remains a challenge. We developed a new method to discriminate blood cancer using hyperspectral imaging (HSI) and the forward searching method (FSM). Four commonly used classification models are applied for four types of blood cancer spectra recognition. The support vector machine (SVM) model with the highest recognition accuracy (94.5%) combined with HSI achieves high-precision tumor identification. For higher recognition accuracy and lower hardware barriers, based on the selection probabilities of spectral lines calculated by a multi-objective atomic orbital search method, the FSM is proposed for HSI feature selection. With the proposed method, the wavelength band range of the spectrum is reduced by at least 50%. Compared with the traditional dimensionality reduction methods, the FSM can obtain a higher accuracy rate with lower hardware requirements. These results show that our proposed method can achieve non-invasive rapid screening of blood cancers with lower hardware requirements. Therefore, HSI assisted with FSM and SVM hybrid models can be a powerful and promising tool for blood cancer detection.
Collapse
Affiliation(s)
- Riheng Chen
- Hunan Provincial Key Laboratory of Grids Operation and Control on Multi-Power Sources Area, Shaoyang University, Shaoyang, Hunan, 422000, China.
| | - Ting Luo
- Hunan Provincial Key Laboratory of Grids Operation and Control on Multi-Power Sources Area, Shaoyang University, Shaoyang, Hunan, 422000, China.
| | - Junfei Nie
- Hunan Provincial Key Laboratory of Grids Operation and Control on Multi-Power Sources Area, Shaoyang University, Shaoyang, Hunan, 422000, China.
| | - Yanwu Chu
- Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, Sichuan, 610209, China.
| |
Collapse
|
25
|
Qin X, Zhang M, Zhou C, Ran T, Pan Y, Deng Y, Xie X, Zhang Y, Gong T, Zhang B, Zhang L, Wang Y, Li Q, Wang D, Gao L, Zou D. A deep learning model using hyperspectral image for EUS-FNA cytology diagnosis in pancreatic ductal adenocarcinoma. Cancer Med 2023; 12:17005-17017. [PMID: 37455599 PMCID: PMC10501295 DOI: 10.1002/cam4.6335] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 06/12/2023] [Accepted: 07/03/2023] [Indexed: 07/18/2023] Open
Abstract
BACKGROUND AND AIMS Endoscopic ultrasonography-guided fine-needle aspiration/biopsy (EUS-FNA/B) is considered to be a first-line procedure for the pathological diagnosis of pancreatic cancer owing to its high accuracy and low complication rate. The number of new cases of pancreatic ductal adenocarcinoma (PDAC) is increasing, and its accurate pathological diagnosis poses a challenge for cytopathologists. Our aim was to develop a hyperspectral imaging (HSI)-based convolution neural network (CNN) algorithm to aid in the diagnosis of pancreatic EUS-FNA cytology specimens. METHODS HSI images were captured of pancreatic EUS-FNA cytological specimens from benign pancreatic tissues (n = 33) and PDAC (n = 39) prepared using a liquid-based cytology method. A CNN was established to test the diagnostic performance, and Attribution Guided Factorization Visualization (AGF-Visualization) was used to visualize the regions of important classification features identified by the model. RESULTS A total of 1913 HSI images were obtained. Our ResNet18-SimSiam model achieved an accuracy of 0.9204, sensitivity of 0.9310 and specificity of 0.9123 (area under the curve of 0.9625) when trained on HSI images for the differentiation of PDAC cytological specimens from benign pancreatic cells. AGF-Visualization confirmed that the diagnoses were based on the features of tumor cell nuclei. CONCLUSIONS An HSI-based model was developed to diagnose cytological PDAC specimens obtained using EUS-guided sampling. Under the supervision of experienced cytopathologists, we performed multi-staged consecutive in-depth learning of the model. Its superior diagnostic performance could be of value for cytologists when diagnosing PDAC.
Collapse
Affiliation(s)
- Xianzheng Qin
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Minmin Zhang
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Chunhua Zhou
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Taojing Ran
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Yundi Pan
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Yingjiao Deng
- Shanghai Key Laboratory of Multidimensional Information ProcessingEast China Normal UniversityShanghaiChina
| | - Xingran Xie
- Shanghai Key Laboratory of Multidimensional Information ProcessingEast China Normal UniversityShanghaiChina
| | - Yao Zhang
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Tingting Gong
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Benyan Zhang
- Department of PathologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Ling Zhang
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Yan Wang
- Shanghai Key Laboratory of Multidimensional Information ProcessingEast China Normal UniversityShanghaiChina
| | - Qingli Li
- Shanghai Key Laboratory of Multidimensional Information ProcessingEast China Normal UniversityShanghaiChina
| | - Dong Wang
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Lili Gao
- Department of PathologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| | - Duowu Zou
- Department of GastroenterologyRuijin Hospital, School of Medicine, Shanghai Jiao Tong UniversityShanghaiChina
| |
Collapse
|
26
|
Liao WC, Mukundan A, Sadiaza C, Tsao YM, Huang CW, Wang HC. Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging. BIOMEDICAL OPTICS EXPRESS 2023; 14:4383-4405. [PMID: 37799695 PMCID: PMC10549751 DOI: 10.1364/boe.492635] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 10/07/2023]
Abstract
One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.
Collapse
Affiliation(s)
- Wei-Chih Liao
- Department of Internal Medicine, National Taiwan University Hospital, National Taiwan University College of Medicine, Taipei, Taiwan
- Graduate Institute of Epidemiology and Preventive Medicine, National Taiwan University, Taipei, Taiwan
| | - Arvind Mukundan
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Cleorita Sadiaza
- Department of Mechanical Engineering, Far Eastern University, P. Paredes St., Sampaloc, Manila, 1015, Philippines
| | - Yu-Ming Tsao
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Chien-Wei Huang
- Department of Gastroenterology, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st.Rd., Lingya District, Kaohsiung City 80284, Taiwan
- Department of Nursing, Tajen University, 20, Weixin Rd., Yanpu Township, Pingtung County 90741, Taiwan
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
- Department of Medical Research, Dalin Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, No. 2, Minsheng Road, Dalin, Chiayi, 62247, Taiwan
- Director of Technology Development, Hitspectra Intelligent Technology Co., Ltd., 4F., No. 2, Fuxing 4th Rd., Qianzhen Dist., Kaohsiung City 80661, Taiwan
| |
Collapse
|
27
|
Puustinen S, Vrzáková H, Hyttinen J, Rauramaa T, Fält P, Hauta-Kasari M, Bednarik R, Koivisto T, Rantala S, von Und Zu Fraunberg M, Jääskeläinen JE, Elomaa AP. Hyperspectral Imaging in Brain Tumor Surgery-Evidence of Machine Learning-Based Performance. World Neurosurg 2023; 175:e614-e635. [PMID: 37030483 DOI: 10.1016/j.wneu.2023.03.149] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Accepted: 03/31/2023] [Indexed: 04/10/2023]
Abstract
BACKGROUND Hyperspectral imaging (HSI) has the potential to enhance surgical tissue detection and diagnostics. Definite utilization of intraoperative HSI guidance demands validated machine learning and public datasets that currently do not exist. Moreover, current imaging conventions are dispersed, and evidence-based paradigms for neurosurgical HSI have not been declared. METHODS We presented the rationale and a detailed clinical paradigm for establishing microneurosurgical HSI guidance. In addition, a systematic literature review was conducted to summarize the current indications and performance of neurosurgical HSI systems, with an emphasis on machine learning-based methods. RESULTS The published data comprised a few case series or case reports aiming to classify tissues during glioma operations. For a multitissue classification problem, the highest overall accuracy of 80% was obtained using deep learning. Our HSI system was capable of intraoperative data acquisition and visualization with minimal disturbance to glioma surgery. CONCLUSIONS In a limited number of publications, neurosurgical HSI has demonstrated unique capabilities in contrast to the established imaging techniques. Multidisciplinary work is required to establish communicable HSI standards and clinical impact. Our HSI paradigm endorses systematic intraoperative HSI data collection, which aims to facilitate the related standards, medical device regulations, and value-based medical imaging systems.
Collapse
Affiliation(s)
- Sami Puustinen
- University of Eastern Finland, Faculty of Health Sciences, School of Medicine, Kuopio, Finland; Kuopio University Hospital, Eastern Finland Microsurgery Center, Kuopio, Finland.
| | - Hana Vrzáková
- Kuopio University Hospital, Eastern Finland Microsurgery Center, Kuopio, Finland; University of Eastern Finland, Faculty of Science and Forestry, School of Computing, Joensuu, Finland
| | - Joni Hyttinen
- University of Eastern Finland, Faculty of Science and Forestry, School of Computing, Joensuu, Finland
| | - Tuomas Rauramaa
- Kuopio University Hospital, Department of Clinical Pathology, Kuopio, Finland
| | - Pauli Fält
- University of Eastern Finland, Faculty of Science and Forestry, School of Computing, Joensuu, Finland
| | - Markku Hauta-Kasari
- University of Eastern Finland, Faculty of Science and Forestry, School of Computing, Joensuu, Finland
| | - Roman Bednarik
- University of Eastern Finland, Faculty of Science and Forestry, School of Computing, Joensuu, Finland
| | - Timo Koivisto
- Kuopio University Hospital, Department of Neurosurgery, Kuopio, Finland
| | - Susanna Rantala
- Kuopio University Hospital, Department of Neurosurgery, Kuopio, Finland
| | - Mikael von Und Zu Fraunberg
- Oulu University Hospital, Department of Neurosurgery, Oulu, Finland; University of Oulu, Faculty of Medicine, Research Unit of Clinical Medicine, Oulu, Finland
| | | | - Antti-Pekka Elomaa
- University of Eastern Finland, Faculty of Health Sciences, School of Medicine, Kuopio, Finland; Kuopio University Hospital, Eastern Finland Microsurgery Center, Kuopio, Finland; Kuopio University Hospital, Department of Neurosurgery, Kuopio, Finland
| |
Collapse
|
28
|
Peterson T, Mann S, Sun BL, Peng L, Cai H, Liang R. Motionless volumetric structured light sheet microscopy. BIOMEDICAL OPTICS EXPRESS 2023; 14:2209-2224. [PMID: 37206125 PMCID: PMC10191636 DOI: 10.1364/boe.489280] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Revised: 04/11/2023] [Accepted: 04/11/2023] [Indexed: 05/21/2023]
Abstract
To meet the increasing need for low-cost, compact imaging technology with cellular resolution, we have developed a microLED-based structured light sheet microscope for three-dimensional ex vivo and in vivo imaging of biological tissue in multiple modalities. All the illumination structure is generated directly at the microLED panel-which serves as the source-so light sheet scanning and modulation is completely digital, yielding a system that is simpler and less prone to error than previously reported methods. Volumetric images with optical sectioning are thus achieved in an inexpensive, compact form factor without any moving parts. We demonstrate the unique properties and general applicability of our technique by ex vivo imaging of porcine and murine tissue from the gastrointestinal tract, kidney, and brain.
Collapse
Affiliation(s)
- Tyler Peterson
- Wyant College of Optical Sciences,
The University of Arizona, Tucson, Arizona 85721, USA
| | - Shivani Mann
- Department of Neuroscience, The University of Arizona, Tucson, Arizona 85721, USA
| | - Belinda L. Sun
- Department of Pathology, College of Medicine, The University of Arizona, Tucson, Arizona 85721, USA
| | - Leilei Peng
- Wyant College of Optical Sciences,
The University of Arizona, Tucson, Arizona 85721, USA
| | - Haijiang Cai
- Department of Neuroscience, The University of Arizona, Tucson, Arizona 85721, USA
| | - Rongguang Liang
- Wyant College of Optical Sciences,
The University of Arizona, Tucson, Arizona 85721, USA
| |
Collapse
|
29
|
Mengu D, Tabassum A, Jarrahi M, Ozcan A. Snapshot multispectral imaging using a diffractive optical network. LIGHT, SCIENCE & APPLICATIONS 2023; 12:86. [PMID: 37024463 PMCID: PMC10079962 DOI: 10.1038/s41377-023-01135-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Revised: 03/14/2023] [Accepted: 03/17/2023] [Indexed: 06/19/2023]
Abstract
Multispectral imaging has been used for numerous applications in e.g., environmental monitoring, aerospace, defense, and biomedicine. Here, we present a diffractive optical network-based multispectral imaging system trained using deep learning to create a virtual spectral filter array at the output image field-of-view. This diffractive multispectral imager performs spatially-coherent imaging over a large spectrum, and at the same time, routes a pre-determined set of spectral channels onto an array of pixels at the output plane, converting a monochrome focal-plane array or image sensor into a multispectral imaging device without any spectral filters or image recovery algorithms. Furthermore, the spectral responsivity of this diffractive multispectral imager is not sensitive to input polarization states. Through numerical simulations, we present different diffractive network designs that achieve snapshot multispectral imaging with 4, 9 and 16 unique spectral bands within the visible spectrum, based on passive spatially-structured diffractive surfaces, with a compact design that axially spans ~72λm, where λm is the mean wavelength of the spectral band of interest. Moreover, we experimentally demonstrate a diffractive multispectral imager based on a 3D-printed diffractive network that creates at its output image plane a spatially repeating virtual spectral filter array with 2 × 2 = 4 unique bands at terahertz spectrum. Due to their compact form factor and computation-free, power-efficient and polarization-insensitive forward operation, diffractive multispectral imagers can be transformative for various imaging and sensing applications and be used at different parts of the electromagnetic spectrum where high-density and wide-area multispectral pixel arrays are not widely available.
Collapse
Affiliation(s)
- Deniz Mengu
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute, University of California, Los Angeles, CA, 90095, USA
| | - Anika Tabassum
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute, University of California, Los Angeles, CA, 90095, USA
| | - Mona Jarrahi
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute, University of California, Los Angeles, CA, 90095, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA.
- California NanoSystems Institute, University of California, Los Angeles, CA, 90095, USA.
| |
Collapse
|
30
|
Liang F, Zhu J, Chai H, Feng Y, Zhao P, Liu S, Yang Y, Lin L, Cao L, Wang W. Non-Invasive and Minute-Frequency 3D Tomographic Imaging Enabling Long-Term Spatiotemporal Observation of Single Cell Fate. SMALL METHODS 2023:e2201492. [PMID: 36950762 DOI: 10.1002/smtd.202201492] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 02/14/2023] [Indexed: 06/18/2023]
Abstract
Non-invasive and rapid imaging technique at subcellular resolution is significantly important for multiple biological applications such as cell fate study. Label-free refractive-index (RI)-based 3D tomographic imaging constitutes an excellent candidate for 3D imaging of cellular structures, but its full potential in long-term spatiotemporal cell fate observation is locked due to the lack of an efficient integrated system. Here, a long-term 3D RI imaging system incorporating a cutting-edge white light diffraction phase microscopy module with spatiotemporal stability, and an acoustofluidic device to roll and culture single cells in a customized live cell culture chamber is reported. Using this system, 3D RI imaging experiments are conducted for 250 cells and demonstrate efficient cell identification with high accuracy. Importantly, long-term and frequency-on-demand 3D RI imaging of K562 and MCF-7 cancer cells reveal different characteristics during normal cell growth, drug-induced cell apoptosis, and necrosis of drug-treated cells. Overall, it is believed that the proposed 3D tomographic imaging technique opens up a new avenue for visualizing intracellular structures and will find many applications such as disease diagnosis and nanomedicine.
Collapse
Affiliation(s)
- Fei Liang
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Junwen Zhu
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Huichao Chai
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Yongxiang Feng
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Peng Zhao
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Shaofeng Liu
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Yuanmu Yang
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Linhan Lin
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Liangcai Cao
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Wenhui Wang
- State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| |
Collapse
|
31
|
In Vitro Antibody Quantification with Hyperspectral Imaging in a Large Field of View for Clinical Applications. Bioengineering (Basel) 2023; 10:bioengineering10030370. [PMID: 36978761 PMCID: PMC10045535 DOI: 10.3390/bioengineering10030370] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Accepted: 03/13/2023] [Indexed: 03/19/2023] Open
Abstract
Hyperspectral imaging (HSI) is a non-invasive, contrast-free optical-based tool that has recently been applied in medical and basic research fields. The opportunity to use HSI to identify exogenous tumor markers in a large field of view (LFOV) could increase precision in oncological diagnosis and surgical treatment. In this study, the anti-high mobility group B1 (HMGB1) labeled with Alexa fluorophore (647 nm) was used as the target molecule. This is the proof-of-concept of HSI’s ability to quantify antibodies via an in vitro setting. A first test was performed to understand whether the relative absorbance provided by the HSI camera was dependent on volume at a 1:1 concentration. A serial dilution of 1:1, 10, 100, 1000, and 10,000 with phosphatase-buffered saline (PBS) was then used to test the sensitivity of the camera at the minimum and maximum volumes. For the analysis, images at 640 nm were extracted from the hypercubes according to peak signals matching the specificities of the antibody manufacturer. The results showed a positive correlation between relative absorbance and volume (r = 0.9709, p = 0.0013). The correlation between concentration and relative absorbance at min (1 µL) and max (20 µL) volume showed r = 0.9925, p < 0.0001, and r = 0.9992, p < 0.0001, respectively. These results demonstrate the HSI potential in quantifying HMGB1, hence deserving further studies in ex vivo and in vivo settings.
Collapse
|
32
|
Sarin JK, Kiema M, Luoto ES, Husso A, Hedman M, Laakkonen JP, Torniainen J. Nondestructive Evaluation of Mechanical and Histological Properties of the Human Aorta With Near-Infrared Spectroscopy. J Surg Res 2023; 287:82-89. [PMID: 36870305 DOI: 10.1016/j.jss.2023.01.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Revised: 12/19/2022] [Accepted: 01/28/2023] [Indexed: 03/06/2023]
Abstract
INTRODUCTION Ascending aortic dilatation is a well-known risk factor for aortic rupture. Indications for aortic replacement in its dilatation concomitant to other open-heart surgery exist; however, cut-off values based solely on aortic diameter may fail to identify patients with weakened aortic tissue. We introduce near-infrared spectroscopy (NIRS) as a diagnostic tool to nondestructively evaluate the structural and compositional properties of the human ascending aorta during open-heart surgeries. During open-heart surgery, NIRS could provide information regarding tissue viability in situ and thus contribute to the decision of optimal surgical repair. MATERIALS AND METHODS Samples were collected from patients with ascending aortic aneurysm (n = 23) undergoing elective aortic reconstruction surgery and from healthy subjects (n = 4). The samples were subjected to spectroscopic measurements, biomechanical testing, and histological analysis. The relationship between the near-infrared spectra and biomechanical and histological properties was investigated by adapting partial least squares regression. RESULTS Moderate prediction performance was achieved with biomechanical properties (r = 0.681, normalized root-mean-square error of cross-validation = 17.9%) and histological properties (r = 0.602, normalized root-mean-square error of cross-validation = 22.2%). Especially the performance with parameters describing the aorta's ultimate strength, for example, failure strain (r = 0.658), and elasticity (phase difference, r = 0.875) were promising and could, therefore, provide quantitative information on the rupture sensitivity of the aorta. For the estimation of histological properties, the results with α-smooth muscle actin (r = 0.581), elastin density (r = 0.973), mucoid extracellular matrix accumulation(r = 0.708), and media thickness (r = 0.866) were promising. CONCLUSIONS NIRS could be a potential technique for in situ evaluation of biomechanical and histological properties of human aorta and therefore useful in patient-specific treatment planning.
Collapse
Affiliation(s)
- Jaakko K Sarin
- Department of Medical Physics, Medical Imaging Center, Tampere University Hospital, Tampere, Finland; Department of Radiology, Tampere University Hospital, Tampere, Finland; Department of Technical Physics, University of Eastern Finland, Kuopio, Finland.
| | - Miika Kiema
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, Kuopio, Finland
| | - Emma-Sofia Luoto
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, Kuopio, Finland
| | - Annastiina Husso
- Department of Cardiothoracic Surgery, Kuopio University Hospital, Kuopio, Finland
| | - Marja Hedman
- Department of Cardiothoracic Surgery, Kuopio University Hospital, Kuopio, Finland; Department of Clinical Radiology, Kuopio University Hospital, Kuopio, Finland; Institute of Clinical Medicine, University of Eastern Finland, Kuopio, Finland
| | - Johanna P Laakkonen
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, Kuopio, Finland
| | - Jari Torniainen
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland; School of Information Technology & Electrical Engineering, The University of Queensland, Brisbane, Australia
| |
Collapse
|
33
|
Catelli E, Li Z, Sciutto G, Oliveri P, Prati S, Occhipinti M, Tocchio A, Alberti R, Frizzi T, Malegori C, Mazzeo R. Towards the non-destructive analysis of multilayered samples: A novel XRF-VNIR-SWIR hyperspectral imaging system combined with multiblock data processing. Anal Chim Acta 2023; 1239:340710. [PMID: 36628716 DOI: 10.1016/j.aca.2022.340710] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Revised: 12/05/2022] [Accepted: 12/06/2022] [Indexed: 12/14/2022]
Abstract
The new challenge in the investigation of cultural heritage is the possibility to obtain stratigraphical information about the distribution of the different organic and inorganic components without sampling. In this paper recently commercialized analytical set-up, which is able to co-register VNIR, SWIR, and XRF spectral data simultaneously, is exploited in combination with an innovative multivariate and multiblock high-throughput data processing for the analysis of multilayered paintings. The instrument allows to obtain elemental and molecular information from superficial to subsurface layers across the investigated area. The chemometric strategy proved to be highly efficient in data reduction and for the extraction and integration of the most useful information coming from the three different spectroscopies, also filling the gap between data acquisition and data understanding through the combination of principal component analysis (PCA), brushing, correlation diagrams and maps (within and between spectral blocks) on the low-level fused. In particular, correlation diagrams and maps provide useful information for the reconstruction of a stratigraphic structure without the need to take any sample, thanks to the effective account for inter-correlation among data (variables), which is able to effectively characterize the possible combinations of components located in the same depth level. The highly innovative technology and the data processing strategy are applied for the multi-level characterization of a complex painting reproduction as an illustrative pilot study.
Collapse
Affiliation(s)
- Emilio Catelli
- Department of Chemistry, University of Bologna-Ravenna Campus, via Guaccimanni, 42, 48121, Ravenna, Italy
| | - Zelan Li
- Department of Chemistry, University of Bologna-Ravenna Campus, via Guaccimanni, 42, 48121, Ravenna, Italy
| | - Giorgia Sciutto
- Department of Chemistry, University of Bologna-Ravenna Campus, via Guaccimanni, 42, 48121, Ravenna, Italy.
| | - Paolo Oliveri
- Department of Pharmacy, University of Genoa, via Cembrano, 4, 16148, Genoa, Italy.
| | - Silvia Prati
- Department of Chemistry, University of Bologna-Ravenna Campus, via Guaccimanni, 42, 48121, Ravenna, Italy
| | - Michele Occhipinti
- XGLab SRL - Bruker Nano Analytics, Via Conte Rosso 23, I-20134 Milano, Italy
| | - Alessandro Tocchio
- XGLab SRL - Bruker Nano Analytics, Via Conte Rosso 23, I-20134 Milano, Italy
| | - Roberto Alberti
- XGLab SRL - Bruker Nano Analytics, Via Conte Rosso 23, I-20134 Milano, Italy
| | - Tommaso Frizzi
- XGLab SRL - Bruker Nano Analytics, Via Conte Rosso 23, I-20134 Milano, Italy
| | - Cristina Malegori
- Department of Pharmacy, University of Genoa, via Cembrano, 4, 16148, Genoa, Italy
| | - Rocco Mazzeo
- Department of Chemistry, University of Bologna-Ravenna Campus, via Guaccimanni, 42, 48121, Ravenna, Italy
| |
Collapse
|
34
|
In vivo evaluation of a hyperspectral imaging system for minimally invasive surgery (HSI-MIS). Surg Endosc 2023; 37:3691-3700. [PMID: 36645484 PMCID: PMC10156625 DOI: 10.1007/s00464-023-09874-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2022] [Accepted: 01/06/2023] [Indexed: 01/17/2023]
Abstract
BACKGROUND Hyperspectral Imaging (HSI) is a reliable and safe imaging method for taking intraoperative perfusion measurements. This is the first study translating intraoperative HSI to an in vivo laparoscopic setting using a CE-certified HSI-system for minimally invasive surgery (HSI-MIS). We aim to compare it to an established HSI-system for open surgery (HSI-Open). METHODS Intraoperative HSI was done using the HSI-MIS and HSI-Open at the Region of Interest (ROI). 19 patients undergoing gastrointestinal resections were analyzed in this study. The HSI-MIS-acquired images were aligned with those from the HSI-Open, and spectra and parameter images were compared pixel-wise. We calculated the Mean Absolute Error (MAE) for Tissue Oxygen Saturation (StO2), Near-Infrared Perfusion Index (NIR-PI), Tissue Water Index (TWI), and Organ Hemoglobin Index (OHI), as well as the Root Mean Squared Error (RMSE) over the whole spectrum. Our analysis of parameters was optimized using partial least squares (PLS) regression. Two experienced surgeons carried out an additional color-change analysis, comparing the ROI images and deciding whether they provided the same (acceptable) or different visual information (rejected). RESULTS HSI and subsequent image registration was possible in 19 patients. MAE results for the original calculation were StO2 orig. 17.2% (± 7.7%), NIR-PIorig. 16.0 (± 9.5), TWIorig. 18.1 (± 7.9), OHIorig. 14.4 (± 4.5). For the PLS calculation, they were StO2 PLS 12.6% (± 5.2%), NIR-PIPLS 10.3 (± 6.0), TWIPLS 10.6 (± 5.1), and OHIPLS 11.6 (± 3.0). The RMSE between both systems was 0.14 (± 0.06). In the color-change analysis; both surgeons accepted more images generated using the PLS method. CONCLUSION Intraoperative HSI-MIS is a new technology and holds great potential for future applications in surgery. Parameter deviations are attributable to technical differences and can be reduced by applying improved calculation methods. This study is an important step toward the clinical implementation of HSI for minimally invasive surgery.
Collapse
|
35
|
Ma L, Srinivas A, Krishnamurthy A, Zhou X, Shah NS, Obaid G, Fei B. Automated Polarized Hyperspectral Imaging (PHSI) for ex-vivo and in-vivo Tissue Assessment. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12391:123910F. [PMID: 38476292 PMCID: PMC10932616 DOI: 10.1117/12.2651011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/14/2024]
Abstract
Polarized light interactions with biological tissues can reveal information regarding tissue structure, while spectral characteristics are closely related to tissue composition. An integration of both modalities in a compact system could better assist tissue assessment. This study aims to develop a polarized hyperspectral imaging (PHSI) system that fulfills both linearly and circularly polarized hyperspectral imaging for in vivo and ex vivo applications. The system is comprised of a white LED, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a hyperspectral snapshot camera. The system was calibrated to compute the full Stokes polarimetry. For tissue differentiation, fresh ex vivo mouse tissue specimens from kidney, liver, spleen, muscle, lung, and salivary gland of mice were imaged. The spectra of three features, named degree of polarization (DOP), degree of linear polarization (DOLP), and degree of circular polarization (DOCP), were generated. A k-nearest neighbor (k-NN) classifier was trained with multi-class spectra and 5-fold cross validation. It was found that DOP better differentiates tissue with an average accuracy of 0.87. Additionally, support vector machine (SVM) classifiers were trained to differentiate between each two of the organs, and it was determined that DOLP better identified kidney, liver, and spleen, whereas DOCP better identified muscle and lung tissues. Then, the setup was employed to image in vivo human fingers with and without a blood occlusion to qualitatively estimate oxygen saturation. Preliminary results demonstrate that both DOLP and DOCP reveal a distinction of oxygen saturation states. These results demonstrate the feasibility of the PHSI system for distinguishing between optical properties of tissues, which has the potential to reveal disease-related information for diverse medical applications.
Collapse
Affiliation(s)
- Ling Ma
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Akhila Srinivas
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Abirami Krishnamurthy
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Ximing Zhou
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | | | - Girgis Obaid
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
36
|
Tomanic T, Rogelj L, Stergar J, Markelc B, Bozic T, Brezar SK, Sersa G, Milanic M. Estimating quantitative physiological and morphological tissue parameters of murine tumor models using hyperspectral imaging and optical profilometry. JOURNAL OF BIOPHOTONICS 2023; 16:e202200181. [PMID: 36054067 DOI: 10.1002/jbio.202200181] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/18/2022] [Revised: 07/27/2022] [Accepted: 08/05/2022] [Indexed: 06/15/2023]
Abstract
Understanding tumors and their microenvironment are essential for successful and accurate disease diagnosis. Tissue physiology and morphology are altered in tumors compared to healthy tissues, and there is a need to monitor tumors and their surrounding tissues, including blood vessels, non-invasively. This preliminary study utilizes a multimodal optical imaging system combining hyperspectral imaging (HSI) and three-dimensional (3D) optical profilometry (OP) to capture hyperspectral images and surface shapes of subcutaneously grown murine tumor models. Hyperspectral images are corrected with 3D OP data and analyzed using the inverse-adding doubling (IAD) method to extract tissue properties such as melanin volume fraction and oxygenation. Blood vessels are segmented using the B-COSFIRE algorithm from oxygenation maps. From 3D OP data, tumor volumes are calculated and compared to manual measurements using a vernier caliper. Results show that tumors can be distinguished from healthy tissue based on most extracted tissue parameters ( p < 0.05 ). Furthermore, blood oxygenation is 50% higher within the blood vessels than in the surrounding tissue, and tumor volumes calculated using 3D OP agree within 26% with manual measurements using a vernier caliper. Results suggest that combining HSI and OP could provide relevant quantitative information about tumors and improve the disease diagnosis.
Collapse
Affiliation(s)
- Tadej Tomanic
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
| | - Luka Rogelj
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
| | - Jost Stergar
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
- Jozef Stefan Institute, Ljubljana, Slovenia
| | - Bostjan Markelc
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Health Sciences, University of Ljubljana, Ljubljana, Slovenia
| | - Tim Bozic
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia
| | - Simona Kranjc Brezar
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia
| | - Gregor Sersa
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Health Sciences, University of Ljubljana, Ljubljana, Slovenia
| | - Matija Milanic
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
- Jozef Stefan Institute, Ljubljana, Slovenia
| |
Collapse
|
37
|
Liu GS, Shenson JA, Farrell JE, Blevins NH. Signal to noise ratio quantifies the contribution of spectral channels to classification of human head and neck tissues ex vivo using deep learning and multispectral imaging. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:016004. [PMID: 36726664 PMCID: PMC9884103 DOI: 10.1117/1.jbo.28.1.016004] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Accepted: 01/06/2023] [Indexed: 05/09/2023]
Abstract
SIGNIFICANCE Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems. AIM Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI. APPROACH MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods. RESULTS Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( p < 0.001 ). Removing spectral channels with SNR > 20 reduced tissue classification accuracy. CONCLUSIONS The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.
Collapse
Affiliation(s)
- George S. Liu
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
| | - Jared A. Shenson
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
| | - Joyce E. Farrell
- Stanford University, Department of Electrical Engineering, Stanford, California, United States
| | - Nikolas H. Blevins
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
- Address all correspondence to Nikolas H. Blevins,
| |
Collapse
|
38
|
Martinez-Vega B, Tkachenko M, Matkabi M, Ortega S, Fabelo H, Balea-Fernandez F, La Salvia M, Torti E, Leporati F, Callico GM, Chalopin C. Evaluation of Preprocessing Methods on Independent Medical Hyperspectral Databases to Improve Analysis. SENSORS (BASEL, SWITZERLAND) 2022; 22:8917. [PMID: 36433516 PMCID: PMC9693077 DOI: 10.3390/s22228917] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 11/10/2022] [Accepted: 11/15/2022] [Indexed: 06/16/2023]
Abstract
Currently, one of the most common causes of death worldwide is cancer. The development of innovative methods to support the early and accurate detection of cancers is required to increase the recovery rate of patients. Several studies have shown that medical Hyperspectral Imaging (HSI) combined with artificial intelligence algorithms is a powerful tool for cancer detection. Various preprocessing methods are commonly applied to hyperspectral data to improve the performance of the algorithms. However, there is currently no standard for these methods, and no studies have compared them so far in the medical field. In this work, we evaluated different combinations of preprocessing steps, including spatial and spectral smoothing, Min-Max scaling, Standard Normal Variate normalization, and a median spatial smoothing technique, with the goal of improving tumor detection in three different HSI databases concerning colorectal, esophagogastric, and brain cancers. Two machine learning and deep learning models were used to perform the pixel-wise classification. The results showed that the choice of preprocessing method affects the performance of tumor identification. The method that showed slightly better results with respect to identifing colorectal tumors was Median Filter preprocessing (0.94 of area under the curve). On the other hand, esophagogastric and brain tumors were more accurately identified using Min-Max scaling preprocessing (0.93 and 0.92 of area under the curve, respectively). However, it is observed that the Median Filter method smooths sharp spectral features, resulting in high variability in the classification performance. Therefore, based on these results, obtained with different databases acquired by different HSI instrumentation, the most relevant preprocessing technique identified in this work is Min-Max scaling.
Collapse
Affiliation(s)
- Beatriz Martinez-Vega
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
| | - Mariia Tkachenko
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany
- Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI), University of Leipzig, 04105 Leipzig, Germany
| | - Marianne Matkabi
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany
- Department of Electrical Engineering, Mechanical Engineering and Industrial Engineering, Anhalt University of Applied Science Anhalt, 06366 Köthen, Germany
| | - Samuel Ortega
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
- Nofima, Norwegian Institute of Food Fisheries and Aquaculture Research, NO-9291 Tromsø, Norway
| | - Himar Fabelo
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
- Fundacion Canaria Instituto de Investigación Sanitaria de Canarias (FIISC), 35019 Las Palmas de Gran Canaria, Spain
| | - Francisco Balea-Fernandez
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
- Department of Psychology, Sociology and Social Work, University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
| | - Marco La Salvia
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, I-27100 Pavia, Italy
| | - Emanuele Torti
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, I-27100 Pavia, Italy
| | - Francesco Leporati
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, I-27100 Pavia, Italy
| | - Gustavo M. Callico
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain
| | - Claire Chalopin
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany
| |
Collapse
|
39
|
Huang SY, Mukundan A, Tsao YM, Kim Y, Lin FC, Wang HC. Recent Advances in Counterfeit Art, Document, Photo, Hologram, and Currency Detection Using Hyperspectral Imaging. SENSORS (BASEL, SWITZERLAND) 2022; 22:7308. [PMID: 36236407 PMCID: PMC9571956 DOI: 10.3390/s22197308] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 09/15/2022] [Accepted: 09/23/2022] [Indexed: 05/08/2023]
Abstract
Forgery and tampering continue to provide unnecessary economic burdens. Although new anti-forgery and counterfeiting technologies arise, they inadvertently lead to the sophistication of forgery techniques over time, to a point where detection is no longer viable without technological aid. Among the various optical techniques, one of the recently used techniques to detect counterfeit products is HSI, which captures a range of electromagnetic data. To aid in the further exploration and eventual application of the technique, this study categorizes and summarizes existing related studies on hyperspectral imaging and creates a mini meta-analysis of this stream of literature. The literature review has been classified based on the product HSI has used in counterfeit documents, photos, holograms, artwork, and currency detection.
Collapse
Affiliation(s)
- Shuan-Yu Huang
- Department of Optometry, Central Taiwan University of Science and Technology, No. 666, Buzih Road, Beitun District, Taichung City 406053, Taiwan
| | - Arvind Mukundan
- Department of Mechanical Engineering, Advanced Institute of Manufacturing with High Tech Innovations (AIM-HI), Center for Innovative Research on Aging Society (CIRAS), National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Yu-Ming Tsao
- Department of Mechanical Engineering, Advanced Institute of Manufacturing with High Tech Innovations (AIM-HI), Center for Innovative Research on Aging Society (CIRAS), National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Youngjo Kim
- Department of Mechanical Engineering, Far Eastern University, P. Paredes St., Sampaloc, Manila 1015, Philippines
| | - Fen-Chi Lin
- Department of Ophthalmology, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st Rd., Lingya District, Kaohsiung City 80284, Taiwan
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, Advanced Institute of Manufacturing with High Tech Innovations (AIM-HI), Center for Innovative Research on Aging Society (CIRAS), National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| |
Collapse
|
40
|
Alafeef M, Pan D. Diagnostic Approaches For COVID-19: Lessons Learned and the Path Forward. ACS NANO 2022; 16:11545-11576. [PMID: 35921264 PMCID: PMC9364978 DOI: 10.1021/acsnano.2c01697] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Accepted: 07/12/2022] [Indexed: 05/17/2023]
Abstract
Coronavirus disease 2019 (COVID-19) is a transmitted respiratory disease caused by the infection of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Although humankind has experienced several outbreaks of infectious diseases, the COVID-19 pandemic has the highest rate of infection and has had high levels of social and economic repercussions. The current COVID-19 pandemic has highlighted the limitations of existing virological tests, which have failed to be adopted at a rate to properly slow the rapid spread of SARS-CoV-2. Pandemic preparedness has developed as a focus of many governments around the world in the event of a future outbreak. Despite the largely widespread availability of vaccines, the importance of testing has not diminished to monitor the evolution of the virus and the resulting stages of the pandemic. Therefore, developing diagnostic technology that serves as a line of defense has become imperative. In particular, that test should satisfy three criteria to be widely adopted: simplicity, economic feasibility, and accessibility. At the heart of it all, it must enable early diagnosis in the course of infection to reduce spread. However, diagnostic manufacturers need guidance on the optimal characteristics of a virological test to ensure pandemic preparedness and to aid in the effective treatment of viral infections. Nanomaterials are a decisive element in developing COVID-19 diagnostic kits as well as a key contributor to enhance the performance of existing tests. Our objective is to develop a profile of the criteria that should be available in a platform as the target product. In this work, virus detection tests were evaluated from the perspective of the COVID-19 pandemic, and then we generalized the requirements to develop a target product profile for a platform for virus detection.
Collapse
Affiliation(s)
- Maha Alafeef
- Department of Chemical, Biochemical and Environmental
Engineering, University of Maryland Baltimore County, Interdisciplinary
Health Sciences Facility, 1000 Hilltop Circle, Baltimore, Maryland 21250,
United States
- Departments of Diagnostic Radiology and Nuclear
Medicine and Pediatrics, Center for Blood Oxygen Transport and Hemostasis,
University of Maryland Baltimore School of Medicine, Health Sciences
Research Facility III, 670 W Baltimore Street, Baltimore, Maryland 21201,
United States
- Department of Bioengineering, the
University of Illinois at Urbana−Champaign, Urbana, Illinois 61801,
United States
- Biomedical Engineering Department, Jordan
University of Science and Technology, Irbid 22110,
Jordan
| | - Dipanjan Pan
- Department of Chemical, Biochemical and Environmental
Engineering, University of Maryland Baltimore County, Interdisciplinary
Health Sciences Facility, 1000 Hilltop Circle, Baltimore, Maryland 21250,
United States
- Departments of Diagnostic Radiology and Nuclear
Medicine and Pediatrics, Center for Blood Oxygen Transport and Hemostasis,
University of Maryland Baltimore School of Medicine, Health Sciences
Research Facility III, 670 W Baltimore Street, Baltimore, Maryland 21201,
United States
- Department of Bioengineering, the
University of Illinois at Urbana−Champaign, Urbana, Illinois 61801,
United States
| |
Collapse
|
41
|
Sun J, Wu Z, Wang L, Yao Q, Li M, Yao G. Adaptive denoising hyperspectral data for visualization enhancement of intraoperative tissue. JOURNAL OF BIOPHOTONICS 2022; 15:e202200083. [PMID: 35460593 DOI: 10.1002/jbio.202200083] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/26/2022] [Revised: 04/21/2022] [Accepted: 04/22/2022] [Indexed: 06/14/2023]
Abstract
The vast amount of reflectance information obtained from the hyperspectral imaging devices offers great opportunities for investigating the function and structure of human tissue. However, the captured hyperspectral data often contain various noises due to the intrinsic imperfection of associated electrical and optical imaging components. This work proposed an automatic total variation algorithm to suppress the noises while preserving the details of the spectral and spatial information. The variation of spectral images at neighboring bands was calculated for regulating the total variation of hyperspectral data so that the spectral-dependent noises can be treated differentially across all bands. Experimental results demonstrated that the proposed method could effectively remove the spectral noises, especially near the ends of those extreme bands. The noise suppressed hyperspectral data could then be used for the visualization enhancement on pathophysiological conditions of intraoperative observed anatomies such as the vessels of brain tissues.
Collapse
Affiliation(s)
- Jiuai Sun
- School of Medical Imaging, Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Zhonghang Wu
- School of Medical Imaging, Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Le Wang
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Qi Yao
- Academy for Engineering and Technology, Fudan University, Shanghai, China
- Research and Development Department, Zhongshan Fudan Joint Innovation Center, Guangdong, China
| | - Min Li
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Guangyu Yao
- Department of Thoracic Surgery, Zhongshan Hospital, Shanghai, China
| |
Collapse
|
42
|
Xu J, Meng Y, Qiu K, Topatana W, Li S, Wei C, Chen T, Chen M, Ding Z, Niu G. Applications of Artificial Intelligence Based on Medical Imaging in Glioma: Current State and Future Challenges. Front Oncol 2022; 12:892056. [PMID: 35965542 PMCID: PMC9363668 DOI: 10.3389/fonc.2022.892056] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 06/22/2022] [Indexed: 12/24/2022] Open
Abstract
Glioma is one of the most fatal primary brain tumors, and it is well-known for its difficulty in diagnosis and management. Medical imaging techniques such as magnetic resonance imaging (MRI), positron emission tomography (PET), and spectral imaging can efficiently aid physicians in diagnosing, treating, and evaluating patients with gliomas. With the increasing clinical records and digital images, the application of artificial intelligence (AI) based on medical imaging has reduced the burden on physicians treating gliomas even further. This review will classify AI technologies and procedures used in medical imaging analysis. Additionally, we will discuss the applications of AI in glioma, including tumor segmentation and classification, prediction of genetic markers, and prediction of treatment response and prognosis, using MRI, PET, and spectral imaging. Despite the benefits of AI in clinical applications, several issues such as data management, incomprehension, safety, clinical efficacy evaluation, and ethical or legal considerations, remain to be solved. In the future, doctors and researchers should collaborate to solve these issues, with a particular emphasis on interdisciplinary teamwork.
Collapse
Affiliation(s)
- Jiaona Xu
- Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Yuting Meng
- Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Kefan Qiu
- Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Win Topatana
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Shijie Li
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Chao Wei
- Department of Neurology, Affiliated Ningbo First Hospital, Ningbo, China
| | - Tianwen Chen
- Department of Neurology, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Mingyu Chen
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
- *Correspondence: Mingyu Chen, ; Zhongxiang Ding, ; Guozhong Niu,
| | - Zhongxiang Ding
- Department of Radiology, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China
- *Correspondence: Mingyu Chen, ; Zhongxiang Ding, ; Guozhong Niu,
| | - Guozhong Niu
- Department of Neurology, Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China
- *Correspondence: Mingyu Chen, ; Zhongxiang Ding, ; Guozhong Niu,
| |
Collapse
|
43
|
Zhang J, Su R, Fu Q, Ren W, Heide F, Nie Y. A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging. Sci Rep 2022; 12:11905. [PMID: 35831474 PMCID: PMC9279412 DOI: 10.1038/s41598-022-16223-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Accepted: 07/06/2022] [Indexed: 11/30/2022] Open
Abstract
Hyperspectral imaging enables many versatile applications for its competence in capturing abundant spatial and spectral information, which is crucial for identifying substances. However, the devices for acquiring hyperspectral images are typically expensive and very complicated, hindering the promotion of their application in consumer electronics, such as daily food inspection and point-of-care medical screening, etc. Recently, many computational spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from widely available RGB images. These reconstruction methods can exclude the usage of burdensome spectral camera hardware while keeping a high spectral resolution and imaging performance. We present a thorough investigation of more than 25 state-of-the-art spectral reconstruction methods which are categorized as prior-based and data-driven methods. Simulations on open-source datasets show that prior-based methods are more suitable for rare data situations, while data-driven methods can unleash the full potential of deep learning in big data cases. We have identified current challenges faced by those methods (e.g., loss function, spectral accuracy, data generalization) and summarized a few trends for future work. With the rapid expansion in datasets and the advent of more advanced neural networks, learnable methods with fine feature representation abilities are very promising. This comprehensive review can serve as a fruitful reference source for peer researchers, thus paving the way for the development of computational hyperspectral imaging.
Collapse
Affiliation(s)
- Jingang Zhang
- School of Future Technology, University of Chinese Academy of Sciences, Beijing, 100039, China
| | - Runmu Su
- School of Future Technology, University of Chinese Academy of Sciences, Beijing, 100039, China
- School of Computer Science and Technology, Xidian University, Xi'an, 710071, China
| | - Qiang Fu
- King Abdullah University of Science and Technology, Thuwal, 23955-6900, Saudi Arabia
| | - Wenqi Ren
- State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100093, China
| | - Felix Heide
- Computational Imaging Lab, Princeton University, Princeton, NJ, 08544, USA
| | - Yunfeng Nie
- Department of Applied Physics and Photonics, Vrije Universiteit Brussel, 1050, Brussels, Belgium.
| |
Collapse
|
44
|
Chalopin C, Nickel F, Pfahl A, Köhler H, Maktabi M, Thieme R, Sucher R, Jansen-Winkeln B, Studier-Fischer A, Seidlitz S, Maier-Hein L, Neumuth T, Melzer A, Müller-Stich BP, Gockel I. [Artificial intelligence and hyperspectral imaging for image-guided assistance in minimally invasive surgery]. CHIRURGIE (HEIDELBERG, GERMANY) 2022; 93:940-947. [PMID: 35798904 DOI: 10.1007/s00104-022-01677-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 06/08/2022] [Indexed: 01/18/2023]
Abstract
BACKGROUND Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article. OBJECTIVE What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery? MATERIAL AND METHODS Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors. RESULTS In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91. DISCUSSION This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland.
| | - Felix Nickel
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - René Thieme
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Robert Sucher
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Boris Jansen-Winkeln
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
- Abteilung für Allgemein‑, Viszeral- und Onkologische Chirurgie, Klinikum St. Georg Leipzig, Leipzig, Deutschland
| | - Alexander Studier-Fischer
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Beat Peter Müller-Stich
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Ines Gockel
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| |
Collapse
|
45
|
Browning CM, Mayes S, Mayes SA, Rich TC, Leavesley SJ. Microscopy is better in color: development of a streamlined spectral light path for real-time multiplex fluorescence microscopy. BIOMEDICAL OPTICS EXPRESS 2022; 13:3751-3772. [PMID: 35991911 PMCID: PMC9352297 DOI: 10.1364/boe.453657] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Revised: 05/24/2022] [Accepted: 05/26/2022] [Indexed: 06/15/2023]
Abstract
Spectroscopic image data has provided molecular discrimination for numerous fields including: remote sensing, food safety and biomedical imaging. Despite the various technologies for acquiring spectral data, there remains a trade-off when acquiring data. Typically, spectral imaging either requires long acquisition times to collect an image stack with high spectral specificity or acquisition times are shortened at the expense of fewer spectral bands or reduced spatial sampling. Hence, new spectral imaging microscope platforms are needed to help mitigate these limitations. Fluorescence excitation-scanning spectral imaging is one such new technology, which allows more of the emitted signal to be detected than comparable emission-scanning spectral imaging systems. Here, we have developed a new optical geometry that provides spectral illumination for use in excitation-scanning spectral imaging microscope systems. This was accomplished using a wavelength-specific LED array to acquire spectral image data. Feasibility of the LED-based spectral illuminator was evaluated through simulation and benchtop testing and assessment of imaging performance when integrated with a widefield fluorescence microscope. Ray tracing simulations (TracePro) were used to determine optimal optical component selection and geometry. Spectral imaging feasibility was evaluated using a series of 6-label fluorescent slides. The LED-based system response was compared to a previously tested thin-film tunable filter (TFTF)-based system. Spectral unmixing successfully discriminated all fluorescent components in spectral image data acquired from both the LED and TFTF systems. Therefore, the LED-based spectral illuminator provided spectral image data sets with comparable information content so as to allow identification of each fluorescent component. These results provide proof-of-principle demonstration of the ability to combine output from many discrete wavelength LED sources using a double-mirror (Cassegrain style) optical configuration that can be further modified to allow for high speed, video-rate spectral image acquisition. Real-time spectral fluorescence microscopy would allow monitoring of rapid cell signaling processes (i.e., Ca2+ and other second messenger signaling) and has potential to be translated to clinical imaging platforms.
Collapse
Affiliation(s)
- Craig M. Browning
- Chemical and Biomolecular Engineering, University of South Alabama, AL 36688, USA
- Systems Engineering, University of South Alabama, AL 36688, USA
- These authors contributed equally to this work
| | - Samantha Mayes
- Chemical and Biomolecular Engineering, University of South Alabama, AL 36688, USA
- These authors contributed equally to this work
| | - Samuel A. Mayes
- Chemical and Biomolecular Engineering, University of South Alabama, AL 36688, USA
- Systems Engineering, University of South Alabama, AL 36688, USA
| | - Thomas C. Rich
- Pharmacology, University of South Alabama, AL 36688, USA
- Center for Lung Biology, University of South Alabama, AL 36688, USA
| | - Silas J. Leavesley
- Chemical and Biomolecular Engineering, University of South Alabama, AL 36688, USA
- Pharmacology, University of South Alabama, AL 36688, USA
- Center for Lung Biology, University of South Alabama, AL 36688, USA
| |
Collapse
|
46
|
Wu Y, Xu Z, Yang W, Ning Z, Dong H. Review on the Application of Hyperspectral Imaging Technology of the Exposed Cortex in Cerebral Surgery. Front Bioeng Biotechnol 2022; 10:906728. [PMID: 35711634 PMCID: PMC9196632 DOI: 10.3389/fbioe.2022.906728] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 05/09/2022] [Indexed: 11/13/2022] Open
Abstract
The study of brain science is vital to human health. The application of hyperspectral imaging in biomedical fields has grown dramatically in recent years due to their unique optical imaging method and multidimensional information acquisition. Hyperspectral imaging technology can acquire two-dimensional spatial information and one-dimensional spectral information of biological samples simultaneously, covering the ultraviolet, visible and infrared spectral ranges with high spectral resolution, which can provide diagnostic information about the physiological, morphological and biochemical components of tissues and organs. This technology also presents finer spectral features for brain imaging studies, and further provides more auxiliary information for cerebral disease research. This paper reviews the recent advance of hyperspectral imaging in cerebral diagnosis. Firstly, the experimental setup, image acquisition and pre-processing, and analysis methods of hyperspectral technology were introduced. Secondly, the latest research progress and applications of hyperspectral imaging in brain tissue metabolism, hemodynamics, and brain cancer diagnosis in recent years were summarized briefly. Finally, the limitations of the application of hyperspectral imaging in cerebral disease diagnosis field were analyzed, and the future development direction was proposed.
Collapse
Affiliation(s)
- Yue Wu
- Research Center for Intelligent Sensing Systems, Zhejiang Lab, Hangzhou, China
| | - Zhongyuan Xu
- Research Center for Intelligent Sensing Systems, Zhejiang Lab, Hangzhou, China
| | - Wenjian Yang
- Research Center for Intelligent Sensing Systems, Zhejiang Lab, Hangzhou, China
| | - Zhiqiang Ning
- Anhui Institute of Optics and Fine Mechanics, Chinese Academy of Sciences (CAS), Hefei, China.,Science Island Branch, Graduate School of USTC, Hefei, China
| | - Hao Dong
- Research Center for Sensing Materials and Devices, Zhejiang Lab, Hangzhou, China
| |
Collapse
|
47
|
Brunner A, Schmidt VM, Zelger B, Woess C, Arora R, Zelger P, Huck CW, Pallua J. Visible and Near-Infrared hyperspectral imaging (HSI) can reliably quantify CD3 and CD45 positive inflammatory cells in myocarditis: Pilot study on formalin-fixed paraffin-embedded specimens from myocard obtained during autopsy. SPECTROCHIMICA ACTA. PART A, MOLECULAR AND BIOMOLECULAR SPECTROSCOPY 2022; 274:121092. [PMID: 35257987 DOI: 10.1016/j.saa.2022.121092] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/24/2021] [Revised: 02/17/2022] [Accepted: 02/25/2022] [Indexed: 06/14/2023]
Abstract
INTRODUCTION To implement Hyperspectral Imaging (HSI) as a tool for quantifying inflammatory cells in tissue specimens by the example of myocarditis in a collective of forensic patients. MATERIAL AND METHODS 44 consecutive patients with suspected myocardial inflammation at autopsy, diagnosed between 2013 and 2018 at the Institute of ForensicMedicine, Medical University of Innsbruck, were selected for this study. Using the IMEC SNAPSCAN camera, visible and near infrared hyperspectral images were collected from slides stained with CD3 and CD45 to assess quantity and spatial distribution of positive cells. Results were compared with visual assessment (VA) and conventional digital image analysis (DIA). RESULTS Finally, specimens of 40 patients were evaluated, of whom 36 patients (90%) suffered from myocarditis, two patients (5%) had suspected healing/healed myocarditis, and two did no have myocarditis (5%). The amount of CD3 and CD45 positive cells did not differ significantly between VA, HSI, and DIA (pVA/HSI/DIA = 0.46 for CD3 and 0.81 for CD45). Coheńs Kappa showed a very high correlation between VA versus HSI, VA versus DIA, and HSI versus DIA for CD3 (Coheńs Kappa = 0.91, 1.00, and 0.91, respectively). For CD45 an almost as high correlation was seen for VA versus HSI and HSI versus DIA (Coheńs Kappa = 0.75 and 0.70) and VA versus DIA (Coheńs Kappa = 0.89). CONCLUSION HSI is a reliable and objective method to count inflammatory cells in tissue slides of suspected myocarditis. Implementation of HSI in digital pathology might further expand the possibility of a sophisticated method.
Collapse
Affiliation(s)
- A Brunner
- Innsbruck Medical University, Institute of Pathology, Neuropathology, and Molecular Pathology, Muellerstrasse 44, 6020 Innsbruck, Austria
| | - V M Schmidt
- Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria
| | - B Zelger
- Innsbruck Medical University, Institute of Pathology, Neuropathology, and Molecular Pathology, Muellerstrasse 44, 6020 Innsbruck, Austria
| | - C Woess
- Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria.
| | - R Arora
- University Hospital for Orthopedics and Traumatology, Medical University of Innsbruck, Anichstraße 35, 6020 Innsbruck, Austria
| | - P Zelger
- University Clinic for Hearing, Voice and Speech Disorders, Medical University of Innsbruck, Anichstrasse 35, Innsbruck, Austria
| | - C W Huck
- Institute of Analytical Chemistry and Radiochemistry, University of Innsbruck, Innrain 80-82, Innsbruck, Austria
| | - J Pallua
- Innsbruck Medical University, Institute of Pathology, Neuropathology, and Molecular Pathology, Muellerstrasse 44, 6020 Innsbruck, Austria; Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria; University Hospital for Orthopedics and Traumatology, Medical University of Innsbruck, Anichstraße 35, 6020 Innsbruck, Austria
| |
Collapse
|
48
|
Grasso V, Willumeit-Rӧmer R, Jose J. Superpixel spectral unmixing framework for the volumetric assessment of tissue chromophores: A photoacoustic data-driven approach. PHOTOACOUSTICS 2022; 26:100367. [PMID: 35601933 PMCID: PMC9120071 DOI: 10.1016/j.pacs.2022.100367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/06/2022] [Revised: 05/04/2022] [Accepted: 05/06/2022] [Indexed: 06/15/2023]
Abstract
The assessment of tissue chromophores at a volumetric scale is vital for an improved diagnosis and treatment of a large number of diseases. Spectral photoacoustic imaging (sPAI) co-registered with high-resolution ultrasound (US) is an innovative technology that has a great potential for clinical translation as it can assess the volumetric distribution of the tissue components. Conventionally, to detect and separate the chromophores from sPAI, an input of the expected tissue absorption spectra is required. However, in pathological conditions, the prediction of the absorption spectra is difficult as it can change with respect to the physiological state. Besides, this conventional approach can also be hampered due to spectral coloring, which is a prominent distortion effect that induces spectral changes at depth. Here, we are proposing a novel data-driven framework that can overcome all these limitations and provide an improved assessment of the tissue chromophores. We have developed a superpixel spectral unmixing (SPAX) approach that can detect the most and less prominent absorber spectra and their volumetric distribution without any user interactions. Within the SPAX framework, we have also implemented an advanced spectral coloring compensation approach by utilizing US image segmentation and Monte Carlo simulations, based on a predefined library of optical properties. The framework has been tested on tissue-mimicking phantoms and also on healthy animals. The obtained results show enhanced specificity and sensitivity for the detection of tissue chromophores. To our knowledge, this is a unique framework that accounts for the spectral coloring and provides automated detection of tissue spectral signatures at a volumetric scale, which can open many possibilities for translational research.
Collapse
Affiliation(s)
- Valeria Grasso
- FUJIFILM VisualSonics, Amsterdam, the Netherlands
- Faculty of Engineering, Institute for Materials Science, Christian-Albrecht University of Kiel, Kiel, Germany
| | - Regine Willumeit-Rӧmer
- Faculty of Engineering, Institute for Materials Science, Christian-Albrecht University of Kiel, Kiel, Germany
- Division Metallic Biomaterials, Institute of Materials Research, Helmholtz-Zentrum Hereon GmbH, Geesthacht, Germany
| | - Jithin Jose
- FUJIFILM VisualSonics, Amsterdam, the Netherlands
| |
Collapse
|
49
|
Pfahl A, Köhler H, Thomaßen MT, Maktabi M, Bloße AM, Mehdorn M, Lyros O, Moulla Y, Niebisch S, Jansen-Winkeln B, Chalopin C, Gockel I. Video: Clinical evaluation of a laparoscopic hyperspectral imaging system. Surg Endosc 2022; 36:7794-7799. [PMID: 35546207 PMCID: PMC9485189 DOI: 10.1007/s00464-022-09282-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 04/16/2022] [Indexed: 11/30/2022]
Abstract
Background Hyperspectral imaging (HSI) during surgical procedures is a new method for perfusion quantification and tissue discrimination. Its use has been limited to open surgery due to large camera sizes, missing color video, or long acquisition times. A hand-held, laparoscopic hyperspectral camera has been developed now to overcome those disadvantages and evaluated clinically for the first time. Methods In a clinical evaluation study, gastrointestinal resectates of ten cancer patients were investigated using the laparoscopic hyperspectral camera. Reference data from corresponding anatomical regions were acquired with a clinically approved HSI system. An image registration process was executed that allowed for pixel-wise comparisons of spectral data and parameter images (StO2: oxygen saturation of tissue, NIR PI: near-infrared perfusion index, OHI: organ hemoglobin index, TWI: tissue water index) provided by both camera systems. The mean absolute error (MAE) and root mean square error (RMSE) served for the quantitative evaluations. Spearman’s rank correlation between factors related to the study design like the time of spectral white balancing and MAE, respectively RMSE, was calculated. Results The obtained mean MAEs between the TIVITA® Tissue and the laparoscopic hyperspectral system resulted in StO2: 11% ± 7%, NIR PI: 14±3, OHI: 14± 5, and TWI: 10 ± 2. The mean RMSE between both systems was 0.1±0.03 from 500 to 750 nm and 0.15 ±0.06 from 750 to 1000 nm. Spearman’s rank correlation coefficients showed no significant correlation between MAE or RMSE and influencing factors related to the study design. Conclusion Qualitatively, parameter images of the laparoscopic system corresponded to those of the system for open surgery. Quantitative deviations were attributed to technical differences rather than the study design. Limitations of the presented study are addressed in current large-scale in vivo trials. Supplementary Information The online version contains supplementary material available at 10.1007/s00464-022-09282-y.
Collapse
Affiliation(s)
- Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103, Leipzig, Germany.
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103, Leipzig, Germany
| | - Madeleine T Thomaßen
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103, Leipzig, Germany
| | - Albrecht M Bloße
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103, Leipzig, Germany
| | - Matthias Mehdorn
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| | - Orestis Lyros
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| | - Stefan Niebisch
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany.,Department of General, Visceral, Thoracic, and Vascular Surgery, Klinikum St. Georg, Leipzig, Germany
| | - Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Semmelweisstr. 14, 04103, Leipzig, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany
| |
Collapse
|
50
|
Raita-Hakola AM, Annala L, Lindholm V, Trops R, Näsilä A, Saari H, Ranki A, Pölönen I. FPI Based Hyperspectral Imager for the Complex Surfaces—Calibration, Illumination and Applications. SENSORS 2022; 22:s22093420. [PMID: 35591109 PMCID: PMC9103796 DOI: 10.3390/s22093420] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 04/13/2022] [Accepted: 04/23/2022] [Indexed: 01/27/2023]
Abstract
Hyperspectral imaging (HSI) applications for biomedical imaging and dermatological applications have been recently under research interest. Medical HSI applications are non-invasive methods with high spatial and spectral resolution. HS imaging can be used to delineate malignant tumours, detect invasions, and classify lesion types. Typical challenges of these applications relate to complex skin surfaces, leaving some skin areas unreachable. In this study, we introduce a novel spectral imaging concept and conduct a clinical pre-test, the findings of which can be used to develop the concept towards a clinical application. The SICSURFIS spectral imager concept combines a piezo-actuated Fabry–Pérot interferometer (FPI) based hyperspectral imager, a specially designed LED module and several sizes of stray light protection cones for reaching and adapting to the complex skin surfaces. The imager is designed for the needs of photometric stereo imaging for providing the skin surface models (3D) for each captured wavelength. The captured HS images contained 33 selected wavelengths (ranging from 477 nm to 891 nm), which were captured simultaneously with accordingly selected LEDs and three specific angles of light. The pre-test results show that the data collected with the new SICSURFIS imager enable the use of the spectral and spatial domains with surface model information. The imager can reach complex skin surfaces. Healthy skin, basal cell carcinomas and intradermal nevi lesions were classified and delineated pixel-wise with promising results, but further studies are needed. The results were obtained with a convolutional neural network.
Collapse
Affiliation(s)
- Anna-Maria Raita-Hakola
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
- Correspondence:
| | - Leevi Annala
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
| | - Vivian Lindholm
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (V.L.); (A.R.)
| | - Roberts Trops
- VTT Technical Research Centre of Finland Ltd., 02150 Espoo, Finland; (R.T.); (A.N.); (H.S.)
| | - Antti Näsilä
- VTT Technical Research Centre of Finland Ltd., 02150 Espoo, Finland; (R.T.); (A.N.); (H.S.)
| | - Heikki Saari
- VTT Technical Research Centre of Finland Ltd., 02150 Espoo, Finland; (R.T.); (A.N.); (H.S.)
| | - Annamari Ranki
- Department of Dermatology and Allergology, University of Helsinki and Helsinki University Hospital, 00290 Helsinki, Finland; (V.L.); (A.R.)
| | - Ilkka Pölönen
- Faculty of Information Technology, University of Jyväskylä, 40100 Jyväskylä, Finland; (L.A.); (I.P.)
| |
Collapse
|