1
|
Bourdillon AT. Computer Vision-Radiomics & Pathognomics. Otolaryngol Clin North Am 2024; 57:719-751. [PMID: 38910065 DOI: 10.1016/j.otc.2024.05.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/25/2024]
Abstract
The role of computer vision in extracting radiographic (radiomics) and histopathologic (pathognomics) features is an extension of molecular biomarkers that have been foundational to our understanding across the spectrum of head and neck disorders. Especially within head and neck cancers, machine learning and deep learning applications have yielded advances in the characterization of tumor features, nodal features, and various outcomes. This review aims to overview the landscape of radiomic and pathognomic applications, informing future work to address gaps. Novel methodologies will be needed to potentially engineer ways of integrating multidimensional data inputs to examine disease features to guide prognosis comprehensively and ultimately clinical management.
Collapse
Affiliation(s)
- Alexandra T Bourdillon
- Department of Otolaryngology-Head & Neck Surgery, University of California-San Francisco, San Francisco, CA 94115, USA.
| |
Collapse
|
2
|
Philips R, Yalamanchi P, Topf MC. Trends and Future Directions in Margin Analysis for Head and Neck Cancers. Surg Oncol Clin N Am 2024; 33:651-667. [PMID: 39244285 DOI: 10.1016/j.soc.2024.04.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/09/2024]
Abstract
Margin status in head and neck cancer has important prognostic implications. Currently, resection is based on manual palpation and gross visualization followed by intraoperative specimen or tumor bed-based margin analysis using frozen sections. While generally effective, this protocol has several limitations including margin sampling and close and positive margin re-localization. There is a lack of evidence on the association of use of frozen section analysis with improved survival in head and neck cancer. This article reviews novel technologies in head and neck margin analysis such as 3-dimensional scanning, augmented reality, molecular margins, optical imaging, spectroscopy, and artificial intelligence.
Collapse
Affiliation(s)
- Ramez Philips
- Department of Otolaryngology-Head and Neck Surgery, Vanderbilt University Medical Center, 1211 Medical Center Drive, Nashville, TN 37232, USA.
| | - Pratyusha Yalamanchi
- Department of Otolaryngology-Head and Neck Surgery, Vanderbilt University Medical Center, 1211 Medical Center Drive, Nashville, TN 37232, USA
| | - Michael C Topf
- Department of Otolaryngology-Head and Neck Surgery, Vanderbilt University Medical Center, 1211 Medical Center Drive, Nashville, TN 37232, USA; Vanderbilt University School of Engineering, 1211 Medical Center Drive, Nashville, TN 37232, USA
| |
Collapse
|
3
|
Tran MH, Ma L, Mubarak H, Gomez O, Yu J, Bryarly M, Fei B. Detection and margin assessment of thyroid carcinoma with microscopic hyperspectral imaging using transformer networks. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:093505. [PMID: 39050615 PMCID: PMC11268383 DOI: 10.1117/1.jbo.29.9.093505] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/01/2024] [Revised: 06/21/2024] [Accepted: 06/28/2024] [Indexed: 07/27/2024]
Abstract
Significance Hyperspectral imaging (HSI) is an emerging imaging modality for oncological applications and can improve cancer detection with digital pathology. Aim The study aims to highlight the increased accuracy and sensitivity of detecting the margin of thyroid carcinoma in hematoxylin and eosin (H&E)-stained histological slides using HSI and data augmentation methods. Approach Using an automated microscopic imaging system, we captured 2599 hyperspectral images from 65 H&E-stained human thyroid slides. Images were then preprocessed into 153,906 image patches of dimension 250 × 250 × 84 pixels . We modified the TimeSformer network architecture, which used alternating spectral attention and spatial attention layers. We implemented several data augmentation methods for HSI based on the RandAugment algorithm. We compared the performances of TimeSformer on HSI against the performances of pretrained ConvNext and pretrained vision transformers (ViT) networks on red, green, and blue (RGB) images. Finally, we applied attention unrolling techniques on the trained TimeSformer network to identify the biological features to which the network paid attention. Results In the testing dataset, TimeSformer achieved an accuracy of 90.87%, a weightedF 1 score of 89.79%, a sensitivity of 91.50%, and an area under the receiving operator characteristic curve (AU-ROC) score of 97.04%. Additionally, TimeSformer produced thyroid carcinoma tumor margins with an average Jaccard score of 0.76 mm. Without data augmentation, TimeSformer achieved an accuracy of 88.23%, a weightedF 1 score of 86.46%, a sensitivity of 85.53%, and an AU-ROC score of 94.94%. In comparison, the ViT network achieved an 89.98% accuracy, an 88.14% weightedF 1 score, an 84.77% sensitivity, and a 96.17% AU-ROC. Our visualization results showed that the network paid attention to biological features. Conclusions The TimeSformer model trained with hyperspectral histological data consistently outperformed conventional RGB-based models, highlighting the superiority of HSI in this context. Our proposed augmentation methods improved the accuracy, theF 1 score, and the sensitivity score.
Collapse
Affiliation(s)
- Minh Ha Tran
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Ling Ma
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Hasan Mubarak
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Ofelia Gomez
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - James Yu
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| | - Michelle Bryarly
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
4
|
Pruitt K, Ma L, Rathgeb A, Gahan JC, Johnson BA, Strand DW, Fei B. Design and validation of a high-speed hyperspectral laparoscopic imaging system. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:093506. [PMID: 39139794 PMCID: PMC11321365 DOI: 10.1117/1.jbo.29.9.093506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/16/2024] [Revised: 07/21/2024] [Accepted: 07/23/2024] [Indexed: 08/15/2024]
Abstract
Significance Minimally invasive surgery (MIS) has shown vast improvement over open surgery by reducing post-operative stays, intraoperative blood loss, and infection rates. However, in spite of these improvements, there are still prevalent issues surrounding MIS that may be addressed through hyperspectral imaging (HSI). We present a laparoscopic HSI system to further advance the field of MIS. Aim We present an imaging system that integrates high-speed HSI technology with a clinical laparoscopic setup and validate the system's accuracy and functionality. Different configurations that cover the visible (VIS) to near-infrared (NIR) range of electromagnetism are assessed by gauging the spectral fidelity and spatial resolution of each hyperspectral camera. Approach Standard Spectralon reflectance tiles were used to provide ground truth spectral footprints to compare with those acquired by our system using the root mean squared error (RMSE). Demosaicing techniques were investigated and used to measure and improve spatial resolution, which was assessed with a USAF resolution test target. A perception-based image quality evaluator was used to assess the demosaicing techniques we developed. Two configurations of the system were developed for evaluation. The functionality of the system was investigated in a phantom study and by imaging ex vivo tissues. Results Multiple configurations of our system were tested, each covering different spectral ranges, including VIS (460 to 600 nm), red/NIR (RNIR) (610 to 850 nm), and NIR (665 to 950 nm). Each configuration is capable of achieving real-time imaging speeds of up to 20 frames per second. RMSE values of 3.51 ± 2.03 % , 3.43 ± 0.84 % , and 3.47% were achieved for the VIS, RNIR, and NIR systems, respectively. We obtained sub-millimeter resolution using our demosaicing techniques. Conclusions We developed and validated a high-speed hyperspectral laparoscopic imaging system. The HSI system can be used as an intraoperative imaging tool for tissue classification during laparoscopic surgery.
Collapse
Affiliation(s)
- Kelden Pruitt
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Ling Ma
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Armand Rathgeb
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Jeffrey C. Gahan
- University of Texas Southwestern Medical Center, Department of Urology, Dallas, Texas, United States
| | - Brett A. Johnson
- University of Texas Southwestern Medical Center, Department of Urology, Dallas, Texas, United States
| | - Douglas W. Strand
- University of Texas Southwestern Medical Center, Department of Urology, Dallas, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
5
|
Butt MHF, Li JP, Ji JC, Riaz W, Anwar N, Butt FF, Ahmad M, Saboor A, Ali A, Uddin MY. Intelligent tumor tissue classification for Hybrid Health Care Units. Front Med (Lausanne) 2024; 11:1385524. [PMID: 38988354 PMCID: PMC11233792 DOI: 10.3389/fmed.2024.1385524] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2024] [Accepted: 04/03/2024] [Indexed: 07/12/2024] Open
Abstract
Introduction In the evolving healthcare landscape, we aim to integrate hyperspectral imaging into Hybrid Health Care Units to advance the diagnosis of medical diseases through the effective fusion of cutting-edge technology. The scarcity of medical hyperspectral data limits the use of hyperspectral imaging in disease classification. Methods Our study innovatively integrates hyperspectral imaging to characterize tumor tissues across diverse body locations, employing the Sharpened Cosine Similarity framework for tumor classification and subsequent healthcare recommendation. The efficiency of the proposed model is evaluated using Cohen's kappa, overall accuracy, and f1-score metrics. Results The proposed model demonstrates remarkable efficiency, with kappa of 91.76%, an overall accuracy of 95.60%, and an f1-score of 96%. These metrics indicate superior performance of our proposed model over existing state-of-the-art methods, even in limited training data. Conclusion This study marks a milestone in hybrid healthcare informatics, improving personalized care and advancing disease classification and recommendations.
Collapse
Affiliation(s)
- Muhammad Hassaan Farooq Butt
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Jian Ping Li
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Jiancheng Charles Ji
- Institute of Intelligent Manufacturing, Shenzhen Polytechnic University, Shenzhen, China
| | - Waqar Riaz
- Institute of Intelligent Manufacturing, Shenzhen Polytechnic University, Shenzhen, China
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Noreen Anwar
- Computer Engineering and Software Engineering Department, Polytechnique Montreal, Montreal, QC, Canada
| | | | - Muhammad Ahmad
- Department of Computer Science, National University of Computer and Emerging Sciences, Chiniot-Faisalabad Campus, Chiniot, Pakistan
| | - Abdus Saboor
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Amjad Ali
- Department of Computer Science and Software Technology, University of Swat, Saidu Sharif, Pakistan
| | - Mohammed Yousuf Uddin
- Department of Information Systems, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, Alkharj, Saudi Arabia
| |
Collapse
|
6
|
Chen Y, Zhong NN, Cao LM, Liu B, Bu LL. Surgical margins in head and neck squamous cell carcinoma: A narrative review. Int J Surg 2024; 110:3680-3700. [PMID: 38935830 PMCID: PMC11175762 DOI: 10.1097/js9.0000000000001306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Accepted: 02/26/2024] [Indexed: 06/29/2024]
Abstract
Head and neck squamous cell carcinoma (HNSCC), a prevalent and frequently recurring malignancy, often necessitates surgical intervention. The surgical margin (SM) plays a pivotal role in determining the postoperative treatment strategy and prognostic evaluation of HNSCC. Nonetheless, the process of clinical appraisal and assessment of the SMs remains a complex and indeterminate endeavor, thereby leading to potential difficulties for surgeons in defining the extent of resection. In this regard, we undertake a comprehensive review of the suggested surgical distance in varying circumstances, diverse methods of margin evaluation, and the delicate balance that must be maintained between tissue resection and preservation in head and neck surgical procedures. This review is intended to provide surgeons with pragmatic guidance in selecting the most suitable resection techniques, and in improving patients' quality of life by achieving optimal functional and aesthetic restoration.
Collapse
Affiliation(s)
- Yang Chen
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology
| | - Nian-Nian Zhong
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology
| | - Lei-Ming Cao
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology
| | - Bing Liu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology
- Department of Oral & Maxillofacial – Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan, People’s Republic of China
| | - Lin-Lin Bu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology
- Department of Oral & Maxillofacial – Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan, People’s Republic of China
| |
Collapse
|
7
|
Courtenay LA, Barbero-García I, Martínez-Lastras S, Del Pozo S, Corral de la Calle M, Garrido A, Guerrero-Sevilla D, Hernandez-Lopez D, González-Aguilera D. Near-infrared hyperspectral imaging and robust statistics for in vivo non-melanoma skin cancer and actinic keratosis characterisation. PLoS One 2024; 19:e0300400. [PMID: 38662718 PMCID: PMC11045066 DOI: 10.1371/journal.pone.0300400] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 02/26/2024] [Indexed: 04/28/2024] Open
Abstract
One of the most common forms of cancer in fair skinned populations is Non-Melanoma Skin Cancer (NMSC), which primarily consists of Basal Cell Carcinoma (BCC), and cutaneous Squamous Cell Carcinoma (SCC). Detecting NMSC early can significantly improve treatment outcomes and reduce medical costs. Similarly, Actinic Keratosis (AK) is a common skin condition that, if left untreated, can develop into more serious conditions, such as SCC. Hyperspectral imagery is at the forefront of research to develop non-invasive techniques for the study and characterisation of skin lesions. This study aims to investigate the potential of near-infrared hyperspectral imagery in the study and identification of BCC, SCC and AK samples in comparison with healthy skin. Here we use a pushbroom hyperspectral camera with a spectral range of ≈ 900 to 1600 nm for the study of these lesions. For this purpose, an ad hoc platform was developed to facilitate image acquisition. This study employed robust statistical methods for the identification of an optimal spectral window where the different samples could be differentiated. To examine these datasets, we first tested for the homogeneity of sample distributions. Depending on these results, either traditional or robust descriptive metrics were used. This was then followed by tests concerning the homoscedasticity, and finally multivariate comparisons of sample variance. The analysis revealed that the spectral regions between 900.66-1085.38 nm, 1109.06-1208.53 nm, 1236.95-1322.21 nm, and 1383.79-1454.83 nm showed the highest differences in this regard, with <1% probability of these observations being a Type I statistical error. Our findings demonstrate that hyperspectral imagery in the near-infrared spectrum is a valuable tool for analyzing, diagnosing, and evaluating non-melanoma skin lesions, contributing significantly to skin cancer research.
Collapse
Affiliation(s)
- Lloyd A. Courtenay
- CNRS, PACEA UMR 5199, Université de Bordeaux, Bât B2, Pessac, 33600, France
| | - Inés Barbero-García
- Department of Cartographic and Land Engineering, Higher Polytechnic School of Ávila, Universidad de Salamanca, Ávila, Spain
| | - Saray Martínez-Lastras
- Department of Cartographic and Land Engineering, Higher Polytechnic School of Ávila, Universidad de Salamanca, Ávila, Spain
| | - Susana Del Pozo
- Department of Cartographic and Land Engineering, Higher Polytechnic School of Ávila, Universidad de Salamanca, Ávila, Spain
| | | | - Alonso Garrido
- Institute of Regional Development, University of Castilla la Mancha, Campus Universitario s/n, Albacete, Spain
| | - Diego Guerrero-Sevilla
- Institute of Regional Development, University of Castilla la Mancha, Campus Universitario s/n, Albacete, Spain
| | - David Hernandez-Lopez
- Institute of Regional Development, University of Castilla la Mancha, Campus Universitario s/n, Albacete, Spain
| | - Diego González-Aguilera
- Department of Cartographic and Land Engineering, Higher Polytechnic School of Ávila, Universidad de Salamanca, Ávila, Spain
| |
Collapse
|
8
|
Pertzborn D, Bali A, Mühlig A, von Eggeling F, Guntinas-Lichius O. Hyperspectral imaging and evaluation of surgical margins: where do we stand? Curr Opin Otolaryngol Head Neck Surg 2024; 32:96-104. [PMID: 38193544 DOI: 10.1097/moo.0000000000000957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2024]
Abstract
PURPOSE OF REVIEW To highlight the recent literature on the use of hyperspectral imaging (HSI) for cancer margin evaluation ex vivo, for head and neck cancer pathology and in vivo during head and neck cancer surgery. RECENT FINDINGS HSI can be used ex vivo on unstained and stained tissue sections to analyze head and neck tissue and tumor cells in combination with machine learning approaches to analyze head and neck cancer cell characteristics and to discriminate the tumor border from normal tissue. Data on in vivo applications during head and neck cancer surgery are preliminary and limited. Even now an accuracy of 80% for tumor versus nonneoplastic tissue classification can be achieved for certain tasks, within the current in vivo settings. SUMMARY Significant progress has been made to introduce HSI for ex vivo head and neck cancer pathology evaluation and for an intraoperative use to define the tumor margins. To optimize the accuracy for in vivo use, larger HSI databases with annotations for head and neck cancer are needed.
Collapse
Affiliation(s)
- David Pertzborn
- Department of Otorhinolaryngology, Jena University Hospital, Jena, Germany
| | | | | | | | | |
Collapse
|
9
|
de Koning KJ, Dankbaar JW, de Keizer B, Willemsen K, van der Toorn A, Breimer GE, van Es RJJ, de Bree R, Noorlag R, Philippens MEP. Feasibility of an MR-based digital specimen for tongue cancer resection specimens: a novel approach for margin evaluation. Front Oncol 2024; 14:1342857. [PMID: 38606095 PMCID: PMC11007136 DOI: 10.3389/fonc.2024.1342857] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Accepted: 03/15/2024] [Indexed: 04/13/2024] Open
Abstract
Objective This study explores the feasibility of ex-vivo high-field magnetic resonance (MR) imaging to create digital a three-dimensional (3D) representations of tongue cancer specimens, referred to as the "MR-based digital specimen" (MR-DS). The aim was to create a method to assist surgeons in identifying and localizing inadequate resection margins during surgery, a critical factor in achieving locoregional control. Methods Fresh resection specimens of nine tongue cancer patients were imaged in a 7 Tesla small-bore MR, using a high-resolution multislice and 3D T2-weighted Turbo Spin Echo. Two independent radiologists (R1 and R2) outlined the tumor and mucosa on the MR-images whereafter the outlines were configured to an MR-DS. A color map was projected on the MR-DS, mapping the inadequate margins according to R1 and R2. We compared the hematoxylin-eosin-based digital specimen (HE-DS), which is a histopathological 3D representation derived from HE stained sections, with its corresponding MR-images. In line with conventional histopathological assessment, all digital specimens were divided into five anatomical regions (anterior, posterior, craniomedial, caudolateral and deep central). Over- and underestimation 95th-percentile Hausdorff-distances were calculated between the radiologist- and histopathologist-determined tumor outlines. The MR-DS' diagnostic accuracy for inadequate margin detection (i.e. sensitivity and specificity) was determined in two ways: with conventional histopathology and HE-DS as reference. Results Using conventional histopathology as a reference, R1 achieved 77% sensitivity and 50% specificity, while R2 achieved 65% sensitivity and 57% specificity. When referencing to the HE-DS, R1 achieved 94% sensitivity and 61% specificity, while R2 achieved 88% sensitivity and 71% specificity. Range of over- and underestimation 95HD was 0.9 mm - 11.8 mm and 0.0 mm - 5.3 mm, respectively. Conclusion This proof of concept for volumetric assessment of resection margins using MR-DSs, demonstrates promising potential for further development. Overall, sensitivity is higher than specificity for inadequate margin detection, because of the radiologist's tendency to overestimate tumor size.
Collapse
Affiliation(s)
- Klijs Jacob de Koning
- Department of Head and Neck Surgical Oncology, University Medical Center Utrecht, Utrecht, Netherlands
| | - Jan Willem Dankbaar
- Department of Radiology, University Medical Center Utrecht, Utrecht, Netherlands
| | - Bart de Keizer
- Department of Radiology, University Medical Center Utrecht, Utrecht, Netherlands
| | - Koen Willemsen
- 3D Lab, University Medical Center Utrecht, Utrecht, Netherlands
| | - Annette van der Toorn
- Translational Neuroimaging Group, Center for Image Sciences, University Medical Center Utrecht & Utrecht University, Utrecht, Netherlands
| | - Gerben Eise Breimer
- Department of Pathology, University Medical Center Utrecht, Utrecht, Netherlands
| | - Robert Jelle Johan van Es
- Department of Head and Neck Surgical Oncology, University Medical Center Utrecht, Utrecht, Netherlands
| | - Remco de Bree
- Department of Head and Neck Surgical Oncology, University Medical Center Utrecht, Utrecht, Netherlands
| | - Rob Noorlag
- Department of Head and Neck Surgical Oncology, University Medical Center Utrecht, Utrecht, Netherlands
| | | |
Collapse
|
10
|
Mubarak HK, Zhou X, Palsgrove D, Sumer BD, Chen AY, Fei B. An Ensemble Learning Method for Detection of Head and Neck Squamous Cell Carcinoma Using Polarized Hyperspectral Microscopic Imaging. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12933:129330P. [PMID: 38711533 PMCID: PMC11073817 DOI: 10.1117/12.3007869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Head and neck squamous cell carcinoma (HNSCC) has a high mortality rate. In this study, we developed a Stokes-vector-derived polarized hyperspectral imaging (PHSI) system for H&E-stained pathological slides with HNSCC and built a dataset to develop a deep learning classification method based on convolutional neural networks (CNN). We use our polarized hyperspectral microscope to collect the four Stokes parameter hypercubes (S0, S1, S2, and S3) from 56 patients and synthesize pseudo-RGB images using a transformation function that approximates the human eye's spectral response to visual stimuli. Each image is divided into patches. Data augmentation is applied using rotations and flipping. We create a four-branch model architecture where each branch is trained on one Stokes parameter individually, then we freeze the branches and fine-tune the top layers of our model to generate final predictions. Our results show high accuracy, sensitivity, and specificity, indicating that our model performed well on our dataset. Future works can improve upon these results by training on more varied data, classifying tumors based on their grade, and introducing more recent architectural techniques.
Collapse
Affiliation(s)
- Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University, Atlanta, GA
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
11
|
Filip P, Lerner DK, Kominsky E, Schupper A, Liu K, Khan NM, Roof S, Hadjipanayis C, Genden E, Iloreta AMC. 5-Aminolevulinic Acid Fluorescence-Guided Surgery in Head and Neck Squamous Cell Carcinoma. Laryngoscope 2024; 134:741-748. [PMID: 37540051 DOI: 10.1002/lary.30910] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 06/30/2023] [Accepted: 07/04/2023] [Indexed: 08/05/2023]
Abstract
OBJECTIVES To determine the utility of 5-aminolevulinic acid (5-ALA) fluorescence for resection of head and neck carcinoma. METHODS In this prospective pilot trial, 5-ALA was administered as an oral suspension 3-5 h prior to induction of anesthesia for resection of head and neck squamous cell carcinoma (HNSCC). Following resection, 405 nm blue light was applied, and fluorescence of the tumor as well as the surgical bed was recorded. Specimen fluorescence intensity was graded categorically as none (score = 0), mild (1), moderate (2), or robust (3) by the operating surgeon intraoperatively and corroborated with final pathologic diagnosis. RESULTS Seven patients underwent resection with 5-ALA. Five (83%) were male with an age range of 33-82 years (mean = 60). Sites included nasal cavity (n = 3), oral cavity (n = 3), and the larynx (n = 1). All specimens demonstrated robust fluorescence when 5-ALA was administered 3-5 h preoperatively. 5-ALA fluorescence predicted the presence of perineural invasion, a positive margin, and metastatic lymphadenopathy. Two patients had acute photosensitivity reactions, and one patient had a temporary elevation of hepatic enzymes. CONCLUSIONS 5-ALA induces robust intraoperative fluorescence of HNSCC, capable of demonstrating a positive margin, perineural invasion, and metastatic nodal disease. Although no conclusions are there about the safety of this drug in the head and neck cancer population, our study parallels the extensive safety data in the neurosurgical literature. Future applications may include intraoperative assessment of margin status, diagnostic accuracy, and impacts on survival. LEVEL OF EVIDENCE 4 Laryngoscope, 134:741-748, 2024.
Collapse
Affiliation(s)
- Peter Filip
- Department of Otolaryngology-Head and Neck Surgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| | - David K Lerner
- Department of Otolaryngology-Head and Neck Surgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| | - Evan Kominsky
- Department of Otolaryngology-Head and Neck Surgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| | - Alexander Schupper
- Department of Neurosurgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| | - Katherine Liu
- Department of Otolaryngology-Head and Neck Surgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| | - Nazir Mohemmed Khan
- Department of Otolaryngology-Head and Neck Surgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| | - Scott Roof
- Department of Otolaryngology-Head and Neck Surgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| | | | - Eric Genden
- Department of Otolaryngology-Head and Neck Surgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| | - Alfred M C Iloreta
- Department of Otolaryngology-Head and Neck Surgery, Icahn School of Medicine at Mount Sinai, New York City, New York, U.S.A
| |
Collapse
|
12
|
Zhou X, Mubarak HK, Kaur J, Dingal PCDP, Fei B. Polarized Hyperspectral Microscopic Imaging for Zebrafish. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12834:1283404. [PMID: 38737328 PMCID: PMC11086558 DOI: 10.1117/12.3007294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2024]
Abstract
Zebrafish is a well-established animal model for developmental and disease studies. Its optical transparency at early developmental stages is ideal for tissue visualization. Interaction of light with zebrafish tissues provides information on their structure and properties. In this study, we developed a microscopic imaging system for improving the visualization of unstained zebrafish tissues on tissue slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector data in two imaging setups (polarized light imaging and polarized hyperspectral imaging) are capable of improving the visualization of different types of zebrafish tissues (brain, muscle, skin cells, blood vessels, and yolk). Using the images collected from larval zebrafish samples by polarized light imaging, we found that DOP and DOLP could show clearer structural information of the brain and of skin cells, muscle and blood vessels in the tail. Furthermore, DOP and DOLP parameters derived from images collected by polarized hyperspectral imaging could show clearer structural information of skin cells developing around yolk as well as the surrounding blood vessel network. In addition, polarized hyperspectral imaging could provide complementary spectral information to the spatial information on Stokes vector data of zebrafish tissues. The polarized light imaging & polarized hyperspectral imaging systems provide a better insight into the microstructures of zebrafish tissues.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Jaideep Kaur
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | | | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
13
|
Pruitt K, Modir N, Nawawithan N, Tran M, Fei B. Optimization of a real-time LED based spectral imaging system. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12853:128530D. [PMID: 38741718 PMCID: PMC11090289 DOI: 10.1117/12.3005452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
We are designing a real-time spectral imaging system using micro-LEDs and a high-speed micro-camera for potential endoscopic applications. Currently, gastrointestinal (GI) endoscopic imaging has largely been limited to white light imaging (WLI), while other endoscopic approaches have seen advancements in imaging techniques including fluorescence imaging, narrow-band imaging, and stereoscopic visualization. To further advance GI endoscopic imaging, we are working towards a high-speed spectral imaging system that can be implemented with a chip-on-tip design for a flexible endoscope. Hyperspectral imaging has potential applications in a variety of imaging procedures with its ability to discern spectral footprints of the imaging field in a series of two-dimensional images at different wavelengths. For investigating the feasibility of a real-time LED-based hyperspectral imaging system for endoscopic applications we designed and developed a large-scale prototype using through-hole LEDs, which is further analyzed as we design our future system. For high quality imaging, the LED array must be designed with the specific illumination patterns and intensities of each LED considered. We present our work in optimizing our current LED array through optical simulations performed in silico. Damped least squares and orthogonal descent algorithms are implemented to maximize irradiance power and improve illumination homogeneity at several working distances by adjusting radial distances of each LED from the camera. With our prototype LED-based hyperspectral imaging system, the simulation and optimization approach achieved an average increase of 8.36 ± 8.79% in irradiance and a 4.3% decrease in standard deviation at multiple working distances and field-of-views for each LED, as compared to the original design, leading to improved image quality and maintained acquisition speeds. This work highlights the value of in silico optical simulation and provides a framework for improved optical system design and will inform design decisions in future works.
Collapse
Affiliation(s)
- Kelden Pruitt
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Naeeme Modir
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Nati Nawawithan
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Minh Tran
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
14
|
Pruitt K, Rathgeb A, Gahan JC, Johnson BA, Strand DW, Fei B. A dual-camera hyperspectral laparoscopic imaging system. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12831:1283107. [PMID: 38708175 PMCID: PMC11069412 DOI: 10.1117/12.3005893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2024]
Abstract
Minimally invasive surgery (MIS) has expanded broadly in the field of abdominal and pelvic surgery. However, there are still prevalent issues surrounding intracorporeal surgery, such as iatrogenic injury, anastomotic leakage, or the presence of positive tumor margins after resection. Current approaches to address these issues and advance laparoscopic imaging techniques often involve fluorescence imaging agents, such as indocyanine green (ICG), to improve visualization, but these have drawbacks. Hyperspectral imaging (HSI) is an emerging optical imaging modality that takes advantage of spectral characteristics of different tissues. Various applications include tissue classification and digital pathology. In this study, we developed a dual-camera system for high-speed hyperspectral imaging. This includes the development of a custom application interface and corresponding hardware setup. Characterization of the system was performed, including spectral accuracy and spatial resolution, showing little sacrifice in speed for the approximate doubling of the covered spectral range, with our system acquiring 29 spectral images from 460-850 nm. Reference color tiles with various reflectance profiles were imaged and a RMSE of 3.56 ± 1.36% was achieved. Sub-millimeter resolution was shown at 7 cm working distance for both hyperspectral cameras. Finally, we image ex vivo tissues, including porcine stomach, liver, intestine, and kidney with our system and use a high-resolution, radiometrically calibrated spectrometer for comparison and evaluation of spectral fidelity. The dual-camera hyperspectral laparoscopic imaging system can have immediate applications in various surgeries.
Collapse
Affiliation(s)
- Kelden Pruitt
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Armand Rathgeb
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Jeffrey C. Gahan
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Brett A. Johnson
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Douglas W. Strand
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
15
|
Ahmad M, Irfan MA, Sadique U, Haq IU, Jan A, Khattak MI, Ghadi YY, Aljuaid H. Multi-Method Analysis of Histopathological Image for Early Diagnosis of Oral Squamous Cell Carcinoma Using Deep Learning and Hybrid Techniques. Cancers (Basel) 2023; 15:5247. [PMID: 37958422 PMCID: PMC10650156 DOI: 10.3390/cancers15215247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2023] [Revised: 10/22/2023] [Accepted: 10/25/2023] [Indexed: 11/15/2023] Open
Abstract
Oral cancer is a fatal disease and ranks seventh among the most common cancers throughout the whole globe. Oral cancer is a type of cancer that usually affects the head and neck. The current gold standard for diagnosis is histopathological investigation, however, the conventional approach is time-consuming and requires professional interpretation. Therefore, early diagnosis of Oral Squamous Cell Carcinoma (OSCC) is crucial for successful therapy, reducing the risk of mortality and morbidity, while improving the patient's chances of survival. Thus, we employed several artificial intelligence techniques to aid clinicians or physicians, thereby significantly reducing the workload of pathologists. This study aimed to develop hybrid methodologies based on fused features to generate better results for early diagnosis of OSCC. This study employed three different strategies, each using five distinct models. The first strategy is transfer learning using the Xception, Inceptionv3, InceptionResNetV2, NASNetLarge, and DenseNet201 models. The second strategy involves using a pre-trained art of CNN for feature extraction coupled with a Support Vector Machine (SVM) for classification. In particular, features were extracted using various pre-trained models, namely Xception, Inceptionv3, InceptionResNetV2, NASNetLarge, and DenseNet201, and were subsequently applied to the SVM algorithm to evaluate the classification accuracy. The final strategy employs a cutting-edge hybrid feature fusion technique, utilizing an art-of-CNN model to extract the deep features of the aforementioned models. These deep features underwent dimensionality reduction through principal component analysis (PCA). Subsequently, low-dimensionality features are combined with shape, color, and texture features extracted using a gray-level co-occurrence matrix (GLCM), Histogram of Oriented Gradient (HOG), and Local Binary Pattern (LBP) methods. Hybrid feature fusion was incorporated into the SVM to enhance the classification performance. The proposed system achieved promising results for rapid diagnosis of OSCC using histological images. The accuracy, precision, sensitivity, specificity, F-1 score, and area under the curve (AUC) of the support vector machine (SVM) algorithm based on the hybrid feature fusion of DenseNet201 with GLCM, HOG, and LBP features were 97.00%, 96.77%, 90.90%, 98.92%, 93.74%, and 96.80%, respectively.
Collapse
Affiliation(s)
- Mehran Ahmad
- Department of Electrical Engineering, University of Engineering and Technology, Peshawar 25000, Pakistan; (M.A.); (A.J.); (M.I.K.)
- AIH, Intelligent Information Processing Lab (NCAI), University of Engineering and Technology, Peshawar 25000, Pakistan; (U.S.); (I.u.H.)
| | - Muhammad Abeer Irfan
- Department of Computer Systems Engineering, University of Engineering and Technology, Peshawar 25000, Pakistan;
| | - Umar Sadique
- AIH, Intelligent Information Processing Lab (NCAI), University of Engineering and Technology, Peshawar 25000, Pakistan; (U.S.); (I.u.H.)
- Department of Computer Systems Engineering, University of Engineering and Technology, Peshawar 25000, Pakistan;
| | - Ihtisham ul Haq
- AIH, Intelligent Information Processing Lab (NCAI), University of Engineering and Technology, Peshawar 25000, Pakistan; (U.S.); (I.u.H.)
- Department of Mechatronics Engineering, University of Engineering and Technology, Peshawar 25000, Pakistan
| | - Atif Jan
- Department of Electrical Engineering, University of Engineering and Technology, Peshawar 25000, Pakistan; (M.A.); (A.J.); (M.I.K.)
| | - Muhammad Irfan Khattak
- Department of Electrical Engineering, University of Engineering and Technology, Peshawar 25000, Pakistan; (M.A.); (A.J.); (M.I.K.)
| | - Yazeed Yasin Ghadi
- Department of Computer Science, Al Ain University, Al Ain 15551, United Arab Emirates;
| | - Hanan Aljuaid
- Computer Sciences Department, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University (PNU), Riyadh 11671, Saudi Arabia
| |
Collapse
|
16
|
Loperfido A, Celebrini A, Marzetti A, Bellocchi G. Current role of artificial intelligence in head and neck cancer surgery: a systematic review of literature. EXPLORATION OF TARGETED ANTI-TUMOR THERAPY 2023; 4:933-940. [PMID: 37970203 PMCID: PMC10645467 DOI: 10.37349/etat.2023.00174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Accepted: 07/19/2023] [Indexed: 11/17/2023] Open
Abstract
Aim Artificial intelligence (AI) is a new field of science in which computers will provide decisions-supporting tools to help doctors make difficult clinical choices. Recent AI applications in otolaryngology include head and neck oncology, rhinology, neurotology, and laryngology. The aim of this systematic review is to describe the potential uses of AI in head and neck oncology with a special focus on the surgical field. Methods The authors performed a systematic review, in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines, in the main medical databases, including PubMed, Scopus, and Cochrane Library, considering all original studies published until February 2023 about the role of AI in head and neck cancer surgery. The search strategy included a combination of the following terms: "artificial intelligence" or "machine learning" and "head and neck cancer". Results Overall, 303 papers were identified and after duplicate removal (12 papers) and excluding papers not written in English (1 paper) and off-topic (4 papers), papers were assessed for eligibility; finally, only 12 papers were included. Three main fields of clinical interest were identified: the most widely investigated included the role of AI in surgical margins assessment (7 papers); the second most frequently evaluated topic was complications assessment (4 papers); finally, only one paper dealt with the indication of salvage laryngectomy after primary radiotherapy. Conclusions The authors report the first systematic review in the literature concerning the role of AI in head and neck cancer surgery. An increasing influx of AI applications to clinical problems in otolaryngology is expected, so specialists should be increasingly prepared to manage the constant changes. It will always remain critical for clinicians to use their skills and knowledge to critically evaluate the additional information provided by AI and make the final decisions on each patient.
Collapse
Affiliation(s)
| | | | - Andrea Marzetti
- Department of Otolaryngology Head and Neck Surgery, Fabrizio Spaziani Hospital, 03100 Frosinone, Italy
| | | |
Collapse
|
17
|
Zhong NN, Wang HQ, Huang XY, Li ZZ, Cao LM, Huo FY, Liu B, Bu LL. Enhancing head and neck tumor management with artificial intelligence: Integration and perspectives. Semin Cancer Biol 2023; 95:52-74. [PMID: 37473825 DOI: 10.1016/j.semcancer.2023.07.002] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Revised: 07/11/2023] [Accepted: 07/15/2023] [Indexed: 07/22/2023]
Abstract
Head and neck tumors (HNTs) constitute a multifaceted ensemble of pathologies that primarily involve regions such as the oral cavity, pharynx, and nasal cavity. The intricate anatomical structure of these regions poses considerable challenges to efficacious treatment strategies. Despite the availability of myriad treatment modalities, the overall therapeutic efficacy for HNTs continues to remain subdued. In recent years, the deployment of artificial intelligence (AI) in healthcare practices has garnered noteworthy attention. AI modalities, inclusive of machine learning (ML), neural networks (NNs), and deep learning (DL), when amalgamated into the holistic management of HNTs, promise to augment the precision, safety, and efficacy of treatment regimens. The integration of AI within HNT management is intricately intertwined with domains such as medical imaging, bioinformatics, and medical robotics. This article intends to scrutinize the cutting-edge advancements and prospective applications of AI in the realm of HNTs, elucidating AI's indispensable role in prevention, diagnosis, treatment, prognostication, research, and inter-sectoral integration. The overarching objective is to stimulate scholarly discourse and invigorate insights among medical practitioners and researchers to propel further exploration, thereby facilitating superior therapeutic alternatives for patients.
Collapse
Affiliation(s)
- Nian-Nian Zhong
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Han-Qi Wang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Xin-Yue Huang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Zi-Zhan Li
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Lei-Ming Cao
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Fang-Yi Huo
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Bing Liu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China; Department of Oral & Maxillofacial - Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China.
| | - Lin-Lin Bu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China; Department of Oral & Maxillofacial - Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China.
| |
Collapse
|
18
|
Liao WC, Mukundan A, Sadiaza C, Tsao YM, Huang CW, Wang HC. Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging. BIOMEDICAL OPTICS EXPRESS 2023; 14:4383-4405. [PMID: 37799695 PMCID: PMC10549751 DOI: 10.1364/boe.492635] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 10/07/2023]
Abstract
One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.
Collapse
Affiliation(s)
- Wei-Chih Liao
- Department of Internal Medicine, National Taiwan University Hospital, National Taiwan University College of Medicine, Taipei, Taiwan
- Graduate Institute of Epidemiology and Preventive Medicine, National Taiwan University, Taipei, Taiwan
| | - Arvind Mukundan
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Cleorita Sadiaza
- Department of Mechanical Engineering, Far Eastern University, P. Paredes St., Sampaloc, Manila, 1015, Philippines
| | - Yu-Ming Tsao
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Chien-Wei Huang
- Department of Gastroenterology, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st.Rd., Lingya District, Kaohsiung City 80284, Taiwan
- Department of Nursing, Tajen University, 20, Weixin Rd., Yanpu Township, Pingtung County 90741, Taiwan
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
- Department of Medical Research, Dalin Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, No. 2, Minsheng Road, Dalin, Chiayi, 62247, Taiwan
- Director of Technology Development, Hitspectra Intelligent Technology Co., Ltd., 4F., No. 2, Fuxing 4th Rd., Qianzhen Dist., Kaohsiung City 80661, Taiwan
| |
Collapse
|
19
|
Tran MH, Gomez O, Fei B. A Video Transformer Network for Thyroid Cancer Detection on Hyperspectral Histologic Images. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12471:1247107. [PMID: 38577581 PMCID: PMC10993530 DOI: 10.1117/12.2654851] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/06/2024]
Abstract
Hyperspectral imaging is a label-free and non-invasive imaging modality that seeks to capture images in different wavelengths. In this study, we used a vision transformer that was pre-trained from video data to detect thyroid cancer on hyperspectral images. We built a dataset of 49 whole slide hyperspectral images (WS-HSI) of thyroid cancer. To improve training, we introduced 5 new data augmentation methods that transform spectra. We achieved an F-1 score of 88.1% and an accuracy of 89.64% on our test dataset. The transformer network and the whole slide hyperspectral imaging technique can have many applications in digital pathology.
Collapse
Affiliation(s)
- Minh Ha Tran
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
| | - Ofelia Gomez
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
20
|
Intraoperative Imaging Techniques to Improve Surgical Resection Margins of Oropharyngeal Squamous Cell Cancer: A Comprehensive Review of Current Literature. Cancers (Basel) 2023; 15:cancers15030896. [PMID: 36765858 PMCID: PMC9913756 DOI: 10.3390/cancers15030896] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 01/24/2023] [Accepted: 01/26/2023] [Indexed: 02/04/2023] Open
Abstract
Inadequate resection margins in head and neck squamous cell carcinoma surgery necessitate adjuvant therapies such as re-resection and radiotherapy with or without chemotherapy and imply increasing morbidity and worse prognosis. On the other hand, taking larger margins by extending the resection also leads to avoidable increased morbidity. Oropharyngeal squamous cell carcinomas (OPSCCs) are often difficult to access; resections are limited by anatomy and functionality and thus carry an increased risk for close or positive margins. Therefore, there is a need to improve intraoperative assessment of resection margins. Several intraoperative techniques are available, but these often lead to prolonged operative time and are only suitable for a subgroup of patients. In recent years, new diagnostic tools have been the subject of investigation. This study reviews the available literature on intraoperative techniques to improve resection margins for OPSCCs. A literature search was performed in Embase, PubMed, and Cochrane. Narrow band imaging (NBI), high-resolution microendoscopic imaging, confocal laser endomicroscopy, frozen section analysis (FSA), ultrasound (US), computed tomography scan (CT), (auto) fluorescence imaging (FI), and augmented reality (AR) have all been used for OPSCC. NBI, FSA, and US are most commonly used and increase the rate of negative margins. Other techniques will become available in the future, of which fluorescence imaging has high potential for use with OPSCC.
Collapse
|
21
|
Spectral Similarity Measures for In Vivo Human Tissue Discrimination Based on Hyperspectral Imaging. Diagnostics (Basel) 2023; 13:diagnostics13020195. [PMID: 36673005 PMCID: PMC9857871 DOI: 10.3390/diagnostics13020195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 12/08/2022] [Accepted: 12/21/2022] [Indexed: 01/06/2023] Open
Abstract
PROBLEM Similarity measures are widely used as an approved method for spectral discrimination or identification with their applications in different areas of scientific research. Even though a range of works have been presented, only a few showed slightly promising results for human tissue, and these were mostly focused on pathological and non-pathological tissue classification. METHODS In this work, several spectral similarity measures on hyperspectral (HS) images of in vivo human tissue were evaluated for tissue discrimination purposes. Moreover, we introduced two new hybrid spectral measures, called SID-JM-TAN(SAM) and SID-JM-TAN(SCA). We analyzed spectral signatures obtained from 13 different human tissue types and two different materials (gauze, instruments), collected from HS images of 100 patients during surgeries. RESULTS The quantitative results showed the reliable performance of the different similarity measures and the proposed hybrid measures for tissue discrimination purposes. The latter produced higher discrimination values, up to 6.7 times more than the classical spectral similarity measures. Moreover, an application of the similarity measures was presented to support the annotations of the HS images. We showed that the automatic checking of tissue-annotated thyroid and colon tissues was successful in 73% and 60% of the total spectra, respectively. The hybrid measures showed the highest performance. Furthermore, the automatic labeling of wrongly annotated tissues was similar for all measures, with an accuracy of up to 90%. CONCLUSION In future work, the proposed spectral similarity measures will be integrated with tools to support physicians in annotations and tissue labeling of HS images.
Collapse
|
22
|
Zhou X, Mubarak HK, Ma L, Palsgrove D, Ortega S, Callicó GM, Medina EA, Brimhall BB, Whitted M, Fei B. Polarized Hyperspectral Microscopic Imaging for White Blood Cells on Wright-Stained Blood Smear Slides. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12382:1238207. [PMID: 38486823 PMCID: PMC10938460 DOI: 10.1117/12.2655708] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
White blood cells, also called leukocytes, are hematopoietic cells of the immune system that are involved in protecting the body against both infectious diseases and foreign materials. The abnormal development and uncontrolled proliferation of these cells can lead to devastating cancers. Their timely recognition in the peripheral blood is critical to diagnosis and treatment. In this study, we developed a microscopic imaging system for improving the visualization of white blood cells on Wright's stained blood smear slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3) of five types of white blood cells (neutrophil, eosinophil, basophil, lymphocyte, and monocyte), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP), and the degree of circular polarization (DOCP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector derived parameters (DOP, DOLP, and DOCP) could improve the visualization of granules in granulocytes (neutrophils, eosinophils, and basophils). Furthermore, Stokes vector derived parameters (DOP, DOLP, and DOCP) could improve the visualization of surface structures (protein patterns) of lymphocytes enabling subclassification of lymphocyte subpopulations. Finally, S2, S3, and DOCP could enhance the morphologic visualization of monocyte nucleus. We also demonstrated that the polarized hyperspectral imaging setup could provide complementary spectral information to the spatial information on different Stokes vector parameters of white blood cells. This work demonstrates that polarized light imaging & polarized hyperspectral imaging has the potential to become a strong imaging tool in the diagnosis of disorders arising from white blood cells.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Samuel Ortega
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Canary Islands, Spain
- Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway
| | - Gustavo Marrero Callicó
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Canary Islands, Spain
| | - Edward A Medina
- Department of Pathology and Laboratory Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX
| | - Bradley B Brimhall
- Department of Pathology and Laboratory Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX
| | - Marisa Whitted
- Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
23
|
Ma L, Srinivas A, Krishnamurthy A, Zhou X, Shah NS, Obaid G, Fei B. Automated Polarized Hyperspectral Imaging (PHSI) for ex-vivo and in-vivo Tissue Assessment. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12391:123910F. [PMID: 38476292 PMCID: PMC10932616 DOI: 10.1117/12.2651011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/14/2024]
Abstract
Polarized light interactions with biological tissues can reveal information regarding tissue structure, while spectral characteristics are closely related to tissue composition. An integration of both modalities in a compact system could better assist tissue assessment. This study aims to develop a polarized hyperspectral imaging (PHSI) system that fulfills both linearly and circularly polarized hyperspectral imaging for in vivo and ex vivo applications. The system is comprised of a white LED, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a hyperspectral snapshot camera. The system was calibrated to compute the full Stokes polarimetry. For tissue differentiation, fresh ex vivo mouse tissue specimens from kidney, liver, spleen, muscle, lung, and salivary gland of mice were imaged. The spectra of three features, named degree of polarization (DOP), degree of linear polarization (DOLP), and degree of circular polarization (DOCP), were generated. A k-nearest neighbor (k-NN) classifier was trained with multi-class spectra and 5-fold cross validation. It was found that DOP better differentiates tissue with an average accuracy of 0.87. Additionally, support vector machine (SVM) classifiers were trained to differentiate between each two of the organs, and it was determined that DOLP better identified kidney, liver, and spleen, whereas DOCP better identified muscle and lung tissues. Then, the setup was employed to image in vivo human fingers with and without a blood occlusion to qualitatively estimate oxygen saturation. Preliminary results demonstrate that both DOLP and DOCP reveal a distinction of oxygen saturation states. These results demonstrate the feasibility of the PHSI system for distinguishing between optical properties of tissues, which has the potential to reveal disease-related information for diverse medical applications.
Collapse
Affiliation(s)
- Ling Ma
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Akhila Srinivas
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Abirami Krishnamurthy
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Ximing Zhou
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | | | - Girgis Obaid
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
24
|
Tran MH, Gomez O, Fei B. An automatic whole-slide hyperspectral imaging microscope. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12391:1239106. [PMID: 38481979 PMCID: PMC10932749 DOI: 10.1117/12.2650815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
Whole slide imaging (WSI) is a common step used in histopathology to quickly digitize stained histological slides. Digital whole-slide images not only improve the efficiency of labeling but also open the door for computer-aided diagnosis, specifically machine learning-based methods. Hyperspectral imaging (HSI) is an imaging modality that captures data in various wavelengths, some beyond the range of visible lights. In this study, we developed and implemented an automated microscopy system that can acquire hyperspectral whole slide images (HWSI). The system is robust since it consists of parts that can be swapped and bought from different manufacturers. We used the automated system and built a database of 49 HWSI of thyroid cancer. The automatic whole-slide hyperspectral imaging microscope can have many potential applications in biological and medical areas.
Collapse
Affiliation(s)
- Minh Ha Tran
- The Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Ofelia Gomez
- The Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Baowei Fei
- The Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
25
|
Zhou X, Mubarak HK, Ma L, Palsgrove D, Sumer BD, Fei B. Polarized hyperspectral microscopic imaging for collagen visualization on pathologic slides of head and neck squamous cell carcinoma. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12382:1238204. [PMID: 38481487 PMCID: PMC10932728 DOI: 10.1117/12.2655831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
We developed a polarized hyperspectral microscope to collect four types of Stokes vector data cubes (S0, S1, S2, and S3) of the pathologic slides with head and neck squamous cell carcinoma (HNSCC). Our system consists of an optical light microscope with a movable stage, two polarizers, two liquid crystal variable retarders (LCVRs), and a SnapScan hyperspectral camera. The polarizers and LCVRs work in tandem with the hyperspectral camera to acquire polarized hyperspectral images. Synthetic pseudo-RGB images are generated from the four Stokes vector data cubes based on a transformation function similar to the spectral response of human eye for the visualization of hyperspectral images. Collagen is the most abundant extracellular matrix (ECM) protein in the human body. A major focus of studying the ECM in tumor microenvironment is the role of collagen in both normal and abnormal function. Collagen tends to accumulate in and around tumors during cancer development and growth. In this study, we acquired images from normal regions containing normal cells and collagen fibers and from tumor regions containing cancerous squamous cells and collagen fibers on HNSCC pathologic slides. The preliminary results demonstrated that our customized polarized hyperspectral microscope is able to improve the visualization of collagen on HNSCC pathologic slides under different situations, including thick fibers of normal stroma, thin fibers of normal stroma, fibers of normal muscle cells, fibers accumulated in tumors, fibers accumulated around tumors. Our preliminary results also demonstrated that the customized polarized hyperspectral microscope is capable of extracting the spectral signatures of collagen based on Stokes vector parameters and can have various applications in pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
26
|
Mat Lazim N, Kandhro AH, Menegaldo A, Spinato G, Verro B, Abdullah B. Autofluorescence Image-Guided Endoscopy in the Management of Upper Aerodigestive Tract Tumors. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 20:159. [PMID: 36612479 PMCID: PMC9819287 DOI: 10.3390/ijerph20010159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 12/16/2022] [Accepted: 12/20/2022] [Indexed: 06/17/2023]
Abstract
At this juncture, autofluorescence and narrow-band imaging have resurfaced in the medicine arena in parallel with current technology advancement. The emergence of newly developed optical instrumentation in addition to the discovery of new fluorescence biomolecules have contributed to a refined management of diseases and tumors, especially in the management of upper aerodigestive tract tumors. The advancement in multispectral imaging and micro-endoscopy has also escalated the trends further in the setting of the management of this tumor, in order to gain not only the best treatment outcomes but also facilitate early tumor diagnosis. This includes the usage of autofluorescence endoscopy for screening, diagnosis and treatment of this tumor. This is crucial, as microtumoral deposit at the periphery of the gross tumor can be only assessed via an enhanced endoscopy and even more precisely with autofluorescence endoscopic techniques. Overall, with this new technique, optimum management can be achieved for these patients. Hence, the treatment outcomes can be improved and patients are able to attain better prognosis and survival.
Collapse
Affiliation(s)
- Norhafiza Mat Lazim
- Department of Otorhinolaryngology-Head and Neck Surgery, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kubang Kerian 16150, Malaysia
| | - Abdul Hafeez Kandhro
- Institute of Medical Technology, Jinnah Sindh Medical University, Karachi 75510, Pakistan
| | - Anna Menegaldo
- Department of Neurosciences, Section of Otolaryngology and Regional Centre for Head and Neck Cancer, University of Padova, 31100 Treviso, Italy
| | - Giacomo Spinato
- Department of Neurosciences, Section of Otolaryngology and Regional Centre for Head and Neck Cancer, University of Padova, 31100 Treviso, Italy
- Department of Surgery, Oncology and Gastroenterology, Section of Oncology and Immunology, University of Padova, 31100 Treviso, Italy
| | - Barbara Verro
- Division of Otorhinolaryngology, Department of Biomedicine, Neuroscience and Advanced Diagnostic, University of Palermo, 90127 Palermo, Italy
| | - Baharudin Abdullah
- Department of Otorhinolaryngology-Head and Neck Surgery, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kubang Kerian 16150, Malaysia
| |
Collapse
|
27
|
Cui R, Yu H, Xu T, Xing X, Cao X, Yan K, Chen J. Deep Learning in Medical Hyperspectral Images: A Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22249790. [PMID: 36560157 PMCID: PMC9784550 DOI: 10.3390/s22249790] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 12/11/2022] [Accepted: 12/12/2022] [Indexed: 06/13/2023]
Abstract
With the continuous progress of development, deep learning has made good progress in the analysis and recognition of images, which has also triggered some researchers to explore the area of combining deep learning with hyperspectral medical images and achieve some progress. This paper introduces the principles and techniques of hyperspectral imaging systems, summarizes the common medical hyperspectral imaging systems, and summarizes the progress of some emerging spectral imaging systems through analyzing the literature. In particular, this article introduces the more frequently used medical hyperspectral images and the pre-processing techniques of the spectra, and in other sections, it discusses the main developments of medical hyperspectral combined with deep learning for disease diagnosis. On the basis of the previous review, tne limited factors in the study on the application of deep learning to hyperspectral medical images are outlined, promising research directions are summarized, and the future research prospects are provided for subsequent scholars.
Collapse
Affiliation(s)
- Rong Cui
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - He Yu
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
- Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun 130022, China
| | - Tingfa Xu
- Image Engineering & Video Technology Lab, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Institute of Technology Chongqing Innovation Center, Chongqing 401120, China
| | - Xiaoxue Xing
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
- Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun 130022, China
| | - Xiaorui Cao
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - Kang Yan
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - Jiexi Chen
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| |
Collapse
|
28
|
Wako BD, Dese K, Ulfata RE, Nigatu TA, Turunbedu SK, Kwa T. Squamous Cell Carcinoma of Skin Cancer Margin Classification From Digital Histopathology Images Using Deep Learning. Cancer Control 2022; 29:10732748221132528. [PMID: 36194624 PMCID: PMC9536105 DOI: 10.1177/10732748221132528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022] Open
Abstract
Objectives Now a days, squamous cell carcinoma (SCC) margin assessment is done by examining histopathology images and inspection of whole slide images (WSI) using a conventional microscope. This is time-consuming, tedious, and depends on experts’ experience which may lead to misdiagnosis and mistreatment plans. This study aims to develop a system for the automatic diagnosis of skin cancer margin for squamous cell carcinoma from histopathology microscopic images by applying deep learning techniques. Methods The system was trained, validated, and tested using histopathology images of SCC cancer locally acquired from Jimma Medical Center Pathology Department from seven different skin sites using an Olympus digital microscope. All images were preprocessed and trained with transfer learning pre-trained models by fine-tuning the hyper-parameter of the selected models. Results The overall best training accuracy of the models become 95.3%, 97.1%, 89.8%, and 89.9% on EffecientNetB0, MobileNetv2, ResNet50, VGG16 respectively. In addition to this, the best validation accuracy of the models was 94.7%, 91.8%, 87.8%, and 86.7% respectively. The best testing accuracy of the models at the same epoch was 95.2%, 91.5%, 87%, and 85.5% respectively. From these models, EfficientNetB0 showed the best average training and testing accuracy than the other models. Conclusions The system assists the pathologist during the margin assessment of SCC by decreasing the diagnosis time from an average of 25 minutes to less than a minute.
Collapse
Affiliation(s)
- Beshatu Debela Wako
- School of Biomedical Engineering, Jimma Institute of Technology, Jimma University, Jimma, Ethiopia,Center of Biomedical Engineering, Jimma University Medical Center, Jimma, Ethiopia
| | - Kokeb Dese
- School of Biomedical Engineering, Jimma Institute of Technology, Jimma University, Jimma, Ethiopia,Artificial Intelligence and Biomedical Imaging Research Lab, Jimma Institute of Technology, Jimma University, Jimma, Ethiopia,Kokeb Dese, Department of Biomedical Engineering, Jimma University, Jimma 378, Ethiopia. ,
| | - Roba Elala Ulfata
- Department of Pathology, Jimma Institute of Health, Jimma University, Jimma, Ethiopia,Department of Pathology, Adama General Hospital and Medical College, Adama, Ethiopia
| | - Tilahun Alemayehu Nigatu
- Department of Biomedical Sciences (Anatomy Course Unit), Jimma Institute of Health, Jimma University, Jimma, Ethiopia
| | | | - Timothy Kwa
- School of Biomedical Engineering, Jimma Institute of Technology, Jimma University, Jimma, Ethiopia,Department of Biomedical Engineering, University of California, 451 Health Sciences, Davis, CA, USA,Medtronic MiniMed, 18000 Devonshire St., Northridge, Los Angeles, CA, USA,Timothy Kwa, Department of Biomedical Engineering, Jimma University, Jimma 378, Ethiopia.
| |
Collapse
|
29
|
Du X, Koronyo Y, Mirzaei N, Yang C, Fuchs DT, Black KL, Koronyo-Hamaoui M, Gao L. Label-free hyperspectral imaging and deep-learning prediction of retinal amyloid β-protein and phosphorylated tau. PNAS NEXUS 2022; 1:pgac164. [PMID: 36157597 PMCID: PMC9491695 DOI: 10.1093/pnasnexus/pgac164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 08/15/2022] [Indexed: 01/16/2023]
Abstract
Alzheimer's disease (AD) is a major risk for the aging population. The pathological hallmarks of AD-an abnormal deposition of amyloid β-protein (Aβ) and phosphorylated tau (pTau)-have been demonstrated in the retinas of AD patients, including in prodromal patients with mild cognitive impairment (MCI). Aβ pathology, especially the accumulation of the amyloidogenic 42-residue long alloform (Aβ42), is considered an early and specific sign of AD, and together with tauopathy, confirms AD diagnosis. To visualize retinal Aβ and pTau, state-of-the-art methods use fluorescence. However, administering contrast agents complicates the imaging procedure. To address this problem from fundamentals, ex-vivo studies were performed to develop a label-free hyperspectral imaging method to detect the spectral signatures of Aβ42 and pS396-Tau, and predicted their abundance in retinal cross-sections. For the first time, we reported the spectral signature of pTau and demonstrated an accurate prediction of Aβ and pTau distribution powered by deep learning. We expect our finding will lay the groundwork for label-free detection of AD.
Collapse
Affiliation(s)
- Xiaoxi Du
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA 90095, USA
| | - Yosef Koronyo
- Department of Neurosurgery, Maxine Dunitz Neurosurgical Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA
| | - Nazanin Mirzaei
- Department of Neurosurgery, Maxine Dunitz Neurosurgical Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA
| | - Chengshuai Yang
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA 90095, USA
| | - Dieu-Trang Fuchs
- Department of Neurosurgery, Maxine Dunitz Neurosurgical Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA
| | - Keith L Black
- Department of Neurosurgery, Maxine Dunitz Neurosurgical Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA
| | - Maya Koronyo-Hamaoui
- Department of Neurosurgery, Maxine Dunitz Neurosurgical Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA
- Department of Biomedical Sciences, Division of Applied Cell Biology and Physiology, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA
| | - Liang Gao
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA 90095, USA
| |
Collapse
|
30
|
Early Diagnosis of Oral Squamous Cell Carcinoma Based on Histopathological Images Using Deep and Hybrid Learning Approaches. Diagnostics (Basel) 2022; 12:diagnostics12081899. [PMID: 36010249 PMCID: PMC9406837 DOI: 10.3390/diagnostics12081899] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 07/30/2022] [Accepted: 08/03/2022] [Indexed: 11/17/2022] Open
Abstract
Oral squamous cell carcinoma (OSCC) is one of the most common head and neck cancer types, which is ranked the seventh most common cancer. As OSCC is a histological tumor, histopathological images are the gold diagnosis standard. However, such diagnosis takes a long time and high-efficiency human experience due to tumor heterogeneity. Thus, artificial intelligence techniques help doctors and experts to make an accurate diagnosis. This study aimed to achieve satisfactory results for the early diagnosis of OSCC by applying hybrid techniques based on fused features. The first proposed method is based on a hybrid method of CNN models (AlexNet and ResNet-18) and the support vector machine (SVM) algorithm. This method achieved superior results in diagnosing the OSCC data set. The second proposed method is based on the hybrid features extracted by CNN models (AlexNet and ResNet-18) combined with the color, texture, and shape features extracted using the fuzzy color histogram (FCH), discrete wavelet transform (DWT), local binary pattern (LBP), and gray-level co-occurrence matrix (GLCM) algorithms. Because of the high dimensionality of the data set features, the principal component analysis (PCA) algorithm was applied to reduce the dimensionality and send it to the artificial neural network (ANN) algorithm to diagnose it with promising accuracy. All the proposed systems achieved superior results in histological image diagnosis of OSCC, the ANN network based on the hybrid features using AlexNet, DWT, LBP, FCH, and GLCM achieved an accuracy of 99.1%, specificity of 99.61%, sensitivity of 99.5%, precision of 99.71%, and AUC of 99.52%.
Collapse
|
31
|
Wu Y, Xu Z, Yang W, Ning Z, Dong H. Review on the Application of Hyperspectral Imaging Technology of the Exposed Cortex in Cerebral Surgery. Front Bioeng Biotechnol 2022; 10:906728. [PMID: 35711634 PMCID: PMC9196632 DOI: 10.3389/fbioe.2022.906728] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 05/09/2022] [Indexed: 11/13/2022] Open
Abstract
The study of brain science is vital to human health. The application of hyperspectral imaging in biomedical fields has grown dramatically in recent years due to their unique optical imaging method and multidimensional information acquisition. Hyperspectral imaging technology can acquire two-dimensional spatial information and one-dimensional spectral information of biological samples simultaneously, covering the ultraviolet, visible and infrared spectral ranges with high spectral resolution, which can provide diagnostic information about the physiological, morphological and biochemical components of tissues and organs. This technology also presents finer spectral features for brain imaging studies, and further provides more auxiliary information for cerebral disease research. This paper reviews the recent advance of hyperspectral imaging in cerebral diagnosis. Firstly, the experimental setup, image acquisition and pre-processing, and analysis methods of hyperspectral technology were introduced. Secondly, the latest research progress and applications of hyperspectral imaging in brain tissue metabolism, hemodynamics, and brain cancer diagnosis in recent years were summarized briefly. Finally, the limitations of the application of hyperspectral imaging in cerebral disease diagnosis field were analyzed, and the future development direction was proposed.
Collapse
Affiliation(s)
- Yue Wu
- Research Center for Intelligent Sensing Systems, Zhejiang Lab, Hangzhou, China
| | - Zhongyuan Xu
- Research Center for Intelligent Sensing Systems, Zhejiang Lab, Hangzhou, China
| | - Wenjian Yang
- Research Center for Intelligent Sensing Systems, Zhejiang Lab, Hangzhou, China
| | - Zhiqiang Ning
- Anhui Institute of Optics and Fine Mechanics, Chinese Academy of Sciences (CAS), Hefei, China.,Science Island Branch, Graduate School of USTC, Hefei, China
| | - Hao Dong
- Research Center for Sensing Materials and Devices, Zhejiang Lab, Hangzhou, China
| |
Collapse
|
32
|
Ma L, Rathgeb A, Mubarak H, Tran M, Fei B. Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-220039GRR. [PMID: 35578386 PMCID: PMC9110022 DOI: 10.1117/1.jbo.27.5.056502] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 04/28/2022] [Indexed: 06/15/2023]
Abstract
SIGNIFICANCE Hyperspectral imaging (HSI) provides rich spectral information for improved histopathological cancer detection. However, acquiring high-resolution HSI data for whole-slide imaging (WSI) can be time-consuming and requires a huge amount of storage space. AIM WSI using a color camera can be achieved with fast speed, high image resolution, and excellent image quality due to the established techniques. We aim to develop an RGB-guided unsupervised hyperspectral super-resolution reconstruction method that is hypothesized to improve image quality while maintaining the spectral characteristics. APPROACH High-resolution hyperspectral images of 32 histologic slides were obtained via automated WSI. High-resolution RGB histology images were registered to the hyperspectral images for RGB guidance. An unsupervised super-resolution network was trained to take the downsampled low-resolution hyperspectral patches (LR-HSI) and high-resolution RGB patches (HR-RGB) as inputs to reconstruct high-resolution hyperspectral patches (HR-HSI). Then, an Inception-based network was trained with the HR-RGB, original HR-HSI, and generated HR-HSI, respectively, for whole-slide histopathological cancer detection. RESULTS Our super-resolution reconstruction network generated high-resolution hyperspectral images with well-maintained spectral characteristics and improved image quality. Image classification using the original hyperspectral data outperformed RGB because of the extra spectral information. The generated hyperspectral image patches further improved the results. CONCLUSIONS The proposed method potentially reduces image acquisition time, saves storage space without compromising image quality, and improves the image classification performance.
Collapse
Affiliation(s)
- Ling Ma
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- Tianjin University, State Key Laboratory of Precision Measurement Technology and Instruments, Tianjin, China
| | - Armand Rathgeb
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Hasan Mubarak
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Minh Tran
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
33
|
Jong LJS, de Kruif N, Geldof F, Veluponnar D, Sanders J, Vrancken Peeters MJTFD, van Duijnhoven F, Sterenborg HJCM, Dashtbozorg B, Ruers TJM. Discriminating healthy from tumor tissue in breast lumpectomy specimens using deep learning-based hyperspectral imaging. BIOMEDICAL OPTICS EXPRESS 2022; 13:2581-2604. [PMID: 35774331 PMCID: PMC9203093 DOI: 10.1364/boe.455208] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 03/09/2022] [Accepted: 03/10/2022] [Indexed: 06/15/2023]
Abstract
Achieving an adequate resection margin during breast-conserving surgery remains challenging due to the lack of intraoperative feedback. Here, we evaluated the use of hyperspectral imaging to discriminate healthy tissue from tumor tissue in lumpectomy specimens. We first used a dataset obtained on tissue slices to develop and evaluate three convolutional neural networks. Second, we fine-tuned the networks with lumpectomy data to predict the tissue percentages of the lumpectomy resection surface. A MCC of 0.92 was achieved on the tissue slices and an RMSE of 9% on the lumpectomy resection surface. This shows the potential of hyperspectral imaging to classify the resection margins of lumpectomy specimens.
Collapse
Affiliation(s)
- Lynn-Jade S. Jong
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
- Equal contributors
| | - Naomi de Kruif
- Department of Biomedical Engineering, Eindhoven University of Technology, PO Box 513, 5600 MB Eindhoven, The Netherlands
- Equal contributors
| | - Freija Geldof
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Dinusha Veluponnar
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Joyce Sanders
- Department of Pathology, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| | | | - Frederieke van Duijnhoven
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| | - Henricus J. C. M. Sterenborg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
| | - Behdad Dashtbozorg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Biomedical Engineering, Eindhoven University of Technology, PO Box 513, 5600 MB Eindhoven, The Netherlands
| | - Theo J. M. Ruers
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| |
Collapse
|
34
|
Tumor cell identification and classification in esophageal adenocarcinoma specimens by hyperspectral imaging. Sci Rep 2022; 12:4508. [PMID: 35296685 PMCID: PMC8927097 DOI: 10.1038/s41598-022-07524-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Accepted: 02/17/2022] [Indexed: 12/24/2022] Open
Abstract
Esophageal cancer is the sixth leading cause of cancer-related death worldwide. Histopathological confirmation is a key step in tumor diagnosis. Therefore, simplification in decision-making by discrimination between malignant and non-malignant cells of histological specimens can be provided by combination of new imaging technology and artificial intelligence (AI). In this work, hyperspectral imaging (HSI) data from 95 patients were used to classify three different histopathological features (squamous epithelium cells, esophageal adenocarcinoma (EAC) cells, and tumor stroma cells), based on a multi-layer perceptron with two hidden layers. We achieved an accuracy of 78% for EAC and stroma cells, and 80% for squamous epithelium. HSI combined with machine learning algorithms is a promising and innovative technique, which allows image acquisition beyond Red–Green–Blue (RGB) images. Further method validation and standardization will be necessary, before automated tumor cell identification algorithms can be used in daily clinical practice.
Collapse
|
35
|
van Vliet-Pérez SM, van de Berg NJ, Manni F, Lai M, Rijstenberg L, Hendriks BHW, Dankelman J, Ewing-Graham PC, Nieuwenhuyzen-de Boer GM, van Beekhuizen HJ. Hyperspectral Imaging for Tissue Classification after Advanced Stage Ovarian Cancer Surgery-A Pilot Study. Cancers (Basel) 2022; 14:cancers14061422. [PMID: 35326577 PMCID: PMC8946803 DOI: 10.3390/cancers14061422] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Revised: 02/24/2022] [Accepted: 03/08/2022] [Indexed: 02/05/2023] Open
Abstract
The most important prognostic factor for the survival of advanced-stage epithelial ovarian cancer (EOC) is the completeness of cytoreductive surgery (CRS). Therefore, an intraoperative technique to detect microscopic tumors would be of great value. The aim of this pilot study is to assess the feasibility of near-infrared hyperspectral imaging (HSI) for EOC detection in ex vivo tissue samples. Images were collected during CRS in 11 patients in the wavelength range of 665−975 nm, and processed by calibration, normalization, and noise filtering. A linear support vector machine (SVM) was employed to classify healthy and tumorous tissue (defined as >50% tumor cells). Classifier performance was evaluated using leave-one-out cross-validation. Images of 26 tissue samples from 10 patients were included, containing 26,446 data points that were matched to histopathology. Tumorous tissue could be classified with an area under the curve of 0.83, a sensitivity of 0.81, a specificity of 0.70, and Matthew’s correlation coefficient of 0.41. This study paves the way to in vivo and intraoperative use of HSI during CRS. Hyperspectral imaging can scan a whole tissue surface in a fast and non-contact way. Our pilot study demonstrates that HSI and SVM learning can be used to discriminate EOC from surrounding tissue.
Collapse
Affiliation(s)
- Sharline M. van Vliet-Pérez
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
- Department of Radiotherapy, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands
- Correspondence:
| | - Nick J. van de Berg
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
| | - Francesca Manni
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (F.M.); (M.L.)
| | - Marco Lai
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (F.M.); (M.L.)
| | - Lucia Rijstenberg
- Department of Pathology, Erasmus University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (L.R.); (P.C.E.-G.)
| | - Benno H. W. Hendriks
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
| | - Jenny Dankelman
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
| | - Patricia C. Ewing-Graham
- Department of Pathology, Erasmus University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (L.R.); (P.C.E.-G.)
| | - Gatske M. Nieuwenhuyzen-de Boer
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
- Department of Gynecology, Albert Schweitzer Hospital, 3318 AT Dordrecht, The Netherlands
| | - Heleen J. van Beekhuizen
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
| |
Collapse
|
36
|
Eggert D, Bengs M, Westermann S, Gessert N, Gerstner AOH, Mueller NA, Bewarder J, Schlaefer A, Betz C, Laffers W. In vivo detection of head and neck tumors by hyperspectral imaging combined with deep learning methods. JOURNAL OF BIOPHOTONICS 2022; 15:e202100167. [PMID: 34889065 DOI: 10.1002/jbio.202100167] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Revised: 11/25/2021] [Accepted: 11/25/2021] [Indexed: 05/24/2023]
Abstract
Currently, there are no fast and accurate screening methods available for head and neck cancer, the eighth most common tumor entity. For this study, we used hyperspectral imaging, an imaging technique for quantitative and objective surface analysis, combined with deep learning methods for automated tissue classification. As part of a prospective clinical observational study, hyperspectral datasets of laryngeal, hypopharyngeal and oropharyngeal mucosa were recorded in 98 patients before surgery in vivo. We established an automated data interpretation pathway that can classify the tissue into healthy and tumorous using convolutional neural networks with 2D spatial or 3D spatio-spectral convolutions combined with a state-of-the-art Densenet architecture. Using 24 patients for testing, our 3D spatio-spectral Densenet classification method achieves an average accuracy of 81%, a sensitivity of 83% and a specificity of 79%.
Collapse
Affiliation(s)
- Dennis Eggert
- Clinic and Polyclinic for Otolaryngology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Marcel Bengs
- Institute of Medical Technology, Hamburg University of Technology, Hamburg, Germany
| | - Stephan Westermann
- Department of Otorhinolaryngology/Head and Neck Surgery, University of Bonn, Bonn, Germany
| | - Nils Gessert
- Institute of Medical Technology, Hamburg University of Technology, Hamburg, Germany
| | | | - Nina A Mueller
- Department of Otorhinolaryngology/Head and Neck Surgery, University of Bonn, Bonn, Germany
| | - Julian Bewarder
- Clinic and Polyclinic for Otolaryngology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Alexander Schlaefer
- Institute of Medical Technology, Hamburg University of Technology, Hamburg, Germany
| | - Christian Betz
- Clinic and Polyclinic for Otolaryngology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Wiebke Laffers
- Clinic and Polyclinic for Otolaryngology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Department of Otorhinolaryngology/Head and Neck Surgery, University of Bonn, Bonn, Germany
- Department of Otorhinolaryngology, Head and Neck Surgery, Evangelisches Krankenhaus, Carl von Ossietzky-University, Oldenburg, Germany
| |
Collapse
|
37
|
Zhou X, Ma L, Mubarak HK, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and deep learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12039:120390G. [PMID: 36798940 PMCID: PMC9930132 DOI: 10.1117/12.2614624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
The study is to incorporate polarized hyperspectral imaging (PHSI) with deep learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we firstly collected the Stokes vector data cubes (S0, S1, S2, and S3) of histologic slides from 17 patients with SCC by the PHSI microscope, under the wavelength range from 467 nm to 750 nm. Secondly, we generated the synthetic RGB images from the original Stokes vector data cubes. Thirdly, we cropped the synthetic RGB images into image patches at the image size of 96×96 pixels, and then set up a ResNet50-based convolutional neural network (CNN) to classify the image patches of the four Stokes vector parameters (S0, S1, S2, and S3) by application of transfer learning. To test the performances of the model, each time we trained the model based on the image patches (S0, S1, S2, and S3) of 16 patients out of 17 patients, and used the trained model to calculate the testing accuracy based on the image patches of the rest 1 patient (S0, S1, S2, and S3). We repeated the process for 6 times and obtained 24 testing accuracies (S0, S1, S2, and S3) from 6 different patients out of the 17 patients. The preliminary results showed that the average testing accuracy (84.2%) on S3 outperformed the average testing accuracy (83.5%) on S0. Furthermore, 4 of 6 testing accuracies of S3 (96.0%, 87.3%, 82.8%, and 86.7%) outperformed the testing accuracies of S0 (93.3%, 85.2%, 80.2%, and 79.0%). The study demonstrated the potential of using polarized hyperspectral imaging and deep learning for automatic detection of head and neck SCC on pathologic slides.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K Mubarak
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - James V. Little
- Emory University, Department of Pathology and Laboratory Medicine, Atlanta, GA
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta, GA
| | - Larry L. Myers
- Univ. of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baran D. Sumer
- Univ. of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX
- Univ. of Texas Southwestern Medical Center, Dept. of Radiology, Dallas, TX
| |
Collapse
|
38
|
Tran MH, Ma L, Litter JV, Chen AY, Fei B. Thyroid Carcinoma Detection on Whole Histologic Slides Using Hyperspectral Imaging and Deep Learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12039:120390H. [PMID: 36798939 PMCID: PMC9929647 DOI: 10.1117/12.2612963] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Hyperspectral imaging (HSI), a non-invasive imaging modality, has been successfully used in many different biological and medical applications. One such application is in the field of oncology, where hyperspectral imaging is being used on histologic samples. This study compares the performances of different image classifiers using different imaging modalities as training data. From a database of 33 fixed tissues from head and neck patients with follicular thyroid carcinoma, we produced three different datasets: an RGB image dataset that was acquired from a whole slide image scanner, a hyperspectral (HS) dataset that was acquired with a compact hyperspectral camera, and an HS-synthesized RGB image dataset. Three separate deep learning classifiers were trained using the three datasets. We show that the deep learning classifier trained on HSI data has an area under the receiver operator characteristic curve (AUC-ROC) of 0.966, higher than that of the classifiers trained on RGB and HSI-synthesized RGB data. This study demonstrates that hyperspectral images improve the performance of cancer classification on whole histologic slides. Hyperspectral imaging and deep learning provide an automatic tool for thyroid cancer detection on whole histologic slides.
Collapse
Affiliation(s)
- Minh Ha Tran
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX
- Univ. of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, TX
| | - Ling Ma
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX
- Univ. of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, TX
| | - James V. Litter
- Emory Univ. School of Medicine, Dept. of Pathology and Laboratory Medicine, Atlanta, GA
| | - Amy Y. Chen
- Emory Univ. School of Medicine, Dept. of Otolaryngology, Atlanta, GA
| | - Baowei Fei
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX
- Univ. of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, TX
- Univ. of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
39
|
Abstract
AbstractMeasuring morphological and biochemical features of tissue is crucial for disease diagnosis and surgical guidance, providing clinically significant information related to pathophysiology. Hyperspectral imaging (HSI) techniques obtain both spatial and spectral features of tissue without labeling molecules such as fluorescent dyes, which provides rich information for improved disease diagnosis and treatment. Recent advances in HSI systems have demonstrated its potential for clinical applications, especially in disease diagnosis and image-guided surgery. This review summarizes the basic principle of HSI and optical systems, deep-learning-based image analysis, and clinical applications of HSI to provide insight into this rapidly growing field of research. In addition, the challenges facing the clinical implementation of HSI techniques are discussed.
Collapse
|
40
|
Modir N, Shahedi M, Dormer J, Ma L, Ghaderi M, Sirsi S, Cheng YSL, Fei B. LED-based Hyperspectral Endoscopic Imaging. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 11954:1195408. [PMID: 36794092 PMCID: PMC9928531 DOI: 10.1117/12.2609023] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Hyperspectral endoscopy can offer multiple advantages as compared to conventional endoscopy. Our goal is to design and develop a real-time hyperspectral endoscopic imaging system for the diagnosis of gastrointestinal (GI) tract cancers using a micro-LED array as an in-situ illumination source. The wavelengths of the system range from ultraviolet to visible and near infrared. To evaluate the use of the LED array for hyperspectral imaging, we designed a prototype system and conducted ex vivo experiments using normal and cancerous tissues of mice, chicken, and sheep. We compared the results of our LED-based approach with our reference hyperspectral camera system. The results confirm the similarity between the LED-based hyperspectral imaging system and the reference HSI camera. Our LED-based hyperspectral imaging system can be used not only as an endoscope but also as a laparoscopic or handheld devices for cancer detection and surgery.
Collapse
Affiliation(s)
- Naeeme Modir
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Maysam Shahedi
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - James Dormer
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Mohammadaref Ghaderi
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Shashank Sirsi
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Yi-Shing Lisa Cheng
- Department of Diagnostic Sciences, College of Dentistry, Texas A&M University, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
41
|
Radiomics and deep learning approach to the differential diagnosis of parotid gland tumors. Curr Opin Otolaryngol Head Neck Surg 2021; 30:107-113. [PMID: 34907957 DOI: 10.1097/moo.0000000000000782] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
PURPOSE OF REVIEW Advances in computer technology and growing expectations from computer-aided systems have led to the evolution of artificial intelligence into subsets, such as deep learning and radiomics, and the use of these systems is revolutionizing modern radiological diagnosis. In this review, artificial intelligence applications developed with radiomics and deep learning methods in the differential diagnosis of parotid gland tumors (PGTs) will be overviewed. RECENT FINDINGS The development of artificial intelligence models has opened new scenarios owing to the possibility of assessing features of medical images that usually are not evaluated by physicians. Radiomics and deep learning models come to the forefront in computer-aided diagnosis of medical images, even though their applications in the differential diagnosis of PGTs have been limited because of the scarcity of data sets related to these rare neoplasms. Nevertheless, recent studies have shown that artificial intelligence tools can classify common PGTs with reasonable accuracy. SUMMARY All studies aimed at the differential diagnosis of benign vs. malignant PGTs or the identification of the commonest PGT subtypes were identified, and five studies were found that focused on deep learning-based differential diagnosis of PGTs. Data sets were created in three of these studies with MRI and in two with computed tomography (CT). Additional seven studies were related to radiomics. Of these, four were on MRI-based radiomics, two on CT-based radiomics, and one compared MRI and CT-based radiomics in the same patients.
Collapse
|
42
|
Horgan C, Jensen M, Nagelkerke A, St-Pierre JP, Vercauteren T, Stevens MM, Bergholt MS. High-Throughput Molecular Imaging via Deep-Learning-Enabled Raman Spectroscopy. Anal Chem 2021; 93:15850-15860. [PMID: 34797972 PMCID: PMC9286315 DOI: 10.1021/acs.analchem.1c02178] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
Raman spectroscopy enables nondestructive, label-free imaging with unprecedented molecular contrast, but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep-learning-enabled Raman spectroscopy, termed DeepeR, trained on a large data set of hyperspectral Raman images, with over 1.5 million spectra (400 h of acquisition) in total. We first perform denoising and reconstruction of low signal-to-noise ratio Raman molecular signatures via deep learning, with a 10× improvement in the mean-squared error over common Raman filtering methods. Next, we develop a neural network for robust 2-4× spatial super-resolution of hyperspectral Raman images that preserve molecular cellular information. Combining these approaches, we achieve Raman imaging speed-ups of up to 40-90×, enabling good-quality cellular imaging with a high-resolution, high signal-to-noise ratio in under 1 min. We further demonstrate Raman imaging speed-up of 160×, useful for lower resolution imaging applications such as the rapid screening of large areas or for spectral pathology. Finally, transfer learning is applied to extend DeepeR from cell to tissue-scale imaging. DeepeR provides a foundation that will enable a host of higher-throughput Raman spectroscopy and molecular imaging applications across biomedicine.
Collapse
Affiliation(s)
- Conor
C. Horgan
- Centre
for Craniofacial and Regenerative Biology, King’s College London, London SE1 9RT, U.K.
- Department
of Materials, Department of Bioengineering, and Institute of Biomedical
Engineering, Imperial College London, London SW7 2AZ, U.K.
| | - Magnus Jensen
- Centre
for Craniofacial and Regenerative Biology, King’s College London, London SE1 9RT, U.K.
| | - Anika Nagelkerke
- Groningen
Research Institute of Pharmacy, Pharmaceutical Analysis, University of Groningen, P.O. Box 196, XB20, Groningen 9700 AD, The Netherlands
- Department
of Materials, Department of Bioengineering, and Institute of Biomedical
Engineering, Imperial College London, London SW7 2AZ, U.K.
| | - Jean-Philippe St-Pierre
- Department
of Chemical and Biological Engineering, University of Ottawa, Ottawa, Ontario K1N 6N5, Canada
- Department
of Materials, Department of Bioengineering, and Institute of Biomedical
Engineering, Imperial College London, London SW7 2AZ, U.K.
| | - Tom Vercauteren
- School
of Biomedical Engineering and Imaging Sciences, King’s College London, London WC2R 2LS, U.K.
| | - Molly M. Stevens
- Department
of Materials, Department of Bioengineering, and Institute of Biomedical
Engineering, Imperial College London, London SW7 2AZ, U.K.
| | - Mads S. Bergholt
- Centre
for Craniofacial and Regenerative Biology, King’s College London, London SE1 9RT, U.K.
| |
Collapse
|
43
|
Asoda S, Miyashita H, Soma T, Munakata K, Yamada Y, Yasui Y, Kudo Y, Usuda S, Hasegawa T, Nakagawa T, Kawana H. Clinical value of entire-circumferential intraoperative frozen section analysis for the complete resection of superficial squamous cell carcinoma of the tongue. Oral Oncol 2021; 123:105629. [PMID: 34784507 DOI: 10.1016/j.oraloncology.2021.105629] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Revised: 10/09/2021] [Accepted: 11/09/2021] [Indexed: 10/19/2022]
Abstract
OBJECTIVES We aimed to evaluate the clinical value of an entire-circumferential intraoperative frozen section analysis (e-IFSA) for the complete resection of superficial squamous cell carcinoma (SCC) of the tongue. MATERIALS AND METHODS A total 276 specimens from 51 patients with pT1-2, N0, mucosal or submucosal invasion SCC were analyzed to evaluate the diagnostic accuracy of the e-IFSA and the added value of the e-IFSA to iodine staining. The e-IFSA results were compared with the final histologic results obtained using permanent sections. All specimens for the e-IFSA were taken over the entire circumference 5 mm outside from the iodine unstained areas. The outline of the main resected specimen after taking these outer mucosal specimens were defined as the surgical margins determined by iodine staining alone. RESULTS The e-IFSA results were in excellent agreement with final histological results (Cohen's kappa value: 0.85) and the e-IFSA showed high sensitivity (100%) and high negative predictive value (100%). The actual complete resection rate with an e-IFSA was 100% (51/51), and no patient required additional resection after surgery. In contrast, 10/51 patients (20%) patients showed residual atypical mucosal epithelium at or beyond the margin determined by iodine staining alone; this difference was statistically significant (P = 0.002). The 5-year local control rate and 5-year overall survival rate after this procedure were both 100%. CONCLUSION An e-IFSA has additional value when performed in conjunction with iodine staining. An e-IFSA would be useful for achieving complete resection of superficial SCC of the tongue.
Collapse
Affiliation(s)
- Seiji Asoda
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Hidetaka Miyashita
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Tomoya Soma
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Kanako Munakata
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Yuka Yamada
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Yuta Yasui
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Yoko Kudo
- Department of Dent-oral Anesthesiology, Tohoku University Graduate School of Dentistry, Sendai, Japan
| | - Shin Usuda
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Toshihiro Hasegawa
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Taneaki Nakagawa
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan
| | - Hiromasa Kawana
- Department of Dentistry and Oral Surgery, Keio University School of Medicine, Tokyo, Japan; Department of Oral and Maxillofacial Implantology, Kanagawa Dental University, Yokosuka, Japan.
| |
Collapse
|
44
|
Aloupogianni E, Ishikawa M, Ichimura T, Sasaki A, Kobayashi N, Obi T. Design of a Hyper-Spectral Imaging System for Gross Pathology of Pigmented Skin Lesions. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3605-3608. [PMID: 34892018 DOI: 10.1109/embc46164.2021.9629512] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
Pigmented skin lesions (PSL) are prevalent in Asian populations and their gross pathology remains a manual, tedious task. Hyper-spectral imaging (HSI) is a non-invasive non-ionizing acquisition technique, allowing malignant tissue to be identified by its spectral signature. We set up a hyper-spectral imaging (HSI) system targeting cancer margin detection of PSL. Because classification among PSL is achieved via comparison of spectral signatures, appropriate calibration is necessary to ensure sufficient data quality. We propose a strategy for system building, calibration and pre-processing, under the requirements of fast acquisition and wide field of view. Preliminary results show that the HSI-based system is able to effectively resolve reflectance signatures of ex-vivo tissue.Clinical Relevance-The imaging system proposed in this study can recover reflectance spectra from PSL during gross pathology, providing a wide imaging area.
Collapse
|
45
|
New Approach to the Old Challenge of Free Flap Monitoring-Hyperspectral Imaging Outperforms Clinical Assessment by Earlier Detection of Perfusion Failure. J Pers Med 2021; 11:jpm11111101. [PMID: 34834453 PMCID: PMC8625540 DOI: 10.3390/jpm11111101] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Revised: 10/24/2021] [Accepted: 10/26/2021] [Indexed: 01/18/2023] Open
Abstract
In reconstructive surgery, free flap failure, especially in complex osteocutaneous reconstructions, represents a significant clinical burden. Therefore, the aim of the presented study was to assess hyperspectral imaging (HSI) for monitoring of free flaps compared to clinical monitoring. In a prospective, non-randomized clinical study, patients with free flap reconstruction of the oro-maxillofacial-complex were included. Monitoring was assessed clinically and by using hyperspectral imaging (TIVITA™ Tissue-System, DiaspectiveVision GmbH, Pepelow, Germany) to determine tissue-oxygen-saturation [StO2], near-infrared-perfusion-index [NPI], distribution of haemoglobin [THI] and water [TWI], and variance to an adjacent reference area (Δreference). A total of 54 primary and 11 secondary reconstructions were performed including fasciocutaneous and osteocutaneous flaps. Re-exploration was performed in 19 cases. A total of seven complete flap failures occurred, resulting in a 63% salvage rate. Mean time from flap inset to decision making for re-exploration based on clinical assessment was 23.1 ± 21.9 vs. 18.2 ± 19.4 h by the appearance of hyperspectral criteria indicating impaired perfusion (StO2 ≤ 32% OR StO2Δreference > −38% OR NPI ≤ 32.9 OR NPIΔreference ≥ −13.4%) resulting in a difference of 4.8 ± 5 h (p < 0.001). HSI seems able to detect perfusion compromise significantly earlier than clinical monitoring. These findings provide an interpretation aid for clinicians to simplify postoperative flap monitoring.
Collapse
|
46
|
Collins T, Maktabi M, Barberio M, Bencteux V, Jansen-Winkeln B, Chalopin C, Marescaux J, Hostettler A, Diana M, Gockel I. Automatic Recognition of Colon and Esophagogastric Cancer with Machine Learning and Hyperspectral Imaging. Diagnostics (Basel) 2021; 11:diagnostics11101810. [PMID: 34679508 PMCID: PMC8535008 DOI: 10.3390/diagnostics11101810] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Revised: 09/18/2021] [Accepted: 09/23/2021] [Indexed: 01/23/2023] Open
Abstract
There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models.
Collapse
Affiliation(s)
- Toby Collins
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
- Correspondence:
| | - Marianne Maktabi
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (M.M.); (C.C.)
| | - Manuel Barberio
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
- General Surgery Department, Card. G. Panico, 73039 Tricase, Italy
| | - Valentin Bencteux
- ICUBE Laboratory, Photonics Instrumentation for Health, University of Strasbourg, 67400 Strasbourg, France;
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (B.J.-W.); (I.G.)
| | - Claire Chalopin
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (M.M.); (C.C.)
| | - Jacques Marescaux
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
| | - Alexandre Hostettler
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
| | - Michele Diana
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.B.); (J.M.); (A.H.); (M.D.)
- ICUBE Laboratory, Photonics Instrumentation for Health, University of Strasbourg, 67400 Strasbourg, France;
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, 67091 Strasbourg, France
- INSERM, Institute of Viral and Liver Disease, 67091 Strasbourg, France
- Mitochondrion, Oxidative Stress and Muscle Protection (MSP)-EA 3072, Institute of Physiology, Faculty of Medicine, University of Strasbourg, 67085 Strasbourg, France
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (B.J.-W.); (I.G.)
| |
Collapse
|
47
|
Linek M, Felicio-Briegel A, Freymüller C, Rühm A, Englhard AS, Sroka R, Volgger V. Evaluation of hyperspectral imaging to quantify perfusion changes during the modified Allen test. Lasers Surg Med 2021; 54:245-255. [PMID: 34541694 DOI: 10.1002/lsm.23479] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 08/29/2021] [Indexed: 12/20/2022]
Abstract
OBJECTIVES To evaluate the capability of hyperspectral imaging (HSI), a contact-less and noninvasive technology, to monitor perfusion changes of the hand during a modified Allen test (MAT) and cuff occlusion test. Furthermore, the study aimed at obtaining objective perfusion parameters of the hand. METHODS HSI of the hand was performed on 20 healthy volunteers with a commercially available HSI system during a MAT and a cuff occlusion test. Besides gathering red-green-blue (RGB) images, the perfusion parameters tissue hemoglobin index (THI), (superficial tissue) hemoglobin oxygenation (StO2), near-infrared perfusion (NIR), and tissue water index (TWI) were calculated for four different regions of interest on the hand. For the MAT, occlusion (OI; the ratio between the condition during occlusion and before occlusion) and reperfusion (RI; the ratio between the non-occlusion state and the prior occlusion state) indices were calculated for each perfusion parameter. All data were correlated to the clinical findings. RESULTS False-color images showed visible differences between the various perfusion conditions during the MAT and cuff occlusion test. THI, StO2, and NIR behaved as expected from physiology, while TWI did not in the context of this study. During rest, mean THI, StO2, and NIR of the hand were 34 ± 2, 72 ± 9, and 61 ± 6, respectively. The RI for THI showed a roundabout threefold increase after reperfusion of both radial and ulnar artery and was thus, distinctly pronounced when compared with StO2 and NIR (~1.25). The OI was lowest for THI when compared with StO2 and NIR. CONCLUSIONS HSI with its parameters THI, StO2, and NIR proved to be suitable to evaluate perfusion of the hand. By this, it could complement visual inspection during the MAT for evaluating the functionality of the superficial palmary arch before radial or ulnar artery harvest. The presented RI might deliver useful comparative values to detect pathological perfusion disorders at an early stage. As microcirculation monitoring is crucial for many medical issues, HSI shows potential to be used, besides further applications, in the monitoring of (free) flaps and transplants and microcirculation monitoring of critically ill patients.
Collapse
Affiliation(s)
- Matthäus Linek
- Laser-Forschungslabor, LIFE Center, University Hospital, LMU Munich, Planegg, Germany
| | | | - Christian Freymüller
- Laser-Forschungslabor, LIFE Center, University Hospital, LMU Munich, Planegg, Germany
| | - Adrian Rühm
- Laser-Forschungslabor, LIFE Center, University Hospital, LMU Munich, Planegg, Germany.,Department of Urology, University Hospital, LMU Munich, Munich, Germany
| | - Anna Sophie Englhard
- Department of Otorhinolaryngology, University Hospital, LMU Munich, Munich, Germany
| | - Ronald Sroka
- Laser-Forschungslabor, LIFE Center, University Hospital, LMU Munich, Planegg, Germany.,Department of Urology, University Hospital, LMU Munich, Munich, Germany
| | - Veronika Volgger
- Department of Otorhinolaryngology, University Hospital, LMU Munich, Munich, Germany
| |
Collapse
|
48
|
Barberio M, Collins T, Bencteux V, Nkusi R, Felli E, Viola MG, Marescaux J, Hostettler A, Diana M. Deep Learning Analysis of In Vivo Hyperspectral Images for Automated Intraoperative Nerve Detection. Diagnostics (Basel) 2021; 11:1508. [PMID: 34441442 PMCID: PMC8391550 DOI: 10.3390/diagnostics11081508] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Revised: 07/27/2021] [Accepted: 08/09/2021] [Indexed: 12/16/2022] Open
Abstract
Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.
Collapse
Affiliation(s)
- Manuel Barberio
- Department of Research, Institute of Image-Guided Surgery, IHU-Strasbourg, 67091 Strasbourg, France; (V.B.); (E.F.)
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
- Department of Surgery, Ospedale Card. G. Panico, 73039 Tricase, Italy;
| | - Toby Collins
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
| | - Valentin Bencteux
- Department of Research, Institute of Image-Guided Surgery, IHU-Strasbourg, 67091 Strasbourg, France; (V.B.); (E.F.)
| | - Richard Nkusi
- Department of Research, Research Institute against Digestive Cancer, IRCAD Africa, Kigali 2 KN 30 ST, Rwanda;
| | - Eric Felli
- Department of Research, Institute of Image-Guided Surgery, IHU-Strasbourg, 67091 Strasbourg, France; (V.B.); (E.F.)
| | | | - Jacques Marescaux
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
| | - Alexandre Hostettler
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
| | - Michele Diana
- Department of Research, Research Institute against Digestive Cancer, IRCAD, 67091 Strasbourg, France; (T.C.); (J.M.); (A.H.); (M.D.)
- ICUBE Laboratory, Photonics Instrumentation for Health, 67412 Strasbourg, France
| |
Collapse
|
49
|
[Artificial intelligence in otorhinolaryngology]. HNO 2021; 70:87-93. [PMID: 34374811 PMCID: PMC8353610 DOI: 10.1007/s00106-021-01095-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/26/2021] [Indexed: 11/24/2022]
Abstract
Hintergrund Die fortschreitende Digitalisierung ermöglicht zunehmend den Einsatz von künstlicher Intelligenz (KI). Sie wird Gesellschaft und Medizin in den nächsten Jahren maßgeblich beeinflussen. Ziel der Arbeit Darstellung des gegenwärtigen Einsatzspektrums von KI in der Hals-Nasen-Ohren-Heilkunde und Skizzierung zukünftiger Entwicklungen bei der Anwendung dieser Technologie. Material und Methoden Es erfolgte die Auswertung und Diskussion wissenschaftlicher Studien und Expertenanalysen. Ergebnisse Durch die Verwendung von KI kann der Nutzen herkömmlicher diagnostischer Werkzeuge in der Hals-Nasen-Ohren-Heilkunde gesteigert werden. Zudem kann der Einsatz dieser Technologie die chirurgische Präzision in der Kopf-Hals-Chirurgie weiter erhöhen. Schlussfolgerungen KI besitzt ein großes Potenzial zur weiteren Verbesserung diagnostischer und therapeutischer Verfahren in der Hals-Nasen-Ohren-Heilkunde. Allerdings ist die Anwendung dieser Technologie auch mit Herausforderungen verbunden, beispielsweise im Bereich des Datenschutzes.
Collapse
|
50
|
Tian L, Hunt B, Bell MAL, Yi J, Smith JT, Ochoa M, Intes X, Durr NJ. Deep Learning in Biomedical Optics. Lasers Surg Med 2021; 53:748-775. [PMID: 34015146 PMCID: PMC8273152 DOI: 10.1002/lsm.23414] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 04/02/2021] [Accepted: 04/15/2021] [Indexed: 01/02/2023]
Abstract
This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.
Collapse
Affiliation(s)
- L. Tian
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA
| | - B. Hunt
- Thayer School of Engineering, Dartmouth College, Hanover, NH, USA
| | - M. A. L. Bell
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - J. Yi
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Ophthalmology, Johns Hopkins University, Baltimore, MD, USA
| | - J. T. Smith
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - M. Ochoa
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - X. Intes
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - N. J. Durr
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|