1
|
Campbell JM, Habibalahi A, Handley S, Agha A, Mahbub SB, Anwer AG, Goldys EM. Emerging clinical applications in oncology for non-invasive multi- and hyperspectral imaging of cell and tissue autofluorescence. JOURNAL OF BIOPHOTONICS 2023; 16:e202300105. [PMID: 37272291 DOI: 10.1002/jbio.202300105] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 05/02/2023] [Accepted: 05/16/2023] [Indexed: 06/06/2023]
Abstract
Hyperspectral and multispectral imaging of cell and tissue autofluorescence is an emerging technology in which fluorescence imaging is applied to biological materials across multiple spectral channels. This produces a stack of images where each matched pixel contains information about the sample's spectral properties at that location. This allows precise collection of molecularly specific data from a broad range of native fluorophores. Importantly, complex information, directly reflective of biological status, is collected without staining and tissues can be characterised in situ, without biopsy. For oncology, this can spare the collection of biopsies from sensitive regions and enable accurate tumour mapping. For in vivo tumour analysis, the greatest focus has been on oral cancer, whereas for ex vivo assessment head-and-neck cancers along with colon cancer have been the most studied, followed by oral and eye cancer. This review details the scope and progress of research undertaken towards clinical translation in oncology.
Collapse
Affiliation(s)
- Jared M Campbell
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Abbas Habibalahi
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Shannon Handley
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Adnan Agha
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Saabah B Mahbub
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Ayad G Anwer
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Ewa M Goldys
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| |
Collapse
|
2
|
Lin Y, Tang M, Liu Y, Jiang M, He S, Zeng D, Cui MY. A narrative review on machine learning in diagnosis and prognosis prediction for tongue squamous cell carcinoma. Transl Cancer Res 2022; 11:4409-4415. [PMID: 36644177 PMCID: PMC9834582 DOI: 10.21037/tcr-22-1669] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 11/07/2022] [Indexed: 12/28/2022]
Abstract
Background Tongue squamous cell carcinoma (TSCC) is the most common subtype of oral cavity squamous cell carcinoma (OCSCC), and it also has the worst prognosis. It is crucial to find an effective way to solve the challenges in diagnosis and prognosis prediction for TSCC. Machine learning (ML) has been widely used in medical research and has shown good performance. It can be used for feature extraction, feature selection, model construction, etc. Radiomics and deep learning (DL), the new components of ML, have also been utilized to explore the relationship between image features and diseases. The current study aimed to highlight the importance of ML as a potential method for addressing the challenges in diagnosis and prognosis prediction of TSCC by reviewing studies on ML in TSCC. Methods The studies on ML in TSCC in PubMed, Scopus, Web of Science, and China National Knowledge Infrastructure published between the dates of inception of these databases and April 30, 2022, were reviewed. Key Content and Findings ML (including radiomics and DL) which was used in diagnosis and prognosis prediction for TSCC, has shown promising performance. Conclusions Despite its limitations, ML is still a potential approach that can help to deal with the challenges in diagnosis and prognosis prediction for TSCC. Nevertheless, more efforts are needed to enhance the usefulness of ML in this field.
Collapse
Affiliation(s)
- Yingyu Lin
- Department of Radiology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Mimi Tang
- Department of Radiology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Yi Liu
- Department of Radiology, Hospital of Stomatology, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Stomatology, Guangzhou, China
| | - Mengjie Jiang
- Department of Radiology, Hospital of Stomatology, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Stomatology, Guangzhou, China
| | - Shuangshuang He
- Department of Radiology, Hospital of Stomatology, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Stomatology, Guangzhou, China
| | - Donglin Zeng
- Department of Radiology, Hospital of Stomatology, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Stomatology, Guangzhou, China
| | - Min-Yi Cui
- Department of Radiology, Hospital of Stomatology, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Stomatology, Guangzhou, China
| |
Collapse
|
3
|
Zhang L, Huang D, Chen X, Zhu L, Chen X, Xie Z, Huang G, Gao J, Shi W, Cui G. Visible near-infrared hyperspectral imaging and supervised classification for the detection of small intestinal necrosis tissue in vivo. BIOMEDICAL OPTICS EXPRESS 2022; 13:6061-6080. [PMID: 36733734 PMCID: PMC9872898 DOI: 10.1364/boe.470202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 09/23/2022] [Accepted: 09/26/2022] [Indexed: 06/18/2023]
Abstract
Complete recognition of necrotic areas during small bowel tissue resection remains challenging due to the lack of optimal intraoperative aid identification techniques. This research utilizes hyperspectral imaging techniques to automatically distinguish normal and necrotic areas of small intestinal tissue. Sample data were obtained from the animal model of small intestinal tissue of eight Japanese large-eared white rabbits developed by experienced physicians. A spectral library of normal and necrotic regions of small intestinal tissue was created and processed using six different supervised classification algorithms. The results show that hyperspectral imaging combined with supervised classification algorithms can be a suitable technique to automatically distinguish between normal and necrotic areas of small intestinal tissue. This new technique could aid physicians in objectively identify normal and necrotic areas of small intestinal tissue.
Collapse
Affiliation(s)
- LeChao Zhang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - DanFei Huang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - XiaoJing Chen
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - LiBin Zhu
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - XiaoQing Chen
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - ZhongHao Xie
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuangZao Huang
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - JunZhao Gao
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - Wen Shi
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuiHua Cui
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| |
Collapse
|
4
|
van Vliet-Pérez SM, van de Berg NJ, Manni F, Lai M, Rijstenberg L, Hendriks BHW, Dankelman J, Ewing-Graham PC, Nieuwenhuyzen-de Boer GM, van Beekhuizen HJ. Hyperspectral Imaging for Tissue Classification after Advanced Stage Ovarian Cancer Surgery-A Pilot Study. Cancers (Basel) 2022; 14:cancers14061422. [PMID: 35326577 PMCID: PMC8946803 DOI: 10.3390/cancers14061422] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Revised: 02/24/2022] [Accepted: 03/08/2022] [Indexed: 02/05/2023] Open
Abstract
The most important prognostic factor for the survival of advanced-stage epithelial ovarian cancer (EOC) is the completeness of cytoreductive surgery (CRS). Therefore, an intraoperative technique to detect microscopic tumors would be of great value. The aim of this pilot study is to assess the feasibility of near-infrared hyperspectral imaging (HSI) for EOC detection in ex vivo tissue samples. Images were collected during CRS in 11 patients in the wavelength range of 665−975 nm, and processed by calibration, normalization, and noise filtering. A linear support vector machine (SVM) was employed to classify healthy and tumorous tissue (defined as >50% tumor cells). Classifier performance was evaluated using leave-one-out cross-validation. Images of 26 tissue samples from 10 patients were included, containing 26,446 data points that were matched to histopathology. Tumorous tissue could be classified with an area under the curve of 0.83, a sensitivity of 0.81, a specificity of 0.70, and Matthew’s correlation coefficient of 0.41. This study paves the way to in vivo and intraoperative use of HSI during CRS. Hyperspectral imaging can scan a whole tissue surface in a fast and non-contact way. Our pilot study demonstrates that HSI and SVM learning can be used to discriminate EOC from surrounding tissue.
Collapse
Affiliation(s)
- Sharline M. van Vliet-Pérez
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
- Department of Radiotherapy, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands
- Correspondence:
| | - Nick J. van de Berg
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
| | - Francesca Manni
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (F.M.); (M.L.)
| | - Marco Lai
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (F.M.); (M.L.)
| | - Lucia Rijstenberg
- Department of Pathology, Erasmus University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (L.R.); (P.C.E.-G.)
| | - Benno H. W. Hendriks
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
| | - Jenny Dankelman
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
| | - Patricia C. Ewing-Graham
- Department of Pathology, Erasmus University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (L.R.); (P.C.E.-G.)
| | - Gatske M. Nieuwenhuyzen-de Boer
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
- Department of Gynecology, Albert Schweitzer Hospital, 3318 AT Dordrecht, The Netherlands
| | - Heleen J. van Beekhuizen
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
| |
Collapse
|
5
|
Dujon AM, Vittecoq M, Bramwell G, Thomas F, Ujvari B. Machine learning is a powerful tool to study the effect of cancer on species and ecosystems. Methods Ecol Evol 2021. [DOI: 10.1111/2041-210x.13703] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Antoine M. Dujon
- Geelong School of Life and Environmental Sciences Centre for Integrative Ecology Deakin University Waurn Ponds Victoria Australia
- CREECUMR IRD 224‐CNRS 5290‐Université de Montpellier Montpellier France
- CANECEV‐Centre de Recherches Ecologiques et Evolutives sur le cancer (CREEC) Montpellier France
| | - Marion Vittecoq
- CREECUMR IRD 224‐CNRS 5290‐Université de Montpellier Montpellier France
- MIVEGECUniversity of MontpellierCNRSIRD Montpellier France
- Tour du Valat Research Institute for the Conservation of Mediterranean Wetlands Arles France
| | - Georgina Bramwell
- Geelong School of Life and Environmental Sciences Centre for Integrative Ecology Deakin University Waurn Ponds Victoria Australia
- CANECEV‐Centre de Recherches Ecologiques et Evolutives sur le cancer (CREEC) Montpellier France
| | - Frédéric Thomas
- CREECUMR IRD 224‐CNRS 5290‐Université de Montpellier Montpellier France
- CANECEV‐Centre de Recherches Ecologiques et Evolutives sur le cancer (CREEC) Montpellier France
- MIVEGECUniversity of MontpellierCNRSIRD Montpellier France
| | - Beata Ujvari
- Geelong School of Life and Environmental Sciences Centre for Integrative Ecology Deakin University Waurn Ponds Victoria Australia
- CANECEV‐Centre de Recherches Ecologiques et Evolutives sur le cancer (CREEC) Montpellier France
| |
Collapse
|
6
|
Wang Q, Wang J, Zhou M, Li Q, Wen Y, Chu J. A 3D attention networks for classification of white blood cells from microscopy hyperspectral images. OPTICS & LASER TECHNOLOGY 2021; 139:106931. [DOI: 10.1016/j.optlastec.2021.106931] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/30/2023]
|
7
|
Ren R, Luo H, Su C, Yao Y, Liao W. Machine learning in dental, oral and craniofacial imaging: a review of recent progress. PeerJ 2021; 9:e11451. [PMID: 34046262 PMCID: PMC8136280 DOI: 10.7717/peerj.11451] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2020] [Accepted: 04/22/2021] [Indexed: 02/05/2023] Open
Abstract
Artificial intelligence has been emerging as an increasingly important aspect of our daily lives and is widely applied in medical science. One major application of artificial intelligence in medical science is medical imaging. As a major component of artificial intelligence, many machine learning models are applied in medical diagnosis and treatment with the advancement of technology and medical imaging facilities. The popularity of convolutional neural network in dental, oral and craniofacial imaging is heightening, as it has been continually applied to a broader spectrum of scientific studies. Our manuscript reviews the fundamental principles and rationales behind machine learning, and summarizes its research progress and its recent applications specifically in dental, oral and craniofacial imaging. It also reviews the problems that remain to be resolved and evaluates the prospect of the future development of this field of scientific study.
Collapse
Affiliation(s)
- Ruiyang Ren
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, West China School of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Haozhe Luo
- School of Computer Science, Sichuan University, Chengdu, Sichuan, China
| | - Chongying Su
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, West China School of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Yang Yao
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Implantology, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Wen Liao
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
- Department of Orthodontics, Osaka Dental University, Hirakata, Osaka, Japan
| |
Collapse
|
8
|
Trajanovski S, Shan C, Weijtmans PJC, de Koning SGB, Ruers TJM. Tongue Tumor Detection in Hyperspectral Images Using Deep Learning Semantic Segmentation. IEEE Trans Biomed Eng 2021; 68:1330-1340. [PMID: 32976092 DOI: 10.1109/tbme.2020.3026683] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
OBJECTIVE The utilization of hyperspectral imaging (HSI) in real-time tumor segmentation during a surgery have recently received much attention, but it remains a very challenging task. METHODS In this work, we propose semantic segmentation methods, and compare them with other relevant deep learning algorithms for tongue tumor segmentation. To the best of our knowledge, this is the first work using deep learning semantic segmentation for tumor detection in HSI data using channel selection, and accounting for more spatial tissue context, and global comparison between the prediction map, and the annotation per sample. Results, and Conclusion: On a clinical data set with tongue squamous cell carcinoma, our best method obtains very strong results of average dice coefficient, and area under the ROC-curve of [Formula: see text], and [Formula: see text], respectively on the original spatial image size. The results show that a very good performance can be achieved even with a limited amount of data. We demonstrate that important information regarding tumor decision is encoded in various channels, but some channel selection, and filtering is beneficial over the full spectra. Moreover, we use both visual (VIS), and near-infrared (NIR) spectrum, rather than commonly used only VIS spectrum; although VIS spectrum is generally of higher significance, we demonstrate NIR spectrum is crucial for tumor capturing in some cases. SIGNIFICANCE The HSI technology augmented with accurate deep learning algorithms has a huge potential to be a promising alternative to digital pathology or a doctors' supportive tool in real-time surgeries.
Collapse
|
9
|
Ilhan B, Guneri P, Wilder-Smith P. The contribution of artificial intelligence to reducing the diagnostic delay in oral cancer. Oral Oncol 2021; 116:105254. [PMID: 33711582 DOI: 10.1016/j.oraloncology.2021.105254] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2020] [Revised: 02/11/2021] [Accepted: 02/24/2021] [Indexed: 02/07/2023]
Abstract
Oral cancer (OC) is the sixth most commonly reported malignant disease globally, with high rates of disease-related morbidity and mortality due to advanced loco-regional stage at diagnosis. Early detection and prompt treatment offer the best outcomes to patients, yet the majority of OC lesions are detected at late stages with 45% survival rate for 2 years. The primary cause of poor OC outcomes is unavailable or ineffective screening and surveillance at the local point-of-care level, leading to delays in specialist referral and subsequent treatment. Lack of adequate awareness of OC among the public and professionals, and barriers to accessing health care services in a timely manner also contribute to delayed diagnosis. As image analysis and diagnostic technologies are evolving, various artificial intelligence (AI) approaches, specific algorithms and predictive models are beginning to have a considerable impact in improving diagnostic accuracy for OC. AI based technologies combined with intraoral photographic images or optical imaging methods are under investigation for automated detection and classification of OC. These new methods and technologies have great potential to improve outcomes, especially in low-resource settings. Such approaches can be used to predict oral cancer risk as an adjunct to population screening by providing real-time risk assessment. The objective of this study is to (1) provide an overview of components of delayed OC diagnosis and (2) evaluate novel AI based approaches with respect to their utility and implications for improving oral cancer detection.
Collapse
Affiliation(s)
- Betul Ilhan
- Ege University, Faculty of Dentistry, Department of Oral & Maxillofacial Radiology, Bornova, Izmir, Turkey.
| | - Pelin Guneri
- Ege University, Faculty of Dentistry, Department of Oral & Maxillofacial Radiology, Bornova, Izmir, Turkey
| | | |
Collapse
|
10
|
Sagheer SH, Whitaker-Menezes D, Han JYS, Curry JM, Martinez-Outschoorn U, Philp NJ. 4NQO induced carcinogenesis: A mouse model for oral squamous cell carcinoma. Methods Cell Biol 2021; 163:93-111. [PMID: 33785171 DOI: 10.1016/bs.mcb.2021.01.001] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Oral squamous cell carcinoma (OSCC) is the most common subsite of head and neck cancer, with a 5-year survival rate of only 50%. There is a pressing need for animal models that recapitulate the human disease to understand the factors driving OSCC carcinogenesis. Many laboratories have used the chemical carcinogen 4-nitroquinoline-1-oxide (4NQO) to investigate OSCC formation. The importance of the 4NQO mouse model is that it mimics the stepwise progression observed in OSCC patients. The 4NQO carcinogen model has the advantage that it can be used with transgenic mice with genetic modification in specific tissue types to investigate their role in driving cancer progression. Herein, we describe the basic approach for administering 4NQO to mice to induce OSCC and methods for assessing the tissue and disease progression.
Collapse
Affiliation(s)
- S Hamad Sagheer
- Department of Otolaryngology-Head and Neck Surgery, Thomas Jefferson University, Philadelphia, PA, United States
| | - Diana Whitaker-Menezes
- Department of Medical Oncology, Thomas Jefferson University, Philadelphia, PA, United States
| | - John Y S Han
- Department of Pathology, Anatomy & Cell Biology, Thomas Jefferson University, Philadelphia, PA, United States
| | - Joseph M Curry
- Department of Otolaryngology-Head and Neck Surgery, Thomas Jefferson University, Philadelphia, PA, United States.
| | | | - Nancy J Philp
- Department of Pathology, Anatomy & Cell Biology, Thomas Jefferson University, Philadelphia, PA, United States.
| |
Collapse
|
11
|
Ma L, Shahedi M, Shi T, Halicek M, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Pixel-level Tumor Margin Assessment of Surgical Specimen with Hyperspectral Imaging and Deep Learning Classification. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11598:1159811. [PMID: 35755403 PMCID: PMC9232191 DOI: 10.1117/12.2581046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Surgery is a major treatment method for squamous cell carcinoma (SCC). During surgery, insufficient tumor margin may lead to local recurrence of cancer. Hyperspectral imaging (HSI) is a promising optical imaging technique for in vivo cancer detection and tumor margin assessment. In this study, a fully convolutional network (FCN) was implemented for tumor classification and margin assessment on hyperspectral images of SCC. The FCN was trained and validated with hyperspectral images of 25 ex vivo SCC surgical specimens from 20 different patients. The network was evaluated per patient and achieved pixel-level tissue classification with an average area under the curve (AUC) of 0.88, as well as 0.83 accuracy, 0.84 sensitivity, and 0.70 specificity across all the 20 patients. The 95% Hausdorff distance of assessed tumor margin in 17 patients was less than 2 mm, and the classification time of each tissue specimen took less than 10 seconds. The proposed methods can potentially facilitate intraoperative tumor margin assessment and improve surgical outcomes.
Collapse
Affiliation(s)
- Ling Ma
- Department of Bioengineering, University of Texas at Dallas
- State Key Laboratory of Precision Measurement Technology and Instrument, Tianjin University
| | - Maysam Shahedi
- Department of Bioengineering, University of Texas at Dallas
| | - Ted Shi
- Department of Bioengineering, University of Texas at Dallas
| | - Martin Halicek
- Department of Bioengineering, University of Texas at Dallas
| | - James V. Little
- Department of Pathology and Laboratory Medicine, Emory University
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University
| | - Larry L. Myers
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center
- Department of Radiology, University of Texas Southwestern Medical Center
| |
Collapse
|
12
|
Wang Q, Sun L, Wang Y, Zhou M, Hu M, Chen J, Wen Y, Li Q. Identification of Melanoma From Hyperspectral Pathology Image Using 3D Convolutional Networks. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:218-227. [PMID: 32956043 DOI: 10.1109/tmi.2020.3024923] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Skin biopsy histopathological analysis is one of the primary methods used for pathologists to assess the presence and deterioration of melanoma in clinical. A comprehensive and reliable pathological analysis is the result of correctly segmented melanoma and its interaction with benign tissues, and therefore providing accurate therapy. In this study, we applied the deep convolution network on the hyperspectral pathology images to perform the segmentation of melanoma. To make the best use of spectral properties of three dimensional hyperspectral data, we proposed a 3D fully convolutional network named Hyper-net to segment melanoma from hyperspectral pathology images. In order to enhance the sensitivity of the model, we made a specific modification to the loss function with caution of false negative in diagnosis. The performance of Hyper-net surpassed the 2D model with the accuracy over 92%. The false negative rate decreased by nearly 66% using Hyper-net with the modified loss function. These findings demonstrated the ability of the Hyper-net for assisting pathologists in diagnosis of melanoma based on hyperspectral pathology images.
Collapse
|
13
|
Ilhan B, Lin K, Guneri P, Wilder-Smith P. Improving Oral Cancer Outcomes with Imaging and Artificial Intelligence. J Dent Res 2020; 99:241-248. [PMID: 32077795 DOI: 10.1177/0022034520902128] [Citation(s) in RCA: 59] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
Early diagnosis is the most important determinant of oral and oropharyngeal squamous cell carcinoma (OPSCC) outcomes, yet most of these cancers are detected late, when outcomes are poor. Typically, nonspecialists such as dentists screen for oral cancer risk, and then they refer high-risk patients to specialists for biopsy-based diagnosis. Because the clinical appearance of oral mucosal lesions is not an adequate indicator of their diagnosis, status, or risk level, this initial triage process is inaccurate, with poor sensitivity and specificity. The objective of this study is to provide an overview of emerging optical imaging modalities and novel artificial intelligence-based approaches, as well as to evaluate their individual and combined utility and implications for improving oral cancer detection and outcomes. The principles of image-based approaches to detecting oral cancer are placed within the context of clinical needs and parameters. A brief overview of artificial intelligence approaches and algorithms is presented, and studies that use these 2 approaches singly and together are cited and evaluated. In recent years, a range of novel imaging modalities has been investigated for their applicability to improving oral cancer outcomes, yet none of them have found widespread adoption or significantly affected clinical practice or outcomes. Artificial intelligence approaches are beginning to have considerable impact in improving diagnostic accuracy in some fields of medicine, but to date, only limited studies apply to oral cancer. These studies demonstrate that artificial intelligence approaches combined with imaging can have considerable impact on oral cancer outcomes, with applications ranging from low-cost screening with smartphone-based probes to algorithm-guided detection of oral lesion heterogeneity and margins using optical coherence tomography. Combined imaging and artificial intelligence approaches can improve oral cancer outcomes through improved detection and diagnosis.
Collapse
Affiliation(s)
- B Ilhan
- Department of Oral & Maxillofacial Radiology, Ege University Faculty of Dentistry, Bornova-Izmir, Turkey
| | - K Lin
- Beckman Laser Institute, University of California, Irvine, CA, USA
| | - P Guneri
- Department of Oral & Maxillofacial Radiology, Ege University Faculty of Dentistry, Bornova-Izmir, Turkey
| | - P Wilder-Smith
- Beckman Laser Institute, University of California, Irvine, CA, USA
| |
Collapse
|
14
|
Tama BA, Kim DH, Kim G, Kim SW, Lee S. Recent Advances in the Application of Artificial Intelligence in Otorhinolaryngology-Head and Neck Surgery. Clin Exp Otorhinolaryngol 2020; 13:326-339. [PMID: 32631041 PMCID: PMC7669308 DOI: 10.21053/ceo.2020.00654] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Revised: 05/24/2020] [Accepted: 06/09/2020] [Indexed: 12/12/2022] Open
Abstract
This study presents an up-to-date survey of the use of artificial intelligence (AI) in the field of otorhinolaryngology, considering opportunities, research challenges, and research directions. We searched PubMed, the Cochrane Central Register of Controlled Trials, Embase, and the Web of Science. We initially retrieved 458 articles. The exclusion of non-English publications and duplicates yielded a total of 90 remaining studies. These 90 studies were divided into those analyzing medical images, voice, medical devices, and clinical diagnoses and treatments. Most studies (42.2%, 38/90) used AI for image-based analysis, followed by clinical diagnoses and treatments (24 studies). Each of the remaining two subcategories included 14 studies. Machine learning and deep learning have been extensively applied in the field of otorhinolaryngology. However, the performance of AI models varies and research challenges remain.
Collapse
Affiliation(s)
- Bayu Adhi Tama
- Department of Mechanical Engineering, Pohang University of Science and Technology, Pohang, Korea
| | - Do Hyun Kim
- Department of Otolaryngology-Head and Neck Surgery, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Gyuwon Kim
- Department of Mechanical Engineering, Pohang University of Science and Technology, Pohang, Korea
| | - Soo Whan Kim
- Department of Otolaryngology-Head and Neck Surgery, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Seungchul Lee
- Department of Mechanical Engineering, Pohang University of Science and Technology, Pohang, Korea
- Graduate School of Artificial Intelligence, Pohang University of Science and Technology, Pohang, Korea
| |
Collapse
|
15
|
Ma L, Halicek M, Fei B. In Vivo Cancer Detection in Animal Model Using Hyperspectral Image Classification with Wavelet Feature Extraction. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11317. [PMID: 32476705 DOI: 10.1117/12.2549397] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Hyperspectral imaging (HSI) is a promising optical imaging technique for cancer detection. However, quantitative methods need to be developed in order to utilize the rich spectral information and subtle spectral variation in such images. In this study, we explore the feasibility of using wavelet-based features from in vivo hyperspectral images for head and neck cancer detection. Hyperspectral reflectance data were collected from 12 mice bearing head and neck cancer. Catenation of 5-level wavelet decomposition outputs of hyperspectral images was used as a feature for tumor discrimination. A support vector machine (SVM) was utilized as the classifier. Seven types of mother wavelets were tested to select the one with the best performance. Classifications with raw reflectance spectra, 1-level wavelet decomposition output, and 2-level wavelet decomposition output, as well as the proposed feature were carried out for comparison. Our results show that the proposed wavelet-based feature yields better classification accuracy, and that using different type and order of mother wavelet achieves different classification results. The wavelet-based classification method provides a new approach for HSI detection of head and neck cancer in the animal model.
Collapse
Affiliation(s)
- Ling Ma
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX 75080.,Tianjin University, State Key Laboratory of Precision Measurement Technology and Instrument, Tianjin, China 300072
| | - Martin Halicek
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX 75080.,Georgia Inst. of Tech. & Emory Univ., Dept. of Biomedical Engineering, Atlanta, GA.,Medical College of Georgia, Augusta University, Augusta, GA
| | - Baowei Fei
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX 75080.,Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX.,Univ. of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
16
|
Identification of oral cancer in OCT images based on an optical attenuation model. Lasers Med Sci 2020; 35:1999-2007. [PMID: 32335743 DOI: 10.1007/s10103-020-03025-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2019] [Accepted: 04/16/2020] [Indexed: 12/21/2022]
Abstract
Surgery is still the first choice to treat oral cancer, where it is important to detect surgical margins in order to reduce cancer recurrence and maintain oral-maxillofacial function simultaneously. As a non-invasive and in situ imaging technique, optical coherence tomography (OCT) can obtain images close to the resolution of histopathology, which makes it have great potential in intraoperative diagnosis. However, it is not enough to find surgical margins accurately just observing OCT images directly and qualitatively. The purpose of this study is to identify oral cancer in OCT images by investigating the quantitative difference of cancer and non-cancer tissue. Based on an available optical attenuation model and the axial confocal PSF of a home-made swept source OCT system, by using fresh ex vivo human oral tissues from 14 patients of oral squamous cell carcinoma (OSCC) as the samples, diagnosis with sensitivity (97.88%) and specificity (83.77%) was achieved at the attenuation threshold of 4.7 mm-1, and the accuracy of identification reached 91.15% in our study. Our preliminary results demonstrated that the oral cancer resection will be guided accurately and the clinical application of OCT will be further promoted by deeply mining the information hidden in OCT images.
Collapse
|
17
|
|
18
|
Zhang Y, Wu X, He L, Meng C, Du S, Bao J, Zheng Y. Applications of hyperspectral imaging in the detection and diagnosis of solid tumors. Transl Cancer Res 2020; 9:1265-1277. [PMID: 35117471 PMCID: PMC8798535 DOI: 10.21037/tcr.2019.12.53] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2019] [Accepted: 11/28/2019] [Indexed: 11/09/2022]
Abstract
Hyperspectral imaging (HSI) is an emerging new technology in solid tumor diagnosis and detection. It incorporates traditional imaging and spectroscopy together to obtain both spatial and spectral information from tissues simultaneously in a non-invasive manner. This imaging modality is based on the principle that different tissues inherit different spectral reflectance responses that present as unique spectral fingerprints. HSI captures those composition-specific fingerprints to identify cancerous and normal tissues. It becomes a promising tool for performing tumor diagnosis and detection from the label-free histopathological examination to real-time intraoperative assistance. This review introduces the basic principles of HSI and summarizes its methodology and recent advances in solid tumor detection. In particular, the advantages of HSI applied to solid tumors are highlighted to show its potential for clinical use.
Collapse
Affiliation(s)
- Yating Zhang
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Xiaoqian Wu
- Department of Liver Surgery, Peking Union Medicine Collage Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| | - Li He
- Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Chan Meng
- Department of Pathology, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Shunda Du
- Department of Liver Surgery, Peking Union Medicine Collage Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| | - Jie Bao
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Yongchang Zheng
- Department of Liver Surgery, Peking Union Medicine Collage Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| |
Collapse
|
19
|
Li Q, Dong H, Yang G, Song Y, Mou Y, Ni Y. Mouse Tumor-Bearing Models as Preclinical Study Platforms for Oral Squamous Cell Carcinoma. Front Oncol 2020; 10:212. [PMID: 32158692 PMCID: PMC7052016 DOI: 10.3389/fonc.2020.00212] [Citation(s) in RCA: 44] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Accepted: 02/06/2020] [Indexed: 12/16/2022] Open
Abstract
Preclinical animal models of oral squamous cell carcinoma (OSCC) have been extensively studied in recent years. Investigating the pathogenesis and potential therapeutic strategies of OSCC is required to further progress in this field, and a suitable research animal model that reflects the intricacies of cancer biology is crucial. Of the animal models established for the study of cancers, mouse tumor-bearing models are among the most popular and widely deployed for their high fertility, low cost, and molecular and physiological similarity to humans, as well as the ease of rearing experimental mice. Currently, the different methods of establishing OSCC mouse models can be divided into three categories: chemical carcinogen-induced, transplanted and genetically engineered mouse models. Each of these methods has unique advantages and limitations, and the appropriate application of these techniques in OSCC research deserves our attention. Therefore, this review comprehensively investigates and summarizes the tumorigenesis mechanisms, characteristics, establishment methods, and current applications of OSCC mouse models in published papers. The objective of this review is to provide foundations and considerations for choosing suitable model establishment methods to study the relevant pathogenesis, early diagnosis, and clinical treatment of OSCC.
Collapse
Affiliation(s)
- Qiang Li
- Central Laboratory, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
| | - Heng Dong
- Central Laboratory, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
- Department of Oral Implantology, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
| | - Guangwen Yang
- Central Laboratory, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
| | - Yuxian Song
- Central Laboratory, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
| | - Yongbin Mou
- Central Laboratory, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
- Department of Oral Implantology, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
- *Correspondence: Yongbin Mou
| | - Yanhong Ni
- Central Laboratory, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
- Yanhong Ni
| |
Collapse
|
20
|
Lu G, Wang D, Qin X, Muller S, Little JV, Wang X, Chen AY, Chen G, Fei B. Histopathology Feature Mining and Association with Hyperspectral Imaging for the Detection of Squamous Neoplasia. Sci Rep 2019; 9:17863. [PMID: 31780698 PMCID: PMC6882850 DOI: 10.1038/s41598-019-54139-5] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2018] [Accepted: 10/31/2019] [Indexed: 12/26/2022] Open
Abstract
Hyperspectral imaging (HSI) is a noninvasive optical modality that holds promise for early detection of tongue lesions. Spectral signatures generated by HSI contain important diagnostic information that can be used to predict the disease status of the examined biological tissue. However, the underlying pathophysiology for the spectral difference between normal and neoplastic tissue is not well understood. Here, we propose to leverage digital pathology and predictive modeling to select the most discriminative features from digitized histological images to differentiate tongue neoplasia from normal tissue, and then correlate these discriminative pathological features with corresponding spectral signatures of the neoplasia. We demonstrated the association between the histological features quantifying the architectural features of neoplasia on a microscopic scale, with the spectral signature of the corresponding tissue measured by HSI on a macroscopic level. This study may provide insight into the pathophysiology underlying the hyperspectral dataset.
Collapse
Affiliation(s)
- Guolan Lu
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, USA
| | - Dongsheng Wang
- Department of Hematology and Medical Oncology, Emory University, Atlanta, GA, USA
| | - Xulei Qin
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA, USA
| | - Susan Muller
- Department of Otolaryngology, Emory University School of Medicine, Atlanta, GA, USA
| | - James V Little
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA, USA
| | - Xu Wang
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA, USA
| | - Amy Y Chen
- Department of Otolaryngology, Emory University School of Medicine, Atlanta, GA, USA
| | - Georgia Chen
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA, USA
| | - Baowei Fei
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, USA.
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA, USA.
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX, USA.
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX, USA.
| |
Collapse
|
21
|
Ma L, Lu G, Wang D, Qin X, Chen ZG, Fei B. Adaptive deep learning for head and neck cancer detection using hyperspectral imaging. Vis Comput Ind Biomed Art 2019; 2:18. [PMID: 32190408 PMCID: PMC7055573 DOI: 10.1186/s42492-019-0023-8] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2019] [Accepted: 10/09/2019] [Indexed: 12/02/2022] Open
Abstract
It can be challenging to detect tumor margins during surgery for complete resection. The purpose of this work is to develop a novel learning method that learns the difference between the tumor and benign tissue adaptively for cancer detection on hyperspectral images in an animal model. Specifically, an auto-encoder network is trained based on the wavelength bands on hyperspectral images to extract the deep information to create a pixel-wise prediction of cancerous and benign pixel. According to the output hypothesis of each pixel, the misclassified pixels would be reclassified in the right prediction direction based on their adaptive weights. The auto-encoder network is again trained based on these updated pixels. The learner can adaptively improve the ability to identify the cancer and benign tissue by focusing on the misclassified pixels, and thus can improve the detection performance. The adaptive deep learning method highlighting the tumor region proved to be accurate in detecting the tumor boundary on hyperspectral images and achieved a sensitivity of 92.32% and a specificity of 91.31% in our animal experiments. This adaptive learning method on hyperspectral imaging has the potential to provide a noninvasive tool for tumor detection, especially, for the tumor whose margin is indistinct and irregular.
Collapse
Affiliation(s)
- Ling Ma
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA 30322 USA
- College of Software, Nankai University, Tianjin, 300350 People’s Republic of China
| | - Guolan Lu
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA 30322 USA
| | - Dongsheng Wang
- Department of Hematology and Medical Oncology, Emory University, Atlanta, GA 30322 USA
| | - Xulei Qin
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA 30322 USA
| | - Zhuo Georgia Chen
- Department of Hematology and Medical Oncology, Emory University, Atlanta, GA 30322 USA
| | - Baowei Fei
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA 30322 USA
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX 75080 USA
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX 75390 USA
| |
Collapse
|
22
|
Fabelo H, Halicek M, Ortega S, Szolna A, Morera J, Sarmiento R, Callico GM, Fei B. Surgical Aid Visualization System for Glioblastoma Tumor Identification based on Deep Learning and In-Vivo Hyperspectral Images of Human Patients. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2019; 10951. [PMID: 31447494 DOI: 10.1117/12.2512569] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Brain cancer surgery has the goal of performing an accurate resection of the tumor and preserving as much as possible the quality of life of the patient. There is a clinical need to develop non-invasive techniques that can provide reliable assistance for tumor resection in real-time during surgical procedures. Hyperspectral imaging (HSI) arises as a new, noninvasive and non-ionizing technique that can assist neurosurgeons during this difficult task. In this paper, we explore the use of deep learning (DL) techniques for processing hyperspectral (HS) images of in-vivo human brain tissue. We developed a surgical aid visualization system capable of offering guidance to the operating surgeon to achieve a successful and accurate tumor resection. The employed HS database is composed of 26 in-vivo hypercubes from 16 different human patients, among which 258,810 labelled pixels were used for evaluation. The proposed DL methods achieve an overall accuracy of 95% and 85% for binary and multiclass classifications, respectively. The proposed visualization system is able to generate a classification map that is formed by the combination of the DL map and an unsupervised clustering via a majority voting algorithm. This map can be adjusted by the operating surgeon to find the suitable configuration for the current situation during the surgical procedure.
Collapse
Affiliation(s)
- Himar Fabelo
- Department of Bioengineering, The University of Texas at Dallas, TX.,Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Spain
| | - Martin Halicek
- Department of Bioengineering, The University of Texas at Dallas, TX.,Department of Biomedical Engineering, Emory Univ. and Georgia Inst. of Tech., Atlanta, GA
| | - Samuel Ortega
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Spain
| | - Adam Szolna
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Spain
| | - Jesus Morera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Spain
| | - Roberto Sarmiento
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Spain
| | - Gustavo M Callico
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Spain
| | - Baowei Fei
- Department of Bioengineering, The University of Texas at Dallas, TX.,Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, TX.,Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
23
|
Fabelo H, Halicek M, Ortega S, Shahedi M, Szolna A, Piñeiro JF, Sosa C, O'Shanahan AJ, Bisshopp S, Espino C, Márquez M, Hernández M, Carrera D, Morera J, Callico GM, Sarmiento R, Fei B. Deep Learning-Based Framework for In Vivo Identification of Glioblastoma Tumor using Hyperspectral Images of Human Brain. SENSORS 2019; 19:s19040920. [PMID: 30813245 PMCID: PMC6412736 DOI: 10.3390/s19040920] [Citation(s) in RCA: 70] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/24/2019] [Revised: 02/18/2019] [Accepted: 02/20/2019] [Indexed: 02/02/2023]
Abstract
The main goal of brain cancer surgery is to perform an accurate resection of the tumor, preserving as much normal brain tissue as possible for the patient. The development of a non-contact and label-free method to provide reliable support for tumor resection in real-time during neurosurgical procedures is a current clinical need. Hyperspectral imaging is a non-contact, non-ionizing, and label-free imaging modality that can assist surgeons during this challenging task without using any contrast agent. In this work, we present a deep learning-based framework for processing hyperspectral images of in vivo human brain tissue. The proposed framework was evaluated by our human image database, which includes 26 in vivo hyperspectral cubes from 16 different patients, among which 258,810 pixels were labeled. The proposed framework is able to generate a thematic map where the parenchymal area of the brain is delineated and the location of the tumor is identified, providing guidance to the operating surgeon for a successful and precise tumor resection. The deep learning pipeline achieves an overall accuracy of 80% for multiclass classification, improving the results obtained with traditional support vector machine (SVM)-based approaches. In addition, an aid visualization system is presented, where the final thematic map can be adjusted by the operating surgeon to find the optimal classification threshold for the current situation during the surgical procedure.
Collapse
Affiliation(s)
- Himar Fabelo
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080, USA.
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain.
| | - Martin Halicek
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080, USA.
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, 1841 Clifton Road NE, Atlanta, GA 30329, USA.
| | - Samuel Ortega
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain.
| | - Maysam Shahedi
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080, USA.
| | - Adam Szolna
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - Juan F Piñeiro
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - Coralia Sosa
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - Aruma J O'Shanahan
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - Sara Bisshopp
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - Carlos Espino
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - Mariano Márquez
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - María Hernández
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - David Carrera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - Jesús Morera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain.
| | - Gustavo M Callico
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain.
| | - Roberto Sarmiento
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain.
| | - Baowei Fei
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080, USA.
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center, 5323 Harry Hine Blvd, Dallas, TX 75390, USA.
- Department of Radiology, University of Texas Southwestern Medical Center, 5323 Harry Hine Blvd, Dallas, TX 75390, USA.
| |
Collapse
|
24
|
Halicek M, Fabelo H, Ortega S, Little JV, Wang X, Chen AY, Callico GM, Myers LL, Sumer BD, Fei B. Cancer Detection Using Hyperspectral Imaging and Evaluation of the Superficial Tumor Margin Variance with Depth. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2019; 10951:109511A. [PMID: 32489227 PMCID: PMC7265739 DOI: 10.1117/12.2512985] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
Head and neck squamous cell carcinoma (SCCa) is primarily managed by surgical resection. Recurrence rates after surgery can be as high as 55% if residual cancer is present. In this study, hyperspectral imaging (HSI) is evaluated for detection of SCCa in ex-vivo surgical specimens. Several methods are investigated, including convolutional neural networks (CNNs) and a spectral-spatial variant of support vector machines. Quantitative results demonstrate that additional processing and unsupervised filtering can improve CNN results to achieve optimal performance. Classifying regions that include specular glare, the average AUC is increased from 0.73 [0.71, 0.75 (95% confidence interval)] to 0.81 [0.80, 0.83] through an unsupervised filtering and majority voting method described. The wavelengths of light used in HSI can penetrate different depths into biological tissue, while the cancer margin may change with depth and create uncertainty in the ground-truth. Through serial histological sectioning, the variance in cancer-margin with depth is also investigated and paired with qualitative classification heat maps using the methods proposed for the testing group SCC patients.
Collapse
Affiliation(s)
- Martin Halicek
- Department of Bioengineering, University of Texas at Dallas, Dallas, TX, USA
- Georgia Inst. of Tech. & Emory Univ., Dept. of Biomedical Engineering, Atlanta, GA
- Medical College of Georgia, Augusta University, Augusta, GA
| | - Himar Fabelo
- Department of Bioengineering, University of Texas at Dallas, Dallas, TX, USA
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Spain
| | - Samuel Ortega
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Spain
| | - James V Little
- Emory Univ. School of Medicine, Dept. of Pathology & Laboratory Medicine, Atlanta, GA
| | - Xu Wang
- Emory Univ. School of Medicine, Dept. of Hematology & Medical Oncology, Atlanta, GA
| | - Amy Y Chen
- Emory University School of Medicine, Dept. of Otolaryngology, Atlanta, GA
| | | | - Larry L Myers
- University of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baran D Sumer
- University of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Dallas, TX, USA
- Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
25
|
Halicek M, Little JV, Wang X, Patel M, Griffith CC, Chen AY, Fei B. Tumor Margin Classification of Head and Neck Cancer Using Hyperspectral Imaging and Convolutional Neural Networks. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2018; 10576:1057605. [PMID: 30245540 PMCID: PMC6149520 DOI: 10.1117/12.2293167] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
One of the largest factors affecting disease recurrence after surgical cancer resection is negative surgical margins. Hyperspectral imaging (HSI) is an optical imaging technique with potential to serve as a computer aided diagnostic tool for identifying cancer in gross ex-vivo specimens. We developed a tissue classifier using three distinct convolutional neural network (CNN) architectures on HSI data to investigate the ability to classify the cancer margins from ex-vivo human surgical specimens, collected from 20 patients undergoing surgical cancer resection as a preliminary validation group. A new approach for generating the HSI ground truth using a registered histological cancer margin is applied in order to create a validation dataset. The CNN-based method classifies the tumor-normal margin of squamous cell carcinoma (SCCa) versus normal oral tissue with an area under the curve (AUC) of 0.86 for inter-patient validation, performing with 81% accuracy, 84% sensitivity, and 77% specificity. Thyroid carcinoma cancer-normal margins are classified with an AUC of 0.94 for inter-patient validation, performing with 90% accuracy, 91% sensitivity, and 88% specificity. Our preliminary results on a limited patient dataset demonstrate the predictive ability of HSI-based cancer margin detection, which warrants further investigation with more patient data and additional processing techniques to optimize the proposed deep learning method.
Collapse
Affiliation(s)
- Martin Halicek
- Georgia Institute of Technology & Emory University, Wallace H. Coulter Department of Biomedical Engineering, Atlanta, GA, USA
- Medical College of Georgia, Augusta University, Augusta, GA, USA
| | - James V Little
- Emory University School of Medicine, Department of Pathology and Laboratory Medicine, Atlanta, GA, USA
| | - Xu Wang
- Emory University School of Medicine, Department of Hematology and Medical Oncology, Atlanta, GA, USA
| | - Mihir Patel
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, GA, USA
- Winship Cancer Institute of Emory University, Atlanta, GA, USA
| | - Christopher C Griffith
- Emory University School of Medicine, Department of Pathology and Laboratory Medicine, Atlanta, GA, USA
| | - Amy Y Chen
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, GA, USA
- Winship Cancer Institute of Emory University, Atlanta, GA, USA
| | - Baowei Fei
- Georgia Institute of Technology & Emory University, Wallace H. Coulter Department of Biomedical Engineering, Atlanta, GA, USA
- Winship Cancer Institute of Emory University, Atlanta, GA, USA
- Emory University, Department of Mathematics and Computer Science, Atlanta, GA, USA
- Emory University, Department of Radiology and Imaging Sciences, Atlanta, GA, USA
| |
Collapse
|
26
|
Halicek M, Little JV, Wang X, Patel M, Griffith CC, El-Deiry MW, Chen AY, Fei B. Optical Biopsy of Head and Neck Cancer Using Hyperspectral Imaging and Convolutional Neural Networks. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2018; 10469:104690X. [PMID: 30197462 PMCID: PMC6123819 DOI: 10.1117/12.2289023] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Successful outcomes of surgical cancer resection necessitate negative, cancer-free surgical margins. Currently, tissue samples are sent to pathology for diagnostic confirmation. Hyperspectral imaging (HSI) is an emerging, non-contact optical imaging technique. A reliable optical method could serve to diagnose and biopsy specimens in real-time. Using convolutional neural networks (CNNs) as a tissue classifier, we developed a method to use HSI to perform an optical biopsy of ex-vivo surgical specimens, collected from 21 patients undergoing surgical cancer resection. Training and testing on samples from different patients, the CNN can distinguish squamous cell carcinoma (SCCa) from normal aerodigestive tract tissues with an area under the curve (AUC) of 0.82, 81% accuracy, 81% sensitivity, and 80% specificity. Additionally, normal oral tissues can be sub-classified into epithelium, muscle, and glandular mucosa using a decision tree method, with an average AUC of 0.94, 90% accuracy, 93% sensitivity, and 89% specificity. After separately training on thyroid tissue, the CNN differentiates between thyroid carcinoma and normal thyroid with an AUC of 0.95, 92% accuracy, 92% sensitivity, and 92% specificity. Moreover, the CNN can discriminate medullary thyroid carcinoma from benign multi-nodular goiter (MNG) with an AUC of 0.93, 87% accuracy, 88% sensitivity, and 85% specificity. Classical-type papillary thyroid carcinoma is differentiated from benign MNG with an AUC of 0.91, 86% accuracy, 86% sensitivity, and 86% specificity. Our preliminary results demonstrate that an HSI-based optical biopsy method using CNNs can provide multi-category diagnostic information for normal head-and-neck tissue, SCCa, and thyroid carcinomas. More patient data are needed in order to fully investigate the proposed technique to establish reliability and generalizability of the work.
Collapse
Affiliation(s)
- Martin Halicek
- Georgia Institute of Technology & Emory University, Wallace H. Coulter Department of Biomedical Engineering, Atlanta, GA, USA
- Medical College of Georgia, Augusta University, Augusta, GA, USA
| | - James V Little
- Emory University School of Medicine, Department of Pathology and Laboratory Medicine, Atlanta, GA, USA
| | - Xu Wang
- Emory University School of Medicine, Department of Hematology and Medical Oncology, Atlanta, GA, USA
| | - Mihir Patel
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, GA, USA
- Winship Cancer Institute of Emory University, Atlanta, GA, USA
| | - Christopher C Griffith
- Emory University School of Medicine, Department of Pathology and Laboratory Medicine, Atlanta, GA, USA
| | - Mark W El-Deiry
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, GA, USA
- Winship Cancer Institute of Emory University, Atlanta, GA, USA
| | - Amy Y Chen
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, GA, USA
- Winship Cancer Institute of Emory University, Atlanta, GA, USA
| | - Baowei Fei
- Georgia Institute of Technology & Emory University, Wallace H. Coulter Department of Biomedical Engineering, Atlanta, GA, USA
- Winship Cancer Institute of Emory University, Atlanta, GA, USA
- Emory University, Department of Mathematics and Computer Science, Atlanta, GA, USA
- Emory University, Department of Radiology and Imaging Sciences, Atlanta, GA, USA
| |
Collapse
|