1
|
Huang J, Zhu X, Chen Z, Lin G, Huang M, Feng Q. Pathological Priors Inspired Network for Vertebral Osteophytes Recognition. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2522-2536. [PMID: 38386579 DOI: 10.1109/tmi.2024.3367868] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/24/2024]
Abstract
Automatic vertebral osteophyte recognition in Digital Radiography is of great importance for the early prediction of degenerative disease but is still a challenge because of the tiny size and high inter-class similarity between normal and osteophyte vertebrae. Meanwhile, common sampling strategies applied in Convolution Neural Network could cause detailed context loss. All of these could lead to an incorrect positioning predicament. In this paper, based on important pathological priors, we define a set of potential lesions of each vertebra and propose a novel Pathological Priors Inspired Network (PPIN) to achieve accurate osteophyte recognition. PPIN comprises a backbone feature extractor integrating with a Wavelet Transform Sampling module for high-frequency detailed context extraction, a detection branch for locating all potential lesions and a classification branch for producing final osteophyte recognition. The Anatomical Map-guided Filter between two branches helps the network focus on the specific anatomical regions via the generated heatmaps of potential lesions in the detection branch to address the incorrect positioning problem. To reduce the inter-class similarity, a Bilateral Augmentation Module based on the graph relationship is proposed to imitate the clinical diagnosis process and to extract discriminative contextual information between adjacent vertebrae in the classification branch. Experiments on the two osteophytes-specific datasets collected from the public VinDr-Spine database show that the proposed PPIN achieves the best recognition performance among multitask frameworks and shows strong generalization. The results on a private dataset demonstrate the potential in clinical application. The Class Activation Maps also show the powerful localization capability of PPIN. The source codes are available in https://github.com/Phalo/PPIN.
Collapse
|
2
|
Zhu G, Shen X, Sun Z, Xiao Z, Zhong J, Yin Z, Li S, Guo H. Deep learning-based automated scan plane positioning for brain magnetic resonance imaging. Quant Imaging Med Surg 2024; 14:4015-4030. [PMID: 38846304 PMCID: PMC11151238 DOI: 10.21037/qims-23-1740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Accepted: 04/10/2024] [Indexed: 06/09/2024]
Abstract
Background Manual planning of scans in clinical magnetic resonance imaging (MRI) exhibits poor accuracy, lacks consistency, and is time-consuming. Meanwhile, classical automated scan plane positioning methods that rely on certain assumptions are not accurate or stable enough, and are computationally inefficient for practical application scenarios. This study aims to develop and evaluate an effective, reliable, and accurate deep learning-based framework that incorporates prior physical knowledge for automatic head scan plane positioning in MRI. Methods A deep learning-based end-to-end automated scan plane positioning framework has been developed for MRI head scans. Our model takes a three-dimensional (3D) pre-scan image input, utilizing a cascaded 3D convolutional neural network to detect anatomical landmarks from coarse to fine. And then, with the determined landmarks, accurate scan plane localization can be achieved. A multi-scale spatial information fusion module was employed to aggregate high- and low-resolution features, combined with physically meaningful point regression loss (PRL) function and direction regression loss (DRL) function. Meanwhile, we simulate complex clinical scenarios to design data augmentation strategies. Results Our proposed approach shows good performance on a clinically wide range of 229 MRI head scans, with a point-to-point absolute error (PAE) of 0.872 mm, a point-to-point relative error (PRE) of 0.10%, and an average angular error (AAE) of 0.502°, 0.381°, and 0.675° for the sagittal, transverse, and coronal planes, respectively. Conclusions The proposed deep learning-based automated scan plane positioning shows high efficiency, accuracy and robustness when evaluated on varied clinical head MRI scans with differences in positioning, contrast, noise levels and pathologies.
Collapse
Affiliation(s)
- Gaojie Zhu
- Center for Biomedical Imaging Research, School of Biomedical Engineering, Tsinghua University, Beijing, China
- Anke High-tech Co., Ltd., Shenzhen, China
| | | | - Zhiguo Sun
- Anke High-tech Co., Ltd., Shenzhen, China
| | | | | | - Zhe Yin
- Anke High-tech Co., Ltd., Shenzhen, China
| | - Shengxiang Li
- Department of Medical Imaging, Hengyang Central Hospital, Hengyang, China
| | - Hua Guo
- Center for Biomedical Imaging Research, School of Biomedical Engineering, Tsinghua University, Beijing, China
| |
Collapse
|
3
|
Spangenberg GW, Uddin F, Faber KJ, Langohr GDG. Automatic bicipital groove identification in arthritic humeri for preoperative planning: A Random Forest Classifier approach. Comput Biol Med 2024; 178:108653. [PMID: 38861894 DOI: 10.1016/j.compbiomed.2024.108653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 05/06/2024] [Accepted: 05/22/2024] [Indexed: 06/13/2024]
Abstract
The bicipital groove is an important anatomical feature of the proximal humerus that needs to be identified during surgical planning for procedures such as shoulder arthroplasty and proximal humeral fracture reconstruction. Current algorithms for automatic identification prove ineffective in arthritic humeri due to the presence of osteophytes, reducing their usefulness for total shoulder arthroplasty. Our methodology involves the use of a Random Forest Classifier (RFC) to automatically detect the bicipital groove on segmented computed tomography scans of humeri. We evaluated our model on two distinct test datasets: one comprising non-arthritic humeri and another with arthritic humeri characterized by significant osteophytes. Our model detected the bicipital groove with a mean absolute error of less than 1mm on arthritic humeri, demonstrating a significant improvement over the previous gold standard approach. Successful identification of the bicipital groove with a high degree of accuracy even in arthritic humeri was accomplished. This model is open source and included in the python package shoulder.
Collapse
Affiliation(s)
- Gregory W Spangenberg
- Department of Mechanical Engineering, Western University, London, ON, Canada; The Roth McFarlane Hand and Upper Limb Centre, St. Joseph's Hospital, London, ON, Canada.
| | - Fares Uddin
- The Roth McFarlane Hand and Upper Limb Centre, St. Joseph's Hospital, London, ON, Canada; Department of Surgery, Western University, London, ON, Canada
| | - Kenneth J Faber
- The Roth McFarlane Hand and Upper Limb Centre, St. Joseph's Hospital, London, ON, Canada; Department of Surgery, Western University, London, ON, Canada
| | - G Daniel G Langohr
- Department of Mechanical Engineering, Western University, London, ON, Canada; The Roth McFarlane Hand and Upper Limb Centre, St. Joseph's Hospital, London, ON, Canada
| |
Collapse
|
4
|
Dritsas S, Chua KWD, Goh ZH, Simpson RE. Classification, registration and segmentation of ear canal impressions using convolutional neural networks. Med Image Anal 2024; 94:103152. [PMID: 38531210 DOI: 10.1016/j.media.2024.103152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 12/12/2023] [Accepted: 03/20/2024] [Indexed: 03/28/2024]
Abstract
Today, fitting bespoke hearing aids involves injecting silicone into patients' ears to produce ear canal molds. These are subsequently 3D scanned to create digital ear canal impressions. However, before digital impressions can be used they require a substantial amount of effort in manual 3D editing. In this article, we present computational methods to pre-process ear canal impressions. The aim is to create automation tools to assist the hearing aid design, manufacturing and fitting processes as well as normalizing anatomical data to assist the study of the outer ear canal's morphology. The methods include classifying the handedness of the impression into left and right ear types, orienting the geometries onto the same coordinate system sense, and removing extraneous artifacts introduced by the silicone mold. We investigate the use of convolutional neural networks for performing these semantic tasks and evaluate their accuracy using a dataset of 3000 ear canal impressions. The neural networks proved highly effective at performing these tasks with 95.8% adjusted accuracy in classification, 92.3% within 20° angular error in registration and 93.4% intersection over union in segmentation.
Collapse
Affiliation(s)
- Stylianos Dritsas
- Singapore University of Technology and Design, 8 Somapah Road, 487372, Singapore.
| | | | - Zhi Hwee Goh
- Singapore University of Technology and Design, 8 Somapah Road, 487372, Singapore
| | | |
Collapse
|
5
|
Quatre R, Schmerber S, Attyé A. Improving rehabilitation of deaf patients by advanced imaging before cochlear implantation. J Neuroradiol 2024; 51:145-154. [PMID: 37806523 DOI: 10.1016/j.neurad.2023.10.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2023] [Revised: 10/05/2023] [Accepted: 10/05/2023] [Indexed: 10/10/2023]
Abstract
INTRODUCTION Cochlear implants have advanced the management of severe to profound deafness. However, there is a strong disparity in hearing performance after implantation from one patient to another. Moreover, there are several advanced kinds of imaging assessment before cochlear implantation. Microstructural white fiber degeneration can be studied with Diffusion weighted MRI (DWI) or tractography of the central auditory pathways. Functional MRI (fMRI) allows us to evaluate brain function, and CT or MRI segmentation to better detect inner ear anomalies. OBJECTIVE This literature review aims to evaluate how helpful pre-implantation anatomic imaging can be to predict hearing rehabilitation outcomes in deaf patients. These techniques include DWI and fMRI of the central auditory pathways, and automated labyrinth segmentation by CT scan, cone beam CT and MRI. DESIGN This systematic review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Studies were selected by searching in PubMed and by checking the reference lists of relevant articles. Inclusion criteria were adults over 18, with unilateral or bilateral hearing loss, who had DWI acquisition or fMRI or CT/ Cone Beam CT/ MRI image segmentation. RESULTS After reviewing 172 articles, we finally included 51. Studies on DWI showed changes in the central auditory pathways affecting the white matter, extending to the primary and non-primary auditory cortices, even in sudden and mild hearing impairment. Hearing loss patients show a reorganization of brain activity in various areas, such as the auditory and visual cortices, as well as regions involved in language and emotions, according to fMRI studies. Deep Learning's automatic segmentation produces the best CT segmentation in just a few seconds. MRI segmentation is mainly used to evaluate fluid space of the inner ear and determine the presence of an endolymphatic hydrops. CONCLUSION Before cochlear implantation, a DWI with tractography can evaluate the central auditory pathways up to the primary and non-primary auditory cortices. This data is then used to generate predictions on the auditory rehabilitation of patients. A CT segmentation with systematic 3D reconstruction allow a better evaluation of cochlear malformations and predictable difficulties during surgery.
Collapse
Affiliation(s)
- Raphaële Quatre
- Department of Oto-Rhino-Laryngology, Head and Neck Surgery, University Hospital, Grenoble, France; BrainTech Lab INSERM UMR 2015, Grenoble, France; GeodAIsics, Grenoble, France.
| | - Sébastien Schmerber
- Department of Oto-Rhino-Laryngology, Head and Neck Surgery, University Hospital, Grenoble, France; BrainTech Lab INSERM UMR 2015, Grenoble, France
| | - Arnaud Attyé
- Department of Neuroradiology, University Hospital, Grenoble, France; GeodAIsics, Grenoble, France
| |
Collapse
|
6
|
Musleh A. Machine learning framework for simulation of artifacts in paranasal sinuses diagnosis using CT images. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2024; 32:839-855. [PMID: 38393882 DOI: 10.3233/xst-230284] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/25/2024]
Abstract
In the medical field, diagnostic tools that make use of deep neural networks have reached a level of performance never before seen. A proper diagnosis of a patient's condition is crucial in modern medicine since it determines whether or not the patient will receive the care they need. Data from a sinus CT scan is uploaded to a computer and displayed on a high-definition monitor to give the surgeon a clear anatomical orientation before endoscopic sinus surgery. In this study, a unique method is presented for detecting and diagnosing paranasal sinus disorders using machine learning. The researchers behind the current study designed their own approach. To speed up diagnosis, one of the primary goals of our study is to create an algorithm that can accurately evaluate the paranasal sinuses in CT scans. The proposed technology makes it feasible to automatically cut down on the number of CT scan images that require investigators to manually search through them all. In addition, the approach offers an automatic segmentation that may be used to locate the paranasal sinus region and crop it accordingly. As a result, the suggested method dramatically reduces the amount of data that is necessary during the training phase. As a result, this results in an increase in the efficiency of the computer while retaining a high degree of performance accuracy. The suggested method not only successfully identifies sinus irregularities but also automatically executes the necessary segmentation without requiring any manual cropping. This eliminates the need for time-consuming and error-prone human labor. When tested with actual CT scans, the method in question was discovered to have an accuracy of 95.16 percent while retaining a sensitivity of 99.14 percent throughout.
Collapse
Affiliation(s)
- Abdullah Musleh
- Department of Surgery, King Khalid University, Abha, Saudi Arabia
| |
Collapse
|
7
|
Wu Q, Wang X, Liang G, Luo X, Zhou M, Deng H, Zhang Y, Huang X, Yang Q. Advances in Image-Based Artificial Intelligence in Otorhinolaryngology-Head and Neck Surgery: A Systematic Review. Otolaryngol Head Neck Surg 2023; 169:1132-1142. [PMID: 37288505 DOI: 10.1002/ohn.391] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Revised: 04/27/2023] [Accepted: 05/13/2023] [Indexed: 06/09/2023]
Abstract
OBJECTIVE To update the literature and provide a systematic review of image-based artificial intelligence (AI) applications in otolaryngology, highlight its advances, and propose future challenges. DATA SOURCES Web of Science, Embase, PubMed, and Cochrane Library. REVIEW METHODS Studies written in English, published between January 2020 and December 2022. Two independent authors screened the search results, extracted data, and assessed studies. RESULTS Overall, 686 studies were identified. After screening titles and abstracts, 325 full-text studies were assessed for eligibility, and 78 studies were included in this systematic review. The studies originated from 16 countries. Among these countries, the top 3 were China (n = 29), Korea (n = 8), the United States, and Japan (n = 7 each). The most common area was otology (n = 35), followed by rhinology (n = 20), pharyngology (n = 18), and head and neck surgery (n = 5). Most applications of AI in otology, rhinology, pharyngology, and head and neck surgery mainly included chronic otitis media (n = 9), nasal polyps (n = 4), laryngeal cancer (n = 12), and head and neck squamous cell carcinoma (n = 3), respectively. The overall performance of AI in accuracy, the area under the curve, sensitivity, and specificity were 88.39 ± 9.78%, 91.91 ± 6.70%, 86.93 ± 11.59%, and 88.62 ± 14.03%, respectively. CONCLUSION This state-of-the-art review aimed to highlight the increasing applications of image-based AI in otorhinolaryngology head and neck surgery. The following steps will entail multicentre collaboration to ensure data reliability, ongoing optimization of AI algorithms, and integration into real-world clinical practice. Future studies should consider 3-dimensional (3D)-based AI, such as 3D surgical AI.
Collapse
Affiliation(s)
- Qingwu Wu
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
- Department of Allergy, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Xinyue Wang
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Guixian Liang
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Xin Luo
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Min Zhou
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
- Department of Allergy, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Huiyi Deng
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Yana Zhang
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Xuekun Huang
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Qintai Yang
- Department of Otorhinolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
- Department of Allergy, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
8
|
Khan MMR, Fan Y, Dawant BM, Noble JH. Cochlear Implant Fold Detection in Intra-operative CT Using Weakly Supervised Multi-task Deep Learning. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2023; 14228:249-259. [PMID: 38515783 PMCID: PMC10953791 DOI: 10.1007/978-3-031-43996-4_24] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/23/2024]
Abstract
In cochlear implant (CI) procedures, an electrode array is surgically inserted into the cochlea. The electrodes are used to stimulate the auditory nerve and restore hearing sensation for the recipient. If the array folds inside the cochlea during the insertion procedure, it can lead to trauma, damage to the residual hearing, and poor hearing restoration. Intraoperative detection of such a case can allow a surgeon to perform reimplantation. However, this intraoperative detection requires experience and electrophysiological tests sometimes fail to detect an array folding. Due to the low incidence of array folding, we generated a dataset of CT images with folded synthetic electrode arrays with realistic metal artifact. The dataset was used to train a multitask custom 3D-UNet model for array fold detection. We tested the trained model on real post-operative CTs (7 with folded arrays and 200 without). Our model could correctly classify all the fold-over cases while misclassifying only 3 non fold-over cases. Therefore, the model is a promising option for array fold detection.
Collapse
Affiliation(s)
- Mohammad M R Khan
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
| | - Yubo Fan
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
| | - Benoit M Dawant
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
| | - Jack H Noble
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
| |
Collapse
|
9
|
Banalagay RA, Labadie RF, Noble JH. Validation of active shape model techniques for intracochlear anatomy segmentation in computed tomography images. J Med Imaging (Bellingham) 2023; 10:044003. [PMID: 37476645 PMCID: PMC10355218 DOI: 10.1117/1.jmi.10.4.044003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2022] [Revised: 05/28/2023] [Accepted: 06/20/2023] [Indexed: 07/22/2023] Open
Abstract
Purpose Cochlear implants (CIs) have been shown to be highly effective restorative devices for patients suffering from severe-to-profound hearing loss. Hearing outcomes with CIs depend on electrode positions with respect to intracochlear anatomy. Intracochlear anatomy can only be directly visualized using high-resolution modalities, such as micro-computed tomography (μ CT ), which cannot be used in vivo. However, active shape models (ASM) have been shown to be robust and effective for segmenting intracochlear anatomy in large scale datasets of patient computed tomographies (CTs). We present an extended dataset of μ CT specimens and aim to evaluate the ASM's performance more comprehensively than has been previously possible. Approach Using a dataset of 16 manually segmented cochlea specimens on μ CTs , we found parameters that optimize mean CT segmentation performance and then evaluate the effect of library size on the ASM. The optimized ASM was further evaluated on a clinical dataset of 134 CT images to assess method reliability. Results Optimized parameters lead to mean CT segmentation performance to 0.36 mm point-to-point error, 0.10 mm surface error, and 0.83 Dice score. Larger library sizes provide diminishing returns on segmentation performance and total variance captured by the ASM. We found our method to be clinically reliable with the main performance limitation that was found to be the candidate search process rather than model representation. Conclusions We have presented a comprehensive validation of the ASM for use in intracochlear anatomy segmentation. These results are critical to understand the limitations of the method for clinical use and for future development.
Collapse
Affiliation(s)
- Rueben A. Banalagay
- Vanderbilt University, Department of Electrical and Computer Engineering, Nashville, Tennessee, United States
| | - Robert F. Labadie
- Medical University of South Carolina, Department of Otolaryngology—Head & Neck Surgery, Charleston, South Carolina, United States
| | - Jack H. Noble
- Vanderbilt University, Department of Electrical and Computer Engineering, Nashville, Tennessee, United States
| |
Collapse
|
10
|
Hasan Z, Key S, Habib AR, Wong E, Aweidah L, Kumar A, Sacks R, Singh N. Convolutional Neural Networks in ENT Radiology: Systematic Review of the Literature. Ann Otol Rhinol Laryngol 2023; 132:417-430. [PMID: 35651308 DOI: 10.1177/00034894221095899] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
INTRODUCTION Convolutional neural networks (CNNs) represent a state-of-the-art methodological technique in AI and deep learning, and were specifically created for image classification and computer vision tasks. CNNs have been applied in radiology in a number of different disciplines, mostly outside otolaryngology, potentially due to a lack of familiarity with this technology within the otolaryngology community. CNNs have the potential to revolutionize clinical practice by reducing the time required to perform manual tasks. This literature search aims to present a comprehensive systematic review of the published literature with regard to CNNs and their utility to date in ENT radiology. METHODS Data were extracted from a variety of databases including PubMED, Proquest, MEDLINE Open Knowledge Maps, and Gale OneFile Computer Science. Medical subject headings (MeSH) terms and keywords were used to extract related literature from each databases inception to October 2020. Inclusion criteria were studies where CNNs were used as the main intervention and CNNs focusing on radiology relevant to ENT. Titles and abstracts were reviewed followed by the contents. Once the final list of articles was obtained, their reference lists were also searched to identify further articles. RESULTS Thirty articles were identified for inclusion in this study. Studies utilizing CNNs in most ENT subspecialties were identified. Studies utilized CNNs for a number of tasks including identification of structures, presence of pathology, and segmentation of tumors for radiotherapy planning. All studies reported a high degree of accuracy of CNNs in performing the chosen task. CONCLUSION This study provides a better understanding of CNN methodology used in ENT radiology demonstrating a myriad of potential uses for this exciting technology including nodule and tumor identification, identification of anatomical variation, and segmentation of tumors. It is anticipated that this field will continue to evolve and these technologies and methodologies will become more entrenched in our everyday practice.
Collapse
Affiliation(s)
- Zubair Hasan
- Faculty of Medicine and Health, University of Sydney, Camperdown, NSW, Australia
- Department of Otolaryngology - Head and Neck Surgery, Westmead Hospital, Westmead, NSW, Australia
| | - Seraphina Key
- Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, VIC, Australia
| | - Al-Rahim Habib
- Faculty of Medicine and Health, University of Sydney, Camperdown, NSW, Australia
- Department of Otolaryngology - Head and Neck Surgery, Westmead Hospital, Westmead, NSW, Australia
- Department of Otolaryngology - Head and Neck Surgery, Princess Alexandra Hospital, Woolloongabba, QLD, Australia
| | - Eugene Wong
- Department of Otolaryngology - Head and Neck Surgery, Westmead Hospital, Westmead, NSW, Australia
| | - Layal Aweidah
- Faculty of Medicine, University of Notre Dame, Darlinghurst, NSW, Australia
| | - Ashnil Kumar
- School of Biomedical Engineering, Faculty of Engineering, University of Sydney, Darlington, NSW, Australia
| | - Raymond Sacks
- Faculty of Medicine and Health, University of Sydney, Camperdown, NSW, Australia
- Department of Otolaryngology - Head and Neck Surgery, Concord Hospital, Concord, NSW, Australia
| | - Narinder Singh
- Faculty of Medicine and Health, University of Sydney, Camperdown, NSW, Australia
- Department of Otolaryngology - Head and Neck Surgery, Westmead Hospital, Westmead, NSW, Australia
| |
Collapse
|
11
|
Zhao Y, Wang X, Che T, Bao G, Li S. Multi-task deep learning for medical image computing and analysis: A review. Comput Biol Med 2023; 153:106496. [PMID: 36634599 DOI: 10.1016/j.compbiomed.2022.106496] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 12/06/2022] [Accepted: 12/27/2022] [Indexed: 12/29/2022]
Abstract
The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.
Collapse
Affiliation(s)
- Yan Zhao
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, 100083, China
| | - Xiuying Wang
- School of Computer Science, The University of Sydney, Sydney, NSW, 2008, Australia.
| | - Tongtong Che
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, 100083, China
| | - Guoqing Bao
- School of Computer Science, The University of Sydney, Sydney, NSW, 2008, Australia
| | - Shuyu Li
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, 100875, China.
| |
Collapse
|
12
|
Su Y, Sun Y, Hosny M, Gao W, Fu Y. Facial landmark-guided surface matching for image-to-patient registration with an RGB-D camera. Int J Med Robot 2022; 18:e2373. [PMID: 35133715 DOI: 10.1002/rcs.2373] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Revised: 01/17/2022] [Accepted: 01/29/2022] [Indexed: 01/01/2023]
Abstract
BACKGROUND Fiducial marker-based image-to-patient registration is the most common way in image-guided neurosurgery, which is labour-intensive, time consuming, invasive and error prone. METHODS We proposed a method of facial landmark-guided surface matching for image-to-patient registration using an RGB-D camera. Five facial landmarks are localised from preoperative magnetic resonance (MR) images using deep learning and RGB image using Adaboost with multi-scale block local binary patterns, respectively. The registration of two facial surface point clouds derived from MR images and RGB-D data is initialised by aligning these five landmarks and further refined by weighted iterative closest point algorithm. RESULTS Phantom experiment results show the target registration error is less than 3 mm when the distance from the camera to the phantom is less than 1000 mm. The registration takes less than 10 s. CONCLUSIONS The proposed method is comparable to the state-of-the-arts in terms of the accuracy yet more time-saving and non-invasive.
Collapse
Affiliation(s)
- Yixian Su
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China
| | - Yu Sun
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China
| | - Mohamed Hosny
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China.,Department of Electrical Engineering, Benha Faculty of Engineering, Benha University, Benha, Egypt
| | - Wenpeng Gao
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China
| | - Yili Fu
- State Key Laboratory of Robotics and System, School of Life Science and Technology, Harbin Institute of Technology, Harbin, China
| |
Collapse
|
13
|
Schurer-Waldheim S, Seebock P, Bogunovic H, Gerendas BS, Schmidt-Erfurth U. Robust Fovea Detection in Retinal OCT Imaging using Deep Learning. IEEE J Biomed Health Inform 2022; 26:3927-3937. [PMID: 35394920 DOI: 10.1109/jbhi.2022.3166068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The fovea centralis is an essential landmark in the retina where the photoreceptor layer is entirely composed of cones responsible for sharp, central vision. The localization of this anatomical landmark in optical coherence tomography (OCT) volumes is important for assessing visual function correlates and treatment guidance in macular disease. In this study, the "PRE U-net" is introduced as a novel approach for a fully automated fovea centralis detection, addressing the localization as a pixel-wise regression task. 2D B-scans are sampled from each image volume and are concatenated with spatial location information to train the deep network. A total of 5586 OCT volumes from 1,541 eyes was used to train, validate and test the deep learning method. The test data is comprised of healthy subjects and patients affected by neovascular age-related macular degeneration (nAMD), diabetic macula edema (DME) and macular edema from retinal vein occlusion (RVO), covering the three major retinal diseases responsible for blindness. Our experiments demonstrate that the PRE U-net significantly outperforms state-of-the-art methods and improves the robustness of automated localization, which is of value for clinical practice.
Collapse
|
14
|
Fan Y, Banalagay RA, Cass ND, Noble JH, Tawfik KO, Labadie RF, Dawant BM. Automatic Segmentation of Intracochlear Anatomy in MR Images Using a Weighted Active Shape Model. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3573-3576. [PMID: 34892011 PMCID: PMC8964074 DOI: 10.1109/embc46164.2021.9630332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
There is evidence that cochlear MR signal intensity may be useful in prognosticating the risk of hearing loss after middle cranial fossa (MCF) resection of acoustic neuroma (AN), but the manual segmentation of this structure is difficult and prone to error. This hampers both large-scale retrospective studies and routine clinical use of this information. To address this issue, we present a fully automatic method that permits the segmentation of the intra-cochlear anatomy in MR images, which uses a weighted active shape model we have developed and validated to segment the intra-cochlear anatomy in CT images. We take advantage of a dataset for which both CT and MR images are available to validate our method on 132 ears in 66 high-resolution T2-weighted MR images. Using the CT segmentation as ground truth, we achieve a mean Dice (DSC) value of 0.81 and 0.79 for the scala tympani (ST) and the scala vestibuli (SV), which are the two main intracochlear structures.Clinical Relevance- The proposed method is accurate and fully automated for MR image segmentation. It can be used to support large retrospective studies that explore relations between MR signal in preoperative images and outcomes. It can also facilitate the routine and clinical use of this information.
Collapse
|
15
|
Chawdhary G, Shoman N. Emerging artificial intelligence applications in otological imaging. Curr Opin Otolaryngol Head Neck Surg 2021; 29:357-364. [PMID: 34459798 DOI: 10.1097/moo.0000000000000754] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
PURPOSE OF REVIEW To highlight the recent literature on artificial intelligence (AI) pertaining to otological imaging and to discuss future directions, obstacles and opportunities. RECENT FINDINGS The main themes in the recent literature centre around automated otoscopic image diagnosis and automated image segmentation for application in virtual reality surgical simulation and planning. Other applications that have been studied include identification of tinnitus MRI biomarkers, facial palsy analysis, intraoperative augmented reality systems, vertigo diagnosis and endolymphatic hydrops ratio calculation in Meniere's disease. Studies are presently at a preclinical, proof-of-concept stage. SUMMARY The recent literature on AI in otological imaging is promising and demonstrates the future potential of this technology in automating certain imaging tasks in a healthcare environment of ever-increasing demand and workload. Some studies have shown equivalence or superiority of the algorithm over physicians, albeit in narrowly defined realms. Future challenges in developing this technology include the compilation of large high quality annotated datasets, fostering strong collaborations between the health and technology sectors, testing the technology within real-world clinical pathways and bolstering trust among patients and physicians in this new method of delivering healthcare.
Collapse
Affiliation(s)
- Gaurav Chawdhary
- ENT Department, Royal Hallamshire Hospital, Broomhall, Sheffield, UK
| | - Nael Shoman
- ENT Department, Queen Elizabeth II Health Sciences Centre, Halifax, Nova Scotia, Canada
| |
Collapse
|
16
|
CNL-UNet: A novel lightweight deep learning architecture for multimodal biomedical image segmentation with false output suppression. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102959] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
17
|
Recognition of Thyroid Ultrasound Standard Plane Images Based on Residual Network. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2021; 2021:5598001. [PMID: 34188673 PMCID: PMC8192196 DOI: 10.1155/2021/5598001] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/17/2021] [Revised: 04/27/2021] [Accepted: 05/14/2021] [Indexed: 01/22/2023]
Abstract
Ultrasound is one of the critical methods for diagnosis and treatment in thyroid examination. In clinical application, many reasons, such as large outpatient traffic, time-consuming training of sonographers, and uneven professional level of physicians, often cause irregularities during the ultrasonic examination, leading to misdiagnosis or missed diagnosis. In order to standardize the thyroid ultrasound examination process, this paper proposes using a deep learning method based on residual network to recognize the Thyroid Ultrasound Standard Plane (TUSP). At first, referring to multiple relevant guidelines, eight TUSP were determined with the advice of clinical ultrasound experts. A total of 5,500 TUSP images of 8 categories were collected with the approval and review of the Ethics Committee and the patient's informed consent. Then, after desensitizing and filling the images, the 18-layer residual network model (ResNet-18) was trained for TUSP image recognition, and five-fold cross-validation was performed. Finally, through indicators like accuracy rate, we compared the recognition effect of other mainstream deep convolutional neural network models. Experimental results showed that ResNet-18 has the best recognition effect on TUSP images with an average accuracy rate of 91.07%. The average macro precision, average macro recall, and average macro F1-score are 91.39%, 91.34%, and 91.30%, respectively. It proves that the deep learning method based on residual network can effectively recognize TUSP images, which is expected to standardize clinical thyroid ultrasound examination and reduce misdiagnosis and missed diagnosis.
Collapse
|
18
|
Fully automated preoperative segmentation of temporal bone structures from clinical CT scans. Sci Rep 2021; 11:116. [PMID: 33420386 PMCID: PMC7794235 DOI: 10.1038/s41598-020-80619-0] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Accepted: 12/23/2020] [Indexed: 11/11/2022] Open
Abstract
Middle- and inner-ear surgery is a vital treatment option in hearing loss, infections, and tumors of the lateral skull base. Segmentation of otologic structures from computed tomography (CT) has many potential applications for improving surgical planning but can be an arduous and time-consuming task. We propose an end-to-end solution for the automated segmentation of temporal bone CT using convolutional neural networks (CNN). Using 150 manually segmented CT scans, a comparison of 3 CNN models (AH-Net, U-Net, ResNet) was conducted to compare Dice coefficient, Hausdorff distance, and speed of segmentation of the inner ear, ossicles, facial nerve and sigmoid sinus. Using AH-Net, the Dice coefficient was 0.91 for the inner ear; 0.85 for the ossicles; 0.75 for the facial nerve; and 0.86 for the sigmoid sinus. The average Hausdorff distance was 0.25, 0.21, 0.24 and 0.45 mm, respectively. Blinded experts assessed the accuracy of both techniques, and there was no statistical difference between the ratings for the two methods (p = 0.93). Objective and subjective assessment confirm good correlation between automated segmentation of otologic structures and manual segmentation performed by a specialist. This end-to-end automated segmentation pipeline can help to advance the systematic application of augmented reality, simulation, and automation in otologic procedures.
Collapse
|
19
|
Noothout JMH, De Vos BD, Wolterink JM, Postma EM, Smeets PAM, Takx RAP, Leiner T, Viergever MA, Isgum I. Deep Learning-Based Regression and Classification for Automatic Landmark Localization in Medical Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:4011-4022. [PMID: 32746142 DOI: 10.1109/tmi.2020.3009002] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Abstract
In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vectors pointing from the center of image patches towards landmark locations are determined. In classification, presence of landmarks of interest in the patch is established. Global landmark locations are obtained by averaging the predicted displacement vectors, where the contribution of each displacement vector is weighted by the posterior classification probability of the patch that it is pointing from. Subsequently, for each landmark localized with global localization, local analysis is performed. Specialized FCNNs refine the global landmark locations by analyzing local sub-images in a similar manner, i.e. by performing regression and classification simultaneously and combining the results. Evaluation was performed through localization of 8 anatomical landmarks in CCTA scans, 2 landmarks in olfactory MR scans, and 19 landmarks in cephalometric X-rays. We demonstrate that the method performs similarly to a second observer and is able to localize landmarks in a diverse set of medical images, differing in image modality, image dimensionality, and anatomical coverage.
Collapse
|
20
|
Lang Y, Lian C, Xiao D, Deng H, Yuan P, Gateno J, Shen SGF, Alfi DM, Yap PT, Xia JJ, Shen D. Automatic Localization of Landmarks in Craniomaxillofacial CBCT Images Using a Local Attention-Based Graph Convolution Network. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2020; 12264:817-826. [PMID: 34927175 PMCID: PMC8675277 DOI: 10.1007/978-3-030-59719-1_79] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
Landmark localization is an important step in quantifying craniomaxillofacial (CMF) deformities and designing treatment plans of reconstructive surgery. However, due to the severity of deformities and defects (partially missing anatomy), it is difficult to automatically and accurately localize a large set of landmarks simultaneously. In this work, we propose two cascaded networks for digitizing 60 anatomical CMF landmarks in cone-beam computed tomography (CBCT) images. The first network is a U-Net that outputs heatmaps for landmark locations and landmark features extracted with a local attention mechanism. The second network is a graph convolution network that takes the features extracted by the first network as input and determines whether each landmark exists via binary classification. We evaluated our approach on 50 sets of CBCT scans of patients with CMF deformities and compared them with state-of-the-art methods. The results indicate that our approach can achieve an average detection error of 1.47mm with a false positive rate of 19%, outperforming related methods.
Collapse
Affiliation(s)
- Yankun Lang
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Chunfeng Lian
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Deqiang Xiao
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Hannah Deng
- Department of Oral and Maxillofacial Surgery, Houston Methodist Hospital, Houston, TX, USA
| | - Peng Yuan
- Department of Oral and Maxillofacial Surgery, Houston Methodist Hospital, Houston, TX, USA
| | - Jaime Gateno
- Department of Oral and Maxillofacial Surgery, Houston Methodist Hospital, Houston, TX, USA
- Department of Surgery (Oral and Maxillofacial Surgery), Weill Medical College, Cornell University, Ithaca, NY, USA
| | - Steve G F Shen
- Department of Oral and Craniofacial Surgery, Shanghai 9th Hospital, Shanghai Jiaotong University College of Medicine, Shanghai, China
| | - David M Alfi
- Department of Oral and Maxillofacial Surgery, Houston Methodist Hospital, Houston, TX, USA
- Department of Surgery (Oral and Maxillofacial Surgery), Weill Medical College, Cornell University, Ithaca, NY, USA
| | - Pew-Thian Yap
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - James J Xia
- Department of Oral and Maxillofacial Surgery, Houston Methodist Hospital, Houston, TX, USA
- Department of Surgery (Oral and Maxillofacial Surgery), Weill Medical College, Cornell University, Ithaca, NY, USA
| | - Dinggang Shen
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| |
Collapse
|