Fully automated preoperative segmentation of temporal bone structures from clinical CT scans

Deep Learning for Automated Image Segmentation of the Middle Ear: A Scoping Review

OBJECTIVE

Convolutional neural networks (CNNs) have revolutionized medical image segmentation in recent years. This scoping review aimed to carry out a comprehensive review of the literature describing automated image segmentation of the middle ear using CNNs from computed tomography (CT) scans.

DATA SOURCES

A comprehensive literature search, generated jointly with a medical librarian, was performed on Medline, Embase, Scopus, Web of Science, and Cochrane, using Medical Subject Heading terms and keywords. Databases were searched from inception to July 2023. Reference lists of included papers were also screened.

REVIEW METHODS

Ten studies were included for analysis, which contained a total of 866 scans which were used in model training/testing. Thirteen different architectures were described to perform automated segmentation. The best Dice similarity coefficient (DSC) for the entire ossicular chain was 0.87 using ResNet. The highest DSC for any structure was the incus using 3D-V-Net at 0.93. The most difficult structure to segment was the stapes, with the highest DSC of 0.84 using 3D-V-Net.

CONCLUSIONS

Numerous architectures have demonstrated good performance in segmenting the middle ear using CNNs. To overcome some of the difficulties in segmenting the stapes, we recommend the development of an architecture trained on cone beam CTs to provide improved spatial resolution to assist with delineating the smallest ossicle.

IMPLICATIONS FOR PRACTICE

This has clinical applications for preoperative planning, diagnosis, and simulation.

Deep Learning Method for Rapid Simultaneous Multistructure Temporal Bone Segmentation

OBJECTIVE

To develop and validate a deep learning algorithm for the automated segmentation of key temporal bone structures from clinical computed tomography (CT) data sets.

STUDY DESIGN

Cross-sectional study.

SETTING

A total of 325 CT scans from a clinical database.

METHOD

A state-of-the-art deep learning (DL) algorithm (SwinUNETR) was used to train a prediction model for rapid segmentation of 9 key temporal bone structures in a data set of 325 clinical CTs. The data set was manually annotated by a specialist to serve as the ground truth. The data set was randomly split into training (n = 260) and testing (n = 65) sets. The model's performance was objectively assessed through external validation on the test set using metrics including Dice, Balanced accuracy, Hausdorff distances, and processing time.

RESULTS

The model achieved an average Dice coefficient of 0.87 for all structures, an average balanced accuracy of 0.94, an average Hausdorff distance of 0.79 mm, and an average processing time of 9.1 seconds per CT.

CONCLUSION

The present DL model for the automated simultaneous segmentation of multiple structures within the temporal bone from CTs achieved high accuracy according to currently commonly employed objective analysis. The results demonstrate the potential of the method to improve preoperative evaluation and intraoperative guidance in otologic surgery.

Clinical application of high-resolution spiral CT scanning in the diagnosis of auriculotemporal and ossicle

Precision and intelligence in evaluating the complexities of middle ear structures are required to diagnose auriculotemporal and ossicle-related diseases within otolaryngology. Due to the complexity of the anatomical details and the varied etiologies of illnesses such as trauma, chronic otitis media, and congenital anomalies, traditional diagnostic procedures may not yield accurate diagnoses. This research intends to enhance the diagnosis of diseases of the auriculotemporal region and ossicles by combining High-Resolution Spiral Computed Tomography (HRSCT) scanning with Deep Learning Techniques (DLT). This study employs a deep learning method, Convolutional Neural Network-UNet (CNN-UNet), to extract sub-pixel information from medical photos. This method equips doctors and researchers with cutting-edge resources, leading to groundbreaking discoveries and better patient healthcare. The research effort is the interaction between the CNN-UNet model and high-resolution Computed Tomography (CT) scans, automating activities including ossicle segmentation, fracture detection, and disruption cause classification, accelerating the diagnostic process and increasing clinical decision-making. The suggested HRSCT-DLT model represents the integration of high-resolution spiral CT scans with the CNN-UNet model, which has been fine-tuned to address the nuances of auriculotemporal and ossicular diseases. This novel combination improves diagnostic efficiency and our overall understanding of these intricate diseases. The results of this study highlight the promise of combining high-resolution CT scanning with the CNN-UNet model in otolaryngology, paving the way for more accurate diagnosis and more individualized treatment plans for patients experiencing auriculotemporal and ossicle-related disruptions.

Surgical Navigation in the Anterior Skull Base Using 3-Dimensional Endoscopy and Surface Reconstruction

Importance

Image guidance is an important adjunct for endoscopic sinus and skull base surgery. However, current systems require bulky external tracking equipment, and their use can interrupt efficient surgical workflow.

Objective

To evaluate a trackerless surgical navigation system using 3-dimensional (3D) endoscopy and simultaneous localization and mapping (SLAM) algorithms in the anterior skull base.

Design, Setting, and Participants

This interventional deceased donor cohort study and retrospective clinical case study was conducted at a tertiary academic medical center with human deceased donor specimens and a patient with anterior skull base pathology.

Exposures

Participants underwent endoscopic endonasal transsphenoidal dissection and surface model reconstruction from stereoscopic video with registration to volumetric models segmented from computed tomography (CT) and magnetic resonance imaging.

Main Outcomes and Measures

To assess the fidelity of surface model reconstruction and accuracy of surgical navigation and surface-CT model coregistration, 3 metrics were calculated: reconstruction error, registration error, and localization error.

Results

In deceased donor models (n = 9), high-fidelity surface models of the posterior wall of the sphenoid sinus were reconstructed from stereoscopic video and coregistered to corresponding volumetric CT models. The mean (SD; range) reconstruction, registration, and localization errors were 0.60 (0.24; 0.36-0.93), 1.11 (0.49; 0.71-1.56) and 1.01 (0.17; 0.78-1.25) mm, respectively. In a clinical case study of a patient who underwent a 3D endoscopic endonasal transsphenoidal resection of a tubercular meningioma, a high-fidelity surface model of the posterior wall of the sphenoid was reconstructed from intraoperative stereoscopic video and coregistered to a volumetric preoperative fused CT magnetic resonance imaging model with a root-mean-square error of 1.38 mm.

Conclusions and Relevance

The results of this study suggest that SLAM algorithm-based endoscopic endonasal surgery navigation is a novel, accurate, and trackerless approach to surgical navigation that uses 3D endoscopy and SLAM-based algorithms in lieu of conventional optical or electromagnetic tracking. While multiple challenges remain before clinical readiness, a SLAM algorithm-based endoscopic endonasal surgery navigation system has the potential to improve surgical efficiency, economy of motion, and safety.

Improving rehabilitation of deaf patients by advanced imaging before cochlear implantation

INTRODUCTION

Cochlear implants have advanced the management of severe to profound deafness. However, there is a strong disparity in hearing performance after implantation from one patient to another. Moreover, there are several advanced kinds of imaging assessment before cochlear implantation. Microstructural white fiber degeneration can be studied with Diffusion weighted MRI (DWI) or tractography of the central auditory pathways. Functional MRI (fMRI) allows us to evaluate brain function, and CT or MRI segmentation to better detect inner ear anomalies.

OBJECTIVE

This literature review aims to evaluate how helpful pre-implantation anatomic imaging can be to predict hearing rehabilitation outcomes in deaf patients. These techniques include DWI and fMRI of the central auditory pathways, and automated labyrinth segmentation by CT scan, cone beam CT and MRI.

DESIGN

This systematic review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Studies were selected by searching in PubMed and by checking the reference lists of relevant articles. Inclusion criteria were adults over 18, with unilateral or bilateral hearing loss, who had DWI acquisition or fMRI or CT/ Cone Beam CT/ MRI image segmentation.

RESULTS

After reviewing 172 articles, we finally included 51. Studies on DWI showed changes in the central auditory pathways affecting the white matter, extending to the primary and non-primary auditory cortices, even in sudden and mild hearing impairment. Hearing loss patients show a reorganization of brain activity in various areas, such as the auditory and visual cortices, as well as regions involved in language and emotions, according to fMRI studies. Deep Learning's automatic segmentation produces the best CT segmentation in just a few seconds. MRI segmentation is mainly used to evaluate fluid space of the inner ear and determine the presence of an endolymphatic hydrops.

CONCLUSION

Before cochlear implantation, a DWI with tractography can evaluate the central auditory pathways up to the primary and non-primary auditory cortices. This data is then used to generate predictions on the auditory rehabilitation of patients. A CT segmentation with systematic 3D reconstruction allow a better evaluation of cochlear malformations and predictable difficulties during surgery.

Variability in Manual Segmentation of Temporal Bone Structures in Cone Beam CT Images

PURPOSE

Manual segmentation of anatomical structures is the accepted "gold standard" for labeling structures in clinical images. However, the variability in manual segmentation of temporal bone structures in CBCT images of the temporal bone has not been systematically evaluated using multiple reviewers. Therefore, we evaluated the intravariability and intervariability of manual segmentation of inner ear structures in CBCT images of the temporal bone.

METHODS

Preoperative CBCTs scans of the inner ear were obtained from 10 patients who had undergone cochlear implant surgery. The cochlea, facial nerve, chorda tympani, mid-modiolar (MM) axis, and round window (RW) were manually segmented by five reviewers in two separate sessions that were at least 1 month apart. Interreviewer and intrareviewer variabilities were assessed using the Dice coefficient (DICE), volume similarity, mean Hausdorff Distance metrics, and visual review.

RESULTS

Manual segmentation of the cochlea was the most consistent within and across reviewers with a mean DICE of 0.91 (SD = 0.02) and 0.89 (SD = 0.01) respectively, followed by the facial nerve with a mean DICE of 0.83 (SD = 0.02) and 0.80 (SD = 0.03), respectively. The chorda tympani had the greatest amount of reviewer variability due to its thin size, and the location of the centroid of the RW and the MM axis were also quite variable between and within reviewers.

CONCLUSIONS

We observed significant variability in manual segmentation of some of the temporal bone structures across reviewers. This variability needs to be considered when interpreting the results in studies using one manual reviewer.

Estimation of Cochlear Implant Insertion Depth Using 2D-3D Registration of Postoperative X-Ray and Preoperative CT Images

OBJECTIVE

To improve estimation of cochlear implant (CI) insertion depth in postoperative skull x-rays using synthesized information from preoperative CT scans.

STUDY DESIGN

Retrospective cohort.

SETTING

Tertiary referral center.

PATIENTS

Ten adult cochlear implant recipients with preoperative and postoperative temporal bone computed tomography (CT)scans and postoperative skull x-ray imaging.

INTERVENTIONS

Postoperative x-rays and digitally reconstructed radiographs (DRR) from preoperative CTs were registered using 3D Slicer and MATLAB to enhance localization of the round window and modiolus. Angular insertion depth (AID) was estimated in unmodified and registration-enhanced x-rays and DRRs in the cochlear view. Linear insertion depth (LID) was estimated in registered images by two methods that localized the proximal CI electrode or segmented the cochlea. Ground truth assessments were made in postoperative CTs.

MAIN OUTCOME MEASURES

Errors of insertion depth estimates were calculated relative to ground truth measurements and compared with paired t t ests. Pearson correlation coefficient was used to assess inter-rater reliability of two reviewer's measurements of AID in unmodified x-rays.

RESULTS

In postoperative x-rays, AID estimation errors were similar with and without registration enhancement (-1.3 ± 20.7° and -4.8 ± 24.9°, respectively; mean ± SD; p = 0.6). AID estimation in unmodified x-rays demonstrated strong interrater agreement (ρ = 0.79, p < 0.05) and interrater differences (-15.0 ± 35.3°) comparable to estimate errors. Registering images allowed measurement of AID in the cochlear view with estimation errors of 14.6 ± 30.6° and measurement of LID, with estimate errors that were similar between proximal electrode localization and cochlear segmentation methods (-0.9 ± 2.2 mm and -2.1 ± 2.7 mm, respectively; p = 0.3).

CONCLUSIONS

2D-3D image registration allows measurement of AID in the cochlear view and LID using postoperative x-rays and preoperative CT imaging. The use of this technique may reduce the need for postimplantation CT studies to assess these metrics of CI electrode position. Further work is needed to improve the accuracy of AID assessment in the postoperative x-ray view with registered images compared with established methods.

Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework

Automated analysis of the inner ear anatomy in radiological data instead of time-consuming manual assessment is a worthwhile goal that could facilitate preoperative planning and clinical research. We propose a framework encompassing joint semantic segmentation of the inner ear and anatomical landmark detection of helicotrema, oval and round window. A fully automated pipeline with a single, dual-headed volumetric 3D U-Net was implemented, trained and evaluated using manually labeled in-house datasets from cadaveric specimen ([Formula: see text]) and clinical practice ([Formula: see text]). The model robustness was further evaluated on three independent open-source datasets ([Formula: see text] scans) consisting of cadaveric specimen scans. For the in-house datasets, Dice scores of [Formula: see text], intersection-over-union scores of [Formula: see text] and average Hausdorff distances of [Formula: see text] and [Formula: see text] voxel units were achieved. The landmark localization task was performed automatically with an average localization error of [Formula: see text] voxel units. A robust, albeit reduced performance could be attained for the catalogue of three open-source datasets. Results of the ablation studies with 43 mono-parametric variations of the basal architecture and training protocol provided task-optimal parameters for both categories. Ablation studies against single-task variants of the basal architecture showed a clear performance benefit of coupling landmark localization with segmentation and a dataset-dependent performance impact on segmentation ability.

Stereoscopic calibration for augmented reality visualization in microscopic surgery

PURPOSE

Middle and inner ear procedures target hearing loss, infections, and tumors of the temporal bone and lateral skull base. Despite the advances in surgical techniques, these procedures remain challenging due to limited haptic and visual feedback. Augmented reality (AR) may improve operative safety by allowing the 3D visualization of anatomical structures from preoperative computed tomography (CT) scans on real intraoperative microscope video feed. The purpose of this work was to develop a real-time CT-augmented stereo microscope system using camera calibration and electromagnetic (EM) tracking.

METHODS

A 3D printed and electromagnetically tracked calibration board was used to compute the intrinsic and extrinsic parameters of the surgical stereo microscope. These parameters were used to establish a transformation between the EM tracker coordinate system and the stereo microscope image space such that any tracked 3D point can be projected onto the left and right images of the microscope video stream. This allowed the augmentation of the microscope feed of a 3D printed temporal bone with its corresponding CT-derived virtual model. Finally, the calibration board was also used for evaluating the accuracy of the calibration.

RESULTS

We evaluated the accuracy of the system by calculating the registration error (RE) in 2D and 3D in a microsurgical laboratory setting. Our calibration workflow achieved a RE of 0.11 ± 0.06 mm in 2D and 0.98 ± 0.13 mm in 3D. In addition, we overlaid a 3D CT model on the microscope feed of a 3D resin printed model of a segmented temporal bone. The system exhibited small latency and good registration accuracy.

CONCLUSION

We present the calibration of an electromagnetically tracked surgical stereo microscope for augmented reality visualization. The calibration method achieved accuracy within a range suitable for otologic procedures. The AR process introduces enhanced visualization of the surgical field while allowing depth perception.

A Self-Configuring Deep Learning Network for Segmentation of Temporal Bone Anatomy in Cone-Beam CT Imaging

OBJECTIVE

Preoperative planning for otologic or neurotologic procedures often requires manual segmentation of relevant structures, which can be tedious and time-consuming. Automated methods for segmenting multiple geometrically complex structures can not only streamline preoperative planning but also augment minimally invasive and/or robot-assisted procedures in this space. This study evaluates a state-of-the-art deep learning pipeline for semantic segmentation of temporal bone anatomy.

STUDY DESIGN

A descriptive study of a segmentation network.

SETTING

Academic institution.

METHODS

A total of 15 high-resolution cone-beam temporal bone computed tomography (CT) data sets were included in this study. All images were co-registered, with relevant anatomical structures (eg, ossicles, inner ear, facial nerve, chorda tympani, bony labyrinth) manually segmented. Predicted segmentations from no new U-Net (nnU-Net), an open-source 3-dimensional semantic segmentation neural network, were compared against ground-truth segmentations using modified Hausdorff distances (mHD) and Dice scores.

RESULTS

Fivefold cross-validation with nnU-Net between predicted and ground-truth labels were as follows: malleus (mHD: 0.044 ± 0.024 mm, dice: 0.914 ± 0.035), incus (mHD: 0.051 ± 0.027 mm, dice: 0.916 ± 0.034), stapes (mHD: 0.147 ± 0.113 mm, dice: 0.560 ± 0.106), bony labyrinth (mHD: 0.038 ± 0.031 mm, dice: 0.952 ± 0.017), and facial nerve (mHD: 0.139 ± 0.072 mm, dice: 0.862 ± 0.039). Comparison against atlas-based segmentation propagation showed significantly higher Dice scores for all structures (p < .05).

CONCLUSION

Using an open-source deep learning pipeline, we demonstrate consistently submillimeter accuracy for semantic CT segmentation of temporal bone anatomy compared to hand-segmented labels. This pipeline has the potential to greatly improve preoperative planning workflows for a variety of otologic and neurotologic procedures and augment existing image guidance and robot-assisted systems for the temporal bone.

Applications of Artificial Intelligence in Temporal Bone Imaging: Advances and Future Challenges

The applications of artificial intelligence (AI) in temporal bone (TB) imaging have gained significant attention in recent years, revolutionizing the field of otolaryngology and radiology. Accurate interpretation of imaging features of TB conditions plays a crucial role in diagnosing and treating a range of ear-related pathologies, including middle and inner ear diseases, otosclerosis, and vestibular schwannomas. According to multiple clinical studies published in the literature, AI-powered algorithms have demonstrated exceptional proficiency in interpreting imaging findings, not only saving time for physicians but also enhancing diagnostic accuracy by reducing human error. Although several challenges remain in routinely relying on AI applications, the collaboration between AI and healthcare professionals holds the key to better patient outcomes and significantly improved patient care. This overview delivers a comprehensive update on the advances of AI in the field of TB imaging, summarizes recent evidence provided by clinical studies, and discusses future insights and challenges in the widespread integration of AI in clinical practice.

Automated Radiomic Analysis of Vestibular Schwannomas and Inner Ears Using Contrast-Enhanced T1-Weighted and T2-Weighted Magnetic Resonance Imaging Sequences and Artificial Intelligence

OBJECTIVE

To objectively evaluate vestibular schwannomas (VSs) and their spatial relationships with the ipsilateral inner ear (IE) in magnetic resonance imaging (MRI) using deep learning.

STUDY DESIGN

Cross-sectional study.

PATIENTS

A total of 490 adults with VS, high-resolution MRI scans, and no previous neurotologic surgery.

INTERVENTIONS

MRI studies of VS patients were split into training (390 patients) and test (100 patients) sets. A three-dimensional convolutional neural network model was trained to segment VS and IE structures using contrast-enhanced T1-weighted and T2-weighted sequences, respectively. Manual segmentations were used as ground truths. Model performance was evaluated on the test set and on an external set of 100 VS patients from a public data set (Vestibular-Schwannoma-SEG).

MAIN OUTCOME MEASURES

Dice score, relative volume error, average symmetric surface distance, 95th-percentile Hausdorff distance, and centroid locations.

RESULTS

Dice scores for VS and IE volume segmentations were 0.91 and 0.90, respectively. On the public data set, the model segmented VS tumors with a Dice score of 0.89 ± 0.06 (mean ± standard deviation), relative volume error of 9.8 ± 9.6%, average symmetric surface distance of 0.31 ± 0.22 mm, and 95th-percentile Hausdorff distance of 1.26 ± 0.76 mm. Predicted VS segmentations overlapped with ground truth segmentations in all test subjects. Mean errors of predicted VS volume, VS centroid location, and IE centroid location were 0.05 cm 3 , 0.52 mm, and 0.85 mm, respectively.

CONCLUSIONS

A deep learning system can segment VS and IE structures in high-resolution MRI scans with excellent accuracy. This technology offers promise to improve the clinical workflow for assessing VS radiomics and enhance the management of VS patients.

Diagnosis, Treatment, and Management of Otitis Media with Artificial Intelligence

A common infectious disease, otitis media (OM) has a low rate of early diagnosis, which significantly increases the difficulty of treating the disease and the likelihood of serious complications developing including hearing loss, speech impairment, and even intracranial infection. Several areas of healthcare have shown great promise in the application of artificial intelligence (AI) systems, such as the accurate detection of diseases, the automated interpretation of images, and the prediction of patient outcomes. Several articles have reported some machine learning (ML) algorithms such as ResNet, InceptionV3 and Unet, were applied to the diagnosis of OM successfully. The use of these techniques in the OM is still in its infancy, but their potential is enormous. We present in this review important concepts related to ML and AI, describe how these technologies are currently being applied to diagnosing, treating, and managing OM, and discuss the challenges associated with developing AI-assisted OM technologies in the future.

The cranial capacity of the Saudi population measured using 3D computed tomography scans

OBJECTIVES

To measure the cranial capacity of members of the Saudi adult population across ages and genders.

METHODS

This was a retrospective cross-sectional study that used 488 Computed Tomography (CT) scans of heads (of which 275 males) to measure cranial volume. The CT slices 0.625 mm thick were uploaded using the freely available software "3D-Slicer", which then reconstructed the images and built a 3D module.

RESULTS

The mean (±SD) cranial capacity of the males was 1481.6 (±110) cm3 (range: 1241-1723 cm3), whereas the cranial capacity of the females was 1375.4 (±104) cm3 (range: 1203-1678 cm3). This study showed that the males had a mean cranial capacity that was 7% greater than that of the females in this study. The average cranial capacity of the males between the ages of 31 and 40 years was statistically significantly larger to that of the males aged 61-80 (p<0.05).

CONCLUSION

This study demonstrated that the average cranial capacity of the males was larger than that of the females. These study results can help to determine the normal cranial capacity of adults in Saudi Arabia. Further work should be carried out to aid in establishing reference data for the Saudi population.

Estimating age at death by Hausdorff distance analyses of the fourth lumbar vertebral bodies using 3D postmortem CT images

The existing methods for determining adult age from human skeletons are mostly qualitative. However, a shift in quantifying age-related skeletal morphology on a quantitative scale is emerging. This study describes an intuitive variable extraction technique and quantifies skeletal morphology in continuous data to understand their aging pattern. A total of 200 postmortem CT images from the deceased aged 25-99 years (130 males, 70 females) who underwent forensic death investigations were used in the study. The 3D volume of the fourth lumbar vertebral body was segmented, smoothed, and post-processed using the open-source software ITK-SNAP and MeshLab, respectively. To measure the extent of 3D shape deformity due to aging, the Hausdorff distance (HD) analysis was performed. In our context, the maximum Hausdorff distance (maxHD) was chosen as a metric, which was subsequently studied for its correlation with age at death. A strong statistically significant positive correlation (P < 0.001) between maxHD and age at death was observed in both sexes (Spearman's rho = 0.742, male; Spearman's rho = 0.729, female). In simple linear regression analyses, the regression equations obtained yielded the standard error of estimates of 12.5 years and 13.1 years for males and females, respectively. Our study demonstrated that age-related vertebral morphology could be described using the HD method. Moreover, it encourages further studies with larger sample sizes and on other population backgrounds to validate the methodology.

Deep learning-based approach for the automatic segmentation of adult and pediatric temporal bone computed tomography images

Background

Automatic segmentation of temporal bone computed tomography (CT) images is fundamental to image-guided otologic surgery and the intelligent analysis of CT images in the field of otology. This study was conducted to test a convolutional neural network (CNN) model that can automatically segment almost all temporal bone anatomy structures in adult and pediatric CT images.

Methods

A dataset comprising 80 annotated CT volumes was collected, of which 40 samples were obtained from adults and 40 from children. A further 60 annotated CT volumes (30 from adults and 30 from children) were used to train the model. The remaining 20 annotated CT volumes were employed to determine the model's generalizability for automatic segmentation. Finally, the Dice coefficient (DC) and average symmetric surface distance (ASSD) were utilized as metrics to evaluate the performance of the CNN model. Two independent-sample t-tests were used to compare the test set results of adults and children.

Results

In the adult test set, the mean DC values of all the structures ranged from 0.714 to 0.912, and the ASSD values were less than 0.24 mm for 11 structures. In the pediatric test set, the mean DC values of all the structures ranged from 0.658 to 0.915, and the ASSD values were less than 0.18 mm for 11 structures. There was no statistically significant difference between the adult and child test sets in most temporal bone structures.

Conclusions

Our CNN model shows excellent automatic segmentation performance and good generalizability for both adult and pediatric temporal bone CT images, which can help to advance otologist education, intelligent imaging diagnosis, surgery simulation, application of augmented reality, and preoperative planning for image-guided otology surgery.

Development and In-Silico and Ex-Vivo Validation of a Software for a Semi-Automated Segmentation of the Round Window Niche to Design a Patient Specific Implant to Treat Inner Ear Disorders

The aim of this study was to develop and validate a semi-automated segmentation approach that identifies the round window niche (RWN) and round window membrane (RWM) for use in the development of patient individualized round window niche implants (RNI) to treat inner ear disorders. Twenty cone beam computed tomography (CBCT) datasets of unilateral temporal bones of patients were included in the study. Defined anatomical landmarks such as the RWM were used to develop a customized 3D Slicer™ plugin for semi-automated segmentation of the RWN. Two otolaryngologists (User 1 and User 2) segmented the datasets manually and semi-automatically using the developed software. Both methods were compared in-silico regarding the resulting RWM area and RWN volume. Finally, the developed software was validated ex-vivo in N = 3 body donor implantation tests with additively manufactured RNI. The independently segmented temporal bones of the different Users showed a strong consistency in the volume of the RWN and the area of the RWM. The volume of the semi-automated RWN segmentations were 48 ± 11% smaller on average than the manual segmentations and the area of the RWM of the semi-automated segmentations was 21 ± 17% smaller on average than the manual segmentation. All additively manufactured implants, based on the semi-automated segmentation method could be implanted successfully in a pressure-tight fit into the RWN. The implants based on the manual segmentations failed to fit into the RWN and this suggests that the larger manual segmentations were over-segmentations. This study presents a semi-automated approach for segmenting the RWN and RWM in temporal bone CBCT scans that is efficient, fast, accurate, and not dependent on trained users. In addition, the manual segmentation, often positioned as the gold-standard, actually failed to pass the implantation validation.

Objective analysis of facial bone fracture CT images using curvature measurement in a surface mesh model

The human facial skeleton consists of multiple segments and causes difficulty during analytic processes. We developed image analysis software to quantify the amount of injury and validate the smooth curvature of the surface after facial bone reduction surgery. Three-dimensional computed tomography images of facial bone were obtained from 40 patients who had undergone open reduction surgery to treat unilateral zygomaticomaxillary fractures. Analytic software was developed based on the discrete curvature of a triangular mesh model. The discrete curvature values were compared before and after surgery using two regions of interest. For the inferior orbital rim, the weighted average of curvature changed from 0.543 ± 0.034 to 0.458 ± 0.042. For the anterior maxilla, the weighted average of curvature changed from 0.596 ± 0.02 to 0.481 ± 0.031, showing a significant decrement (P < 0.05). The curvature was further compared with the unaffected side using the Bray-Curtis similarity index (BCSI). The BCSI of the inferior orbital rim changed from 0.802 ± 0.041 to 0.904 ± 0.015, and that for the anterior maxilla changed from 0.797 ± 0.029 to 0.84 ± 0.025, demonstrating increased similarity (P < 0.05). In computational biology, adequate analytic software is crucial. The newly developed software demonstrated significant differentiation between pre- and postoperative curvature values. Modification of formulas and software will lead to further advancements.

The user experience design of a novel microscope within SurgiSim, a virtual reality surgical simulator

PURPOSE

Virtual reality (VR) simulation has the potential to advance surgical education, procedural planning, and intraoperative guidance. "SurgiSim" is a VR platform developed for the rehearsal of complex procedures using patient-specific anatomy, high-fidelity stereoscopic graphics, and haptic feedback. SurgiSim is the first VR simulator to include a virtual operating room microscope. We describe the process of designing and refining the VR microscope user experience (UX) and user interaction (UI) to optimize surgical rehearsal and education.

METHODS

Human-centered VR design principles were applied in the design of the SurgiSim microscope to optimize the user's sense of presence. Throughout the UX's development, the team of developers met regularly with surgeons to gather end-user feedback. Supplemental testing was performed on four participants.

RESULTS

Through observation and participant feedback, we made iterative design upgrades to the SurgiSim platform. We identified the following key characteristics of the VR microscope UI: overall appearance, hand controller interface, and microscope movement.

CONCLUSION

Our design process identified challenges arising from the disparity between VR and physical environments that pertain to microscope education and deployment. These roadblocks were addressed using creative solutions. Future studies will investigate the efficacy of VR surgical microscope training on real-world microscope skills as assessed by validated performance metrics.

The role of artificial intelligence in surgical simulation

Artificial Intelligence (AI) plays an integral role in enhancing the quality of surgical simulation, which is increasingly becoming a popular tool for enriching the training experience of a surgeon. This spans the spectrum from facilitating preoperative planning, to intraoperative visualisation and guidance, ultimately with the aim of improving patient safety. Although arguably still in its early stages of widespread clinical application, AI technology enables personal evaluation and provides personalised feedback in surgical training simulations. Several forms of surgical visualisation technologies currently in use for anatomical education and presurgical assessment rely on different AI algorithms. However, while it is promising to see clinical examples and technological reports attesting to the efficacy of AI-supported surgical simulators, barriers to wide-spread commercialisation of such devices and software remain complex and multifactorial. High implementation and production costs, scarcity of reports evidencing the superiority of such technology, and intrinsic technological limitations remain at the forefront. As AI technology is key to driving the future of surgical simulation, this paper will review the literature delineating its current state, challenges, and prospects. In addition, a consolidated list of FDA/CE approved AI-powered medical devices for surgical simulation is presented, in order to shed light on the existing gap between academic achievements and the universal commercialisation of AI-enabled simulators. We call for further clinical assessment of AI-supported surgical simulators to support novel regulatory body approved devices and usher surgery into a new era of surgical education.

Automated Extraction of Anatomical Measurements From Temporal Bone CT Imaging

OBJECTIVE

Proposed methods of minimally invasive and robot-assisted procedures within the temporal bone require measurements of surgically relevant distances and angles, which often require time-consuming manual segmentation of preoperative imaging. This study aims to describe an automatic segmentation and measurement extraction pipeline of temporal bone cone-beam computed tomography (CT) scans.

STUDY DESIGN

Descriptive study of temporal bone measurements.

SETTING

Academic institution.

METHODS

A propagation template composed of 16 temporal bone CT scans was formed with relevant anatomical structures and landmarks manually segmented. Next, 52 temporal bone CT scans were autonomously segmented using deformable registration techniques from the Advanced Normalization Tools Python package. Anatomical measurements were extracted via in-house Python algorithms. Extracted measurements were compared to ground truth values from manual segmentations.

RESULTS

Paired t test analyses showed no statistical difference between measurements using this pipeline and ground truth measurements from manually segmented images. Mean (SD) malleus manubrium length was 4.39 (0.34) mm. Mean (SD) incus short and long processes were 2.91 (0.18) mm and 3.53 (0.38) mm, respectively. The mean (SD) maximal diameter of the incus long process was 0.74 (0.17) mm. The first and second facial nerve genus had mean (SD) angles of 68.6 (6.7) degrees and 111.1 (5.3) degrees, respectively. The facial recess had a mean (SD) span of 3.21 (0.46) mm. Mean (SD) minimum distance between the external auditory canal and tegmen was 3.79 (1.05) mm.

CONCLUSIONS

This is the first study to automatically extract relevant temporal bone anatomical measurements from CT scans using segmentation propagation. Measurements from these models can streamline preoperative planning, improve future segmentation techniques, and help develop future image-guided or robot-assisted systems for temporal bone procedures.

Machine Learning in the Management of Lateral Skull Base Tumors: A Systematic Review

The application of machine learning (ML) techniques to otolaryngology remains a topic of interest and prevalence in the literature, though no previous articles have summarized the current state of ML application to management and the diagnosis of lateral skull base (LSB) tumors. Subsequently, we present a systematic overview of previous applications of ML techniques to the management of LSB tumors. Independent searches were conducted on PubMed and Web of Science between August 2020 and February 2021 to identify the literature pertaining to the use of ML techniques in LSB tumor surgery written in the English language. All articles were assessed in regard to their application task, ML methodology, and their outcomes. A total of 32 articles were examined. The number of articles involving applications of ML techniques to LSB tumor surgeries has significantly increased since the first article relevant to this field was published in 1994. The most commonly employed ML category was tree-based algorithms. Most articles were included in the category of surgical management (13; 40.6%), followed by those in disease classification (8; 25%). Overall, the application of ML techniques to the management of LSB tumor has evolved rapidly over the past two decades, and the anticipated growth in the future could significantly augment the surgical outcomes and management of LSB tumors.

A geometric alignment for human temporal bone CT images via lateral semicircular canals segmentation

PURPOSE

Due to the different posture of the subject and settings of CT scanners, the CT images of the human temporal bone should be geometrically aligned with multi-planar reconstruction to ensure the symmetry of the bilateral anatomical structure. Manual alignment is a time-consuming task for radiologists and an important pre-processing step for further computer-aided CT analysis. We propose a fully automatic alignment algorithm for temporal bone CT images via Lateral Semicircular Canals (LSCs) segmentation.

METHODS

The LSCs are segmented with our proposed multi-feature fusion network as anchors at first. Then, we define a standard 3D coordinate system and propose an alignment procedure.

RESULTS

The experimental results show that our LSC segmentation network achieved a higher segmentation accuracy. The acceptable rate is achieved 85% over 910 raw temporal bone CT sequences. The alignment speed is reduced from 10 minutes by manual to 60s.

CONCLUSIONS

Aiming at the problem of bilateral asymmetry in the raw temporal bone CT images, we propose an automatic geometric alignment method. Our proposed method can help to perform alignment of temporal bone CT images efficiently. This article is protected by copyright. All rights reserved.

Automated Registration-Based Temporal Bone Computed Tomography Segmentation for Applications in Neurotologic Surgery

OBJECTIVE

This study investigates the accuracy of an automated method to rapidly segment relevant temporal bone anatomy from cone beam computed tomography (CT) images. Implementation of this segmentation pipeline has potential to improve surgical safety and decrease operative time by augmenting preoperative planning and interfacing with image-guided robotic surgical systems.

STUDY DESIGN

Descriptive study of predicted segmentations.

SETTING

Academic institution.

METHODS

We have developed a computational pipeline based on the symmetric normalization registration method that predicts segmentations of anatomic structures in temporal bone CT scans using a labeled atlas. To evaluate accuracy, we created a data set by manually labeling relevant anatomic structures (eg, ossicles, labyrinth, facial nerve, external auditory canal, dura) for 16 deidentified high-resolution cone beam temporal bone CT images. Automated segmentations from this pipeline were compared against ground-truth manual segmentations by using modified Hausdorff distances and Dice scores. Runtimes were documented to determine the computational requirements of this method.

RESULTS

Modified Hausdorff distances and Dice scores between predicted and ground-truth labels were as follows: malleus (0.100 ± 0.054 mm; Dice, 0.827 ± 0.068), incus (0.100 ± 0.033 mm; Dice, 0.837 ± 0.068), stapes (0.157 ± 0.048 mm; Dice, 0.358 ± 0.100), labyrinth (0.169 ± 0.100 mm; Dice, 0.838 ± 0.060), and facial nerve (0.522 ± 0.278 mm; Dice, 0.567 ± 0.130). A quad-core 16GB RAM workstation completed this segmentation pipeline in 10 minutes.

CONCLUSIONS

We demonstrated submillimeter accuracy for automated segmentation of temporal bone anatomy when compared against hand-segmented ground truth using our template registration pipeline. This method is not dependent on the training data volume that plagues many complex deep learning models. Favorable runtime and low computational requirements underscore this method's translational potential.

Towards accurate facial nerve segmentation with decoupling optimization

Robotic cochlear implantation is an effective way to restore the hearing of hearing-impaired patients, and facial nerve recognition is the key to the operation. However, accurate facial nerve segmentation is a challenging task, mainly for two key issues: (1) the facial nerve area is very small in image, and there are many similar areas; (2) low contrast of the border between the facial nerve and the surrounding tissues increases the difficulty. In this work, we propose an end-to-end neural network, called FNSegNet, with two stages to solve these problems. Specifically, in the coarse segmentation stage, we first adopt three search identification modules to capture small objects by expanding the receptive field from high-level features and combine an effective pyramid fusion module to fuse. In the refine segmentation stage, we use a decoupling optimization module to establish the relationship between the central region and the boundary details of facial nerve by decoupling the boundary and center area. Meanwhile, we feed them into a spatial attention module to correct the conflict regions. Extensive experiments on the challenging dataset demonstrate that the proposed FNSegNet significantly improves the segmentation accuracy (0.858 on Dice, 0.363 mm on 95% Hausdorff distance), and reduces the computational complexity (13.33G on FLOPs, 9.86M parameters).

Automated objective surgical planning for lateral skull base tumors

PURPOSE

Surgical removal of pathology at the lateral skull base is challenging because of the proximity of critical anatomical structures which can lead to significant morbidity when damaged or traversed. Pre-operative computed surgical approach planning has the potential to aid in selection of the optimal approach to remove pathology and minimize complications.

METHODS

We propose an automated surgical approach planning algorithm to derive the optimal approach to vestibular schwannomas in the internal auditory canal for hearing preservation surgery. The algorithm selects between the middle cranial fossa and retrosigmoid approach by utilizing a unique segmentation of each patient's anatomy and a cost function to minimize potential surgical morbidity.

RESULTS

Patients who underwent hearing preservation surgery for vestibular schwannoma resection (n = 9) were included in the cohort. Middle cranial fossa surgery was performed in 5 patients, and retrosigmoid surgery was performed in 4. The algorithm favored the performed surgical approach in 6 of 9 patients.

CONCLUSION

We developed a method for computing morbidity costs of surgical paths to objectively analyze surgical approaches at the lateral skull base. Computed pre-operative planning may assist in surgical decision making, trainee education, and improving clinical outcomes.

Pediatric chest-abdomen-pelvis and abdomen-pelvis CT images with expert organ contours

PURPOSE

Organ autosegmentation efforts to date have largely been focused on adult populations, due to limited availability of pediatric training data. Pediatric patients may present additional challenges for organ segmentation. This paper describes a dataset of 359 pediatric chest-abdomen-pelvis and abdomen-pelvis CT images with expert contours of up to 29 anatomical organ structures to aid in the evaluation and development of autosegmentation algorithms for pediatric CT imaging.

ACQUISITION AND VALIDATION METHODS

The dataset collection consists of axial CT images in DICOM format of 180 male and 179 female pediatric chest-abdomen-pelvis or abdomen-pelvis exams acquired from one of three CT scanners at Children's Wisconsin. The datasets represent random pediatric cases based upon routine clinical indications. Subjects ranged in age from 5 days to 16 years, with a mean age of seven years. The CT acquisition, contrast, and reconstruction protocols varied across the scanner models and patients, with specifications available in the DICOM headers. Expert contours were manually labeled for up to 29 organ structures per subject. Not all contours are available for all subjects, due to limited field of view or unreliable contouring due to high noise.

DATA FORMAT AND USAGE NOTES

The data are available on TCIA (https://www.cancerimagingarchive.net/) under the collection Pediatric-CT-SEG. The axial CT image slices for each subject are available in DICOM format. The expert contours are stored in a single DICOM RTSTRUCT file for each subject. The contours are names as listed in Table 2.

POTENTIAL APPLICATIONS

This dataset will enable the evaluation and development of organ autosegmentation algorithms for pediatric populations, which exhibit variations in organ shape and size across age. Automated organ segmentation from CT images has numerous applications including radiation therapy, diagnostic tasks, surgical planning, and patient-specific organ dose estimation. This article is protected by copyright. All rights reserved.

Quantitative Analysis of Temporal Bone Density and Thickness for Robotic Ear Surgery

Background and Objective: Quantitative assessment of bone density and thickness in computed-tomography images offers great potential for preoperative planning procedures in robotic ear surgery. Methods: We retrospectively analyzed computed-tomography scans of subjects undergoing cochlear implantation (N = 39). In addition, scans of Thiel-fixated ex-vivo specimens were analyzed (N = 15). To estimate bone mineral density, quantitative computed-tomography data were obtained using a calibration phantom. The temporal bone thickness and cortical bone density were systematically assessed at retroauricular positions using an automated algorithm referenced by an anatomy-based coordinate system. Two indices are proposed to include information of bone density and thickness for the preoperative assessment of safe screw positions (Screw Implantation Safety Index, SISI) and mass distribution (Column Density Index, CODI). Linear mixed-effects models were used to assess the effects of age, gender, ear side and position on bone thickness, cortical bone density and the distribution of the indices. Results: Age, gender, and ear side only had negligible effects on temporal bone thickness and cortical bone density. The average radiodensity of cortical bone was 1,511 Hounsfield units, corresponding to a bone mineral density of 1,145 mg HA/cm³. Temporal bone thickness and cortical bone density depend on the distance from Henle's spine in posterior direction. Moreover, safe screw placement locations can be identified by computation of the SISI distribution. A local maximum in mass distribution was observed posteriorly to the supramastoid crest. Conclusions: We provide quantitative information about temporal bone density and thickness for applications in robotic and computer-assisted ear surgery. The proposed preoperative indices (SISI and CODI) can be applied to patient-specific cases to identify optimal regions with respect to bone density and thickness for safe screw placement and effective implant positioning.

Emerging artificial intelligence applications in otological imaging

PURPOSE OF REVIEW

To highlight the recent literature on artificial intelligence (AI) pertaining to otological imaging and to discuss future directions, obstacles and opportunities.

RECENT FINDINGS

The main themes in the recent literature centre around automated otoscopic image diagnosis and automated image segmentation for application in virtual reality surgical simulation and planning. Other applications that have been studied include identification of tinnitus MRI biomarkers, facial palsy analysis, intraoperative augmented reality systems, vertigo diagnosis and endolymphatic hydrops ratio calculation in Meniere's disease. Studies are presently at a preclinical, proof-of-concept stage.

SUMMARY

The recent literature on AI in otological imaging is promising and demonstrates the future potential of this technology in automating certain imaging tasks in a healthcare environment of ever-increasing demand and workload. Some studies have shown equivalence or superiority of the algorithm over physicians, albeit in narrowly defined realms. Future challenges in developing this technology include the compilation of large high quality annotated datasets, fostering strong collaborations between the health and technology sectors, testing the technology within real-world clinical pathways and bolstering trust among patients and physicians in this new method of delivering healthcare.

Automatic Semicircular Canal Segmentation of CT Volumes Using Improved 3D U-Net with Attention Mechanism

The vestibular system is the sensory apparatus that helps the body maintain its postural equilibrium, and semicircular canal is an important organ of the vestibular system. The semicircular canals are three membranous tubes, each forming approximately two-thirds of a circle with a diameter of approximately 6.5 mm, and segmenting them accurately is of great benefit for auxiliary diagnosis, surgery, and treatment of vestibular disease. However, the semicircular canal has small volume, which accounts for less than 1% of the overall computed tomography image. Doctors have to annotate the image in a slice-by-slice manner, which is time-consuming and labor-intensive. To solve this problem, we propose a novel 3D convolutional neural network based on 3D U-Net to automatically segment the semicircular canal. We added the spatial attention mechanism of 3D spatial squeeze and excitation modules, as well as channel attention mechanism of 3D global attention upsample modules to improve the network performance. Our network achieved an average dice coefficient of 92.5% on the test dataset, which shows competitive performance in semicircular canals segmentation task.

Fully Automated Measurement of Cochlear Duct Length From Clinical Temporal Bone Computed Tomography

OBJECTIVES/HYPOTHESIS

To present and validate a novel fully automated method to measure cochlear dimensions, including cochlear duct length (CDL).

STUDY DESIGN

Cross-sectional study.

METHODS

The computational method combined 1) a deep learning (DL) algorithm to segment the cochlea and otic capsule and 2) geometric analysis to measure anti-modiolar distances from the round window to the apex. The algorithm was trained using 165 manually segmented clinical computed tomography (CT). A Testing group of 159 CTs were then measured for cochlear diameter and width (A- and B-values) and CDL using the automated system and compared against manual measurements. The results were also compared with existing approaches and historical data. In addition, pre- and post-implantation scans from 27 cochlear implant recipients were studied to compare predicted versus actual array insertion depth.

RESULTS

Measurements were successfully obtained in 98.1% of scans. The mean CDL to 900° was 35.52 mm (SD, 2.06; range, [30.91-40.50]), the mean A-value was 8.88 mm (0.47; [7.67-10.49]), and mean B-value was 6.38 mm (0.42; [5.16-7.38]). The R² fit of the automated to manual measurements was 0.87 for A-value, 0.70 for B-value, and 0.71 for CDL. For anti-modiolar arrays, the distance between the imaged and predicted array tip location was 0.57 mm (1.25; [0.13-5.28]).

CONCLUSION

Our method provides a fully automated means of cochlear analysis from clinical CTs. The distribution of CDL, dimensions, and cochlear quadrant lengths is similar to those from historical data. This approach requires no radiographic experience and is free from user-related variation.

LEVEL OF EVIDENCE

3 Laryngoscope, 2021.

Accuracy of free-hand positioned patient specific implants (PSI) in primary reconstruction after inferior and/or medial orbital wall fractures

BACKGROUND

To assess the accuracy with which CAD/CAM-fabricated patient-specific titanium implants (PSI) are positioned for inferior and/or medial orbital wall reconstruction without the use of intraoperative navigation.

METHODS

Patients who underwent a primary reconstruction of the orbital walls with PSI due to fractures were enrolled in this retrospective cohort analysis. The primary outcome variables were the mean surface distances (MSD) between virtually planned and postoperative PSI position and single linear deviations in the x-, y- and z-axis at corresponding reference points. Secondary outcome variables included demographic data, classification of orbital wall defects and clinical outcomes.

RESULTS

A total of 33 PSI (orbital floor n = 22; medial wall, n = 11) were examined in 27 patients. MSD was on a comparable level for the orbital floor and medial wall (median 0.39 mm, range 0.22-1.53 mm vs. median 0.42 mm, range 0.21-0.98 mm; p = 0.56). Single linear deviations were lower for reconstructions of the orbital floor compared to the medial wall (median 0.45 vs. 0.79 mm; p < 0.05). There was no association between the occurrence of diplopia and the accuracy level (p = 0.418).

CONCLUSIONS

Free-hand positioning of PSI reaches a clinically appropriate level of accuracy, limiting the necessity of navigational systems to selected cases.