Segmentation algorithms of subcortical brain structures on MRI for radiotherapy and radiosurgery: A survey

A Narrative Review on 3-Dimensional Visualization Techniques in Neurosurgical Education, Simulation, and Planning

BACKGROUND

High-fidelity visualization of anatomical organs is crucial for neurosurgical education, simulation, and planning. This becomes much more important for minimally invasive neurosurgical procedures. Realistic anatomical visualization can allow resident surgeons to learn visual cues and orient themselves with the complex 3-dimensional (3D) anatomy. Achieving full fidelity in 3D medical visualization is an active area of research; however, the prior reviews focus on the application area and lack the underlying technical principles. Accordingly, the present study attempts to bridge this gap by providing a narrative review of the techniques used for 3D visualization.

METHODS

We conducted a literature review on 3D medical visualization technology from 2018 to 2023 using the PubMed and Google Scholar search engines. The cross-referenced manuscripts were extensively studied to find literature that discusses technology relevant to 3D medical visualization. We also compiled and ran software applications that were accessible to us in order to better understand them.

RESULTS

We present the underlying fundamental technology used in 3D medical visualization in the context of neurosurgical education, simulation, and planning. Further, we discuss and categorize a few important applications based on the 3D visualization techniques they use.

CONCLUSIONS

The visualization of virtual human organs has not yet achieved a level of realism close to reality. This gap is largely due to the interdisciplinary nature of this research, population diversity, and validation complexities. With the advancements in computational resources and automation of 3D visualization pipelines, next-gen applications may offer enhanced medical 3D visualization fidelity.

Improved segmentation of basal ganglia from MR images using convolutional neural network with crossover-typed skip connection

PURPOSE

Accurate and automatic segmentation of basal ganglia from magnetic resonance (MR) images is important for diagnosis and treatment of various brain disorders. However, the basal ganglia segmentation is a challenging task because of the class imbalance and the unclear boundaries among basal ganglia anatomical structures. Thus, we aim to present an encoder-decoder convolutional neural network (CNN)-based method for improved segmentation of basal ganglia by focusing on skip connections that determine the segmentation performance of encoder-decoder CNNs. We also aim to reveal the effect of skip connections on the segmentation of basal ganglia with unclear boundaries.

METHODS

We used the encoder-decoder CNNs with the following five patterns of skip connections: without skip connection, with full-resolution horizontal skip connection, with horizontal skip connections, with vertical skip connections, and with crossover-typed skip connections (the proposed method). We compared and evaluated the performance of the CNNs in the experiment of basal ganglia segmentation using T1-weighted MR brain images of 79 patients.

RESULTS

The experimental results showed that the skip connections at each scale level help CNNs to acquire multi-scale image features, the vertical skip connections contribute on acquiring finer image features for segmentation of smaller anatomical structures with more blurred boundaries, and the crossover-typed skip connections, a combination of horizontal and vertical skip connections, provided better segmentation accuracy.

CONCLUSION

This paper investigated the effect of skip connections on the basal ganglia segmentation and revealed the crossover-typed skip connections might be effective for improving the segmentation of basal ganglia with the class imbalance and the unclear boundaries.

Multi-Scale Squeeze U-SegNet with Multi Global Attention for Brain MRI Segmentation

In this paper, we propose a multi-scale feature extraction with novel attention-based convolutional learning using the U-SegNet architecture to achieve segmentation of brain tissue from a magnetic resonance image (MRI). Although convolutional neural networks (CNNs) show enormous growth in medical image segmentation, there are some drawbacks with the conventional CNN models. In particular, the conventional use of encoder-decoder approaches leads to the extraction of similar low-level features multiple times, causing redundant use of information. Moreover, due to inefficient modeling of long-range dependencies, each semantic class is likely to be associated with non-accurate discriminative feature representations, resulting in low accuracy of segmentation. The proposed global attention module refines the feature extraction and improves the representational power of the convolutional neural network. Moreover, the attention-based multi-scale fusion strategy can integrate local features with their corresponding global dependencies. The integration of fire modules in both the encoder and decoder paths can significantly reduce the computational complexity owing to fewer model parameters. The proposed method was evaluated on publicly accessible datasets for brain tissue segmentation. The experimental results show that our proposed model achieves segmentation accuracies of 94.81% for cerebrospinal fluid (CSF), 95.54% for gray matter (GM), and 96.33% for white matter (WM) with a noticeably reduced number of learnable parameters. Our study shows better segmentation performance, improving the prediction accuracy by 2.5% in terms of dice similarity index while achieving a 4.5 times reduction in the number of learnable parameters compared to previously developed U-SegNet based segmentation approaches. This demonstrates that the proposed approach can achieve reliable and precise automatic segmentation of brain MRI images.

Multi-Scale Self-Guided Attention for Medical Image Segmentation

Even though convolutional neural networks (CNNs) are driving progress in medical image segmentation, standard models still have some drawbacks. First, the use of multi-scale approaches, i.e., encoder-decoder architectures, leads to a redundant use of information, where similar low-level features are extracted multiple times at multiple scales. Second, long-range feature dependencies are not efficiently modeled, resulting in non-optimal discriminative feature representations associated with each semantic class. In this paper we attempt to overcome these limitations with the proposed architecture, by capturing richer contextual dependencies based on the use of guided self-attention mechanisms. This approach is able to integrate local features with their corresponding global dependencies, as well as highlight interdependent channel maps in an adaptive manner. Further, the additional loss between different modules guides the attention mechanisms to neglect irrelevant information and focus on more discriminant regions of the image by emphasizing relevant feature associations. We evaluate the proposed model in the context of semantic segmentation on three different datasets: abdominal organs, cardiovascular structures and brain tumors. A series of ablation experiments support the importance of these attention modules in the proposed architecture. In addition, compared to other state-of-the-art segmentation networks our model yields better segmentation performance, increasing the accuracy of the predictions while reducing the standard deviation. This demonstrates the efficiency of our approach to generate precise and reliable automatic segmentations of medical images. Our code is made publicly available at: https://github.com/sinAshish/Multi-Scale-Attention.

Fully automated brain resection cavity delineation for radiation target volume definition in glioblastoma patients using deep learning

Background

Automated brain tumor segmentation methods are computational algorithms that yield tumor delineation from, in this case, multimodal magnetic resonance imaging (MRI). We present an automated segmentation method and its results for resection cavity (RC) in glioblastoma multiforme (GBM) patients using deep learning (DL) technologies.

Methods

Post-operative, T1w with and without contrast, T2w and fluid attenuated inversion recovery MRI studies of 30 GBM patients were included. Three radiation oncologists manually delineated the RC to obtain a reference segmentation. We developed a DL cavity segmentation method, which utilizes all four MRI sequences and the reference segmentation to learn to perform RC delineations. We evaluated the segmentation method in terms of Dice coefficient (DC) and estimated volume measurements.

Results

Median DC of the three radiation oncologist were 0.85 (interquartile range [IQR]: 0.08), 0.84 (IQR: 0.07), and 0.86 (IQR: 0.07). The results of the automatic segmentation compared to the three different raters were 0.83 (IQR: 0.14), 0.81 (IQR: 0.12), and 0.81 (IQR: 0.13) which was significantly lower compared to the DC among raters (chi-square = 11.63, p = 0.04). We did not detect a statistically significant difference of the measured RC volumes for the different raters and the automated method (Kruskal-Wallis test: chi-square = 1.46, p = 0.69). The main sources of error were due to signal inhomogeneity and similar intensity patterns between cavity and brain tissues.

Conclusions

The proposed DL approach yields promising results for automated RC segmentation in this proof of concept study. Compared to human experts, the DC are still subpar.

Role of deep learning in infant brain MRI analysis

Deep learning algorithms and in particular convolutional networks have shown tremendous success in medical image analysis applications, though relatively few methods have been applied to infant MRI data due numerous inherent challenges such as inhomogenous tissue appearance across the image, considerable image intensity variability across the first year of life, and a low signal to noise setting. This paper presents methods addressing these challenges in two selected applications, specifically infant brain tissue segmentation at the isointense stage and presymptomatic disease prediction in neurodevelopmental disorders. Corresponding methods are reviewed and compared, and open issues are identified, namely low data size restrictions, class imbalance problems, and lack of interpretation of the resulting deep learning solutions. We discuss how existing solutions can be adapted to approach these issues as well as how generative models seem to be a particularly strong contender to address them.

Deep CNN ensembles and suggestive annotations for infant brain MRI segmentation

Precise 3D segmentation of infant brain tissues is an essential step towards comprehensive volumetric studies and quantitative analysis of early brain development. However, computing such segmentations is very challenging, especially for 6-month infant brain, due to the poor image quality, among other difficulties inherent to infant brain MRI, e.g., the isointense contrast between white and gray matter and the severe partial volume effect due to small brain sizes. This study investigates the problem with an ensemble of semi-dense fully convolutional neural networks (CNNs), which employs T1-weighted and T2-weighted MR images as input. We demonstrate that the ensemble agreement is highly correlated with the segmentation errors. Therefore, our method provides measures that can guide local user corrections. To the best of our knowledge, this work is the first ensemble of 3D CNNs for suggesting annotations within images. Our quasi-dense architecture allows the efficient propagation of gradients during training, while limiting the number of parameters, requiring one order of magnitude less parameters than popular medical image segmentation networks such as 3D U-Net (Çiçek, et al.). We also investigated the impact that early or late fusions of multiple image modalities might have on the performances of deep architectures. We report evaluations of our method on the public data of the MICCAI iSEG-2017 Challenge on 6-month infant brain MRI segmentation, and show very competitive results among 21 teams, ranking first or second in most metrics.

Improved dosimetric accuracy with semi-automatic contour propagation of organs-at-risk in glioblastoma patients undergoing chemoradiation

Background

We study the changes in organs‐at‐risk (OARs) morphology as contoured on serial MRIs during chemoradiation therapy (CRT) of glioblastoma (GBM). The dosimetric implication of assuming non‐deformable OAR changes and the accuracy and feasibility of semi‐automatic OAR contour propagation are investigated.

Methods

Fourteen GBM patients who were treated with adjuvant CRT for GBM prospectively underwent MRIs on fractions 0 (i.e., planning), 10, 20, and 1 month post last fraction of CRT. Three sets of OAR contours — (a) manual, (b) rigidly registered (static), and (c) semi‐automatically propagated — were compared using Dice similarity coefficient (DSC) and Hausdorff distance (HD). Dosimetric impact was determined by comparing the minimum dose to the 0.03 cc receiving the highest dose (D0.03 cc) on a clinically approved reference, non‐adapted radiation therapy plan.

Results

The DSC between the manual contours and the static contours decreased significantly over time (fraction 10: [mean ± 1 SD] 0.78 ± 0.17, post 1 month: 0.76 ± 0.17, P = 0.02) while the HD (P = 0.74) and the difference in D0.03cc did not change significantly (P = 0.51). Using the manual contours as reference, compared to static contours, propagated contours have a significantly higher DSC (propagated: [mean ± 1 SD] 0.81 ± 0.15, static: 0.77 ± 0.17, P < 0.001), lower HD (propagated: 3.77 ± 1.8 mm, static: 3.96 ± 1.6 mm, P = 0.002), and a significantly lower absolute difference in D0.03cc (propagated: 101 ± 159 cGy, static: 136 ± 243 cGy, P = 0.019).

Conclusions

Nonrigid changes in OARs over time lead to different maximum doses than planned. By using semi‐automatic OAR contour propagation, OARs are more accurately delineated on subsequent fractions, with corresponding improved accuracy of the reported dose to the OARs.

Automatic Segmentation of Neonatal Ventricles from Cranial Ultrasound for Prediction of Intraventricular Hemorrhage Outcome

Intraventricular hemorrhage (IVH) followed by post hemorrhagic hydrocephalus (PHH) in premature neonates is one of the recognized reasons of brain injury in newborns. Cranial ultrasound (CUS) is a noninvasive imaging tool that has been used widely to diagnose and monitor neonates with IVH. In our previous work, we showed the potential of quantitative morphological analysis of lateral ventricles from early CUS to predict the PHH outcome in neonates with IVH. In this paper, we first present a new automatic method for ventricle segmentation in 2D CUS images. We detect the brain bounding box and brain mid-line to estimate the anatomical positions of ventricles and correct the brain rotation. The ventricles are segmented using a combination of fuzzy c-means, phase congruency, and active contour algorithms. Finally, we compare this fully automated approach with our previous work for the prediction of the outcome of PHH on a set of 2D CUS images taken from 60 premature neonates with different IVH grades. Experimental results showed that our method could segment ventricles with an average Dice similarity coefficient of 0.8 ± 0.12. In addition, our fully automated method could predict the outcome of PHH based on the extracted ventricle regions with similar accuracy to our previous semi-automated approach (83% vs. 84%, respectively, p-value = 0.8). This method has the potential to standardize the evaluation of CUS images and can be a helpful clinical tool for early monitoring and treatment of IVH and PHH.

Quantifying deep grey matter atrophy using automated segmentation approaches: A systematic review of structural MRI studies

The deep grey matter (DGM) nuclei of the brain play a crucial role in learning, behaviour, cognition, movement and memory. Although automated segmentation strategies can provide insight into the impact of multiple neurological conditions affecting these structures, such as Multiple Sclerosis (MS), Huntington's disease (HD), Alzheimer's disease (AD), Parkinson's disease (PD) and Cerebral Palsy (CP), there are a number of technical challenges limiting an accurate automated segmentation of the DGM. Namely, the insufficient contrast of T1 sequences to completely identify the boundaries of these structures, as well as the presence of iso-intense white matter lesions or extensive tissue loss caused by brain injury. Therefore in this systematic review, 269 eligible studies were analysed and compared to determine the optimal approaches for addressing these technical challenges. The automated approaches used among the reviewed studies fall into three broad categories, atlas-based approaches focusing on the accurate alignment of atlas priors, algorithmic approaches which utilise intensity information to a greater extent, and learning-based approaches that require an annotated training set. Studies that utilise freely available software packages such as FIRST, FreeSurfer and LesionTOADS were also eligible, and their performance compared. Overall, deep learning approaches achieved the best overall performance, however these strategies are currently hampered by the lack of large-scale annotated data. Improving model generalisability to new datasets could be achieved in future studies with data augmentation and transfer learning. Multi-atlas approaches provided the second-best performance overall, and may be utilised to construct a "silver standard" annotated training set for deep learning. To address the technical challenges, providing robustness to injury can be improved by using multiple channels, highly elastic diffeomorphic transformations such as LDDMM, and by following atlas-based approaches with an intensity driven refinement of the segmentation, which has been done with the Expectation Maximisation (EM) and level sets methods. Accounting for potential lesions should be achieved with a separate lesion segmentation approach, as in LesionTOADS. Finally, to address the issue of limited contrast, R2*, T2* and QSM sequences could be used to better highlight the DGM due to its higher iron content. Future studies could look to additionally acquire these sequences by retaining the phase information from standard structural scans, or alternatively acquiring these sequences for only a training set, allowing models to learn the "improved" segmentation from T1-sequences alone.

Image Based Brain Segmentation: From Multi-Atlas Fusion to Deep Learning

Background:

This review aims to identify the development of the algorithms for brain tissue and structure segmentation in MRI images.

Discussion:

Starting from the results of the Grand Challenges on brain tissue and structure segmentation held in Medical Image Computing and Computer-Assisted Intervention (MICCAI), this review analyses the development of the algorithms and discusses the tendency from multi-atlas label fusion to deep learning. The intrinsic characteristics of the winners’ algorithms on the Grand Challenges from the year 2012 to 2018 are analyzed and the results are compared carefully.

Conclusion:

Although deep learning has got higher rankings in the challenge, it has not yet met the expectations in terms of accuracy. More effective and specialized work should be done in the future.

A modality-adaptive method for segmenting brain tumors and organs-at-risk in radiation therapy planning

In this paper we present a method for simultaneously segmenting brain tumors and an extensive set of organs-at-risk for radiation therapy planning of glioblastomas. The method combines a contrast-adaptive generative model for whole-brain segmentation with a new spatial regularization model of tumor shape using convolutional restricted Boltzmann machines. We demonstrate experimentally that the method is able to adapt to image acquisitions that differ substantially from any available training data, ensuring its applicability across treatment sites; that its tumor segmentation accuracy is comparable to that of the current state of the art; and that it captures most organs-at-risk sufficiently well for radiation therapy planning purposes. The proposed method may be a valuable step towards automating the delineation of brain tumors and organs-at-risk in glioblastoma patients undergoing radiation therapy.

Cognitive impairment and morphological changes after radiation therapy in brain tumors: A review

Life expectancy of patients treated for brain tumors has lengthened due to the therapeutic improvements. Cognitive impairment has been described following brain radiotherapy, but the mechanisms leading to this adverse event remain mostly unknown. Technical evolutions aim at enhancing the therapeutic ratio. Sparing of the healthy tissues has been improved using various approaches; however, few dose constraints have been established regarding brain structures associated with cognitive functions. The aims of this literature review are to report the main brain areas involved in cognitive adverse effects induced by radiotherapy as described in literature, to better understand brain radiosensitivity and to describe potential future improvements.