Fully automated 3D segmentation of MR-imaged calf muscle compartments: Neighborhood relationship enhanced fully convolutional network

Efficient musculoskeletal annotation using free-form deformation

Traditionally, constructing training datasets for automatic muscle segmentation from medical images involved skilled operators, leading to high labor costs and limited scalability. To address this issue, we developed a tool that enables efficient annotation by non-experts and assessed its effectiveness for training an automatic segmentation network. Our system allows users to deform a template three-dimensional (3D) anatomical model to fit a target magnetic-resonance image using free-form deformation with independent control points for axial, sagittal, and coronal directions. This method simplifies the annotation process by allowing non-experts to intuitively adjust the model, enabling simultaneous annotation of all muscles in the template. We evaluated the quality of the tool-assisted segmentation performed by non-experts, which achieved a Dice coefficient greater than 0.75 compared to expert segmentation, without significant errors such as mislabeling adjacent muscles or omitting musculature. An automatic segmentation network trained with datasets created using this tool demonstrated performance comparable to or superior to that of networks trained with expert-generated datasets. This innovative tool significantly reduces the time and labor costs associated with dataset creation for automatic muscle segmentation, potentially revolutionizing medical image annotation and accelerating the development of deep learning-based segmentation networks in various clinical applications.

Reviewing 3D convolutional neural network approaches for medical image segmentation

Background

Convolutional neural networks (CNNs) assume pivotal roles in aiding clinicians in diagnosis and treatment decisions. The rapid evolution of imaging technology has established three-dimensional (3D) CNNs as a formidable framework for delineating organs and anomalies in medical images. The prominence of 3D CNN frameworks is steadily growing within medical image segmentation and classification. Thus, our proposition entails a comprehensive review, encapsulating diverse 3D CNN algorithms for the segmentation of medical image anomalies and organs.

Methods

This study systematically presents an exhaustive review of recent 3D CNN methodologies. Rigorous screening of abstracts and titles were carried out to establish their relevance. Research papers disseminated across academic repositories were meticulously chosen, analyzed, and appraised against specific criteria. Insights into the realm of anomalies and organ segmentation were derived, encompassing details such as network architecture and achieved accuracies.

Results

This paper offers an all-encompassing analysis, unveiling the prevailing trends in 3D CNN segmentation. In-depth elucidations encompass essential insights, constraints, observations, and avenues for future exploration. A discerning examination indicates the preponderance of the encoder-decoder network in segmentation tasks. The encoder-decoder framework affords a coherent methodology for the segmentation of medical images.

Conclusion

The findings of this study are poised to find application in clinical diagnosis and therapeutic interventions. Despite inherent limitations, CNN algorithms showcase commendable accuracy levels, solidifying their potential in medical image segmentation and classification endeavors.

The Impact of Fatty Infiltration on MRI Segmentation of Lower Limb Muscles in Neuromuscular Diseases: A Comparative Study of Deep Learning Approaches

BACKGROUND

Deep learning methods have been shown to be useful for segmentation of lower limb muscle MRIs of healthy subjects but, have not been sufficiently evaluated on neuromuscular disease (NDM) patients.

PURPOSE

Evaluate the influence of fat infiltration on convolutional neural network (CNN) segmentation of MRIs from NMD patients.

STUDY TYPE

Retrospective study.

SUBJECTS

Data were collected from a hospital database of 67 patients with NMDs and 14 controls (age: 53 ± 17 years, sex: 48 M, 33 F). Ten individual muscles were segmented from the thigh and six from the calf (20 slices, 200 cm section).

FIELD STRENGTH/SEQUENCE

A 1.5 T. Sequences: 2D T1 -weighted fast spin echo. Fat fraction (FF): three-point Dixon 3D GRE, magnetization transfer ratio (MTR): 3D MT-prepared GRE, T2: 2D multispin-echo sequence.

ASSESSMENT

U-Net 2D, U-Net 3D, TransUNet, and HRNet were trained to segment thigh and leg muscles (101/11 and 95/11 training/validation images, 10-fold cross-validation). Automatic and manual segmentations were compared based on geometric criteria (Dice coefficient [DSC], outlier rate, absence rate) and reliability of measured MRI quantities (FF, MTR, T2, volume).

STATISTICAL TESTS

Bland-Altman plots were chosen to describe agreement between manual vs. automatic estimated FF, MTR, T2 and volume. Comparisons were made between muscle populations with an FF greater than 20% (G20+) and lower than 20% (G20-).

RESULTS

The CNNs achieved equivalent results, yet only HRNet recognized every muscle in the database, with a DSC of 0.91 ± 0.08, and measurement biases reaching -0.32% ± 0.92% for FF, 0.19 ± 0.77 for MTR, -0.55 ± 1.95 msec for T2, and - 0.38 ± 3.67 cm3 for volume. The performances of HRNet, between G20- and G20+ decreased significantly.

DATA CONCLUSION

HRNet was the most appropriate network, as it did not omit any muscle. The accuracy obtained shows that CNNs could provide fully automated methods for studying NMDs. However, the accuracy of the methods may be degraded on the most infiltrated muscles (>20%).

EVIDENCE LEVEL

4.

TECHNICAL EFFICACY

Stage 1.

Probing muscle recovery following downhill running using precise mapping of MRI T2 relaxation times

PURPOSE

Postexercise recovery rate is a vital component of designing personalized training protocols and rehabilitation plans. Tracking exercise-induced muscle damage and recovery requires sensitive tools that can probe the muscles' state and composition noninvasively.

METHODS

Twenty-four physically active males completed a running protocol consisting of a 60-min downhill run on a treadmill at -10% incline and 65% of maximal heart rate. Quantitative mapping of MRI T2 was performed using the echo-modulation-curve algorithm before exercise, and at two time points: 1 h and 48 h after exercise.

RESULTS

T2 values increased by 2%-4% following exercise in the primary mover muscles and exhibited further elevation of 1% after 48 h. For the antagonist muscles, T2 values increased only at the 48-h time point (2%-3%). Statistically significant decrease in the SD of T2 values was found following exercise for all tested muscles after 1 h (16%-21%), indicating a short-term decrease in the heterogeneity of the muscle tissue.

CONCLUSION

MRI T2 relaxation time constitutes a useful quantitative marker for microstructural muscle damage, enabling region-specific identification for short-term and long-term systemic processes, and sensitive assessment of muscle recovery following exercise-induced muscle damage. The variability in T2 changes across different muscle groups can be attributed to their different role during downhill running, with immediate T2 elevation occurring in primary movers, followed by delayed elevation in both primary and antagonist muscle groups, presumably due to secondary damage caused by systemic processes.

Assisted annotation in Deep LOGISMOS: Simultaneous multi-compartment 3D MRI segmentation of calf muscles

BACKGROUND

Automated segmentation of individual calf muscle compartments in 3D MR images is gaining importance in diagnosing muscle disease, monitoring its progression, and prediction of the disease course. Although deep convolutional neural networks have ushered in a revolution in medical image segmentation, achieving clinically acceptable results is a challenging task and the availability of sufficiently large annotated datasets still limits their applicability.

PURPOSE

In this paper, we present a novel approach combing deep learning and graph optimization in the paradigm of assisted annotation for solving general segmentation problems in 3D, 4D, and generally n-D with limited annotation cost.

METHODS

Deep LOGISMOS combines deep-learning-based pre-segmentation of objects of interest provided by our convolutional neural network, FilterNet+, and our 3D multi-objects LOGISMOS framework (layered optimal graph image segmentation of multiple objects and surfaces) that uses newly designed trainable machine-learned cost functions. In the paradigm of assisted annotation, multi-object JEI for efficient editing of automated Deep LOGISMOS segmentation was employed to form a new larger training set with significant decrease of manual tracing effort.

RESULTS

We have evaluated our method on 350 lower leg (left/right) T1-weighted MR images from 93 subjects (47 healthy, 46 patients with muscular morbidity) by fourfold cross-validation. Compared with the fully manual annotation approach, the annotation cost with assisted annotation is reduced by 95%, from 8 h to 25 min in this study. The experimental results showed average Dice similarity coefficient (DSC) of96.56 ± 0.26 % $96.56\pm 0.26 \%$ and average absolute surface positioning error of 0.63 pixels (0.44 mm) for the five 3D muscle compartments for each leg. These results significantly improve our previously reported method and outperform the state-of-the-art nnUNet method.

CONCLUSIONS

Our proposed approach can not only dramatically reduce the expert's annotation efforts but also significantly improve the segmentation performance compared to the state-of-the-art nnUNet method. The notable performance improvements suggest the clinical-use potential of our new fully automated simultaneous segmentation of calf muscle compartments.

Hexagonal-Grid-Layout Image Segmentation Using Shock Filters: Computational Complexity Case Study for Microarray Image Analysis Related to Machine Learning Approaches

Hexagonal grid layouts are advantageous in microarray technology; however, hexagonal grids appear in many fields, especially given the rise of new nanostructures and metamaterials, leading to the need for image analysis on such structures. This work proposes a shock-filter-based approach driven by mathematical morphology for the segmentation of image objects disposed in a hexagonal grid. The original image is decomposed into a pair of rectangular grids, such that their superposition generates the initial image. Within each rectangular grid, the shock-filters are once again used to confine the foreground information for each image object into an area of interest. The proposed methodology was successfully applied for microarray spot segmentation, whereas its character of generality is underlined by the segmentation results obtained for two other types of hexagonal grid layouts. Considering the segmentation accuracy through specific quality measures for microarray images, such as the mean absolute error and the coefficient of variation, high correlations of our computed spot intensity features with the annotated reference values were found, indicating the reliability of the proposed approach. Moreover, taking into account that the shock-filter PDE formalism is targeting the one-dimensional luminance profile function, the computational complexity to determine the grid is minimized. The order of growth for the computational complexity of our approach is at least one order of magnitude lower when compared with state-of-the-art microarray segmentation approaches, ranging from classical to machine learning ones.

KCB-Net: A 3D knee cartilage and bone segmentation network via sparse annotation

Knee cartilage and bone segmentation is critical for physicians to analyze and diagnose articular damage and knee osteoarthritis (OA). Deep learning (DL) methods for medical image segmentation have largely outperformed traditional methods, but they often need large amounts of annotated data for model training, which is very costly and time-consuming for medical experts, especially on 3D images. In this paper, we report a new knee cartilage and bone segmentation framework, KCB-Net, for 3D MR images based on sparse annotation. KCB-Net selects a small subset of slices from 3D images for annotation, and seeks to bridge the performance gap between sparse annotation and full annotation. Specifically, it first identifies a subset of the most effective and representative slices with an unsupervised scheme; it then trains an ensemble model using the annotated slices; next, it self-trains the model using 3D images containing pseudo-labels generated by the ensemble method and improved by a bi-directional hierarchical earth mover's distance (bi-HEMD) algorithm; finally, it fine-tunes the segmentation results using the primal-dual Internal Point Method (IPM). Experiments on four 3D MR knee joint datasets (the SKI10 dataset, OAI ZIB dataset, Iowa dataset, and iMorphics dataset) show that our new framework outperforms state-of-the-art methods on full annotation, and yields high quality results for small annotation ratios even as low as 10%.

CMC-Net: 3D calf muscle compartment segmentation with sparse annotation

Accurate 3D segmentation of calf muscle compartments in volumetric MR images is essential to diagnose as well as assess progression of muscular diseases. Recently, good segmentation performance was achieved using state-of-the-art deep learning approaches, which, however, require large amounts of annotated data for training. Considering that obtaining sufficiently large medical image annotation datasets is often difficult, time-consuming, and requires expert knowledge, minimizing the necessary sizes of expert-annotated training datasets is of great importance. This paper reports CMC-Net, a new deep learning framework for calf muscle compartment segmentation in 3D MR images that selects an effective small subset of 2D slices from the 3D images to be labelled, while also utilizing unannotated slices to facilitate proper generalization of the subsequent training steps. Our model consists of three parts: (1) an unsupervised method to select the most representative 2D slices on which expert annotation is performed; (2) ensemble model training employing these annotated as well as additional unannotated 2D slices; (3) a model-tuning method using pseudo-labels generated by the ensemble model that results in a trained deep network capable of accurate 3D segmentations. Experiments on segmentation of calf muscle compartments in 3D MR images show that our new approach achieves good performance with very small annotation ratios, and when utilizing full annotation, it outperforms state-of-the-art full annotation segmentation methods. Additional experiments on a 3D MR thigh dataset further verify the ability of our method in segmenting leg muscle groups with sparse annotation.

Robustness and stability of volume-based tractography in a multicenter setting

Muscle diffusion tensor imaging (mDTI)-based tractography is a promising tool with which to detect subclinical changes in muscle injuries and to evaluate pathophysiology in neuromuscular diseases. Classic region of interest (ROI)-based tractography is very time-consuming and requires an examiner with extensive experience. (Semi)automatic approaches such as volume-based tractography (VBT) can diminish this problem but its robustness and stability are unknown. The aim of the current study was to assess the performance of VBT in a multicenter setting and to evaluate semiautomatic segmentation approaches in the analysis of VBT-derived data in terms of the comparability of the outcome measures. Five traveling volunteers underwent 3-T mDTI of seven calf muscles of both legs at six different MR sites. Tract properties and diffusion metrics were calculated using VBT. Within-subject coefficients of variance (wsCVs) and intraclass correlation coefficients (ICCs) were calculated to assess the multicenter reproducibility of tract properties such as tract density (TD), mean tract length, volume and tract propagation angle, and diffusion metrics such as fractional anisotropy, mean diffusivity, axial diffusivity (λ₁ ) and radial diffusivity in traveling subjects. Furthermore, 50 individual datasets from five different centers (10 datasets per center) were pooled to assess the feasibility of VBT with manual and semiautomatic segmentation. To assess the differences of tract properties and diffusion metrics between segmentation approaches an ANOVA was performed, and ICC and Bland-Altman plots were analyzed. wsCVs and ICCs showed good reproducibility of the tract properties TD and volume, as well as diffusion metrics. ANOVA showed no significant differences between manual and semiautomatic approaches. ICCs were excellent (≥ 0.992) and Bland-Altman analysis did not reveal any systemic bias between the methods. Tract properties and diffusion metrics derived from VBT showed good comparability among centers. Semiautomatic approaches revealed excellent agreement with gold standard of manual segmentation. These findings suggest that pooling data from different centers to construct a reference database for tractography results is feasible using semiautomatic segmentation approaches.

Deep learning methods for automatic segmentation of lower leg muscles and bones from MRI scans of children with and without cerebral palsy

Cerebral palsy is a neurological condition that is known to affect muscle growth. Detailed investigations of muscle growth require segmentation of muscles from MRI scans, which is typically done manually. In this study, we evaluated the performance of 2D, 3D, and hybrid deep learning models for automatic segmentation of 11 lower leg muscles and two bones from MRI scans of children with and without cerebral palsy. All six models were trained and evaluated on manually segmented T₁ -weighted MRI scans of the lower legs of 20 children, six of whom had cerebral palsy. The segmentation results were assessed using the median Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), and volume error (VError) of all 13 labels of every scan. The best performance was achieved by H-DenseUNet, a hybrid model (DSC 0.90, ASSD 0.5 mm, and VError 2.6 cm³ ). The performance was equivalent to the inter-rater performance of manual segmentation (DSC 0.89, ASSD 0.6 mm, and VError 3.3 cm³ ). Models trained with the Dice loss function outperformed models trained with the cross-entropy loss function. Near-optimal performance could be attained using only 11 scans for training. Segmentation performance was similar for scans of typically developing children (DSC 0.90, ASSD 0.5 mm, and VError 2.8 cm³ ) and children with cerebral palsy (DSC 0.85, ASSD 0.6 mm, and VError 2.4 cm³ ). These findings demonstrate the feasibility of fully automatic segmentation of individual muscles and bones from MRI scans of children with and without cerebral palsy.

3D Automated Segmentation of Lower Leg Muscles Using Machine Learning on a Heterogeneous Dataset

Quantitative MRI combines non-invasive imaging techniques to reveal alterations in muscle pathophysiology. Creating muscle-specific labels manually is time consuming and requires an experienced examiner. Semi-automatic and fully automatic methods reduce segmentation time significantly. Current machine learning solutions are commonly trained on data from healthy subjects using homogeneous databases with the same image contrast. While yielding high Dice scores (DS), those solutions are not applicable to different image contrasts and acquisitions. Therefore, the aim of our study was to evaluate the feasibility of automatic segmentation of a heterogeneous database. To create a heterogeneous dataset, we pooled lower leg muscle images from different studies with different contrasts and fields-of-view, containing healthy controls and diagnosed patients with various neuromuscular diseases. A second homogenous database with uniform contrasts was created as a subset of the first database. We trained three 3D-convolutional neuronal networks (CNN) on those databases to test performance as compared to manual segmentation. All networks, training on heterogeneous data, were able to predict seven muscles with a minimum average DS of 0.75. U-Net performed best when trained on the heterogeneous dataset (DS: 0.80 ± 0.10, AHD: 0.39 ± 0.35). ResNet and DenseNet yielded higher DS, when trained on a heterogeneous dataset (both DS: 0.86), as compared to a homogeneous dataset (ResNet DS: 0.83, DenseNet DS: 0.76). In conclusion, a CNN trained on a heterogeneous dataset achieves more accurate labels for predicting a heterogeneous database of lower leg muscles than a CNN trained on a homogenous dataset. We propose that a large heterogeneous database is needed, to make automated segmentation feasible for different kinds of image acquisitions.

High Inter-Rater Reliability of Manual Segmentation and Volume-Based Tractography in Healthy and Dystrophic Human Calf Muscle

BACKGROUND

Muscle diffusion tensor imaging (mDTI) is a promising surrogate biomarker in the evaluation of muscular injuries and neuromuscular diseases. Since mDTI metrics are known to vary between different muscles, separation of different muscles is essential to achieve muscle-specific diffusion parameters. The commonly used technique to assess DTI metrics is parameter maps based on manual segmentation (MSB). Other techniques comprise tract-based approaches, which can be performed in a previously defined volume. This so-called volume-based tractography (VBT) may offer a more robust assessment of diffusion metrics and additional information about muscle architecture through tract properties. The purpose of this study was to assess DTI metrics of human calf muscles calculated with two segmentation techniques-MSB and VBT-regarding their inter-rater reliability in healthy and dystrophic calf muscles.

METHODS

20 healthy controls and 18 individuals with different neuromuscular diseases underwent an MRI examination in a 3T scanner using a 16-channel Torso XL coil. DTI metrics were assessed in seven calf muscles using MSB and VBT. Coefficients of variation (CV) were calculated for both techniques. MSB and VBT were performed by two independent raters to assess inter-rater reliability by ICC analysis and Bland-Altman plots. Next to analysis of DTI metrics, the same assessments were also performed for tract properties extracted with VBT.

RESULTS

For both techniques, low CV were found for healthy controls (≤13%) and neuromuscular diseases (≤17%). Significant differences between methods were found for all diffusion metrics except for λ₁. High inter-rater reliability was found for both MSB and VBT (ICC ≥ 0.972). Assessment of tract properties revealed high inter-rater reliability (ICC ≥ 0.974).

CONCLUSIONS

Both segmentation techniques can be used in the evaluation of DTI metrics in healthy controls and different NMD with low rater dependency and high precision but differ significantly from each other. Our findings underline that the same segmentation protocol must be used to ensure comparability of mDTI data.

Evaluation of Amyotrophic Lateral Sclerosis-Induced Muscle Degeneration Using Magnetic Resonance-Based Relaxivity Contrast Imaging (RCI)

(1) Background: This work characterizes the sensitivity of magnetic resonance-based Relaxivity Contrast Imaging (RCI) to Amyotrophic Lateral Sclerosis (ALS)-induced changes in myofiber microstructure. Transverse Relaxivity at Tracer Equilibrium (TRATE), an RCI-based parameter, was evaluated in the lower extremities of ALS patients and healthy subjects. (2) Methods: In this IRB-approved study, 23 subjects (12 ALS patients and 11 healthy controls) were scanned at 3T (Philips, The Netherlands). RCI data were obtained during injection of a gadolinium-based contrast agent. TRATE, fat fraction and T₂ measures, were compared in five muscle groups of the calf muscle, between ALS and control populations. TRATE was also evaluated longitudinally (baseline and 6 months) and was compared to clinical measures, namely ALS Functional Rating Scale (ALSFRS-R) and Hand-Held Dynamometry (HHD), in a subset of the ALS population. (3) Results: TRATE was significantly lower (p < 0.001) in ALS-affected muscle than in healthy muscle in all muscle groups. Fat fraction differences between ALS and healthy muscle were statistically significant for the tibialis anterior (p = 0.01), tibialis posterior (p = 0.004), and peroneus longus (p = 0.02) muscle groups but were not statistically significant for the medial (p = 0.07) and lateral gastrocnemius (p = 0.06) muscles. T₂ differences between ALS and healthy muscle were statistically significant for the tibialis anterior (p = 0.004), peroneus longus (p = 0.004) and lateral gastrocnemius (p = 0.03) muscle groups but were not statistically significant for the tibialis posterior (p = 0.06) and medial gastrocnemius (p = 0.07) muscles. Longitudinally, TRATE, averaged over all patients, decreased by 28 ± 16% in the tibialis anterior, 47 ± 18% in the peroneus longus, 25 ± 19% in the tibialis posterior, 29 ± 14% in the medial gastrocnemius and 35 ± 18% in the lateral gastrocnemius muscles between two timepoints. ALSFRS-R scores were stable in two of four ALS patients. HHD scores decreased in three of four ALS patients. (4) Conclusion: RCI-based TRATE was shown to consistently differentiate ALS-affected muscle from healthy muscle and also provide a quantitative measure of longitudinal muscle degeneration.

Quantitative muscle MRI as a sensitive marker of early muscle pathology in myotonic dystrophy type 1

BACKGROUND

Quantitative muscle MRI as a sensitive marker of early muscle pathology and disease progression in adult-onset myotonic dystrophy type 1. The utility of muscle MRI as a marker of muscle pathology and disease progression in adult-onset myotonic dystrophy type 1 (DM1) was evaluated.

METHODS

This prospective, longitudinal study included 67 observations from 36 DM1 patients (50% female), and 92 observations from 49 healthy adults (49% female). Lower-leg 3T magnetic resonance imaging (MRI) scans were acquired. Volume and fat fraction (FF) were estimated using a three-point Dixon method, and T2-relaxometry was determined using a multi-echo spin-echo sequence. Muscles were segmented automatically. Mixed linear models were conducted to determine group differences across muscles and image modality, accounting for age, sex, and repeated observations. Differences in rate of change in volume, T2-relaxometry, and FF were also determined with mixed linear regression that included a group by elapsed time interaction.

RESULTS

Compared with healthy adults, DM1 patients exhibited reduced volume of the tibialis anterior, soleus, and gastrocnemius (GAS) (all, P < .05). T2-relaxometry and FF were increased across all calf muscles in DM1 compared to controls. (all, P < .01). Signs of muscle pathology, including reduced volume, and increased T2-relaxometry and FF were already noted in DM1 patients who did not exhibit clinical motor symptoms of DM1. As a group, DM1 patients exhibited a more rapid change than did controls in tibialis posterior volume (P = .05) and GAS T2-relaxometry (P = .03) and FF (P = .06).

CONCLUSIONS

Muscle MRI renders sensitive, early markers of muscle pathology and disease progression in DM1. T2 relaxometry may be particularly sensitive to early muscle changes related to DM1.

Overview of MR Image Segmentation Strategies in Neuromuscular Disorders

Neuromuscular disorders are rare diseases for which few therapeutic strategies currently exist. Assessment of therapeutic strategies efficiency is limited by the lack of biomarkers sensitive to the slow progression of neuromuscular diseases (NMD). Magnetic resonance imaging (MRI) has emerged as a tool of choice for the development of qualitative scores for the study of NMD. The recent emergence of quantitative MRI has enabled to provide quantitative biomarkers more sensitive to the evaluation of pathological changes in muscle tissue. However, in order to extract these biomarkers from specific regions of interest, muscle segmentation is mandatory. The time-consuming aspect of manual segmentation has limited the evaluation of these biomarkers on large cohorts. In recent years, several methods have been proposed to make the segmentation step automatic or semi-automatic. The purpose of this study was to review these methods and discuss their reliability, reproducibility, and limitations in the context of NMD. A particular attention has been paid to recent deep learning methods, as they have emerged as an effective method of image segmentation in many other clinical contexts.