Stacked Bidirectional Convolutional LSTMs for Deriving 3D Non-Contrast CT From Spatiotemporal 4D CT

Application of emerging technologies in ischemic stroke: from clinical study to basic research

Stroke is a primary cause of noncommunicable disease-related death and disability worldwide. The most common form, ischemic stroke, is increasing in incidence resulting in a significant burden on patients and society. Urgent action is thus needed to address preventable risk factors and improve treatment methods. This review examines emerging technologies used in the management of ischemic stroke, including neuroimaging, regenerative medicine, biology, and nanomedicine, highlighting their benefits, clinical applications, and limitations. Additionally, we suggest strategies for technological development for the prevention, diagnosis, and treatment of ischemic stroke.

Feasibility Study of Convolutional Long ShortTerm Memory Network for Pulmonary Movement Prediction in CT Images

Background

During X-ray imaging, pulmonary movements can cause many image artifacts. To tackle this issue, several studies, including mathematical algorithms and 2D-3D image registration methods, have been presented. Recently, the application of deep artificial neural networks has been considered for image generation and prediction.

Objective

In this study, a convolutional long short-term memory (ConvLSTM) neural network is used to predict spatiotemporal 4DCT images.

Material and Methods

In this analytical analysis study, two ConvLSTM structures, consisting of stacked ConvLSTM models along with the hyperparameter optimizer algorithm and a new design of the ConvLSTM model are proposed. The hyperparameter optimizer algorithm in the conventional ConvLSTM includes the number of layers, number of filters, kernel size, epoch number, optimizer, and learning rate. The two ConvLSTM structures were also evaluated through six experiments based on Root Mean Square Error (RMSE) and structural similarity index (SSIM).

Results

Comparing the two networks demonstrates that the new design of the ConvLSTM network is faster, more accurate, and more reliable in comparison to the tuned-stacked ConvLSTM model. For all patients, the estimated RMSE and SSIM were 3.17 and 0.988, respectively, and a significant improvement can be observed in comparison to the previous studies.

Conclusion

Overall, the results of the new design of the ConvLSTM network show excellent performances in terms of RMSE and SSIM. Also, the generated CT images with the new design of the ConvLSTM model show a good consistency with the corresponding references regarding registration accuracy and robustness.

STANet: Spatio-Temporal Adaptive Network and Clinical Prior Embedding Learning for 3D+T CMR Segmentation

The segmentation of cardiac structure in magnetic resonance images (CMR) is paramount in diagnosing and managing cardiovascular illnesses, given its 3D+Time (3D+T) sequence. The existing deep learning methods are constrained in their ability to 3D+T CMR segmentation, due to: (1) Limited motion perception. The complexity of heart beating renders the motion perception in 3D+T CMR, including the long-range and cross-slice motions. The existing methods' local perception and slice-fixed perception directly limit the performance of 3D+T CMR perception. (2) Lack of labels. Due to the expensive labeling cost of the 3D+T CMR sequence, the labels of 3D+T CMR only contain the end-diastolic and end-systolic frames. The incomplete labeling scheme causes inefficient supervision. Hence, we propose a novel spatio-temporal adaptation network with clinical prior embedding learning (STANet) to ensure efficient spatio-temporal perception and optimization on 3D+T CMR segmentation. (1) A spatio-temporal adaptive convolution (STAC) treats the 3D+T CMR sequence as a whole for perception. The long-distance motion correlation is embedded into the structural perception by learnable weight regularization to balance long-range motion perception. The structural similarity is measured by cross-attention to adaptively correlate the cross-slice motion. (2) A clinical prior embedding learning strategy (CPE) is proposed to optimize the partially labeled 3D+T CMR segmentation dynamically by embedding clinical priors into optimization. STANet achieves outstanding performance with Dice of 0.917 and 0.94 on two public datasets (ACDC and STACOM), which indicates STANet has the potential to be incorporated into computer-aided diagnosis tools for clinical application.

Brain image segmentation of the corpus callosum by combining Bi-Directional Convolutional LSTM and U-Net using multi-slice CT and MRI

BACKGROUND AND OBJECTIVE

Traditional disease diagnosis is usually performed by experienced physicians, but misdiagnosis or missed diagnosis still exists. Exploring the relationship between changes in the corpus callosum and multiple brain infarcts requires extracting corpus callosum features from brain image data, which requires addressing three key issues. (1) automation, (2) completeness, and (3) accuracy. Residual learning can facilitate network training, Bi-Directional Convolutional LSTM (BDC-LSTM) can exploit interlayer spatial dependencies, and HDC can expand the receptive domain without losing resolution.

METHODS

In this paper, we propose a segmentation method by combining BDC-LSTM and U-Net to segment the corpus callosum from multiple angles of brain images based on computed tomography (CT) and magnetic resonance imaging (MRI) in which two types of sequence, namely T2-weighted imaging as well as the Fluid Attenuated Inversion Recovery (Flair), were utilized. The two-dimensional slice sequences are segmented in the cross-sectional plane, and the segmentation results are combined to obtain the final results. Encoding, BDC- LSTM, and decoding include convolutional neural networks. The coding part uses asymmetric convolutional layers of different sizes and dilated convolutions to get multi-slice information and extend the convolutional layers' perceptual field.

RESULTS

This paper uses BDC-LSTM between the encoding and decoding parts of the algorithm. On the image segmentation of the brain in multiple cerebral infarcts dataset, accuracy rates of 0.876, 0.881, 0.887, and 0.912 were attained for the intersection of union (IOU), dice similarity coefficient (DS), sensitivity (SE), and predictive positivity value (PPV). The experimental findings demonstrate that the algorithm outperforms its rivals in accuracy.

CONCLUSION

This paper obtained segmentation results for three images using three models, ConvLSTM, Pyramid-LSTM, and BDC-LSTM, and compared them to verify that BDC-LSTM is the best method to perform the segmentation task for faster and more accurate detection of 3D medical images. We improve the convolutional neural network segmentation method to obtain medical images with high segmentation accuracy by solving the over-segmentation problem.

Multi-scale feature pyramid fusion network for medical image segmentation

PURPOSE

Medical image segmentation is the most widely used technique in diagnostic and clinical research. However, accurate segmentation of target organs from blurred border regions and low-contrast adjacent organs in Computed tomography (CT) imaging is crucial for clinical diagnosis and treatment.

METHODS

In this article, we propose a Multi-Scale Feature Pyramid Fusion Network (MS-Net) based on the codec structure formed by the combination of Multi-Scale Attention Module (MSAM) and Stacked Feature Pyramid Module (SFPM). Among them, MSAM is used to skip connections, which aims to extract different levels of context details by dynamically adjusting the receptive fields under different network depths; the SFPM including multi-scale strategies and multi-layer Feature Perception Module (FPM) is nested in the network at the deepest point, which aims to better focus the network's attention on the target organ by adaptively increasing the weight of the features of interest.

RESULTS

Experiments demonstrate that the proposed MS-Net significantly improved the Dice score from 91.74% to 94.54% on CHAOS, from 97.59% to 98.59% on Lung, and from 82.55% to 86.06% on ISIC 2018, compared with U-Net. Additionally, comparisons with other six state-of-the-art codec structures also show the presented network has great advantages on evaluation indicators such as Miou, Dice, ACC and AUC.

CONCLUSION

The experimental results show that both the MSAM and SFPM techniques proposed in this paper can assist the network to improve the segmentation effect, so that the proposed MS-Net method achieves better results in the CHAOS, Lung and ISIC 2018 segmentation tasks.

Parallel non-Cartesian spatial-temporal dictionary learning neural networks (stDLNN) for accelerating 4D-MRI

Dynamic magnetic resonance imaging (MRI) acquisitions are relatively slow due to physical and physiological limitations. The spatial-temporal dictionary learning (DL) approach accelerates dynamic MRI by learning spatial-temporal correlations, but the regularization parameters need to be manually adjusted, the performance at high acceleration rate is limited, and the reconstruction can be time-consuming. Deep learning techniques have shown good performance in accelerating MRI due to the powerful representational capabilities of neural networks. In this work, we propose a parallel non-Cartesian spatial-temporal dictionary learning neural networks (stDLNN) framework that combines dictionary learning with deep learning algorithms and utilizes the spatial-temporal prior information of dynamic MRI data to achieve better reconstruction quality and efficiency. The coefficient estimation modules (CEM) are designed in the framework to adaptively adjust the regularization coefficients. Experimental results show that combining dictionary learning with deep neural networks and using spatial-temporal dictionaries can obviously improve the image quality and computational efficiency compared with the state-of-the-art non-Cartesian imaging methods for accelerating the 4D-MRI especially at high acceleration rate.

Predicting treatment-specific lesion outcomes in acute ischemic stroke from 4D CT perfusion imaging using spatio-temporal convolutional neural networks

For the diagnosis and precise treatment of acute ischemic stroke, predicting the final location and volume of lesions is of great clinical interest. Current deep learning-based prediction methods mainly use perfusion parameter maps, which can be calculated from spatio-temporal (4D) CT perfusion (CTP) imaging data, to estimate the tissue outcome of an acute ischemic stroke. However, this calculation relies on a deconvolution operation, an ill-posed problem requiring strong regularization and definition of an arterial input function. Thus, improved predictions might be achievable if the deep learning models were applied directly to acute 4D CTP data rather than perfusion maps. In this work, a novel deep spatio-temporal convolutional neural network is proposed for predicting treatment-dependent stroke lesion outcomes by making full use of raw 4D CTP data. By merging a U-Net-like architecture with temporal convolutional networks, we efficiently process the spatio-temporal information available in CTP datasets to make a tissue outcome prediction. The proposed method was evaluated on 147 patients using a 10-fold cross validation, which demonstrated that the proposed 3D+time model (mean Dice=0.45) significantly outperforms both a 2D+time variant of our approach (mean Dice=0.43) and a state-of-the-art method that uses perfusion maps (mean Dice=0.38). These results show that 4D CTP datasets include more predictive information than perfusion parameter maps, and that the proposed method is an efficient approach to make use of this complex data.

Strength prediction and application of cemented paste backfill based on machine learning and strength correction

Cemented paste backfill (CPB) is wildly used in mines production practices around the world. The strength of CPB is the core of research which is affected by factors such as slurry concentration and cement content. In this paper, a research on the UCS is conducted by means of a combination of laboratory experiments and machine learning. BPNN, RBFNN, GRNN and LSTM are trained and used for UCS prediction based on 180 sets of experimental UCS data. The simulation results show that LSTM is the neural network with the optimal prediction performance (The total rank is 11). The trial-and-error, PSO, GWO and SSA are used to optimize the learning rate and the hidden layer nodes for LSTM. The comparison results show that GWO-LSTM is the optimal model which can effectively express the non-linear relationship between underflow productivity, slurry concentration, cement content and UCS in experiments (R=0.9915, RMSE = 0.0204, VAF = 98.2847 and T = 16.37 s). The correction coefficient (k) is defined to adjust the error between predicted UCS in laboratory (UCS_M) and predicted UCS in actual engineering (UCS_A) based on extensive engineering and experimental experience. Using GWO-LSTM combined with k, the strength of the filling body is successfully predicted for 153 different filled stopes with different stowing gradient at different curing times. This study provides both effective guidance and a new intelligent method for the support of safety mining.

Deep Learning-Based Computed Tomography Perfusion Imaging to Evaluate the Effectiveness and Safety of Thrombolytic Therapy for Cerebral Infarct with Unknown Time of Onset

This study was aimed to discuss the effectiveness and safety of deep learning-based computed tomography perfusion (CTP) imaging in the thrombolytic therapy for acute cerebral infarct with unknown time of onset. A total of 100 patients with acute cerebral infarct with unknown time of onset were selected as the research objects. All patients received thrombolytic therapy. According to different image processing methods, they were divided into the algorithm group (artificial intelligence algorithm-based image processing group) and the control group (conventional method-based image processing group). After that, the evaluations of effectiveness and safety of thrombolytic therapy for the patients with acute cerebral infarct in the two groups were compared. The research results demonstrated that artificial intelligence algorithm-based CTP imaging showed significant diagnostic effects and the image quality in the algorithm group was remarkably higher than that in the control group (P < 0.05). Besides, the overall image quality of algorithm group was relatively higher. The differences in the National Institute of Health stroke scale (NIHSS) scores for the two groups indicated that the thrombolytic effect on the algorithm group was superior to that on the control group. Thrombolytic therapy for the algorithm group showed therapeutic effects on neurologic impairment. The symptomatic intracranial hemorrhage rate of the algorithm group within 24 hours was lower than the hemorrhage conversion rate of the control group, and the difference between the two groups was 14%. The data differences between the two groups showed statistical significance (P < 0.05). The results demonstrated that the safety of guided thrombolytic therapy for the algorithm group was higher than that in the control group. To sum up, deep learning-based CTP images showed the clinical application values in the diagnosis of cerebral infarct.

Robust Deep 3D Blood Vessel Segmentation Using Structural Priors

Deep learning has enabled significant improvements in the accuracy of 3D blood vessel segmentation. Open challenges remain in scenarios where labeled 3D segmentation maps for training are severely limited, as is often the case in practice, and in ensuring robustness to noise. Inspired by the observation that 3D vessel structures project onto 2D image slices with informative and unique edge profiles, we propose a novel deep 3D vessel segmentation network guided by edge profiles. Our network architecture comprises a shared encoder and two decoders that learn segmentation maps and edge profiles jointly. 3D context is mined in both the segmentation and edge prediction branches by employing bidirectional convolutional long-short term memory (BCLSTM) modules. 3D features from the two branches are concatenated to facilitate learning of the segmentation map. As a key contribution, we introduce new regularization terms that: a) capture the local homogeneity of 3D blood vessel volumes in the presence of biomarkers; and b) ensure performance robustness to domain-specific noise by suppressing false positive responses. Experiments on benchmark datasets with ground truth labels reveal that the proposed approach outperforms state-of-the-art techniques on standard measures such as DICE overlap and mean Intersection-over-Union. The performance gains of our method are even more pronounced when training is limited. Furthermore, the computational cost of our network inference is among the lowest compared with state-of-the-art.

A Tensor-Based Framework for rs-fMRI Classification and Functional Connectivity Construction

Recently, machine learning methods have gained lots of attention from researchers seeking to analyze brain images such as Resting-State Functional Magnetic Resonance Imaging (rs-fMRI) to obtain a deeper understanding of the brain and such related diseases, for example, Alzheimer's disease. Finding the common patterns caused by a brain disorder through analysis of the functional connectivity (FC) network along with discriminating brain diseases from normal controls have long been the two principal goals in studying rs-fMRI data. The majority of FC extraction methods calculate the FC matrix for each subject and then use simple techniques to combine them and obtain a general FC matrix. In addition, the state-of-the-art classification techniques for finding subjects with brain disorders also rely on calculating an FC for each subject, vectorizing, and feeding them to the classifier. Considering these problems and based on multi-dimensional nature of the data, we have come up with a novel tensor framework in which a general FC matrix is obtained without the need to construct an FC matrix for each sample. This framework also allows us to reduce the dimensionality and create a novel discriminant function that rather than using FCs works directly with each sample, avoids vectorization in any step, and uses the test data in the training process without forcing any prior knowledge of its label into the classifier. Extensive experiments using the ADNI dataset demonstrate that our proposed framework effectively boosts the fMRI classification performance and reveals novel connectivity patterns in Alzheimer's disease at its early stages.

CBCT image quality QA: Establishing a quantitative program

Purpose

Routine quality assurance (QA) of cone‐beam computed tomography (CBCT) scans used for image‐guided radiotherapy is prescribed by the American Association of Physicists in Medicine Task Group (TG)‐142 report. For CBCT image quality, TG‐142 recommends using clinically established baseline values as QA tolerances. This work examined how image quality parameters vary both across machines of the same model and across different CBCT techniques. Additionally, this work investigated how image quality values are affected by imager recalibration and repeated exposures during routine QA.

Methods

Cone‐beam computed tomography scans of the Catphan 604 phantom were taken on four TrueBeam® and one Edge™ linear accelerator using four manufacturer‐provided techniques. TG‐142 image quality parameters were calculated for each CBCT scan using SunCHECK Machine™. The variability of each parameter with machine and technique was evaluated using a two‐way ANOVA test on a dataset consisting of 200 CBCT scans. The impact of imager calibration on image quality parameters was examined for a subset of three machines using an unpaired Student’s t‐test. The effect of artifacts appearing on CBCTs taken in rapid succession was characterized and an approach to reduce their appearance was evaluated. Additionally, a set of baselines and tolerances for all image quality metrics was presented.

Results

All imaging parameters except geometric distortion varied with technique (P < 0.05) and all imaging parameters except slice thickness varied with machine (P < 0.05). Imager calibration can change the expected value of all imaging parameters, though it does not consistently do so. While changes are statistically significant, they may not be clinically significant. Finally, rapid acquisition of CBCT scans can introduce image artifacts that degrade CBCT uniformity.

Conclusions

This work characterized the variability of acquired CBCT data across machines and CBCT techniques along with the impact of imager calibration and rapid CBCT acquisition on image quality.

Spatio-temporal deep learning methods for motion estimation using 4D OCT image data

PURPOSE

Localizing structures and estimating the motion of a specific target region are common problems for navigation during surgical interventions. Optical coherence tomography (OCT) is an imaging modality with a high spatial and temporal resolution that has been used for intraoperative imaging and also for motion estimation, for example, in the context of ophthalmic surgery or cochleostomy. Recently, motion estimation between a template and a moving OCT image has been studied with deep learning methods to overcome the shortcomings of conventional, feature-based methods.

METHODS

We investigate whether using a temporal stream of OCT image volumes can improve deep learning-based motion estimation performance. For this purpose, we design and evaluate several 3D and 4D deep learning methods and we propose a new deep learning approach. Also, we propose a temporal regularization strategy at the model output.

RESULTS

Using a tissue dataset without additional markers, our deep learning methods using 4D data outperform previous approaches. The best performing 4D architecture achieves an correlation coefficient (aCC) of 98.58% compared to 85.0% of a previous 3D deep learning method. Also, our temporal regularization strategy at the output further improves 4D model performance to an aCC of 99.06%. In particular, our 4D method works well for larger motion and is robust toward image rotations and motion distortions.

CONCLUSIONS

We propose 4D spatio-temporal deep learning for OCT-based motion estimation. On a tissue dataset, we find that using 4D information for the model input improves performance while maintaining reasonable inference times. Our regularization strategy demonstrates that additional temporal information is also beneficial at the model output.