A modality-collaborative convolution and transformer hybrid network for unpaired multi-modal medical image segmentation with limited annotations

BACKGROUND

Multi-modal learning is widely adopted to learn the latent complementary information between different modalities in multi-modal medical image segmentation tasks. Nevertheless, the traditional multi-modal learning methods require spatially well-aligned and paired multi-modal images for supervised training, which cannot leverage unpaired multi-modal images with spatial misalignment and modality discrepancy. For training accurate multi-modal segmentation networks using easily accessible and low-cost unpaired multi-modal images in clinical practice, unpaired multi-modal learning has received comprehensive attention recently.

PURPOSE

Existing unpaired multi-modal learning methods usually focus on the intensity distribution gap but ignore the scale variation problem between different modalities. Besides, within existing methods, shared convolutional kernels are frequently employed to capture common patterns in all modalities, but they are typically inefficient at learning global contextual information. On the other hand, existing methods highly rely on a large number of labeled unpaired multi-modal scans for training, which ignores the practical scenario when labeled data is limited. To solve the above problems, we propose a modality-collaborative convolution and transformer hybrid network (MCTHNet) using semi-supervised learning for unpaired multi-modal segmentation with limited annotations, which not only collaboratively learns modality-specific and modality-invariant representations, but also could automatically leverage extensive unlabeled scans for improving performance.

METHODS

We make three main contributions to the proposed method. First, to alleviate the intensity distribution gap and scale variation problems across modalities, we develop a modality-specific scale-aware convolution (MSSC) module that can adaptively adjust the receptive field sizes and feature normalization parameters according to the input. Secondly, we propose a modality-invariant vision transformer (MIViT) module as the shared bottleneck layer for all modalities, which implicitly incorporates convolution-like local operations with the global processing of transformers for learning generalizable modality-invariant representations. Third, we design a multi-modal cross pseudo supervision (MCPS) method for semi-supervised learning, which enforces the consistency between the pseudo segmentation maps generated by two perturbed networks to acquire abundant annotation information from unlabeled unpaired multi-modal scans.

RESULTS

Extensive experiments are performed on two unpaired CT and MR segmentation datasets, including a cardiac substructure dataset derived from the MMWHS-2017 dataset and an abdominal multi-organ dataset consisting of the BTCV and CHAOS datasets. Experiment results show that our proposed method significantly outperforms other existing state-of-the-art methods under various labeling ratios, and achieves a comparable segmentation performance close to single-modal methods with fully labeled data by only leveraging a small portion of labeled data. Specifically, when the labeling ratio is 25%, our proposed method achieves overall mean DSC values of 78.56% and 76.18% in cardiac and abdominal segmentation, respectively, which significantly improves the average DSC value of two tasks by 12.84% compared to single-modal U-Net models.

CONCLUSIONS

Our proposed method is beneficial for reducing the annotation burden of unpaired multi-modal medical images in clinical applications.

Effects of voluntary and involuntary exercise on cognitive functions, and VEGF and BDNF levels in adolescent rats

Regular treadmill running during adolescence improves learning and memory in rats. During adolescence, the baseline level of stress is thought to be greater than during other periods of life. We investigated the effects of voluntary and involuntary exercise on the prefrontal cortex and hippocampus, vascular endothelial growth factor (VEGF), brain-derived neurotrophic factor (BDNF) levels, and spatial learning, memory and anxiety in adolescent male and female rats. The voluntary exercise group was given free access to a running wheel for 6 weeks. The involuntary exercise group was forced to run on a treadmill for 30 min at 8 m/min 5 days/week for 6 weeks. Improved learning was demonstrated in both exercise groups compared to controls. Neuron density in the CA1 region of the hippocampus, dentate gyrus and prefrontal cortex were increased. Hippocampal VEGF and BDNF levels were increased in both exercise groups compared to controls. In females, anxiety and corticosterone levels were decreased; BDNF and VEGF levels were higher in the voluntary exercise group than in the involuntary exercise group. The adolescent hippocampus is affected favorably by regular exercise. Although no difference was found in anxiety levels as a result of involuntary exercise in males, females showed increased anxiety levels, and decreased VEGF and BDNF levels in the prefrontal cortex after involuntary exercise.

Continuous hypoxic culturing maintains activation of Notch and allows long-term propagation of human embryonic stem cells without spontaneous differentiation

OBJECTIVE

The maintenance of pluripotency of human embryonic stem cells (hESCs) requires a high efficiency of self-renewal. During in vitro propagation, however, hESCs have a propensity to differentiate spontaneously. In this study, we assessed the nature of hESC responses to hypoxic conditions.

MATERIALS AND METHODS

Human embryonic stem cells were grown in normoxic and hypoxic conditions, and the cells expressing Oct4 and stage-specific embryonic antigen-1 were identified by indirect immunofluorescence. The transcriptional expression of Nanog, Notch1, and Oct4 was determined by a real-time reverse transcription-polymerase chain reaction, and the inhibition of Notch-mediated signalling was achieved with a gamma-secretase inhibitor.

RESULTS

In contrast to culture at 21% oxygen, where the colonies displayed a marked degree of differentiation, we found that during exposure to 5% oxygen, the hESC colonies displayed a homogenous and flat morphology that was consistent with the presence of Oct4-positive phenotype, indicating no spontaneous differentiation. When cultured at 5% oxygen for either 4 weeks or up to 18 months, high levels of Nanog and Notch1 transcriptional expression were detected, albeit the expression was significantly lower during longer exposure. The suppression of differentiation was rapidly reversed on transfer of the hypoxic cultures to normoxic conditions. Looking into the molecular mechanisms of the maintenance of self-renewal at low oxygen tensions, we found that inhibition of Notch signalling fully abrogated the hypoxic induction of undifferentiated phenotype.

CONCLUSION

Our data, thus, indicate that hypoxic exposure has the capacity to sustain long-term self-renewal of hESCs and that this effect is mediated through activation of Notch.

Kinetic analysis of recombinant mammalian ?1 and ?1? glycine receptor channels

To analyze the influence of the beta-subunit on the kinetic properties of GlyR channel currents, alpha(1)-subunits and alpha(1)beta-subunits were transiently expressed in HEK 293 cells. A piezo dimorph was used for fast application of glycine to outside-out patches. The rise time of activation was dose dependent for both receptors and decreased with increasing glycine concentrations. Subunit composition had no effect on the time course of activation. Coexpression of alpha(1)- and beta-subunits resulted in a significantly lower EC(50) and a reduced slope of the dose-response curve of glycine compared with expression of alpha(1)-subunits alone. For both receptor subtypes, the time course of desensitization was concentration dependent. Desensitization was best fitted with a single time constant at 10-30 micro M, with two at 0.1 mM, and at saturating concentrations (0.3-3 mM) with three time constants. Desensitization of homomeric alpha(1)-receptor channels was significantly slower than that of alpha(1)beta-receptor channels. The time course of current decay after the end of glycine pulses was tested at different pulse durations of 1 mM glycine. It was best fitted with two time constants for both alpha(1) and alpha(1)beta GlyR channels, and increased significantly with increasing pulse duration.

Interferon-gamma cooperates with retinoic acid and phorbol ester to induce differentiation and growth inhibition of human neuroblastoma cells

The prognosis of patients with advanced stages of neuroblastoma with N-myc amplification remains poor despite escalated therapy, a situation that has called for alternative therapeutic approaches. Neuroblastoma cells, which represent immature peripheral neuronal cells, treated with certain physiologic and nonphysiologic agents such as retinoic acid (RA), phorbol esters and interferons (IFN) in vitro undergo cellular differentiation and stop to divide, a process that mimics normal neuronal development. Such "differentiation therapy" using RA after autologous bone marrow transplantation has recently given encouraging results in neuroblastoma patients with advanced disease. Considering approaches for improved differentiation therapy, we investigated possible synergistic effects of combining agents known to influence neuroblastoma growth and differentiation in vitro. Our results show that combined treatment with IFN-gamma and RA or the phorbol ester 12-O-tetradecanoyl-phorbol acetate (TPA) had synergistic or enhancing effects on morphologic differentiation and neurite outgrowth in 5 of 5 neuroblastoma cell lines, 3 of which expressed very high levels of N-myc mRNA due to N-myc amplification. The combinations RA+IFN-gamma or TPA+IFN-gamma also enhanced induced growth inhibition in all 5 cell lines, in several cases resulting in complete growth arrest under conditions where cells stimulated with either agent alone continued to grow. The phenotypic effects of the combined RA+IFN-gamma or TPA+IFN-gamma treatments were in most, but not all, investigated cases accompanied by moderate reductions in N-myc expression, suggesting that the cooperative signals may counteract N-Myc activity at several levels. The cooperativity between IFN-gamma and other differentiation signals may be relevant for approaches to improve the therapy for high-risk neuroblastoma with N-myc-amplification.

c-Myc hot spot mutations in lymphomas result in inefficient ubiquitination and decreased proteasome-mediated turnover

The c-myc proto-oncogene encodes a short-lived transcription factor that plays an important role in cell cycle regulation, differentiation and apoptosis. c-myc is often rearranged in tumors resulting in deregulated expression. In addition, mutations in the coding region of c-myc are frequently found in human lymphomas, a hot spot being the Thr58 phosphorylation site, a mutation shown to enhance the transforming capacity of c-Myc. It is, however, still unclear in what way this mutation affects c-Myc activity. Our results show that proteasome-mediated turnover of c-Myc is substantially impaired in Burkitt's lymphoma cells with mutated Thr58 or other mutations that abolish Thr58 phosphorylation, whereas endogenous or ectopically expressed wild type c-Myc proteins turn over at normal rates in these cells. Myc Thr58 mutants expressed ectopically in other cell types also exhibit reduced proteasome-mediated degradation, which correlates with a substantial decrease in their ubiquitination. These results suggest that ubiquitin/proteasome-mediated degradation of c-Myc is triggered by Thr58 phosphorylation revealing a new important level of control of c-Myc activity. Mutation of Thr58 in lymphoma thus escapes this regulation resulting in accumulation of c-Myc protein, likely as part of the tumor progression. (Blood. 2000;95:2104-2110)