Exploring simple triplet representation learning

Fully supervised learning methods necessitate a substantial volume of labelled training instances, a process that is typically both labour-intensive and costly. In the realm of medical image analysis, this issue is further amplified, as annotated medical images are considerably more scarce than their unlabelled counterparts. Consequently, leveraging unlabelled images to extract meaningful underlying knowledge presents a formidable challenge in medical image analysis. This paper introduces a simple triple-view unsupervised representation learning model (SimTrip) combined with a triple-view architecture and loss function, aiming to learn meaningful inherent knowledge efficiently from unlabelled data with small batch size. With the meaningful representation extracted from unlabelled data, our model demonstrates exemplary performance across two medical image datasets. It achieves this using only partial labels and outperforms other state-of-the-art methods. The method we present herein offers a novel paradigm for unsupervised representation learning, establishing a baseline that is poised to inspire the development of more intricate SimTrip-based methods across a spectrum of computer vision applications. Code and user guide are released at https://github.com/JerryRollingUp/SimTripSystem, the system also runs at http://43.131.9.159:5000/.

PatchCL-AE: Anomaly detection for medical images using patch-wise contrastive learning-based auto-encoder

Anomaly detection is an important yet challenging task in medical image analysis. Most anomaly detection methods are based on reconstruction, but the performance of reconstruction-based methods is limited due to over-reliance on pixel-level losses. To address the limitation, we propose a patch-wise contrastive learning-based auto-encoder for medical anomaly detection. The key contribution is the patch-wise contrastive learning loss that provides supervision on local semantics to enforce semantic consistency between corresponding input-output patches. Contrastive learning pulls corresponding patch pairs closer while pushing non-corresponding ones apart between input and output, enabling the model to learn local normal features better and improve discriminability on anomalous regions. Additionally, we design an anomaly score based on local semantic discrepancies to pinpoint abnormalities by comparing feature difference rather than pixel variations. Extensive experiments on three public datasets (i.e., brain MRI, retinal OCT, and chest X-ray) achieve state-of-the-art performance, with our method achieving over 99% AUC on retinal and brain images. Both the contrastive patch-wise supervision and patch-discrepancy score provide targeted advancements to overcome the weaknesses in existing approaches.

Contrastive representation learning on dynamic networks

Representation learning for dynamic networks is designed to learn the low-dimensional embeddings of nodes that can well preserve the snapshot structure, properties and temporal evolution of dynamic networks. However, current dynamic network representation learning methods tend to focus on estimating or generating observed snapshot structures, paying excessive attention to network details, and disregarding distinctions between snapshots with larger time intervals, resulting in less robustness for sparse or noisy networks. To alleviate these challenges, this paper proposes a contrastive mechanism for temporal representation learning on dynamic networks, inspired by the success of contrastive learning in visual and static network representation learning. This paper proposes a novel Dynamic Network Contrastive representation Learning (DNCL) model. Specifically, contrast objective functions are constructed using intra-snapshot and inter-snapshot contrasts to capture the network topology, node feature information, and network evolution information, respectively. Rather than estimating or generating ground-truth network features, the proposed approach maximizes mutual information between nodes from different time steps and views generated. The experimental results of link prediction, node classification, and clustering on several real-world and synthetic networks demonstrate the superiority of DNCL over state-of-the-art methods, indicating the effectiveness of the proposed approach for dynamic network representation learning.

INTransformer: Data augmentation-based contrastive learning by injecting noise into transformer for molecular property prediction

Molecular property prediction plays an essential role in drug discovery for identifying the candidate molecules with target properties. Deep learning models usually require sufficient labeled data to train good prediction models. However, the size of labeled data is usually small for molecular property prediction, which brings great challenges to deep learning-based molecular property prediction methods. Furthermore, the global information of molecules is critical for predicting molecular properties. Therefore, we propose INTransformer for molecular property prediction, which is a data augmentation method via contrastive learning to alleviate the limitations of the labeled molecular data while enhancing the ability to capture global information. Specifically, INTransformer consists of two identical Transformer sub-encoders to extract the molecular representation from the original SMILES and noisy SMILES respectively, while achieving the goal of data augmentation. To reduce the influence of noise, we use contrastive learning to ensure the molecular encoding of noisy SMILES is consistent with that of the original input so that the molecular representation information can be better extracted by INTransformer. Experiments on various benchmark datasets show that INTransformer achieved competitive performance for molecular property prediction tasks compared with the baselines and state-of-the-art methods.

SCMEA: A stacked co-enhanced model for entity alignment based on multi-aspect information fusion and bidirectional contrastive learning

Entity alignment refers to discovering the entity pairs with the same realistic meaning in different knowledge graphs. This technology is of great significance for completing and fusing knowledge graphs. Recently, methods based on knowledge representation learning have achieved remarkable achievements in entity alignment. However, most existing approaches do not mine hidden information in the knowledge graph as much as possible. This paper suggests SCMEA, a novel cross-lingual entity alignment framework based on multi-aspect information fusion and bidirectional contrastive learning. SCMEA initially adopts diverse representation learning models to embed multi-aspect information of entities and integrates them into a unified embedding space with an adaptive weighted mechanism to overcome the missing information and the problem of different-aspect information are not uniform. Then, we propose a stacked relation-entity co-enhanced model to further improve the representations of entities, wherein relation representation is modeled using an Entity Collector with Global Entity Attention. Finally, a combined loss function based on improved bidirectional contrastive learning is introduced to optimize model parameters and entity representation, effectively mitigating the hubness problem and accelerating model convergence. We conduct extensive experiments to evaluate the alignment performance of SCMEA. The overall experimental results, ablation studies, and analysis performed on five cross-lingual datasets demonstrate that our model achieves varying degrees of performance improvement and verifies the effectiveness and robustness of the model.

Non-local degradation modeling for spatially adaptive single image super-resolution

Existing methods for single image super-resolution (SISR) model the blur kernel as spatially invariant across the entire image, and are susceptible to the adverse effects of textureless patches. To achieve improved results, adaptive estimation of the degradation kernel is necessary. We explore the synergy of joint global and local degradation modeling for spatially adaptive blind SISR. Our model, named spatially adaptive network for blind super-resolution (SASR), employs a simple encoder to estimate global degradation representations and a decoder to extract local degradation. These two representations are fused with a cross-attention mechanism and applied using spatially adaptive filtering to enhance the local image detail. Specifically, SASR contains two novel features: (1) a non-local degradation modeling with contrastive learning to learn global and local degradation representations, and (2) a non-local spatially adaptive filtering module (SAFM) that incorporates the global degradation and spatial-detail factors to preserve and enhance local details. We demonstrate that SASR can efficiently estimate degradation representations and handle multiple types of degradation. The local representations avoid the detrimental effect of estimating the entire super-resolved image with only one kernel through locally adaptive adjustments. Extensive experiments are performed to quantitatively and qualitatively demonstrate that SASR not only performs favorably for degradation estimation but also leads to state-of-the-art blind SISR performance when compared to alternative approaches.

On the use of contrastive learning for standard-plane classification in fetal ultrasound imaging

BACKGROUND

To investigate the effectiveness of contrastive learning, in particular SimClr, in reducing the need for large annotated ultrasound (US) image datasets for fetal standard plane identification.

METHODS

We explore SimClr advantage in the cases of both low and high inter-class variability, considering at the same time how classification performance varies according to different amounts of labels used. This evaluation is performed by exploiting contrastive learning through different training strategies. We apply both quantitative and qualitative analyses, using standard metrics (F1-score, sensitivity, and precision), Class Activation Mapping (CAM), and t-Distributed Stochastic Neighbor Embedding (t-SNE).

RESULTS

When dealing with high inter-class variability classification tasks, contrastive learning does not bring a significant advantage; whereas it results to be relevant for low inter-class variability classification, specifically when initialized with ImageNet weights.

CONCLUSIONS

Contrastive learning approaches are typically used when a large number of unlabeled data is available, which is not representative of US datasets. We proved that SimClr either as pre-training with backbone initialized via ImageNet weights or used in an end-to-end dual-task may impact positively the performance over standard transfer learning approaches, under a scenario in which the dataset is small and characterized by low inter-class variability.

Spectral intelligent detection for aflatoxin B1 via contrastive learning based on Siamese network

Aflatoxins, harmful substances found in peanuts, corn, and their derivatives, pose significant health risks. Addressing this, the presented research introduces an innovative MSGhostDNN model, merging contrastive learning with multi-scale convolutional networks for precise aflatoxin detection. The method significantly enhances feature discrimination, achieving an impressive 97.87% detection accuracy with a pre-trained model. By applying Grad-CAM, it further refines the model to identify key wavelengths, particularly 416 nm, and focuses on 40 key wavelengths for optimal performance with 97.46% accuracy. The study also incorporates a task dimensionality reduction approach for continuous learning, allowing effective ongoing aflatoxin spectrum monitoring in peanuts and corn. This approach not only boosts aflatoxin detection efficiency but also sets a precedent for rapid online detection of similar toxins, offering a promising solution to mitigate the health risks associated with aflatoxin exposure.

Self-supervised anomaly detection in computer vision and beyond: A survey and outlook

Anomaly detection (AD) plays a crucial role in various domains, including cybersecurity, finance, and healthcare, by identifying patterns or events that deviate from normal behavior. In recent years, significant progress has been made in this field due to the remarkable growth of deep learning models. Notably, the advent of self-supervised learning has sparked the development of novel AD algorithms that outperform the existing state-of-the-art approaches by a considerable margin. This paper aims to provide a comprehensive review of the current methodologies in self-supervised anomaly detection. We present technical details of the standard methods and discuss their strengths and drawbacks. We also compare the performance of these models against each other and other state-of-the-art anomaly detection models. Finally, the paper concludes with a discussion of future directions for self-supervised anomaly detection, including the development of more effective and efficient algorithms and the integration of these techniques with other related fields, such as multi-modal learning.

Structural deep multi-view clustering with integrated abstraction and detail

Deep multi-view clustering, which can obtain complementary information from different views, has received considerable attention in recent years. Although some efforts have been made and achieve decent performances, most of them overlook the structural information and are susceptible to poor quality views, which may seriously restrict the capacity for clustering. To this end, we propose Structural deep Multi-View Clustering with integrated abstraction and detail (SMVC). Specifically, multi-layer perceptrons are used to extract features from specific views, which are then concatenated to form the global features. Besides, a global target distribution is constructed and guides the soft cluster assignments of specific views. In addition to the exploitation of the top-level abstraction, we also design the mining of the underlying details. We construct instance-level contrastive learning using high-order adjacency matrices, which has an equivalent effect to graph attention network and reduces feature redundancy. By integrating the top-level abstraction and underlying detail into a unified framework, our model can jointly optimize the cluster assignments and feature embeddings. Extensive experiments on four benchmark datasets have demonstrated that the proposed SMVC consistently outperforms the state-of-the-art methods.

Contrastive learning of graphs under label noise

In the domain of graph-structured data learning, semi-supervised node classification serves as a critical task, relying mainly on the information from unlabeled nodes and a minor fraction of labeled nodes for training. However, real-world graph-structured data often suffer from label noise, which significantly undermines the performance of Graph Neural Networks (GNNs). This problem becomes increasingly severe in situations where labels are scarce. To tackle this issue of sparse and noisy labels, we propose a novel approach Contrastive Robust Graph Neural Network (CR-GNN), Firstly, considering label sparsity and noise, we employ unsupervised contrastive loss and further incorporate homophily in the graph structure, thus introducing neighbor contrastive loss. Moreover, data augmentation is typically used to construct positive and negative samples in contrastive learning, which may result in inconsistent prediction outcomes. Based on this, we propose a dynamic cross-entropy loss, which selects the nodes with consistent predictions as reliable nodes for cross-entropy loss and benefits to mitigate the overfitting to labeling noise. Finally, we propose cross-space consistency to narrow the semantic gap between the contrast and classification spaces. Extensive experiments on multiple publicly available datasets demonstrate that CR-GNN notably outperforms existing methods in resisting label noise.

Contrastive and adversarial regularized multi-level representation learning for incomplete multi-view clustering

Incomplete multi-view clustering is a significant task in machine learning, given that complex systems in nature and society cannot be fully observed; it provides an opportunity to exploit the structure and functions of underlying systems. Current algorithms are criticized for failing either to balance data restoration and clustering or to capture the consistency of the representation of various views. To address these problems, a novel Multi-level Representation Learning Contrastive and Adversarial Learning (aka MRL_CAL) for incomplete multi-view clustering is proposed, in which data restoration, consistent representation, and clustering are jointly learned by exploiting features in various subspaces. Specifically, MRL_CAL employs v auto-encoder to obtain a low-level specific-view representation of instances, which restores data by estimating the distribution of the original incomplete data with adversarial learning. Then, MRL_CAL extracts a high-level representation of instances, in which the consistency of various views and labels of clusters is incorporated with contrastive learning. In this case, MRL_CAL simultaneously learns multi-level features of instances in various subspaces, which not only overcomes the confliction of representations but also improves the quality of features. Finally, MRL_CAL transforms incomplete multi-view clustering into an overall objective, where features are learned under the guidance of clustering. Extensive experimental results indicate that MRL_CAL outperforms state-of-the-art algorithms in terms of various measurements, implying that the proposed method is promising for incomplete multi-view clustering.

Generation, division and training: A promising method for source-free unsupervised domain adaptation

Conventional unsupervised domain adaptation (UDA) methods often presuppose the existence of labeled source domain samples while adapting the source model to the target domain. Nevertheless, this premise is not always tenable in the context of source-free UDA (SFUDA) attributed to data privacy considerations. Some existing methods address this challenging SFUDA problem by self-supervised learning. But inaccurate pseudo-labels are always unavoidable to degrade the performance of the target model among these methods. Therefore, we propose a promising SFUDA method, namely Generation, Division and Training (GDT) which aims to promote the reliability of pseudo-labels for self-supervised learning and encourage similar features to have closer predictions than dissimilar ones by contrastive learning. Specifically in our GDT method, we first refine pseudo-labels with deep clustering for target samples and then split them into reliable samples and unreliable samples. After that, we adopt self-supervised learning and information maximization for reliable samples training. And for unreliable samples, we conduct contrastive learning via the perspective of similarity and disparity to attract similar samples and repulse dissimilar samples, which helps pull the similar features closed and push the dissimilar features away, leading to efficient feature clustering. Thorough experimentation on three benchmark datasets substantiates the excellence of our proposed approach.

Cryo-forum: A framework for orientation recovery with uncertainty measure with the application in cryo-EM image analysis

In single-particle cryo-electron microscopy (cryo-EM), efficient determination of orientation parameters for particle images poses a significant challenge yet is crucial for reconstructing 3D structures. This task is complicated by the high noise levels in the datasets, which often include outliers, necessitating several time-consuming 2D clean-up processes. Recently, solutions based on deep learning have emerged, offering a more streamlined approach to the traditionally laborious task of orientation estimation. These solutions employ amortized inference, eliminating the need to estimate parameters individually for each image. However, these methods frequently overlook the presence of outliers and may not adequately concentrate on the components used within the network. This paper introduces a novel method using a 10-dimensional feature vector for orientation representation, extracting orientations as unit quaternions with an accompanying uncertainty metric. Furthermore, we propose a unique loss function that considers the pairwise distances between orientations, thereby enhancing the accuracy of our method. Finally, we also comprehensively evaluate the design choices in constructing the encoder network, a topic that has not received sufficient attention in the literature. Our numerical analysis demonstrates that our methodology effectively recovers orientations from 2D cryo-EM images in an end-to-end manner. Notably, the inclusion of uncertainty quantification allows for direct clean-up of the dataset at the 3D level. Lastly, we package our proposed methods into a user-friendly software suite named cryo-forum, designed for easy access by developers.

CLCLSA: Cross-omics linked embedding with contrastive learning and self attention for integration with incomplete multi-omics data

Integration of heterogeneous and high-dimensional multi-omics data is becoming increasingly important in understanding etiology of complex genetic diseases. Each omics technique only provides a limited view of the underlying biological process and integrating heterogeneous omics layers simultaneously would lead to a more comprehensive and detailed understanding of diseases and phenotypes. However, one obstacle faced when performing multi-omics data integration is the existence of unpaired multi-omics data due to instrument sensitivity and cost. Studies may fail if certain aspects of the subjects are missing or incomplete. In this paper, we propose a deep learning method for multi-omics integration with incomplete data by Cross-omics Linked unified embedding with Contrastive Learning and Self Attention (CLCLSA). Utilizing complete multi-omics data as supervision, the model employs cross-omics autoencoders to learn the feature representation across different types of biological data. The multi-omics contrastive learning is employed, which maximizes the mutual information between different types of omics. In addition, the feature-level self-attention and omics-level self-attention are employed to dynamically identify the most informative features for multi-omics data integration. Finally, a Softmax classifier is employed to perform multi-omics data classification. Extensive experiments were conducted on four public multi-omics datasets. The experimental results indicate that our proposed CLCLSA produces promising results in multi-omics data classification using both complete and incomplete multi-omics data.

Contrastive learning-based histopathological features infer molecular subtypes and clinical outcomes of breast cancer from unannotated whole slide images

The artificial intelligence-powered computational pathology has led to significant improvements in the speed and precision of tumor diagnosis, while also exhibiting substantial potential to infer genetic mutations and gene expression levels. However, current studies remain limited in predicting molecular subtypes and clinical outcomes in breast cancer. In this paper, we proposed a weakly supervised contrastive learning framework to address this challenge. Our framework first performed contrastive learning pretraining on a large number of unlabeled patches tiled from whole slide images (WSIs) to extract patch-level features. The gated attention mechanism was leveraged to aggregate patch-level features to produce slide feature that was then applied to various downstream tasks. To confirm the effectiveness of the proposed method, three public cohorts and one external independent cohort of breast cancer have been used to conducted evaluation experiments. The predictive powers of our model to infer gene expression, molecular subtypes, recurrence events and drug responses were validated across cohorts. In addition, the learned patch-level attention scores enabled us to generate heatmaps that were highly consistent with pathologist annotations and spatial transcriptomic data. These findings demonstrated that our model effectively established the high-order genotype-phenotype associations, thereby potentially extend the application of digital pathology in clinical practice.

Cross-domain fault diagnosis method for rolling bearings based on contrastive universal domain adaptation

To address the unknown spatial relationship between source and target domain labels, which leads to poor fault diagnosis accuracy, a contrastive universal domain adaptation model and rolling bearing fault diagnosis approach are proposed. The approach introduces bootstrap your own latent network to mine the data-specific structure of the target domain and proposes rejecting unknown class samples using an entropy separation strategy. Simultaneously, a source class weighting mechanism is designed to improve the transferable semantics augmentation method by assigning various class-level weights to source categories, which improves the alignment of the feature distributions in the shared label space to further construct fault diagnosis models. Experimental validation on two rolling bearing datasets confirmed the superior fault diagnosis accuracy of the proposed method under diverse working conditions.

Coarse-to-fine visual representation learning for medical images via class activation maps

The value of coarsely labeled datasets in learning transferable representations for medical images is investigated in this work. Compared to fine labels which require meticulous effort to annotate, coarse labels can be acquired at a significantly lower cost and can provide useful training signals for data-hungry deep neural networks. We consider coarse labels in the form of binary labels differentiating a normal (healthy) image from an abnormal (diseased) image and propose CAMContrast, a two-stage representation learning framework for medical images. Using class activation maps, CAMContrast makes use of the binary labels to generate heatmaps as positive views for contrastive representation learning. Specifically, the learning objective is optimized to maximize the agreement within fixed crops of image-heatmap pair to learn fine-grained representations that are generalizable to different downstream tasks. We empirically validate the transfer learning performance of CAMContrast on several public datasets, covering classification and segmentation tasks on fundus photographs and chest X-ray images. The experimental results showed that our method outperforms other self-supervised and supervised pretrain methods in terms of data efficiency and downstream performance.

Cross comparison representation learning for semi-supervised segmentation of cellular nuclei in immunofluorescence staining

The morphological analysis of cells from optical images is vital for interpreting brain function in disease states. Extracting comprehensive cell morphology from intricate backgrounds, common in neural and some medical images, poses a significant challenge. Due to the huge workload of manual recognition, automated neuron cell segmentation using deep learning algorithms with labeled data is integral to neural image analysis tools. To combat the high cost of acquiring labeled data, we propose a novel semi-supervised cell segmentation algorithm for immunofluorescence-stained cell image datasets (ISC), utilizing a mean-teacher semi-supervised learning framework. We include a "cross comparison representation learning block" to enhance the teacher-student model comparison on high-dimensional channels, thereby improving feature compactness and separability, which results in the extraction of higher-dimensional features from unlabeled data. We also suggest a new network, the Multi Pooling Layer Attention Dense Network (MPAD-Net), serving as the backbone of the student model to augment segmentation accuracy. Evaluations on the immunofluorescence staining datasets and the public CRAG dataset illustrate our method surpasses other top semi-supervised learning methods, achieving average Jaccard, Dice and Normalized Surface Dice (NSD) indicators of 83.22%, 90.95% and 81.90% with only 20% labeled data. The datasets and code are available on the website at https://github.com/Brainsmatics/CCRL.

Multi-view representation learning for tabular data integration using inter-feature relationships

OBJECTIVE

An applied problem facing all areas of data science is harmonizing data sources. Joining data from multiple origins with unmapped and only partially overlapping features is a prerequisite to developing and testing robust, generalizable algorithms, especially in healthcare. This integrating is usually resolved using meta-data such as feature names, which may be unavailable or ambiguous. Our goal is to design methods that create a mapping between structured tabular datasets derived from electronic health records independent of meta-data.

METHODS

We evaluate methods in the challenging case of numeric features without reliable and distinctive univariate summaries, such as nearly Gaussian and binary features. We assume that a small set of features are a priori mapped between two datasets, which share unknown identical features and possibly many unrelated features. Inter-feature relationships are the main source of identification which we expect. We compare the performance of contrastive learning methods for feature representations, novel partial auto-encoders, mutual-information graph optimizers, and simple statistical baselines on simulated data, public datasets, the MIMIC-III medical-record changeover, and perioperative records from before and after a medical-record system change. Performance was evaluated using both mapping of identical features and reconstruction accuracy of examples in the format of the other dataset.

RESULTS

Contrastive learning-based methods overall performed the best, often substantially beating the literature baseline in matching and reconstruction, especially in the more challenging real data experiments. Partial auto-encoder methods showed on-par matching with contrastive methods in all synthetic and some real datasets, along with good reconstruction. However, the statistical method we created performed reasonably well in many cases, with much less dependence on hyperparameter tuning. When validating feature match output in the EHR dataset we found that some mistakes were actually a surrogate or related feature as reviewed by two subject matter experts.

CONCLUSION

In simulation studies and real-world examples, we find that inter-feature relationships are effective at identifying matching or closely related features across tabular datasets when meta-data is not available. Decoder architectures are also reasonably effective at imputing features without an exact match.

Approximating Intermediate Feature Maps of Self-Supervised Convolution Neural Network to Learn Hard Positive Representations in Chest Radiography

Recent advances in contrastive learning have significantly improved the performance of deep learning models. In contrastive learning of medical images, dealing with positive representation is sometimes difficult because some strong augmentation techniques can disrupt contrastive learning owing to the subtle differences between other standardized CXRs compared to augmented positive pairs; therefore, additional efforts are required. In this study, we propose intermediate feature approximation (IFA) loss, which improves the performance of contrastive convolutional neural networks by focusing more on positive representations of CXRs without additional augmentations. The IFA loss encourages the feature maps of a query image and its positive pair to resemble each other by maximizing the cosine similarity between the intermediate feature outputs of the original data and the positive pairs. Therefore, we used the InfoNCE loss, which is commonly used loss to address negative representations, and the IFA loss, which addresses positive representations, together to improve the contrastive network. We evaluated the performance of the network using various downstream tasks, including classification, object detection, and a generative adversarial network (GAN) inversion task. The downstream task results demonstrated that IFA loss can improve the performance of effectively overcoming data imbalance and data scarcity; furthermore, it can serve as a perceptual loss encoder for GAN inversion. In addition, we have made our model publicly available to facilitate access and encourage further research and collaboration in the field.

Simcryocluster: a semantic similarity clustering method of cryo-EM images by adopting contrastive learning

BACKGROUND

Cryo-electron microscopy (Cryo-EM) plays an increasingly important role in the determination of the three-dimensional (3D) structure of macromolecules. In order to achieve 3D reconstruction results close to atomic resolution, 2D single-particle image classification is not only conducive to single-particle selection, but also a key step that affects 3D reconstruction. The main task is to cluster and align 2D single-grain images into non-heterogeneous groups to obtain sharper single-grain images by averaging calculations. The main difficulties are that the cryo-EM single-particle image has a low signal-to-noise ratio (SNR), cannot manually label the data, and the projection direction is random and the distribution is unknown. Therefore, in the low SNR scenario, how to obtain the characteristic information of the effective particles, improve the clustering accuracy, and thus improve the reconstruction accuracy, is a key problem in the 2D image analysis of single particles of cryo-EM.

RESULTS

Aiming at the above problems, we propose a learnable deep clustering method and a fast alignment weighted averaging method based on frequency domain space to effectively improve the class averaging results and improve the reconstruction accuracy. In particular, it is very prominent in the feature extraction and dimensionality reduction module. Compared with the classification method based on Bayesian and great likelihood, a large amount of single particle data is required to estimate the relative angle orientation of macromolecular single particles in the 3D structure, and we propose that the clustering method shows good results.

CONCLUSIONS

SimcryoCluster can use the contrastive learning method to perform well in the unlabeled high-noise cryo-EM single particle image classification task, making it an important tool for cryo-EM protein structure determination.

Contrastive image adaptation for acquisition shift reduction in medical imaging

The domain shift, or acquisition shift in medical imaging, is responsible for potentially harmful differences between development and deployment conditions of medical image analysis techniques. There is a growing need in the community for advanced methods that could mitigate this issue better than conventional approaches. In this paper, we consider configurations in which we can expose a learning-based pixel level adaptor to a large variability of unlabeled images during its training, i.e. sufficient to span the acquisition shift expected during the training or testing of a downstream task model. We leverage the ability of convolutional architectures to efficiently learn domain-agnostic features and train a many-to-one unsupervised mapping between a source collection of heterogeneous images from multiple unknown domains subjected to the acquisition shift and a homogeneous subset of this source set of lower cardinality, potentially constituted of a single image. To this end, we propose a new cycle-free image-to-image architecture based on a combination of three loss functions : a contrastive PatchNCE loss, an adversarial loss and an edge preserving loss allowing for rich domain adaptation to the target image even under strong domain imbalance and low data regimes. Experiments support the interest of the proposed contrastive image adaptation approach for the regularization of downstream deep supervised segmentation and cross-modality synthesis models.

Boosting semi-supervised learning with Contrastive Complementary Labeling

Semi-supervised learning (SSL) approaches have achieved great success in leveraging a large amount of unlabeled data to learn deep models. Among them, one popular approach is pseudo-labeling which generates pseudo labels only for those unlabeled data with high-confidence predictions. As for the low-confidence ones, existing methods often simply discard them because these unreliable pseudo labels may mislead the model. Unlike existing methods, we highlight that these low-confidence data can be still beneficial to the training process. Specifically, although we cannot determine which class a low-confidence sample belongs to, we can assume that this sample should be very unlikely to belong to those classes with the lowest probabilities (often called complementary classes/labels). Inspired by this, we propose a novel Contrastive Complementary Labeling (CCL) method that constructs a large number of reliable negative pairs based on the complementary labels and adopts contrastive learning to make use of all the unlabeled data. Extensive experiments demonstrate that CCL significantly improves the performance on top of existing advanced methods and is particularly effective under the label-scarce settings. For example, CCL yields an improvement of 2.43% over FixMatch on CIFAR-10 only with 40 labeled data.

HPFG: semi-supervised medical image segmentation framework based on hybrid pseudo-label and feature-guiding

Semi-supervised learning methods have been attracting much attention in medical image segmentation due to the lack of high-quality annotation. To cope with the noise problem of pseudo-label in semi-supervised medical image segmentation and the limitations of contrastive learning applications, we propose a semi-supervised medical image segmentation framework, HPFG, based on hybrid pseudo-label and feature-guiding, which consists of a hybrid pseudo-label strategy and two different feature-guiding modules. The hybrid pseudo-label strategy uses the CutMix operation and an auxiliary network to enable the labeled images to guide the unlabeled images to generate high-quality pseudo-label and reduce the impact of pseudo-label noise. In addition, a feature-guiding encoder module based on feature-level contrastive learning is designed to guide the encoder to mine useful local and global image features, thus effectively enhancing the feature extraction capability of the model. At the same time, a feature-guiding decoder module based on adaptive class-level contrastive learning is designed to guide the decoder in better extracting class information, achieving intra-class affinity and inter-class separation, and effectively alleviating the class imbalance problem in medical datasets. Extensive experimental results show that the segmentation performance of the HPFG framework proposed in this paper outperforms existing semi-supervised medical image segmentation methods on three public datasets: ACDC, LIDC, and ISIC. Code is available at https://github.com/fakerlove1/HPFG .

DrasCLR: A self-supervised framework of learning disease-related and anatomy-specific representation for 3D lung CT images

Large-scale volumetric medical images with annotation are rare, costly, and time prohibitive to acquire. Self-supervised learning (SSL) offers a promising pre-training and feature extraction solution for many downstream tasks, as it only uses unlabeled data. Recently, SSL methods based on instance discrimination have gained popularity in the medical imaging domain. However, SSL pre-trained encoders may use many clues in the image to discriminate an instance that are not necessarily disease-related. Moreover, pathological patterns are often subtle and heterogeneous, requiring the ability of the desired method to represent anatomy-specific features that are sensitive to abnormal changes in different body parts. In this work, we present a novel SSL framework, named DrasCLR, for 3D lung CT images to overcome these challenges. We propose two domain-specific contrastive learning strategies: one aims to capture subtle disease patterns inside a local anatomical region, and the other aims to represent severe disease patterns that span larger regions. We formulate the encoder using conditional hyper-parameterized network, in which the parameters are dependant on the anatomical location, to extract anatomically sensitive features. Extensive experiments on large-scale datasets of lung CT scans show that our method improves the performance of many downstream prediction and segmentation tasks. The patient-level representation improves the performance of the patient survival prediction task. We show how our method can detect emphysema subtypes via dense prediction. We demonstrate that fine-tuning the pre-trained model can significantly reduce annotation efforts without sacrificing emphysema detection accuracy. Our ablation study highlights the importance of incorporating anatomical context into the SSL framework. Our codes are available at https://github.com/batmanlab/DrasCLR.

Predicting miRNA-disease association via graph attention learning and multiplex adaptive modality fusion

miRNAs are a class of small non-coding RNA molecules that play important roles in gene regulation. They are crucial for maintaining normal cellular functions, and dysregulation or dysfunction of miRNAs which are linked to the onset and advancement of multiple human diseases. Research on miRNAs has unveiled novel avenues in the realm of the diagnosis, treatment, and prevention of human diseases. However, clinical trials pose challenges and drawbacks, such as complexity and time-consuming processes, which create obstacles for many researchers. Graph Attention Network (GAT) has shown excellent performance in handling graph-structured data for tasks such as link prediction. Some studies have successfully applied GAT to miRNA-disease association prediction. However, there are several drawbacks to existing methods. Firstly, most of the previous models rely solely on concatenation operations to merge features of miRNAs and diseases, which results in the deprivation of significant modality-specific information and even the inclusion of redundant information. Secondly, as the number of layers in GAT increases, there is a possibility of excessive smoothing in the feature extraction process, which significantly affects the prediction accuracy. To address these issues and effectively complete miRNA disease prediction tasks, we propose an innovative model called Multiplex Adaptive Modality Fusion Graph Attention Network (MAMFGAT). MAMFGAT utilizes GAT as the main structure for feature aggregation and incorporates a multi-modal adaptive fusion module to extract features from three interconnected networks: the miRNA-disease association network, the miRNA similarity network, and the disease similarity network. It employs adaptive learning and cross-modality contrastive learning to fuse more effective miRNA and disease feature embeddings as well as incorporates multi-modal residual feature fusion to tackle the problem of excessive feature smoothing in GATs. Finally, we employ a Multi-Layer Perceptron (MLP) model that takes the embeddings of miRNA and disease features as input to anticipate the presence of potential miRNA-disease associations. Extensive experimental results provide evidence of the superior performance of MAMFGAT in comparison to other state-of-the-art methods. To validate the significance of various modalities and assess the efficacy of the designed modules, we performed an ablation analysis. Furthermore, MAMFGAT shows outstanding performance in three cancer case studies, indicating that it is a reliable method for studying the association between miRNA and diseases. The implementation of MAMFGAT can be accessed at the following GitHub repository: https://github.com/zixiaojin66/MAMFGAT-master.

Learning self-supervised molecular representations for drug-drug interaction prediction

Drug-drug interactions (DDI) are a critical concern in healthcare due to their potential to cause adverse effects and compromise patient safety. Supervised machine learning models for DDI prediction need to be optimized to learn abstract, transferable features, and generalize to larger chemical spaces, primarily due to the scarcity of high-quality labeled DDI data. Inspired by recent advances in computer vision, we present SMR-DDI, a self-supervised framework that leverages contrastive learning to embed drugs into a scaffold-based feature space. Molecular scaffolds represent the core structural motifs that drive pharmacological activities, making them valuable for learning informative representations. Specifically, we pre-trained SMR-DDI on a large-scale unlabeled molecular dataset. We generated augmented views for each molecule via SMILES enumeration and optimized the embedding process through contrastive loss minimization between views. This enables the model to capture relevant and robust molecular features while reducing noise. We then transfer the learned representations for the downstream prediction of DDI. Experiments show that the new feature space has comparable expressivity to state-of-the-art molecular representations and achieved competitive DDI prediction results while training on less data. Additional investigations also revealed that pre-training on more extensive and diverse unlabeled molecular datasets improved the model's capability to embed molecules more effectively. Our results highlight contrastive learning as a promising approach for DDI prediction that can identify potentially hazardous drug combinations using only structural information.

Enhancing breast cancer classification via histopathological image analysis: Leveraging self-supervised contrastive learning and transfer learning

Breast cancer, a significant threat to women's health, demands early detection. Automating histopathological image analysis offers a promising solution to enhance efficiency and accuracy in diagnosis. This study addresses the challenge of breast cancer histopathological image classification by leveraging the ResNet architecture, known for its depth and skip connections. In this work, two distinct approaches were pursued, each driven by unique motivations. The first approach aimed to improve the learning process through self-supervised contrastive learning. It utilizes a small subset of the training data for initial model training and progressively expands the training set by incorporating confidently labeled data from the unlabeled pool, ultimately achieving a reliable model with limited training data. The second approach focused on optimizing the architecture by combining ResNet50 and Inception module to get a lightweight and efficient classifier. The dataset utilized in this work comprises histopathological images categorized into benign and malignant classes at varying magnification levels (40X, 100X, 200X, 400X), all originating from the same source image. The results demonstrate state-of-the-art performance, achieving 98% accuracy for images magnified at 40X and 200X, and 94% for 100X and 400X. Notably, the proposed architecture boasts a substantially reduced parameter count of approximately 3.6 million, contrasting with existing leading architectures, which possess parameter sizes at least twice as large.

Improving cell type identification with Gaussian noise-augmented single-cell RNA-seq contrastive learning

Cell type identification is an important task for single-cell RNA-sequencing (scRNA-seq) data analysis. Many prediction methods have recently been proposed, but the predictive accuracy of difficult cell type identification tasks is still low. In this work, we proposed a novel Gaussian noise augmentation-based scRNA-seq contrastive learning method (GsRCL) to learn a type of discriminative feature representations for cell type identification tasks. A large-scale computational evaluation suggests that GsRCL successfully outperformed other state-of-the-art predictive methods on difficult cell type identification tasks, while the conventional random genes masking augmentation-based contrastive learning method also improved the accuracy of easy cell type identification tasks in general.

Consistency regularisation in varying contexts and feature perturbations for semi-supervised semantic segmentation of histology images

Semantic segmentation of various tissue and nuclei types in histology images is fundamental to many downstream tasks in the area of computational pathology (CPath). In recent years, Deep Learning (DL) methods have been shown to perform well on segmentation tasks but DL methods generally require a large amount of pixel-wise annotated data. Pixel-wise annotation sometimes requires expert's knowledge and time which is laborious and costly to obtain. In this paper, we present a consistency based semi-supervised learning (SSL) approach that can help mitigate this challenge by exploiting a large amount of unlabelled data for model training thus alleviating the need for a large annotated dataset. However, SSL models might also be susceptible to changing context and features perturbations exhibiting poor generalisation due to the limited training data. We propose an SSL method that learns robust features from both labelled and unlabelled images by enforcing consistency against varying contexts and feature perturbations. The proposed method incorporates context-aware consistency by contrasting pairs of overlapping images in a pixel-wise manner from changing contexts resulting in robust and context invariant features. We show that cross-consistency training makes the encoder features invariant to different perturbations and improves the prediction confidence. Finally, entropy minimisation is employed to further boost the confidence of the final prediction maps from unlabelled data. We conduct an extensive set of experiments on two publicly available large datasets (BCSS and MoNuSeg) and show superior performance compared to the state-of-the-art methods.

HG-PerCon: Cross-view contrastive learning for personality prediction

Personality prediction task not only helps us to better understand personal needs and preferences but also is essential for many fields such as psychology and behavioral economics. Current personality prediction primarily focuses on discovering personality traits through user posts. Additionally, there are also methods that utilize psychological information to uncover certain underlying personality traits. Although significant progress has been made in personality prediction, we believe that current solutions still overlook the long-term sustainability of personality and are constrained by the challenge of capturing consistent personality-related clues across different views in a simple and efficient manner. To this end, we propose HG-PerCon, which utilizes user representations based on historical semantic information and psychological knowledge for cross-view contrastive learning. Specifically, we design a transformer-based module to obtain user representations with long-lasting personality-related information from their historical posts. We leverage a psychological knowledge graph which incorporates language styles to generate user representations guided by psychological knowledge. Additionally, we employ contrastive learning to capture the consistency of user personality-related clues across views. To evaluate the effectiveness of our model, and our approach achieved a reduction of 2%, 4%, and 6% in RMSE compared to the second-best baseline method.

Improving few-shot relation extraction through semantics-guided learning

Few-shot relation extraction (few-shot RE) aims to recognize relations between the entity pair in a given text by utilizing very few annotated instances. As a simple yet efficient approach, prototype network-based methods often directly incorporate relation information to enhance prototype representation or leverage contrastive learning to mitigate prediction confusion. Despite achieving good results, the above methods are still susceptible to false judgments of outlier samples and confusion of similar classes. To address these issues, we propose a novel Semantics-Guided Learning (SemGL) method that more effectively utilizes relation information to enhance both the representations of instances and prototypes for improving the performance of few-shot RE. First, SemGL employs the prompt encoder to encode various prompt templates of instances and relation information and obtains more accurate semantic representations of instances, instance prototypes, and concept prototypes via the prompt enhancement from large language models. Then, SemGL introduces a novel technique called relation graph learning, which leverages concept prototypes to cluster homogeneous instances together, emphasizing relation-specific features of concrete instances. Simultaneously, SemGL employs instance-level contrastive learning between instance prototypes and support instances to distinguish between intra-class instances and inter-class instances to promote shared features among intra-class instances. Additionally, prototype-level contrastive learning leverages concept prototypes to pull closer relation-specific features of the concept prototype and shared features of the instance prototype from the same relation. Finally, SemGL utilizes new relation prototypes that integrate interpretable features of concept prototypes and shared features of instance prototypes for prediction. Experimental results on two publicly available few-shot RE datasets demonstrate the effectiveness and efficiency of SemGL in introducing relation information, with particularly promising results for the domain adaptation challenge task.

ComCo: Complementary supervised contrastive learning for complementary label learning

Complementary label learning (CLL) is an important problem that aims to reduce the cost of obtaining large-scale accurate datasets by only allowing each training sample to be equipped with labels the sample does not belong. Despite its promise, CLL remains a challenging task. Previous methods have proposed new loss functions or introduced deep learning-based models to CLL, but they mostly overlook the semantic information that may be implicit in the complementary labels. In this work, we propose a novel method, ComCo, which leverages a contrastive learning framework to assist CLL. Our method includes two key strategies: a positive selection strategy that identifies reliable positive samples and a negative selection strategy that skillfully integrates and leverages the information in the complementary labels to construct a negative set. These strategies bring ComCo closer to supervised contrastive learning. Empirically, ComCo significantly achieves better representation learning and outperforms the baseline models and the current state-of-the-art by up to 14.61% in CLL.

Contrastive learning and dynamics embedding neural network for label-free interpretable machine fault diagnosis

Industrial machinery often produces vibration signals that can serve as indicators of underlying faults. However, these signals often need to be labeled, presenting a challenge for accurate and interpretable fault diagnosis. While supervised learning methods, such as deep neural networks, have been applied for fault diagnosis, they need help in effectively distinguishing between different vibration-related faults. In response to this issue, our study introduces an innovative approach for automatic fault diagnosis through the application of the Bootstrap Your Own Latent and Dynamical Systems Model Discovery algorithm (BYOLDIS). This method not only addresses the challenge of unlabelled signals but also provides readily interpretable results. The proposed methodology consists of three fundamental steps. First, we derive a matrix of differential equations to capture the dynamic behavior of faulty bearings. Second, we employ a contrastive learning network alongside a time-delay embedding matrix to reconstruct the coordinates of the fault-dynamical system. Lastly, we construct a library of fault machine dynamic polynomial equations, incorporating prior constraints based on physical models. To assess the effectiveness and robustness of our proposed method, we conducted both simulations and experiments. The results of these case studies affirm that BYOLDIS can accurately diagnose bearing faults and offer dynamic explanations for the diagnostic outcomes. This suggests that BYOLDIS holds substantial promise as a diagnostic tool for processing unlabelled vibrational data.

Hyperbolic graph convolutional neural network with contrastive learning for automated ICD coding

The International Classification of Diseases (ICD) is a widely used criterion for disease classification, health monitoring, and medical data analysis. Deep learning-based automated ICD coding has gained attention due to the time-consuming and costly nature of manual coding. The main challenges of automated ICD coding include imbalanced label distribution, code hierarchy and noisy texts. Recent works have considered using code hierarchy or description for better label representation to solve the problem of imbalanced label distribution. However, these methods are still ineffective and redundant since they only interact with a constant label representation. In this work, we introduce a novel Hyperbolic Graph Convolutional Network with Contrastive Learning (HGCN-CL) to solve the above problems and the shortcomings of the previous methods. We adopt a Hyperbolic graph convolutional network on ICD coding to capture the hierarchical structure of codes, which can solve the problem of large distortions when embedding hierarchical structure with graph convolutional network. Besides, we introduce contrastive learning for automatic ICD coding by injecting code features into text encoder to generate hierarchical-aware positive samples to solve the problem of interacting with constant code features. We conduct experiments on the public MIMIC-III and MIMIC-II datasets. The results on MIMIC III show that HGCN-CL outperforms previous state-of-art methods for automatic ICD coding, which achieves a 2.7% and 3.6% improvement respectively compared to previous best results (Hypercore). We also provide ablation experiments and hierarchy visualization to verify the effectiveness of components in our model.

Contrastive Transfer Learning for Prediction of Adverse Events in Hospitalized Patients

OBJECTIVE

Deterioration index (DI) is a computer-generated score at a specific frequency that represents the overall condition of hospitalized patients using a variety of clinical, laboratory and physiologic data. In this paper, a contrastive transfer learning method is proposed and validated for early prediction of adverse events in hospitalized patients using DI scores.

METHODS AND PROCEDURES

An unsupervised contrastive learning (CL) model with a classifier is proposed to predict adverse outcome using a single temporal variable (DI scores). The model is pretrained on an unsupervised fashion with large-scale time series data and fine-tuned with retrospective DI score data.

RESULTS

The performance of this model is compared with supervised deep learning models for time series classification. Results show that unsupervised contrastive transfer learning with a classifier outperforms supervised deep learning solutions. Pretraining of the proposed CL model with large-scale time series data and fine-tuning that with DI scores can enhance prediction accuracy.

CONCLUSION

A relationship exists between longitudinal DI scores of a patient and the corresponding outcome. DI scores and contrastive transfer learning can be used to predict and prevent adverse outcomes in hospitalized patients.

CLINICAL IMPACT

This paper successfully developed an unsupervised contrastive transfer learning algorithm for prediction of adverse events in hospitalized patients. The proposed model can be deployed in hospitals as an early warning system for preemptive intervention in hospitalized patients, which can mitigate the likelihood of adverse outcomes.

MNCLCDA: predicting circRNA-drug sensitivity associations by using mixed neighbourhood information and contrastive learning

BACKGROUND

circRNAs play an important role in drug resistance and cancer development. Recently, many studies have shown that the expressions of circRNAs in human cells can affect the sensitivity of cells to therapeutic drugs, thus significantly influencing the therapeutic effects of these drugs. Traditional biomedical experiments required to verify this sensitivity relationship are not only time-consuming but also expensive. Hence, the development of an efficient computational approach that can accurately predict the novel associations between drug sensitivities and circRNAs is a crucial and pressing need.

METHODS

In this research, we present a novel computational framework called MNCLCDA, which aims to predict the potential associations between drug sensitivities and circRNAs to assist with medical research. First, MNCLCDA quantifies the similarity between the given drug and circRNA using drug structure information, circRNA gene sequence information, and GIP kernel information. Due to the existence of noise in similarity information, we employ a preprocessing approach based on random walk with restart for similarity networks to efficiently capture the useful features of circRNAs and drugs. Second, we use a mixed neighbourhood graph convolutional network to obtain the neighbourhood information of nodes. Then, a graph-based contrastive learning method is used to enhance the robustness of the model, and finally, a double Laplace-regularized least-squares method is used to predict potential circRNA-drug associations through the kernel matrices in the circRNA and drug spaces.

RESULTS

Numerous experimental results show that MNCLCDA outperforms six other advanced methods. In addition, the excellent performance of our proposed model in case studies illustrates that MNCLCDA also has the ability to predict the associations between drug sensitivity and circRNA in practical situations.

CONCLUSIONS

After a large number of experiments, it is illustrated that MNCLCDA is an efficient tool for predicting the potential associations between drug sensitivities and circRNAs, thereby can provide some guidance for clinical trials.

StableDNAm: towards a stable and efficient model for predicting DNA methylation based on adaptive feature correction learning

BACKGROUND

DNA methylation, instrumental in numerous life processes, underscores the paramount importance of its accurate prediction. Recent studies suggest that deep learning, due to its capacity to extract profound insights, provides a more precise DNA methylation prediction. However, issues related to the stability and generalization performance of these models persist.

RESULTS

In this study, we introduce an efficient and stable DNA methylation prediction model. This model incorporates a feature fusion approach, adaptive feature correction technology, and a contrastive learning strategy. The proposed model presents several advantages. First, DNA sequences are encoded at four levels to comprehensively capture intricate information across multi-scale and low-span features. Second, we design a sequence-specific feature correction module that adaptively adjusts the weights of sequence features. This improvement enhances the model's stability and scalability, or its generality. Third, our contrastive learning strategy mitigates the instability issues resulting from sparse data. To validate our model, we conducted multiple sets of experiments on commonly used datasets, demonstrating the model's robustness and stability. Simultaneously, we amalgamate various datasets into a single, unified dataset. The experimental outcomes from this combined dataset substantiate the model's robust adaptability.

CONCLUSIONS

Our research findings affirm that the StableDNAm model is a general, stable, and effective instrument for DNA methylation prediction. It holds substantial promise for providing invaluable assistance in future methylation-related research and analyses.

Self-Supervised EEG Representation Learning with Contrastive Predictive Coding for Post-Stroke Patients

Stroke patients are prone to fatigue during the EEG acquisition procedure, and experiments have high requirements on cognition and physical limitations of subjects. Therefore, how to learn effective feature representation is very important. Deep learning networks have been widely used in motor imagery (MI) based brain-computer interface (BCI). This paper proposes a contrast predictive coding (CPC) framework based on the modified s-transform (MST) to generate MST-CPC feature representations. MST is used to acquire the temporal-frequency feature to improve the decoding performance for MI task recognition. EEG2Image is used to convert multi-channel one-dimensional EEG into two-dimensional EEG topography. High-level feature representations are generated by CPC which consists of an encoder and autoregressive model. Finally, the effectiveness of generated features is verified by the k-means clustering algorithm. It can be found that our model generates features with high efficiency and a good clustering effect. After classification performance evaluation, the average classification accuracy of MI tasks is 89% based on 40 subjects. The proposed method can obtain effective feature representations and improve the performance of MI-BCI systems. By comparing several self-supervised methods on the public dataset, it can be concluded that the MST-CPC model has the highest average accuracy. This is a breakthrough in the combination of self-supervised learning and image processing of EEG signals. It is helpful to provide effective rehabilitation training for stroke patients to promote motor function recovery.

Dual CNN cross-teaching semi-supervised segmentation network with multi-kernels and global contrastive loss in ACDC

The cross-teaching based on Convolutional Neural Network (CNN) and Transformer has been successful in semi-supervised learning; however, the information interaction between local and global relations ignores the semantic features of the medium scale, and at the same time, the information in the process of feature coding is not fully utilized. To solve these problems, we proposed a new semi-supervised segmentation network. Based on the principle of complementary modeling information of different kernel convolutions, we design a dual CNN cross-supervised network with different kernel sizes under cross-teaching. We introduce global feature contrastive learning and generate contrast samples with the help of dual CNN architecture to make efficient use of coding features. We conducted plenty of experiments on the Automated Cardiac Diagnosis Challenge (ACDC) dataset to evaluate our approach. Our method achieves an average Dice Similarity Coefficient (DSC) of 87.2% and Hausdorff distance ([Formula: see text]) of 6.1 mm on 10% labeled data, which is significantly improved compared with many current popular models. Supervised learning is performed on the labeled data, and dual CNN cross-teaching supervised learning is performed on the unlabeled data. All data would be mapped by the two CNNs to generate features, which are used for contrastive learning to optimize the parameters.

PaCL: Patient-aware contrastive learning through metadata refinement for generalized early disease diagnosis

Early diagnosis plays a pivotal role in effectively treating numerous diseases, especially in healthcare scenarios where prompt and accurate diagnoses are essential. Contrastive learning (CL) has emerged as a promising approach for medical tasks, offering advantages over traditional supervised learning methods. However, in healthcare, patient metadata contains valuable clinical information that can enhance representations, yet existing CL methods often overlook this data. In this study, we propose an novel approach that leverages both clinical information and imaging data in contrastive learning to enhance model generalization and interpretability. Furthermore, existing contrastive methods may be prone to sampling bias, which can lead to the model capturing spurious relationships and exhibiting unequal performance across protected subgroups frequently encountered in medical settings. To address these limitations, we introduce Patient-aware Contrastive Learning (PaCL), featuring an inter-class separability objective (IeSO) and an intra-class diversity objective (IaDO). IeSO harnesses rich clinical information to refine samples, while IaDO ensures the necessary diversity among samples to prevent class collapse. We demonstrate the effectiveness of PaCL both theoretically through causal refinements and empirically across six real-world medical imaging tasks spanning three imaging modalities: ophthalmology, radiology, and dermatology. Notably, PaCL outperforms previous techniques across all six tasks.

DC-SiamNet: Deep contrastive Siamese network for self-supervised MRI reconstruction

Reconstruction methods based on deep learning have greatly shortened the data acquisition time of magnetic resonance imaging (MRI). However, these methods typically utilize massive fully sampled data for supervised training, restricting their application in certain clinical scenarios and posing challenges to the reconstruction effect when high-quality MR images are unavailable. Recently, self-supervised methods have been developed that only undersampled MRI images participate in the network training. Nevertheless, due to the lack of complete referable MR image data, self-supervised reconstruction is prone to produce incorrect structure contents, such as unnatural texture details and over-smoothed tissue sites. To solve this problem, we propose a self-supervised Deep Contrastive Siamese Network (DC-SiamNet) for fast MR imaging. First, DC-SiamNet performs the reconstruction with a Siamese unrolled structure and obtains visual representations in different iterative phases. Particularly, an attention-weighted average pooling module is employed at the bottleneck layer of the U-shape regularization unit, which can effectively aggregate valuable local information of the underlying feature map in the generated representation vector. Then, a novel hybrid loss function is designed to drive the self-supervised reconstruction and contrastive learning simultaneously by forcing the output consistency across different branches in the frequency domain, the image domain, and the latent space. The proposed method is extensively evaluated with different sampling patterns on the IXI brain dataset and the MRINet knee dataset. Experimental results show that DC-SiamNet can achieve 0.93 in structural similarity and 33.984 dB in peak signal-to-noise ratio on the IXI brain dataset under 8x acceleration. It has better reconstruction accuracy than other methods, and the performance is close to the corresponding model trained with full supervision, especially when the sampling rate is low. In addition, generalization experiments verify that our method has a strong cross-domain reconstruction ability for different contrast brain images.

Multimodal contrastive representation learning for drug-target binding affinity prediction

In the biomedical field, the efficacy of most drugs is demonstrated by their interactions with targets, meanwhile, accurate prediction of the strength of drug-target binding is extremely important for drug development efforts. Traditional bioassay-based drug-target binding affinity (DTA) prediction methods cannot meet the needs of drug R&D in the era of big data. Recent years we have witnessed significant success on deep learning-based models for drug-target binding affinity prediction task. However, these models only considered a single modality of drug and target information, and some valuable information was not fully utilized. In fact, the information of different modalities of drug and target can complement each other, and more valuable information can be obtained by fusing the information of different modalities. In this paper, we introduce a multimodal information fusion model for DTA prediction that is called FMDTA, which fully considers drug/target information in both string and graph modalities and balances the feature representations of different modalities by a contrastive learning approach. In addition, we exploited the alignment information of drug atoms and target residues to capture the positional information of string patterns, which can extract more useful feature information in SMILES and target sequences. Experimental results on two benchmark datasets show that FMDTA outperforms the state-of-the-art model, demonstrating the feasibility and excellent feature capture capability of FMDTA. The code of FMDTA and the data are available at: https://github.com/bestdoubleLin/FMDTA.

BBLN: A bilateral-branch learning network for unknown protein-protein interaction prediction

Unknown Protein-Protein Interactions (PPIs) prediction has a huge demand in the biological analysis field. Since the effect of the limited availability of protein data is severe, transferable representations are highly demanded to be learned from various data. The latest works enhance the model performance on unknown PPIs prediction and have achieved certain improvements by combining protein information and relation information on PPI graph. However, such methods inevitably suffer from a so-called information monotonicity problem that limits the improvements when encountering large amounts of unknown PPIs. The prediction performance cannot be actually increased without considering the complementary information and relationship information among various modalities of protein data. To this end, we propose a bilateral-branch learning network to deeply enhance the both complementary and relationship information based on the amino acid sequence and gene ontology from multi- and cross-modal views. Experimental results on massive real-world datasets show that our method significantly outperforms the previous state-of-the-art on both traditional and novel unknown PPIs prediction.

pathCLIP: Detection of Genes and Gene Relations from Biological Pathway Figures through Image-Text Contrastive Learning

In biomedical literature, biological pathways are commonly described through a combination of images and text. These pathways contain valuable information, including genes and their relationships, which provide insight into biological mechanisms and precision medicine. Curating pathway information across the literature enables the integration of this information to build a comprehensive knowledge base. While some studies have extracted pathway information from images and text independently, they often overlook the correspondence between the two modalities. In this paper, we present a pathway figure curation system named pathCLIP for identifying genes and gene relations from pathway figures. Our key innovation is the use of an image-text contrastive learning model to learn coordinated embeddings of image snippets and text descriptions of genes and gene relations, thereby improving curation. Our validation results, using pathway figures from PubMed, showed that our multimodal model outperforms models using only a single modality. Additionally, our system effectively curates genes and gene relations from multiple literature sources. A case study on extracting pathway information from non-small cell lung cancer literature further demonstrates the usefulness of our curated pathway information in enhancing related pathways in the KEGG database.

RepCo: Replenish sample views with better consistency for contrastive learning

Contrastive learning methods aim to learn shared representations by minimizing distances between positive pairs, and maximizing distances between negative pairs in the embedding space. To achieve better performance of contrastive learning, one of the key problems is to design appropriate sample pairs. In most previous works, random cropping on the input image is utilized to obtain two views as positive pairs. However, such strategies lead to suboptimal performance since the sampled crops may have inconsistent semantic information, which consequently degrades the quality of contrastive views. To address this limitation, we explore to replenish sample views with better consistency of the image and propose a novel self-supervised learning (SSL) framework RepCo. Instead of searching for semantically consistent patches between two different views, we select patches on the same image as the replenishment of positive/negative pairs, encourage patches that are similar but come from different positions as positive pairs, and force patches that are dissimilar but come from adjacent positions to have different representations, i.e. construct negative pairs to enrich the learned representations. Our method effectively generates high-quality contrastive views, explores the untapped semantic consistency on images, and provides more informative representations for downstream tasks. Experiments on adequate downstream tasks have shown that, our approach achieves ＋2.1 AP50 (COCO pre-trained) and ＋1.6 AP50 (ImageNet pre-trained) gains on Pascal VOC object detection, ＋2.3 mIoU gains on Cityscapes semantic segmentation, respectively.

A dual-branch model with inter- and intra-branch contrastive loss for long-tailed recognition

Real-world data often exhibits a long-tailed distribution, in which head classes occupy most of the data, while tail classes only have very few samples. Models trained on long-tailed datasets have poor adaptability to tail classes and the decision boundaries are ambiguous. Therefore, in this paper, we propose a simple yet effective model, named Dual-Branch Long-Tailed Recognition (DB-LTR), which includes an imbalanced learning branch and a Contrastive Learning Branch (CoLB). The imbalanced learning branch, which consists of a shared backbone and a linear classifier, leverages common imbalanced learning approaches to tackle the data imbalance issue. In CoLB, we learn a prototype for each tail class, and calculate an inter-branch contrastive loss, an intra-branch contrastive loss and a metric loss. CoLB can improve the capability of the model in adapting to tail classes and assist the imbalanced learning branch to learn a well-represented feature space and discriminative decision boundary. Extensive experiments on three long-tailed benchmark datasets, i.e., CIFAR100-LT, ImageNet-LT and Places-LT, show that our DB-LTR is competitive and superior to the comparative methods.

Contrastive diagnostic embedding (CDE) model for automated coding - A case study using emergency department encounters

BACKGROUND

Billing codes are utilized for medical reimbursement, clinical quality metric valuation and for epidemiologic purposes to report and follow disease trends and outcomes. The current paradigm of manual coding can be expensive, time-consuming, and subject to human error. Though automation of the billing codes has been widely reported in the literature via rule-based and supervised approaches, existing strategies lack generalizability and robustness towards large and constantly changing ICD hierarchical structure.

METHOD

We propose a weakly supervised training strategy by leveraging contrastive learning, contrastive diagnosis embedding (CDE) to capture the fine semantic variations between the diagnosis codes. The approach consists of a two-phase contrastive training for generating the semantic embedding space adapted to incorporate hierarchical information of ICD-10 vocabulary and a weakly supervised retrieval scheme. Core strength of the proposed method is that it puts no limit on the 70 K ICD-10 codes set and can handle all rare codes for coding the diagnosis.

RESULTS

Our CDE model outperformed string-based partial matching and ClinicalBERT embedding on three test cases (a retrospective testset, a prospective testset, and external testset) and produced an accurate prediction of rare and newly introduced diagnosis codes. A detailed ablation study showed the importance of each phase of the proposed multi-phase training. Each successive phase of training - ICD-10 group sensitive training (phase 1.1), ICD-10 subgroup sensitive training (phase 1.2), free-text diagnosis description-based training (phase 2) - improved performance beyond the previous phase of training. The model also outperformed existing supervised models like CAML and PLM-ICD and produced satisfactory performance on the rare codes.

CONCLUSION

Compared to the existing rule-based and supervised models, the proposed weakly supervised contrastive learning overcomes the limitations in terms of generalization capability and increases the robustness of the automated billing. Such a model will allow flexibility through accurate billing code automation for practice convergence and gains efficiencies in a value-based care payment environment.

Unsupervised anomaly detection by densely contrastive learning for time series data

Time series data continuously collected by different sensors play an essential role in monitoring and predicting events in many real-world applications, and anomaly detection for time series has received increasing attention during the past decades. In this paper, we propose an anomaly detection method by densely contrasting the whole time series with its sub-sequences at different timestamps in a latent space. Our approach leverages the locality property of convolutional neural networks (CNN) and integrates position embedding to effectively capture local features for sub-sequences. Simultaneously, we employ an attention mechanism to extract global features from the entire time series. By combining these local and global features, our model is trained using both instance-level contrastive learning loss and distribution-level alignment loss. Furthermore, we introduce a reconstruction loss applied to the extracted global features to prevent the potential loss of information. To validate the efficacy of our proposed technique, we conduct experiments on publicly available time-series datasets for anomaly detection. Additionally, we evaluate our method on an in-house mobile phone dataset aimed at monitoring the status of Parkinson's disease, all within an unsupervised learning framework. Our results demonstrate the effectiveness and potential of the proposed approach in tackling anomaly detection in time series data, offering promising applications in real-world scenarios.