Combining Sparse and Dense Features to Improve Multi-Modal Registration for Brain DTI Images

An NLP-based technique to extract meaningful features from drug SMILES

NLP is a well-established field in ML for developing language models that capture the sequence of words in a sentence. Similarly, drug molecule structures can also be represented as sequences using the SMILES notation. However, unlike natural language texts, special characters in drug SMILES have specific meanings and cannot be ignored. We introduce a novel NLP-based method that extracts interpretable sequences and essential features from drug SMILES notation using N-grams. Our method compares these features to Morgan fingerprint bit-vectors using UMAP-based embedding, and we validate its effectiveness through two personalized drug screening (PSD) case studies. Our NLP-based features are sparse and, when combined with gene expressions and disease phenotype features, produce better ML models for PSD. This approach provides a new way to analyze drug molecule structures represented as SMILES notation, which can help accelerate drug discovery efforts. We have also made our method accessible through a Python library.

Combining CNNs and Markov-like Models for Facial Landmark Detection with Spatial Consistency Estimates

The accurate localization of facial landmarks is essential for several tasks, including face recognition, head pose estimation, facial region extraction, and emotion detection. Although the number of required landmarks is task-specific, models are typically trained on all available landmarks in the datasets, limiting efficiency. Furthermore, model performance is strongly influenced by scale-dependent local appearance information around landmarks and the global shape information generated by them. To account for this, we propose a lightweight hybrid model for facial landmark detection designed specifically for pupil region extraction. Our design combines a convolutional neural network (CNN) with a Markov random field (MRF)-like process trained on only 17 carefully selected landmarks. The advantage of our model is the ability to run different image scales on the same convolutional layers, resulting in a significant reduction in model size. In addition, we employ an approximation of the MRF that is run on a subset of landmarks to validate the spatial consistency of the generated shape. This validation process is performed against a learned conditional distribution, expressing the location of one landmark relative to its neighbor. Experimental results on popular facial landmark localization datasets such as 300 w, WFLW, and HELEN demonstrate the accuracy of our proposed model. Furthermore, our model achieves state-of-the-art performance on a well-defined robustness metric. In conclusion, the results demonstrate the ability of our lightweight model to filter out spatially inconsistent predictions, even with significantly fewer training landmarks.

A Medical Image Registration Method Based on Progressive Images

Background

Medical image registration is an essential task for medical image analysis in various applications. In this work, we develop a coarse-to-fine medical image registration method based on progressive images and SURF algorithm (PI-SURF) for higher registration accuracy.

Methods

As a first step, the reference image and the floating image are fused to generate multiple progressive images. Thereafter, the floating image and progressive image are registered to get the coarse registration result based on the SURF algorithm. For further improvement, the coarse registration result and the reference image are registered to perform fine image registration. The appropriate progressive image has been investigated by experiments. The mutual information (MI), normal mutual information (NMI), normalized correlation coefficient (NCC), and mean square difference (MSD) similarity metrics are used to demonstrate the potential of the PI-SURF method.

Results

For the unimodal and multimodal registration, the PI-SURF method achieves the best results compared with the mutual information method, Demons method, Demons+B-spline method, and SURF method. The MI, NMI, and NCC of PI-SURF are improved by 15.5%, 1.31%, and 7.3%, respectively, while MSD decreased by 13.2% for the multimodal registration compared with the optimal result of the state-of-the-art methods.

Conclusions

The extensive experiments show that the proposed PI-SURF method achieves higher quality of registration.