Enhanced Nuclei Segmentation and Classification via Category Descriptors in the SAM Model

Segmenting and classifying nuclei in H&E histopathology images is often limited by the long-tailed distribution of nuclei types. However, the strong generalization ability of image segmentation foundation models like the Segment Anything Model (SAM) can help improve the detection quality of rare types of nuclei. In this work, we introduce category descriptors to perform nuclei segmentation and classification by prompting the SAM model. We close the domain gap between histopathology and natural scene images by aligning features in low-level space while preserving the high-level representations of SAM. We performed extensive experiments on the Lizard dataset, validating the ability of our model to perform automatic nuclei segmentation and classification, especially for rare nuclei types, where achieved a significant detection improvement in the F1 score of up to 12%. Our model also maintains compatibility with manual point prompts for interactive refinement during inference without requiring any additional training.

WA-ResUNet: A Focused Tail Class MRI Medical Image Segmentation Algorithm

Medical image segmentation can effectively identify lesions in medicine, but some small and rare lesions cannot be well identified. Existing studies do not take into account the uncertainty of the occurrence of diseased tissue, and the problem of long-tailed distribution of medical data. Meanwhile, the grayscale image obtained from Magnetic Resonance Imaging (MRI) detection has problems, such as the features being difficult to extract and invalid features being difficult to distinguish. In order to solve these problems, we propose a new weighted attention ResUNet (WA-ResUNet) and a class weight formula based on the number of images contained in the class, which improves the performance of the model in the low-frequency class and the overall effect of the model by improving the degree of attention paid to the valid features and invalid ones and rebalancing the learning efficiency among the classes. We evaluated our method on an uterine MRI dataset and compared it with the ResUNet. WA-ResUNet increased Intersection over Union (IoU) in the low-frequency class (Nabothian cysts) by 21.87%, and the overall mIoU increased by more than 6.5%.

A Universal Decoupled Training Framework for Human Parsing

Human parsing is an important technology in human-robot interaction systems. At present, the distribution of multi-category human parsing datasets is unbalanced, and the samples present a long-tailed distribution, which directly affects the performance of human parsing. Meanwhile, the similarity between different categories leads the model to predict false parsing results. To solve the above problems, a general decoupled training framework called Decoupled Training framework based on Pixel Resampling (DTPR) was proposed to solve the long-tailed distribution, and a new sampling method named Pixel Resampling based on Accuracy distribution (PRA) for semantic segmentation was also proposed and applied to this decoupled training framework. The framework divides the training process into two phases, the first phase is to improve the model feature extraction ability, and the second phase is to improve the performance of the model on tail categories. The training framework was evaluated in MHPv2.0 and LIP datasets, and tested in both high-precision and real-time SOTA models. The MPA metric of model trained by DTPR in above two datasets increased by more than 6%, and the mIoU metric increased by more than 1% without changing the model structure.