Identifying genotype-phenotype relationships in biomedical text

BACKGROUND

One important type of information contained in biomedical research literature is the newly discovered relationships between phenotypes and genotypes. Because of the large quantity of literature, a reliable automatic system to identify this information for future curation is essential. Such a system provides important and up to date data for database construction and updating, and even text summarization. In this paper we present a machine learning method to identify these genotype-phenotype relationships. No large human-annotated corpus of genotype-phenotype relationships currently exists. So, a semi-automatic approach has been used to annotate a small labelled training set and a self-training method is proposed to annotate more sentences and enlarge the training set.

RESULTS

The resulting machine-learned model was evaluated using a separate test set annotated by an expert. The results show that using only the small training set in a supervised learning method achieves good results (precision: 76.47, recall: 77.61, F-measure: 77.03) which are improved by applying a self-training method (precision: 77.70, recall: 77.84, F-measure: 77.77).

CONCLUSIONS

Relationships between genotypes and phenotypes is biomedical information pivotal to the understanding of a patient's situation. Our proposed method is the first attempt to make a specialized system to identify genotype-phenotype relationships in biomedical literature. We achieve good results using a small training set. To improve the results other linguistic contexts need to be explored and an appropriately enlarged training set is required.

High-fidelity simulation self-training enables novice bronchoscopists to acquire basic bronchoscopy skills comparable to their moderately and highly experienced counterparts

BACKGROUND

We sought to determine whether a self-training program on a high-fidelity flexible bronchoscopy (FB) simulator would allow residents who were novices in bronchoscopy to acquire competencies similar to those of experienced bronchoscopists as concerns the visualization of the bronchial tree and the identification of its anatomical elements.

METHODS

We performed a prospective cohort study, categorizing bronchoscopists into three groups according to their experience level: novice (Group A, no FBs performed, n = 8), moderate (Group B, 30 ≤ FBs performed ≤200, n = 17) or high (Group C, > 200 FBs performed, n = 9). All were initially evaluated on their ability to perform on a high-fidelity FB simulator a complete visualization/identification of the bronchial tree in the least amount of time possible. The residents in Group A then completed a simulation-based self-training program and underwent a final evaluation thereafter.

RESULTS

The median total procedure time for Group A fell from 561 s (IQR = 134) in the initial evaluation to 216 s (IQR = 257) in the final evaluation (P = 0.002). The visualization and identification scores for Group A also improved significantly in the final evaluation. Resultantly, the overall performance score for Group A climbed from 5.9% (IQR = 5.1) before self-training to 25.5% (IQR = 26.3) after (P = 0.002), thus becoming comparable to the overall performance scores of Group B (25.3%, IQR = 13.8) and Group C (22.2%, IQR = 5.5).

CONCLUSIONS

Novice bronchoscopists who self-train on a high-fidelity simulator acquire basic competencies similar to those of moderately or even highly experienced bronchoscopists. High-fidelity simulation should be rapidly integrated within the learning curriculum and replace traditional, in-patient learning methods.

A self-training deep neural network for early prediction of cognitive deficits in very preterm infants using brain functional connectome data

BACKGROUND

Deep learning has been employed using brain functional connectome data for evaluating the risk of cognitive deficits in very preterm infants. Although promising, training these deep learning models typically requires a large amount of labeled data, and labeled medical data are often very difficult and expensive to obtain.

OBJECTIVE

This study aimed to develop a self-training deep neural network (DNN) model for early prediction of cognitive deficits at 2 years of corrected age in very preterm infants (gestational age ≤32 weeks) using both labeled and unlabeled brain functional connectome data.

MATERIALS AND METHODS

We collected brain functional connectome data from 343 very preterm infants at a mean (standard deviation) postmenstrual age of 42.7 (2.5) weeks, among whom 103 children had a cognitive assessment at 2 years (i.e. labeled data), and the remaining 240 children had not received 2-year assessments at the time this study was conducted (i.e. unlabeled data). To develop a self-training DNN model, we built an initial student model using labeled brain functional connectome data. Then, we applied the trained model as a teacher model to generate pseudo-labels for unlabeled brain functional connectome data. Next, we combined labeled and pseudo-labeled data to train a new student model. We iterated this procedure to obtain the best student model for the early prediction task in very preterm infants.

RESULTS

In our cross-validation experiments, the proposed self-training DNN model achieved an accuracy of 71.0%, a specificity of 71.5%, a sensitivity of 70.4% and an area under the curve of 0.75, significantly outperforming transfer learning models through pre-training approaches.

CONCLUSION

We report the first self-training prognostic study in very preterm infants, efficiently utilizing a small amount of labeled data with a larger share of unlabeled data to aid the model training. The proposed technique is expected to facilitate deep learning with insufficient training data.

PICO entity extraction for preclinical animal literature

BACKGROUND

Natural language processing could assist multiple tasks in systematic reviews to reduce workflow, including the extraction of PICO elements such as study populations, interventions, comparators and outcomes. The PICO framework provides a basis for the retrieval and selection for inclusion of evidence relevant to a specific systematic review question, and automatic approaches to PICO extraction have been developed particularly for reviews of clinical trial findings. Considering the difference between preclinical animal studies and clinical trials, developing separate approaches is necessary. Facilitating preclinical systematic reviews will inform the translation from preclinical to clinical research.

METHODS

We randomly selected 400 abstracts from the PubMed Central Open Access database which described in vivo animal research and manually annotated these with PICO phrases for Species, Strain, methods of Induction of disease model, Intervention, Comparator and Outcome. We developed a two-stage workflow for preclinical PICO extraction. Firstly we fine-tuned BERT with different pre-trained modules for PICO sentence classification. Then, after removing the text irrelevant to PICO features, we explored LSTM-, CRF- and BERT-based models for PICO entity recognition. We also explored a self-training approach because of the small training corpus.

RESULTS

For PICO sentence classification, BERT models using all pre-trained modules achieved an F1 score of over 80%, and models pre-trained on PubMed abstracts achieved the highest F1 of 85%. For PICO entity recognition, fine-tuning BERT pre-trained on PubMed abstracts achieved an overall F1 of 71% and satisfactory F1 for Species (98%), Strain (70%), Intervention (70%) and Outcome (67%). The score of Induction and Comparator is less satisfactory, but F1 of Comparator can be improved to 50% by applying self-training.

CONCLUSIONS

Our study indicates that of the approaches tested, BERT pre-trained on PubMed abstracts is the best for both PICO sentence classification and PICO entity recognition in the preclinical abstracts. Self-training yields better performance for identifying comparators and strains.

Appropriate use of a dry powder inhaler based on inhalation flow pattern

Background

An optimal inhalation flow pattern is essential for effective use of a dry powder inhaler (DPI). We wondered whether DPI instructors inhale from a DPI with an appropriate pattern, and if not, whether self-training with visual feedback is effective.

Methods

Subjects were 14 pharmacists regularly engaged in instruction in DPI use. A newly designed handy inhalation flow visualizer (Visual Trainer: VT) was used to assess inhalation profiles and to assist in self-training. With a peak inhalation flow rate (PIFR) > 50 L/min, time reaching PIFR (T_PF) < 0.4 s, inhalation volume (V_I) > 1 L, and flow at 0.3 s after the onset of inhalation (F_0.3) > 50 L/min, the pattern was considered optimal.

Results

Using Diskus or Turbuhaler 12 and 10 subjects respectively inhaled with a suitable PIFR. Those with a satisfactory F_0.3 were 10 and 7 respectively. The T_PF was short enough in only 1 and 2 respectively. All 14 subjects inhaled deeply (V_I) through Diskus, and 10 did so through Turbuhaler. In the self-training session, only 3 subjects satisfied all three variables at the first trial, while 2 or 3 trials were required in other subjects. Among the three variables, optimal T_PF was the most difficult to attain. Once a satisfactory inhalation pattern was achieved using one DPI, eleven out of 12 subjects inhaled with a satisfactory pattern through the other DPI.

Conclusion

Visualization of the inhalation flow pattern facilitates the learning of proper inhalation technique through a DPI.

Feature Alignment by Uncertainty and Self-Training for Source-Free Unsupervised Domain Adaptation

Most unsupervised domain adaptation (UDA) methods assume that labeled source images are available during model adaptation. However, this assumption is often infeasible owing to confidentiality issues or memory constraints on mobile devices. Some recently developed approaches do not require source images during adaptation, but they show limited performance on perturbed images. To address these problems, we propose a novel source-free UDA method that uses only a pre-trained source model and unlabeled target images. Our method captures the aleatoric uncertainty by incorporating data augmentation and trains the feature generator with two consistency objectives. The feature generator is encouraged to learn consistent visual features away from the decision boundaries of the head classifier. Thus, the adapted model becomes more robust to image perturbations. Inspired by self-supervised learning, our method promotes inter-space alignment between the prediction space and the feature space while incorporating intra-space consistency within the feature space to reduce the domain gap between the source and target domains. We also consider epistemic uncertainty to boost the model adaptation performance. Extensive experiments on popular UDA benchmark datasets demonstrate that the proposed source-free method is comparable or even superior to vanilla UDA methods. Moreover, the adapted models show more robust results when input images are perturbed.

Self-learning followed by telepresence instruction of focused cardiac ultrasound with a handheld device for medical students: a preliminary study

Purpose

This study aimed to assess the feasibility and efficiency of self-learning with or without self-training (subjects performed scans on themselves) and telepresence instruction in focused cardiac ultrasound (FOCUS) education for medical students.

Methods

This study included 24 medical students. The participants initially completed a written pre-test and were randomized into a video lecture (participants watched a video lecture) or self-training (participants watched a video lecture and self-performed FOCUS) group. After finishing self-learning, they completed a written post-test. Then they undertook a skill pre-test and a first perception survey. Telepresence instruction was then provided. Finally, they undertook a skill post-test and a second perception survey.

Results

The written post-test total scores were significantly higher than the pre-test total scores (P < 0.001). In the skill pre-test, the scores for the video lecture and self-training groups were not significantly different (P = 0.542). The skill post-test total scores were significantly higher than the skill pre-test total scores (P = 0.008). Forty-two percent of the video lecture group participants agreed that the video lecture was effective preparation for the skill pre-test, while all participants in the same group agreed that the combination of the video lecture and telepresence instruction was effective preparation for the skill post-test.

Conclusion

This study demonstrated the feasibility and efficiency of self-learning followed by telepresence instruction on FOCUS for medical students.

Iterative confidence relabeling with deep ConvNets for organ segmentation with partial labels

Training deep ConvNets requires large labeled datasets. However, collecting pixel-level labels for medical image segmentation is very expensive and requires a high level of expertise. In addition, most existing segmentation masks provided by clinical experts focus on specific anatomical structures. In this paper, we propose a method dedicated to handle such partially labeled medical image datasets. We propose a strategy to identify pixels for which labels are correct, and to train Fully Convolutional Neural Networks with a multi-label loss adapted to this context. In addition, we introduce an iterative confidence self-training approach inspired by curriculum learning to relabel missing pixel labels, which relies on selecting the most confident prediction with a specifically designed confidence network that learns an uncertainty measure which is leveraged in our relabeling process. Our approach, INERRANT for Iterative coNfidencE Relabeling of paRtial ANnoTations, is thoroughly evaluated on two public datasets (TCAI and LITS), and one internal dataset with seven abdominal organ classes. We show that INERRANT robustly deals with partial labels, performing similarly to a model trained on all labels even for large missing label proportions. We also highlight the importance of our iterative learning scheme and the proposed confidence measure for optimal performance. Finally we show a practical use case where a limited number of completely labeled data are enriched by publicly available but partially labeled data.

Memory consistent unsupervised off-the-shelf model adaptation for source-relaxed medical image segmentation

Unsupervised domain adaptation (UDA) has been a vital protocol for migrating information learned from a labeled source domain to facilitate the implementation in an unlabeled heterogeneous target domain. Although UDA is typically jointly trained on data from both domains, accessing the labeled source domain data is often restricted, due to concerns over patient data privacy or intellectual property. To sidestep this, we propose "off-the-shelf (OS)" UDA (OSUDA), aimed at image segmentation, by adapting an OS segmentor trained in a source domain to a target domain, in the absence of source domain data in adaptation. Toward this goal, we aim to develop a novel batch-wise normalization (BN) statistics adaptation framework. In particular, we gradually adapt the domain-specific low-order BN statistics, e.g., mean and variance, through an exponential momentum decay strategy, while explicitly enforcing the consistency of the domain shareable high-order BN statistics, e.g., scaling and shifting factors, via our optimization objective. We also adaptively quantify the channel-wise transferability to gauge the importance of each channel, via both low-order statistics divergence and a scaling factor. Furthermore, we incorporate unsupervised self-entropy minimization into our framework to boost performance alongside a novel queued, memory-consistent self-training strategy to utilize the reliable pseudo label for stable and efficient unsupervised adaptation. We evaluated our OSUDA-based framework on both cross-modality and cross-subtype brain tumor segmentation and cardiac MR to CT segmentation tasks. Our experimental results showed that our memory consistent OSUDA performs better than existing source-relaxed UDA methods and yields similar performance to UDA methods with source data.

CASPIANET++: A multidimensional Channel-Spatial Asymmetric attention network with Noisy Student Curriculum Learning paradigm for brain tumor segmentation

Convolutional neural networks (CNNs) have been used quite successfully for semantic segmentation of brain tumors. However, current CNNs and attention mechanisms are stochastic in nature and neglect the morphological indicators used by radiologists to manually annotate regions of interest. In this paper, we introduce a channel and spatial wise asymmetric attention (CASPIAN) by leveraging the inherent structure of tumors to detect regions of saliency. To demonstrate the efficacy of our proposed layer, we integrate this into a well-established convolutional neural network (CNN) architecture to achieve higher Dice scores, with less GPU resources. Also, we investigate the inclusion of auxiliary multiscale and multiplanar attention branches to increase the spatial context crucial in semantic segmentation tasks. The resulting architecture is the new CASPIANET++, which achieves Dice Scores of 91.19%, 87.6% and 81.03% for whole tumor, tumor core and enhancing tumor respectively. Furthermore, driven by the scarcity of brain tumor data, we investigate the Noisy Student method for segmentation tasks. Our new Noisy Student Curriculum Learning paradigm, which infuses noise incrementally to increase the complexity of the training images exposed to the network, further boosts the enhancing tumor region to 81.53%. Additional validation performed on the BraTS2020 data shows that the Noisy Student Curriculum Learning method works well without any additional training or finetuning.

Self-training adversarial learning for cross-domain retinal OCT fluid segmentation

Accurate measurements of the size, shape and volume of macular edema can provide important biomarkers to jointly assess disease progression and treatment outcome. Although many deep learning-based segmentation algorithms have achieved remarkable success in semantic segmentation, these methods have difficulty obtaining satisfactory segmentation results in retinal optical coherence tomography (OCT) fluid segmentation tasks due to low contrast, blurred boundaries, and varied distribution. Moreover, directly applying a well-trained model on one device to test the images from other devices may cause the performance degradation in the joint analysis of multi-domain OCT images. In this paper, we propose a self-training adversarial learning framework for unsupervised domain adaptation in retinal OCT fluid segmentation tasks. Specifically, we develop an image style transfer module and a fine-grained feature transfer module to reduce discrepancies in the appearance and high-level features of images from different devices. Importantly, we transfer the target images to the appearance of source images to ensure that no image information of the source domain for supervised training is lost. To capture specific features of the target domain, we design a self-training module based on a discrepancy and similarity strategy to select the images with better segmentation results from the target domain and then introduce them into the source domain for the iterative training segmentation model. Extensive experiments on two challenging datasets demonstrate the effectiveness of our proposed method. In Particular, our proposed method achieves comparable results on cross-domain retinal OCT fluid segmentation compared with the state-of-the-art methods.

Source-free domain adaptive segmentation with class-balanced complementary self-training

Unsupervised domain adaptation (UDA) plays a crucial role in transferring knowledge gained from a labeled source domain to effectively apply it in an unlabeled and diverse target domain. While UDA commonly involves training on data from both domains, accessing labeled data from the source domain is frequently constrained, citing concerns related to patient data privacy or intellectual property. The source-free UDA (SFUDA) can be promising to sidestep this difficulty. However, without the source domain supervision, the SFUDA methods can easily fall into the dilemma of "winner takes all", in which the majority category can dominate the deep segmentor, and the minority categories are largely ignored. In addition, the over-confident pseudo-label noise in self-training-based UDA is a long-lasting problem. To sidestep these difficulties, we propose a novel class-balanced complementary self-training (CBCOST) framework for SFUDA segmentation. Specifically, we jointly optimize the pseudo-label-based self-training with two mutually reinforced components. The first class-wise balanced pseudo-label training (CBT) explicitly exploits the fine-grained class-wise confidence to select the class-wise balanced pseudo-labeled pixels with the adaptive within-class thresholds. Second, to alleviate the pseudo-labeled noise, we propose a complementary self-training (COST) to exclude the classes that do not belong to, with a heuristic complementary label selection scheme. We evaluated our CBCOST framework on both 2D and 3D cross-modality cardiac anatomical segmentation tasks and brain tumor segmentation tasks. Our experimental results showed that our CBCOST performs better than existing SFUDA methods and yields similar performance, compared with UDA methods with the source data.

Augmented reality self-training system for suturing in open surgery: A randomized controlled trial

BACKGROUND

Existing self-training materials are insufficient to learn open surgical procedures, and a new self-training system that provides three-dimensional procedural information is needed. The effectiveness and usability of a self-training system providing three-dimensional information by augmented reality (AR) were compared to those of an existing self-training system, instructional video, in self-learning of suturing in open surgery.

MATERIALS AND METHODS

This was a prospective, evaluator-blinded, randomized, controlled study. Medical students who were suturing novices were randomized into 2 groups: practice with the AR training system (AR group) or an instructional video (video group). Participants were instructed in subcuticular interrupted suture and each training system and watched the instructional video once. They then completed a pretest performing the suture on a skin pad. Participants in each group practiced the procedure 10 times using each training system, followed by a posttest. The pretest and posttest were video-recorded and graded by blinded evaluators using a validated scoring form composed of global rating (GR) and task-specific (TS) subscales. Students completed a post-study questionnaire assessing system usability, each system's usefulness, and their confidence and interest in surgery.

RESULTS

Nineteen participants in each group completed the trial. No significant difference was found between the AR and video groups on the improvement of the scores from pretest to posttest (GR: p = 0.54, TS: p = 0.91). The posttest scores of both GR and TS improved significantly from pretest in both groups (GR: both p < 0.001, TS: both p < 0.001). There was no significant difference between the groups in the system usability scale scores (p = 0.38). The motion provided in the AR system was more helpful for manipulating surgical instruments than the video (p = 0.02).

CONCLUSION

The AR system was considered as understandable and easy to use as the instructional video in learning suture technique in open surgery for novices.

A medical unsupervised domain adaptation framework based on Fourier transform image translation and multi-model ensemble self-training strategy

PURPOSE

Well-established segmentation models will suffer performance degradation when deployed on data with heterogeneous features, especially in the field of medical image analysis. Although researchers have proposed many approaches to address this problem in recent years, most of them are feature-adaptation-based adversarial networks, the problems such as training instability often arise in adversarial training. To ameliorate this challenge and improve the robustness of processing data with different distributions, we propose a novel unsupervised domain adaptation framework for cross-domain medical image segmentation.

METHODS

In our proposed approach, Fourier transform guided images translation and multi-model ensemble self-training are integrated into a unified framework. First, after Fourier transform, the amplitude spectrum of source image is replaced with that of target image, and reconstructed by the inverse Fourier transform. Second, we augment target dataset with the synthetic cross-domain images, performing supervised learning using the original source set labels while implementing regularization by entropy minimization on predictions of unlabeled target data. We employ several segmentation networks with different hyperparameters simultaneously, pseudo-labels are generated by averaging their outputs and comparing to confidence threshold, and gradually optimize the quality of pseudo-labels through multiple rounds self-training.

RESULTS

We employed our framework to two liver CT datasets for bidirectional adaptation experiments. In both experiments, compared to the segmentation network without domain alignment, dice similarity coefficient (DSC) increased by nearly 34% and average symmetric surface distance (ASSD) decreased by about 10. The DSC values were also improved by 10.8% and 6.7%, respectively, compared to the existing model.

CONCLUSION

We propose a Fourier transform-based UDA framework, the experimental results and comparisons demonstrate that the proposed method can effectively diminish the performance degradation caused by domain shift and performs best on the cross-domain segmentation tasks. Our proposed multi-model ensemble training strategy can also improve the robustness of the segmentation system.

Image-level supervision and self-training for transformer-based cross-modality tumor segmentation

Deep neural networks are commonly used for automated medical image segmentation, but models will frequently struggle to generalize well across different imaging modalities. This issue is particularly problematic due to the limited availability of annotated data, both in the target as well as the source modality, making it difficult to deploy these models on a larger scale. To overcome these challenges, we propose a new semi-supervised training strategy called MoDATTS. Our approach is designed for accurate cross-modality 3D tumor segmentation on unpaired bi-modal datasets. An image-to-image translation strategy between modalities is used to produce synthetic but annotated images and labels in the desired modality and improve generalization to the unannotated target modality. We also use powerful vision transformer architectures for both image translation (TransUNet) and segmentation (Medformer) tasks and introduce an iterative self-training procedure in the later task to further close the domain gap between modalities, thus also training on unlabeled images in the target modality. MoDATTS additionally allows the possibility to exploit image-level labels with a semi-supervised objective that encourages the model to disentangle tumors from the background. This semi-supervised methodology helps in particular to maintain downstream segmentation performance when pixel-level label scarcity is also present in the source modality dataset, or when the source dataset contains healthy controls. The proposed model achieves superior performance compared to other methods from participating teams in the CrossMoDA 2022 vestibular schwannoma (VS) segmentation challenge, as evidenced by its reported top Dice score of 0.87±0.04 for the VS segmentation. MoDATTS also yields consistent improvements in Dice scores over baselines on a cross-modality adult brain gliomas segmentation task composed of four different contrasts from the BraTS 2020 challenge dataset, where 95% of a target supervised model performance is reached when no target modality annotations are available. We report that 99% and 100% of this maximum performance can be attained if 20% and 50% of the target data is additionally annotated, which further demonstrates that MoDATTS can be leveraged to reduce the annotation burden.

Experiences of Older Adults with Mild Cognitive Impairment from Cognitive Self-Training Using Touchscreen Tablets

Background: "Tablet Enhancement of Cognition and Health" (TECH) is a cognitive intervention that includes two components: 5 weeks of daily self-training using puzzle-game apps on a touch screen tablet and weekly group sessions. This study aimed to (i) explore experiences of older adults with mild cognitive impairment (MCI) following their participation in TECH, (ii) identify hindering and enabling factors to self-training, and (iii) describe participants' perceived and objective cognitive changes and examine factors associated with their satisfaction from TECH. Materials and Methods: We used quantitative and qualitative measures; a phenomenological qualitative design using focus groups and interviews of 14 older adults with MCI and a focus group of the TECH facilitators. Satisfaction with TECH, self-training time, and perceived and objective cognitive changes (using the Montreal Cognitive Assessment) were evaluated. Results: Qualitative data were classified into three categories: Memory problems, Hindering and enabling factors to self-training, and Meaningful group sessions. The TECH facilitators reported positive changes, less cognitive complaints, and commitment and satisfaction of the participants. Participants reported overall satisfaction from TECH and performed a median interquartile range of 22.6 (19.9-42.8) self-training hours. Higher satisfaction was correlated with a higher objective cognitive change (r = 0.95, P < 0.01) and less training time (r = -0.91, P < 0.01). Discussion and Conclusions: Participants in the current study actively engaged in daily self-training using touch screen-tablet-puzzle-game and functional apps, driven by both internal and external motivators. Despite the lack of cognitive improvement, they expressed satisfaction with their participation in TECH. Therefore, encouraging older adults to engage in meaningful cognitive stimulating activities is recommended.

Class-Specific Distribution Alignment for semi-supervised medical image classification

Despite the success of deep neural networks in medical image classification, the problem remains challenging as data annotation is time-consuming, and the class distribution is imbalanced due to the relative scarcity of diseases. To address this problem, we propose Class-Specific Distribution Alignment (CSDA), a semi-supervised learning framework based on self-training that is suitable to learn from highly imbalanced datasets. Specifically, we first provide a new perspective to distribution alignment by considering the process as a change of basis in the vector space spanned by marginal predictions, and then derive CSDA to capture class-dependent marginal predictions on both labeled and unlabeled data, in order to avoid the bias towards majority classes. Furthermore, we propose a Variable Condition Queue (VCQ) module to maintain a proportionately balanced number of unlabeled samples for each class. Experiments on three public datasets HAM10000, CheXpert and Kvasir show that our method provides competitive performance on semi-supervised skin disease, thoracic disease, and endoscopic image classification tasks.

Enhanced Domain Adaptation for Foot Ulcer Segmentation Through Mixing Self-Trained Weak Labels

Wound management requires the measurement of the wound parameters such as its shape and area. However, computerized analysis of the wound suffers the challenge of inexact segmentation of the wound images due to limited or inaccurate labels. It is a common scenario that the source domain provides an abundance of labeled data, while the target domain provides only limited labels. To overcome this, we propose a novel approach that combines self-training learning and mixup augmentation. The neural network is trained on the source domain to generate weak labels on the target domain via the self-training process. In the second stage, generated labels are mixed up with labels from the source domain to retrain the neural network and enhance generalization across diverse datasets. The efficacy of our approach was evaluated using the DFUC 2022, FUSeg, and RMIT datasets, demonstrating substantial improvements in segmentation accuracy and robustness across different data distributions. Specifically, in single-domain experiments, segmentation on the DFUC 2022 dataset scored a dice score of 0.711, while the score on the FUSeg dataset achieved 0.859. For domain adaptation, when these datasets were used as target datasets, the dice scores were 0.714 for DFUC 2022 and 0.561 for FUSeg.

Weakly supervised temporal action localization with actionness-guided false positive suppression

Weakly supervised temporal action localization aims to locate the temporal boundaries of action instances in untrimmed videos using video-level labels and assign them the corresponding action category. Generally, it is solved by a pipeline called "localization-by-classification", which finds the action instances by classifying video snippets. However, since this approach optimizes the video-level classification objective, the generated activation sequences often suffer interference from class-related scenes, resulting in a large number of false positives in the prediction results. Many existing works treat background as an independent category, forcing models to learn to distinguish background snippets. However, under weakly supervised conditions, the background information is fuzzy and uncertain, making this method extremely difficult. To alleviate the impact of false positives, we propose a new actionness-guided false positive suppression framework. Our method seeks to suppress false positive backgrounds without introducing the background category. Firstly, we propose a self-training actionness branch to learn class-agnostic actionness, which can minimize the interference of class-related scene information by ignoring the video labels. Secondly, we propose a false positive suppression module to mine false positive snippets and suppress them. Finally, we introduce the foreground enhancement module, which guides the model to learn the foreground with the help of the attention mechanism as well as class-agnostic actionness. We conduct extensive experiments on three benchmarks (THUMOS14, ActivityNet1.2, and ActivityNet1.3). The results demonstrate the effectiveness of our method in suppressing false positives and it achieves the state-of-the-art performance. Code: https://github.com/lizhilin-ustc/AFPS.

Instance segmentation using semi-supervised learning for fire recognition

Fire disaster brings enormous danger to the safety of human life and property, and it is important to identify the fire situation in time through image processing technology. The current instance segmentation algorithms suffer from problems such as inadequate fire images and annotations, low recognition accuracy, and slow inference speed for fire recognition tasks. In this paper, we propose a semi-supervised learning-based fire instance segmentation method based on deep learning image processing technology. We used a lightweight version of the SOLOv2 network and optimized the network structure to improve accuracy. We propose a semi-supervised learning method based on fire features. To reduce the negative impact of error pseudo-labels on the model training, the pseudo-labels are matched by the color and morphological features of flames and smoke at the pseudo-label generation stage, and some images are screened for strong image enhancement before entering the next round of training for the student model. We further exploit the potential of the model with a limited dataset and improve the model accuracy without affecting the inference efficiency of the model. Experiments show that our proposed algorithm can successfully improve the accuracy of fire instance segmentation with good inference speed.