UGLS: an uncertainty guided deep learning strategy for accurate image segmentation

Accurate image segmentation plays a crucial role in computer vision and medical image analysis. In this study, we developed a novel uncertainty guided deep learning strategy (UGLS) to enhance the performance of an existing neural network (i.e., U-Net) in segmenting multiple objects of interest from images with varying modalities. In the developed UGLS, a boundary uncertainty map was introduced for each object based on its coarse segmentation (obtained by the U-Net) and then combined with input images for the fine segmentation of the objects. We validated the developed method by segmenting optic cup (OC) regions from color fundus images and left and right lung regions from Xray images. Experiments on public fundus and Xray image datasets showed that the developed method achieved a average Dice Score (DS) of 0.8791 and a sensitivity (SEN) of 0.8858 for the OC segmentation, and 0.9605, 0.9607, 0.9621, and 0.9668 for the left and right lung segmentation, respectively. Our method significantly improved the segmentation performance of the U-Net, making it comparable or superior to five sophisticated networks (i.e., AU-Net, BiO-Net, AS-Net, Swin-Unet, and TransUNet).

DILS: depth incremental learning strategy

There exist various methods for transferring knowledge between neural networks, such as parameter transfer, feature sharing, and knowledge distillation. However, these methods are typically applied when transferring knowledge between networks of equal size or from larger networks to smaller ones. Currently, there is a lack of methods for transferring knowledge from shallower networks to deeper ones, which is crucial in real-world scenarios such as system upgrades where network size increases for better performance. End-to-end training is the commonly used method for network training. However, in this training strategy, the deeper network cannot inherit the knowledge from the existing shallower network. As a result, not only is the flexibility of the network limited but there is also a significant waste of computing power and time. Therefore, it is imperative to develop new methods that enable the transfer of knowledge from shallower to deeper networks. To address the aforementioned issue, we propose an depth incremental learning strategy (DILS). It starts from a shallower net and deepens the net gradually by inserting new layers each time until reaching requested performance. We also derive an analytical method and a network approximation method for training new added parameters to guarantee the new deeper net can inherit the knowledge learned by the old shallower net. It enables knowledge transfer from smaller to larger networks and provides good initialization of layers in the larger network to stabilize the performance of large models and accelerate their training process. Its reasonability can be guaranteed by information projection theory and is verified by a series of synthetic and real-data experiments.

Recognition of Wheat Leaf Diseases Using Lightweight Convolutional Neural Networks against Complex Backgrounds

Wheat leaf diseases are considered to be the foremost threat to wheat yield. In the realm of crop disease detection, convolutional neural networks (CNNs) have emerged as important tools. The training strategy and the initial learning rate are key factors that impact the performance and training speed of the model in CNNs. This study employed six training strategies, including Adam, SGD, Adam + StepLR, SGD + StepLR, Warm-up + Cosine annealing + SGD, Warm-up + Cosine, and annealing + Adam, with three initial learning rates (0.05, 0.01, and 0.001). Using the wheat stripe rust, wheat powdery mildew, and healthy wheat datasets, five lightweight CNN models, namely MobileNetV3, ShuffleNetV2, GhostNet, MnasNet, and EfficientNetV2, were evaluated. The results showed that upon combining the SGD + StepLR with the initial learning rate of 0.001, the MnasNet obtained the highest recognition accuracy of 98.65%. The accuracy increased by 1.1% as compared to that obtained with the training strategy with a fixed learning rate, and the size of the parameters was only 19.09 M. The above results indicated that the MnasNet was appropriate for porting to the mobile terminal and efficient for automatically identifying wheat leaf diseases.

Continuous joint velocity estimation using CNN-based deep learning for multi-DoF prosthetic wrist for activities of daily living

Introduction

Myoelectric control of prostheses is a long-established technique, using surface electromyography (sEMG) to detect user intention and perform subsequent mechanical actions. Most machine learning models utilized in control systems are trained using isolated movements that do not reflect the natural movements occurring during daily activities. Moreover, movements are often affected by arm postures, the duration of activities, and personal habits. It is crucial to have a control system for multi-degree-of-freedom (DoF) prosthetic arms that is trained using sEMG data collected from activities of daily living (ADL) tasks.

Method

This work focuses on two major functional wrist movements: pronation-supination and dart-throwing movement (DTM), and introduces a new wrist control system that directly maps sEMG signals to the joint velocities of the multi-DoF wrist. Additionally, a specific training strategy (Quick training) is proposed that enables the controller to be applied to new subjects and handle situations where sensors may displace during daily living, muscles can become fatigued, or sensors can become contaminated (e.g., due to sweat). The prosthetic wrist controller is designed based on data from 24 participants and its performance is evaluated using the Root Mean Square Error (RMSE) and Pearson Correlation.

Result

The results are found to depend on the characteristics of the tasks. For example, tasks with dart-throwing motion show smaller RSME values (Hammer: 6.68 deg/s and Cup: 7.92 deg/s) compared to tasks with pronation-supination (Bulb: 43.98 deg/s and Screw: 53.64 deg/s). The proposed control technique utilizing Quick training demonstrates a decrease in the average root mean square error (RMSE) value by 35% and an increase in the average Pearson correlation value by 40% across all four ADL tasks.

Few-shot learning approach with multi-scale feature fusion and attention for plant disease recognition

Image-based deep learning method for plant disease diagnosing is promising but relies on large-scale dataset. Currently, the shortage of data has become an obstacle to leverage deep learning methods. Few-shot learning can generalize to new categories with the supports of few samples, which is very helpful for those plant disease categories where only few samples are available. However, two challenging problems are existing in few-shot learning: (1) the feature extracted from few shots is very limited; (2) generalizing to new categories, especially to another domain is very tough. In response to the two issues, we propose a network based on the Meta-Baseline few-shot learning method, and combine cascaded multi-scale features and channel attention. The network takes advantage of multi-scale features to rich the feature representation, uses channel attention as a compensation module efficiently to learn more from the significant channels of the fused features. Meanwhile, we propose a group of training strategies from data configuration perspective to match various generalization requirements. Through extensive experiments, it is verified that the combination of multi-scale feature fusion and channel attention can alleviate the problem of limited features caused by few shots. To imitate different generalization scenarios, we set different data settings and suggest the optimal training strategies for intra-domain case and cross-domain case, respectively. The effects of important factors in few-shot learning paradigm are analyzed. With the optimal configuration, the accuracy of 1-shot task and 5-shot task achieve at 61.24% and 77.43% respectively in the task targeting to single-plant, and achieve at 82.52% and 92.83% in the task targeting to multi-plants. Our results outperform the existing related works. It demonstrates that the few-shot learning is a feasible potential solution for plant disease recognition in the future application.

Detecting Proximal Caries on Periapical Radiographs Using Convolutional Neural Networks with Different Training Strategies on Small Datasets

The present study aimed to evaluate the performance of convolutional neural networks (CNNs) that were trained with small datasets using different strategies in the detection of proximal caries at different levels of severity on periapical radiographs. Small datasets containing 800 periapical radiographs were randomly categorized into a training and validation dataset (n = 600) and a test dataset (n = 200). A pretrained Cifar-10Net CNN was used in the present study. Different training strategies were used to train the CNN model independently; these strategies were defined as image recognition (IR), edge extraction (EE), and image segmentation (IS). Different metrics, such as sensitivity and area under the receiver operating characteristic curve (AUC), for the trained CNN and human observers were analysed to evaluate the performance in detecting proximal caries. IR, EE, and IS recognition modes and human eyes achieved AUCs of 0.805, 0.860, 0.549, and 0.767, respectively, with the EE recognition mode having the highest values (p all < 0.05). The EE recognition mode was significantly more sensitive in detecting both enamel and dentin caries than human eyes (p all < 0.05). The CNN trained with the EE strategy, the best performer in the present study, showed potential utility in detecting proximal caries on periapical radiographs when using small datasets.

Heat Acclimation Following Heat Acclimatization Elicits Additional Physiological Improvements in Male Endurance Athletes

The purpose of this study was to assess the effectiveness of heat acclimatization (HAz) followed by heat acclimation (HA) on physiological adaptations. 25 male endurance athletes (age 36 ± 12 y, height 178.8 ± 6.39 cm, body mass 73.03 ± 8.97 kg, and VO_2peak 57.5 ± 7.0 mL·kg^-1·min^-1) completed HAz and HA. HAz was 3 months of self-directed summer training. In the laboratory, a 5-day HA prescribed exercise to target a hyperthermic zone (HZHA) of T_rec between 38.50 and 39.75 °C for 60 min. Exercise trials were 60 min of running (59% ± 2% VO_2peak) in an environmental chamber (wet bulb globe temperature 29.53 ± 0.63 °C) and administered at: baseline, post-HAz, and post-HAz+HA. Measured variables included internal body temperature (T_rec), heart rate (HR), and sweat rate (SR). Repeated measure ANOVAs and post hoc comparisons were used to assess statistically significant (p < 0.05) differences. T_rec was lower post-HAz+HA (38.03 ± 0.39 °C) than post-HAz (38.25 ± 0.42 °C, p = 0.009) and baseline (38.29 ± 0.37 °C, p = 0.005). There were no differences between baseline and post-HAz (p = 0.479) in T_rec. HR was lower post-HAz (143 ± 12 bpm, p = 0.002) and post-HAz+HA (134 ± 11 bpm, p < 0.001) than baseline (138 ± 14 bpm). HR was lower post-HAz+HA than post-HAz (p = 0.013). SR was higher post-HAz+HA (1.93 ± 0.47 L·h^-1) than post-HAz (1.76 ± 0.43 L·h^-1, p = 0.027). Combination HAz and HA increased physiological outcomes above HAz. This method can be used to improve performance and safety in addition to HAz alone.

Artificial classification of cervical squamous lesions in ThinPrep cytologic tests using a deep convolutional neural network

The diagnosis of squamous cell carcinoma requires the accurate classification of cervical squamous lesions in the ThinPrep cytologic test (TCT). It primarily relies on a pathologist's interpretation under a microscope. Deep convolutional neural networks (DCNN) have played an increasingly important role in digital pathology. However, they have not been applied to diverse datasets and externally validated. In the present study, a DCNN model based on VGG16 and an ensemble training strategy (ETS) based on 5-fold cross-validation was employed to automatically classify normal and abnormal cervical squamous cells from a multi-center dataset. First, we collected a dataset comprising 82 TCT samples from four hospitals and fine-tuned our model twice on the dataset with and without the ETS. Then, we compared the classifications obtained from the models with those provided by two skilled pathologists to discriminate the performance of the models in terms of classification accuracy and efficiency. Finally, paired sample t-tests were used to validate the consistency between the classification provided by the proposed methods and that of the pathologists. The results showed that ETS slightly, though not significantly, improved the classification accuracy compared with that of the pathologists: P₀=0.387>0.05 (DCNN without ETS vs. DCNN with ETS), P₁=0.771>0.05 (DCNN with ETS vs. pathologist 1), P₂=0.489>0.05 (DCNN with ETS vs. pathologist 2). The DCNN model was almost 6-fold faster than that of the pathologists. The accuracy of our automated scheme was similar to that of the pathologists, but a higher efficiency in the accurate identification of cervical squamous lesions was provided by the scheme. This result allows for wider and more efficient screening and may provide a replacement for pathologists in the future. Future research should address the viability of the practical implementation of such DCNN models in the laboratory setting.

Methods for training collaborative biostatisticians

The emphasis on team science in clinical and translational research increases the importance of collaborative biostatisticians (CBs) in healthcare. Adequate training and development of CBs ensure appropriate conduct of robust and meaningful research and, therefore, should be considered as a high-priority focus for biostatistics groups. Comprehensive training enhances clinical and translational research by facilitating more productive and efficient collaborations. While many graduate programs in Biostatistics and Epidemiology include training in research collaboration, it is often limited in scope and duration. Therefore, additional training is often required once a CB is hired into a full-time position. This article presents a comprehensive CB training strategy that can be adapted to any collaborative biostatistics group. This strategy follows a roadmap of the biostatistics collaboration process, which is also presented. A TIE approach (Teach the necessary skills, monitor the Implementation of these skills, and Evaluate the proficiency of these skills) was developed to support the adoption of key principles. The training strategy also incorporates a "train the trainer" approach to enable CBs who have successfully completed training to train new staff or faculty.

Divide and Conquer: Stratifying Training Data by Tumor Grade Improves Deep Learning-Based Brain Tumor Segmentation

It is a general assumption in deep learning that more training data leads to better performance, and that models will learn to generalize well across heterogeneous input data as long as that variety is represented in the training set. Segmentation of brain tumors is a well-investigated topic in medical image computing, owing primarily to the availability of a large publicly-available dataset arising from the long-running yearly Multimodal Brain Tumor Segmentation (BraTS) challenge. Research efforts and publications addressing this dataset focus predominantly on technical improvements of model architectures and less on properties of the underlying data. Using the dataset and the method ranked third in the BraTS 2018 challenge, we performed experiments to examine the impact of tumor type on segmentation performance. We propose to stratify the training dataset into high-grade glioma (HGG) and low-grade glioma (LGG) subjects and train two separate models. Although we observed only minor gains in overall mean dice scores by this stratification, examining case-wise rankings of individual subjects revealed statistically significant improvements. Compared to a baseline model trained on both HGG and LGG cases, two separately trained models led to better performance in 64.9% of cases (p < 0.0001) for the tumor core. An analysis of subjects which did not profit from stratified training revealed that cases were missegmented which had poor image quality, or which presented clinically particularly challenging cases (e.g., underrepresented subtypes such as IDH1-mutant tumors), underlining the importance of such latent variables in the context of tumor segmentation. In summary, we found that segmentation models trained on the BraTS 2018 dataset, stratified according to tumor type, lead to a significant increase in segmentation performance. Furthermore, we demonstrated that this gain in segmentation performance is evident in the case-wise ranking of individual subjects but not in summary statistics. We conclude that it may be useful to consider the segmentation of brain tumors of different types or grades as separate tasks, rather than developing one tool to segment them all. Consequently, making this information available for the test data should be considered, potentially leading to a more clinically relevant BraTS competition.

The strategy of training staff for a new type of helicopter as an element of raising the security level of flight operations

In 2008, the Polish Medical Air Rescue started replacing its fleet with modern EC135 machines. To ensure the maximum possible safety of the missions performed both in the period of implementing the change and later on, the management prepared a strategy of training its crews to use the new type of helicopter. The analysis of incidents that occurred during 2006–2009 showed that both the human and the technical factors must be carefully considered. Moreover, a risk analysis was conducted to reduce the risk both during general crew training and in the course of particular flight operations. A four-stage strategy of training pilots and crew members was worked out by weighing up all the risks. The analysis of data from 2010 to 2013 confirmed that the risk connected with flying and with all the activities involved in direct support aircraft operations is under control and lowered to an acceptable level.