Cross-scale multi-instance learning for pathological image diagnosis

Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20× magnification) of WSIs, disregarding the vital inter-scale information that is key to diagnoses by human pathologists. In this study, we propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis. The contribution of this paper is three-fold: (1) A novel cross-scale MIL (CS-MIL) algorithm that integrates the multi-scale information and the inter-scale relationships is proposed; (2) A toy dataset with scale-specific morphological features is created and released to examine and visualize differential cross-scale attention; (3) Superior performance on both in-house and public datasets is demonstrated by our simple cross-scale MIL strategy. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.

An end-to-end multi-scale airway segmentation framework based on pulmonary CT image

OBJECTIVE

Automatic and accurate airway segmentation is necessary for lung disease diagnosis. The complex tree-like structures leads to gaps in the different generations of the airway tree, and thus airway segmentation is also considered to be a multi-scale problem. In recent years, convolutional neural networks have facilitated the development of medical image segmentation. In particular, 2D CNNs and 3D CNNs can extract different scale features. Hence, we propose a two-stage and 2D+3D framework for multi-scale airway tree segmentation.

APPROACH

In stage 1, we use a 2D Full Airway SegNet(2D FA-SegNet) to segment the complete airway tree. Multi-scale Atros Spatial Pyramid (MASP) and Atros Residual Skip connection (ARSc) modules are inserted to extract different scales feature. We designed a hard sample selection strategy to increase the proportion of intrapulmonary airway samples in stage 2. 3D Airway RefineNet (3D ARNet) as stage 2 takes the results of stage 1 as a priori information. Spatial information extracted by 3D convolutional kernel compensates for the loss of in 2D FA-SegNet. Furthermore, we added False Positive losses and False Negative losses to improve the segmentation performance of airway branches within the lungs.

MAIN RESULTS

We performed data enhancement on the publicly available dataset of ISICDM 2020 Challenge 3, and on which evaluated our method. Comprehensive experiments show that the proposed method has the highest DSC of 0.931, and IoU of 0.871 for the whole airway tree and DSC of 0.699, and IoU of 0.543 for the intrapulmonary bronchi tree. In addition, 3D ARNet proposed in this paper cascaded with other State-Of-The-Art methods to increase DLR by up to 46.33% and DBR by up to 42.97%.

SIGNIFICANCE

The quantitative and qualitative evaluation results show that our proposed method performs well in segmenting the airway at different scales.

GMAlignNet: multi-scale lightweight brain tumor image segmentation with enhanced semantic information consistency

Although the U-shaped architecture, represented by UNet, has become a major network model for brain tumor segmentation, the repeated convolution and sampling operations can easily lead to the loss of crucial information. Additionally, directly fusing features from different levels without distinction can easily result in feature misalignment, affecting segmentation accuracy. On the other hand, traditional convolutional blocks used for feature extraction cannot capture the abundant multi-scale information present in brain tumor images. This paper proposes a multi-scale feature-aligned segmentation model called GMAlignNet that fully utilizes Ghost convolution to solve these problems. Ghost Hierarchical Decoupled Fusion Unit and Ghost Hierarchical Decoupled Unit are used instead of standard convolutions in the encoding and decoding paths. This transformation replaces the holistic learning of volume structures by traditional convolutional blocks with multi-level learning on a specific view, facilitating the acquisition of abundant multi-scale contextual information through low-cost operations. Furthermore, a feature alignment unit is proposed that can utilize semantic information flow to guide the recovery of upsampled features. It performs pixel-level semantic information correction on misaligned features due to feature fusion. The proposed method is also employed to optimize three classic networks, namely DMFNet, HDCNet, and 3D UNet, demonstrating its effectiveness in automatic brain tumor segmentation. The proposed network model was applied to the BraTS 2018 dataset, and the results indicate that the proposed GMAlignNet achieved Dice coefficients of 81.65%, 90.07%, and 85.16% for enhancing tumor, whole tumor, and tumor core segmentation, respectively. Moreover, with only 0.29M parameters and 26.88G FLOPs, it demonstrates better potential in terms of computational efficiency and possesses the advantages of lightweight. Extensive experiments on the BraTS 2018, BraTS 2019, and BraTS 2020 datasets suggest that the proposed model exhibits better potential in handling edge details and contour recognition.

A review of social-ecological system vulnerability in desertified regions: Assessment, simulation, and sustainable management

Desertified regions face considerable vulnerability due to the combined effects of climate change and human activities, which threaten regional ecological security and societal development. It is therefore necessary to assess, simulate, and manage the vulnerability of desertified regions from the perspective of the social-ecological system, to support desertification control and sustainable development. This study is a systematic review of the vulnerability of the social-ecological system in desertified regions (SESDR) based on a bibliometric analysis, and a summary of the research progresses in vulnerability assessment, simulation, and sustainable management is provided. It was found that SESDR vulnerability research started relatively late, but has developed rapidly in recent years, with an emphasis on the coupling between natural systems and human activities, and multi-scale interactions and dynamics. Using various indicators at different scales, SESDR vulnerability could be assessed in terms of exposure, sensitivity, and adaptability. Modeling the complex interactions among natural and human factors across multiple scales is essential to simulate the vulnerability dynamics of the SESDR. The sustainable management of SESDR vulnerability focuses on rational spatial planning to achieve the maximum benefits, with the right measures in the right places. Four priority research directions were proposed to develop a better understanding of the mechanisms of vulnerability and smart restoration of desertified land. The findings of this study will enable researchers, land managers, and policymakers to develop a more comprehensive understanding of SESDR vulnerability, thereby enabling them to better address the challenges posed by complex resource and environmental issues.

MRUNet-3D: A multi-stride residual 3D UNet for lung nodule segmentation

Obtaining an accurate segmentation of the pulmonary nodules in computed tomography (CT) images is challenging. This is due to: (1) the heterogeneous nature of the lung nodules; (2) comparable visual characteristics between the nodules and their surroundings. A robust multi-scale feature extraction mechanism that can effectively obtain multi-scale representations at a granular level can improve segmentation accuracy. As the most commonly used network in lung nodule segmentation, UNet, its variants, and other image segmentation methods lack this robust feature extraction mechanism. In this study, we propose a multi-stride residual 3D UNet (MRUNet-3D) to improve the segmentation accuracy of lung nodules in CT images. It incorporates a multi-slide Res2Net block (MSR), which replaces the simple sequence of convolution layers in each encoder stage to effectively extract multi-scale features at a granular level from different receptive fields and resolutions while conserving the strengths of 3D UNet. The proposed method has been extensively evaluated on the publicly available LUNA16 dataset. Experimental results show that it achieves competitive segmentation performance with an average dice similarity coefficient of 83.47 % and an average surface distance of 0.35 mm on the dataset. More notably, our method has proven to be robust to the heterogeneity of lung nodules. It has also proven to perform better at segmenting small lung nodules. Ablation studies have shown that the proposed MSR and RFIA modules are fundamental to improving the performance of the proposed model.

Tailoring left ventricular assist device cannula implantation using coupled multi-scale multi-objective optimization

BACKGROUND

The frequent occurrence of thromboembolic cerebral events continues to limit the widespread implementation of Ventricular Assist Devices (VAD) despite continued advancements in VAD design and anti-coagulation treatments. Recent studies point to the optimal positioning of the outflow graft (OG) as a potential mitigator of post implantation thromboembolism.

OBJECTIVE

This study aims to examine the tailoring of the OG implantation orientation with the goal of minimizing the number of thrombi reaching the cerebral vessels by means of a formal shape optimization scheme incorporated into a multi-scale hemodynamics analysis.

METHODS

A 3-D patient-specific computational fluid dynamics model is loosely coupled in a two-way manner to a 0-D lumped parameter model of the peripheral circulation. A Lagrangian particle-tracking scheme models and tracks thrombi as non-interacting solid spheres. The loose coupling between CFD and LPM is integrated into a geometric shape optimization scheme which aims to optimize an objective function that targets a drop in cerebral embolization, and an overall reduction in particle residence times.

RESULTS

The results elucidate the importance of OG anastomosis orientation and placement particularly in the case that studied particle release from the OG, as a fivefold decrease in cerebral embolization was observed between the optimal and non-optimal implantations. Another case considered particle release from the ventricle and aortic root walls, in which optimal implantation was achieved with a shallow insertion angle. Particle release from all three origins was investigated in the third case, demonstrating that the optimal configurations were generally characterized by VAD flow directed along the central lumen of the aortic arch. Because optimal configurations depended on the anatomic origin of the thrombus, it is important to determine, in clinical studies, the most likely sites of thrombus formation in VAD patients.

FAFS-UNet: Redesigning skip connections in UNet with feature aggregation and feature selection

In recent years, the encoder-decoder U-shaped network architecture has become a mainstream structure for medical image segmentation. Its biggest advantage lies in the incorporation of shallow features into deeper layers of the network through skip connections. However, according to our research, there are still some limitations in the skip connection part of the network: (1) The information from the encoder stage is not completely and effectively supplemented to the decoder stage; (2) The decoder receives the supplemented feature information from the encoder indiscriminately, which sometimes leads to the poor performance of the model. Therefore, to effectively address these limitations, we have redesigned the skip connections in UNet using a feature aggregation and feature selection approach. We firstly design the FA module to aggregate all encoder features and perform local multi-scale information extraction to obtain the complete multi-scale aggregated features. Further, we design the FS module to actively perform specific selection of these aggregated features through the decoder, thus effectively guiding the semantic recovery of the decoder. Finally, we conduct experiments on several medical image datasets, and the results show that our method has higher segmentation accuracy compared with other methods.

[Multi-modal physiological time-frequency feature extraction network for accurate sleep stage classification]

Sleep stage classification is essential for clinical disease diagnosis and sleep quality assessment. Most of the existing methods for sleep stage classification are based on single-channel or single-modal signal, and extract features using a single-branch, deep convolutional network, which not only hinders the capture of the diversity features related to sleep and increase the computational cost, but also has a certain impact on the accuracy of sleep stage classification. To solve this problem, this paper proposes an end-to-end multi-modal physiological time-frequency feature extraction network (MTFF-Net) for accurate sleep stage classification. First, multi-modal physiological signal containing electroencephalogram (EEG), electrocardiogram (ECG), electrooculogram (EOG) and electromyogram (EMG) are converted into two-dimensional time-frequency images containing time-frequency features by using short time Fourier transform (STFT). Then, the time-frequency feature extraction network combining multi-scale EEG compact convolution network (Ms-EEGNet) and bidirectional gated recurrent units (Bi-GRU) network is used to obtain multi-scale spectral features related to sleep feature waveforms and time series features related to sleep stage transition. According to the American Academy of Sleep Medicine (AASM) EEG sleep stage classification criterion, the model achieved 84.3% accuracy in the five-classification task on the third subgroup of the Institute of Systems and Robotics of the University of Coimbra Sleep Dataset (ISRUC-S3), with 83.1% macro F1 score value and 79.8% Cohen's Kappa coefficient. The experimental results show that the proposed model achieves higher classification accuracy and promotes the application of deep learning algorithms in assisting clinical decision-making.

MulCNN-HSP: A multi-scale convolutional neural networks-based deep learning method for classification of heat shock proteins

Heat shock proteins (HSPs) are crucial cellular stress proteins that react to environmental cues, ensuring the preservation of cellular functions. They also play pivotal roles in orchestrating the immune response and participating in processes associated with cancer. Consequently, the classification of HSPs holds immense significance in enhancing our understanding of their biological functions and in various diseases. However, the use of computational methods for identifying and classifying HSPs still faces challenges related to accuracy and interpretability. In this study, we introduced MulCNN-HSP, a novel deep learning model based on multi-scale convolutional neural networks, for identifying and classifying of HSPs. Comparative results showed that MulCNN-HSP outperforms or matches existing models in the identification and classification of HSPs. Furthermore, MulCNN-HSP can extract and analyze essential features for the prediction task, enhancing its interpretability. To facilitate its accessibility, we have made MulCNN-HSP available at http://cbcb.cdutcm.edu.cn/HSP/. We hope that MulCNN-HSP will contribute to advancing the study of HSPs and their roles in various biological processes and diseases.

TiCNet: Transformer in Convolutional Neural Network for Pulmonary Nodule Detection on CT Images

Lung cancer is the leading cause of cancer death. Since lung cancer appears as nodules in the early stage, detecting the pulmonary nodules in an early phase could enhance the treatment efficiency and improve the survival rate of patients. The development of computer-aided analysis technology has made it possible to automatically detect lung nodules in Computed Tomography (CT) screening. In this paper, we propose a novel detection network, TiCNet. It is attempted to embed a transformer module in the 3D Convolutional Neural Network (CNN) for pulmonary nodule detection on CT images. First, we integrate the transformer and CNN in an end-to-end structure to capture both the short- and long-range dependency to provide rich information on the characteristics of nodules. Second, we design the attention block and multi-scale skip pathways for improving the detection of small nodules. Last, we develop a two-head detector to guarantee high sensitivity and specificity. Experimental results on the LUNA16 dataset and PN9 dataset showed that our proposed TiCNet achieved superior performance compared with existing lung nodule detection methods. Moreover, the effectiveness of each module has been proven. The proposed TiCNet model is an effective tool for pulmonary nodule detection. Validation revealed that this model exhibited excellent performance, suggesting its potential usefulness to support lung cancer screening.

MSCAN: multi-scale self- and cross-attention network for RNA methylation site prediction

BACKGROUND

Epi-transcriptome regulation through post-transcriptional RNA modifications is essential for all RNA types. Precise recognition of RNA modifications is critical for understanding their functions and regulatory mechanisms. However, wet experimental methods are often costly and time-consuming, limiting their wide range of applications. Therefore, recent research has focused on developing computational methods, particularly deep learning (DL). Bidirectional long short-term memory (BiLSTM), convolutional neural network (CNN), and the transformer have demonstrated achievements in modification site prediction. However, BiLSTM cannot achieve parallel computation, leading to a long training time, CNN cannot learn the dependencies of the long distance of the sequence, and the Transformer lacks information interaction with sequences at different scales. This insight underscores the necessity for continued research and development in natural language processing (NLP) and DL to devise an enhanced prediction framework that can effectively address the challenges presented.

RESULTS

This study presents a multi-scale self- and cross-attention network (MSCAN) to identify the RNA methylation site using an NLP and DL way. Experiment results on twelve RNA modification sites (m6A, m1A, m5C, m5U, m6Am, m7G, Ψ, I, Am, Cm, Gm, and Um) reveal that the area under the receiver operating characteristic of MSCAN obtains respectively 98.34%, 85.41%, 97.29%, 96.74%, 99.04%, 79.94%, 76.22%, 65.69%, 92.92%, 92.03%, 95.77%, 89.66%, which is better than the state-of-the-art prediction model. This indicates that the model has strong generalization capabilities. Furthermore, MSCAN reveals a strong association among different types of RNA modifications from an experimental perspective. A user-friendly web server for predicting twelve widely occurring human RNA modification sites (m6A, m1A, m5C, m5U, m6Am, m7G, Ψ, I, Am, Cm, Gm, and Um) is available at http://47.242.23.141/MSCAN/index.php .

CONCLUSIONS

A predictor framework has been developed through binary classification to predict RNA methylation sites.

m5UMCB: Prediction of RNA 5-methyluridine sites using multi-scale convolutional neural network with BiLSTM

As a prevalent RNA modification, 5-methyluridine (m5U) plays a critical role in diverse biological processes and disease pathogenesis. High-throughput identification of m5U typically relies on labor-intensive biochemical experiments using various sequencing-based techniques, which are not only time-consuming but also expensive. Consequently, there is a pressing need for more efficient and cost-effective computational methods to complement these high-throughput techniques. In this study, we present m5UMCB, a novel approach that harnesses a multi-scale convolutional neural network (CNN) in tandem with bidirectional long short-term memory (BiLSTM) to recognize m5U sites. Our method involves segmenting RNA sequences into smaller fragments based on a 3-mer length and subsequently mapping each fragment to a lower-dimensional vector representation using the global vectors for word representation (GloVe) technique. Through a series of multi-scale convolution and pooling operations, local features are extracted from RNA sequences and transformed into abstract, high-level features. The feature matrix is then inputted into a BiLSTM network, enabling the capture of contextual information and long-term dependencies within the sequence. Ultimately, a fully connected layer is employed to classify m5U sites. The validation results from 5-fold cross-validation (5-fold CV) test indicate that m5UMCB outperforms existing state-of-the-art predictive methods, demonstrating a 1.98% increase in the area under ROC curve (AUC) and significant improvements in relevant evaluation metrics. We are confident that m5UMCB will serve as a valuable tool for m5U prediction.

MST-net: A multi-scale swin transformer network for EEG-based cognitive load assessment

Cognitive load assessment plays a crucial role in monitoring safe production, resource allocation, and subjective initiative in human-computer interaction. Due to its high time resolution and convenient acquisition, Electroencephalography (EEG) is widely applied in brain monitoring and cognitive state assessment. In this study, a multi-scale Swin Transformer network (MST-Net) was proposed for cognitive load assessment, which extracts local features with different sensory fields using a multi-scale parallel convolution model and introduces the attention mechanism of the Swin Transformer to obtain the feature correlations among multi-scale local features. The performance of the proposed network was validated using the EEG signals collected during cognitive tasks and N-back tasks with three different load levels. Results show that the MST-Net network achieved the best classification accuracy on both local and public datasets, and was higher than the mainstream Swin Transformer and CNN. Furthermore, results of ablation experiments and feature visualization revealed that the proposed MST-Net could well characterize different cognitive loads, which not only provided novel and powerful tools for cognitive load assessment but also showed potential for broad application in brain-computer interface (BCI) systems.

Scale-dependent responses of ecosystem service trade-offs to urbanization in Erhai Lake Basin, China

Urbanization is an important factor affecting ecosystem services (ESs) and their trade-offs. However, little is known about the responses of ES trade-offs to urbanization at different scales. Here, the Integrated Valuation of Ecosystem Services and Trade-offs (InVEST) model was used to evaluate water yield (WY), water purification (WP), carbon storage (CS), and habitat quality (HQ) in Erhai Lake Basin using earth observation data, and the percentage of urban land (PUL), population density (POP), gross domestic product (GDP), and night light index (NLI) were used as urbanization indicators. We quantified the ES trade-offs using the root mean square error and analyzed spatiotemporal changes in urbanization indicators, ESs, and their trade-offs. Finally, we characterized the relationship between urbanization and ES trade-offs using correlation analysis and curve regression at the grid and town scales. From 2000 to 2020, values of PUL/GDP/NLI/POP were high in the south and low in the north; specifically, they were 15, 8, 2, and 0.42 times higher in the south than in the north, respectively. The urban expansion area in the Erhai Basin from 2000 to 2020 resulted in a 123.24% and 77.03% increase in WY and WP, respectively, and a 32.38% and 100% decrease in CS and HQ, respectively. The trade-offs between WY and CS and between WY and HQ increased, and other ES trade-offs decreased. Urbanization was significantly correlated with most ES trade-offs at the grid scale, but not at the town scale. There was a significant positive correlation between all urbanization indicators and the trade-off between CS and WP (p < 0.05), and the magnitude of the correlation increased with scale. The relationship between ES trade-offs and urbanization was mostly U-shaped and inverted U-shaped at the grid scale, but N-shaped and inverted N-shaped at the town scale. This study provides information that could be used for multi-scale urban planning.

4mCFSNet: A Feature Fusion-Based Multi-Scale Representations for Predicting DNA N4-Methylcytosine Sites

N4-methylcytosine (4mC) is a very important epigenetic modification that regulates DNA expression, repair and replication. Traditional experimental methods for 4mC site detection are both time consuming and laborious. Therefore, the development of computational methods is necessary. But mining the internal information of DNA sequences remains a great challenge. In this paper, we propose a novel 4mC deep learning prediction method, named 4mCFSNet. Firstly, we encode the sequences using one-hot. Secondly, we construct multi-scale fusion modules to fully extract biological sequence information by overlapping multi-scale channel input features. Finally, we use fully connected layers and class weights for multi-species classification prediction. The average MCC of our proposed method on six species is about 2% higher than the optimal method, and the average ACC is about 1% higher.

A multi-scale self-supervised hypergraph contrastive learning framework for video question answering

Video question answering (VideoQA) is a challenging video understanding task that requires a comprehensive understanding of multimodal information and accurate answers to related questions. Most existing VideoQA models use Graph Neural Networks (GNN) to capture temporal-spatial interactions between objects. Despite achieving certain success, we argue that current schemes have two limitations: (i) existing graph-based methods require stacking multi-layers of GNN to capture high-order relations between objects, which inevitably introduces irrelevant noise; (ii) neglecting the unique self-supervised signals in the high-order relational structures among multiple objects that can facilitate more accurate QA. To this end, we propose a novel Multi-scale Self-supervised Hypergraph Contrastive Learning (MSHCL) framework for VideoQA. Specifically, we first segment the video from multiple temporal dimensions to obtain multiple frame groups. For different frame groups, we design appearance and motion hyperedges based on node semantics to connect object nodes. In this way, we construct a multi-scale temporal-spatial hypergraph to directly capture high-order relations among multiple objects. Furthermore, the node features after hypergraph convolution are injected into a Transformer to capture the global information of the input sequence. Second, we design a self-supervised hypergraph contrastive learning task based on the node- and hyperedge-dropping data augmentation and an improved question-guided multimodal interaction module to enhance the accuracy and robustness of the VideoQA model. Finally, extensive experiments on three benchmark datasets demonstrate the superiority of our proposed MSHCL compared with stat-of-the-art methods.

A multiscale orchestrated computational framework to reveal emergent phenomena in neuroblastoma

Neuroblastoma is a complex and aggressive type of cancer that affects children. Current treatments involve a combination of surgery, chemotherapy, radiotherapy, and stem cell transplantation. However, treatment outcomes vary due to the heterogeneous nature of the disease. Computational models have been used to analyse data, simulate biological processes, and predict disease progression and treatment outcomes. While continuum cancer models capture the overall behaviour of tumours, and agent-based models represent the complex behaviour of individual cells, multiscale models represent interactions at different organisational levels, providing a more comprehensive understanding of the system. In 2018, the PRIMAGE consortium was formed to build a cloud-based decision support system for neuroblastoma, including a multi-scale model for patient-specific simulations of disease progression. In this work we have developed this multi-scale model that includes data such as patient's tumour geometry, cellularity, vascularization, genetics and type of chemotherapy treatment, and integrated it into an online platform that runs the simulations on a high-performance computation cluster using Onedata and Kubernetes technologies. This infrastructure will allow clinicians to optimise treatment regimens and reduce the number of costly and time-consuming clinical trials. This manuscript outlines the challenging framework's model architecture, data workflow, hypothesis, and resources employed in its development.

MM-GLCM-CNN: A multi-scale and multi-level based GLCM-CNN for polyp classification

Distinguishing malignant from benign lesions has significant clinical impacts on both early detection and optimal management of those early detections. Convolutional neural network (CNN) has shown great potential in medical imaging applications due to its powerful feature learning capability. However, it is very challenging to obtain pathological ground truth, addition to collected in vivo medical images, to construct objective training labels for feature learning, leading to the difficulty of performing lesion diagnosis. This is contrary to the requirement that CNN algorithms need a large number of datasets for the training. To explore the ability to learn features from small pathologically-proven datasets for differentiation of malignant from benign polyps, we propose a Multi-scale and Multi-level based Gray-level Co-occurrence Matrix CNN (MM-GLCM-CNN). Specifically, instead of inputting the lesions' medical images, the GLCM, which characterizes the lesion heterogeneity in terms of image texture characteristics, is fed into the MM-GLCN-CNN model for the training. This aims to improve feature extraction by introducing multi-scale and multi-level analysis into the construction of lesion texture characteristic descriptors (LTCDs). To learn and fuse multiple sets of LTCDs from small datasets for lesion diagnosis, we further propose an adaptive multi-input CNN learning framework. Furthermore, an Adaptive Weight Network is used to highlight important information and suppress redundant information after the fusion of the LTCDs. We evaluated the performance of MM-GLCM-CNN by the area under the receiver operating characteristic curve (AUC) merit on small private lesion datasets of colon polyps. The AUC score reaches 93.99% with a gain of 1.49% over current state-of-the-art lesion classification methods on the same dataset. This gain indicates the importance of incorporating lesion characteristic heterogeneity for the prediction of lesion malignancy using small pathologically-proven datasets.

Fully feature fusion based neural network for COVID-19 lesion segmentation in CT images

Coronavirus Disease 2019 (COVID-19) spreads around the world, seriously affecting people's health. Computed tomography (CT) images contain rich semantic information as an auxiliary diagnosis method. However, the automatic segmentation of COVID-19 lesions in CT images faces several challenges, including inconsistency in size and shape of the lesion, the high variability of the lesion, and the low contrast of pixel values between the lesion and normal tissue surrounding the lesion. Therefore, this paper proposes a Fully Feature Fusion Based Neural Network for COVID-19 Lesion Segmentation in CT Images (F3-Net). F3-Net uses an encoder-decoder architecture. In F3-Net, the Multiple Scale Module (MSM) can sense features of different scales, and Dense Path Module (DPM) is used to eliminate the semantic gap between features. The Attention Fusion Module (AFM) is the attention module, which can better fuse the multiple features. Furthermore, we proposed an improved loss function L o s s C o v i d - B C E that pays more attention to the lesions based on the prior knowledge of the distribution of COVID-19 lesions in the lungs. Finally, we verified the superior performance of F3-Net on a COVID-19 segmentation dataset, experiments demonstrate that the proposed model can segment COVID-19 lesions more accurately in CT images than benchmarks of state of the art.

Lightweight image de-snowing: A better trade-off between network capacity and performance

The single image de-snowing task is an essential topic in computer vision, as images captured on snowy days degrade the performance of current vision-based intelligent systems. Existing methods build complex network structures with numerous parameters to pursue continuous performance improvement. Nonetheless, they generally ignore the negative impact of large memory consumption in real applications. This paper aims to address the above problem by making a trade-off between network capacity and performance. We propose two novel networks suitable for different application scenarios. For devices with small memory and requiring fast inference speed, we propose an extremely lightweight recursive network (XLRNet). XLRNet is constructed by a single recursive strategy and two novel lightweight modules. For devices with large memory and pursuing better de-snowing performance, we propose a coupled lightweight dual recursive network (CLDRNet). CLDRNet cascades two XLRNets by a novel dual recursive strategy and a novel dual coupled LSTM module (DC-LSTM). Extensive experiments demonstrate the effectiveness and superiority of our two models on three synthetic datasets and real-world datasets.

Multi-Scale Feature Fusion Network for Low-Dose CT Denoising

Computed tomography (CT) is an imaging technique extensively used in medical treatment, but too much radiation dose in a CT scan will cause harm to the human body. Decreasing the dose of radiation will result in increased noise and artifacts in the reconstructed image, blurring the internal tissue and edge details. To get high-quality CT images, we present a multi-scale feature fusion network (MSFLNet) for low-dose CT (LDCT) denoising. In our MSFLNet, we combined multiple feature extraction modules, effective noise reduction modules, and fusion modules constructed using the attention mechanism to construct a horizontally connected multi-scale structure as the overall architecture of the network, which is used to construct different levels of feature maps at all scales. We innovatively define a composite loss function composed of pixel-level loss based on MS-SSIM-L1 and edge-based edge loss for LDCT denoising. In short, our approach learns a rich set of features that combine contextual information from multiple scales while maintaining the spatial details of denoised CT images. Our laboratory results indicate that compared with the existing methods, the peak signal-to-noise ratio (PSNR) value of CT images of the AAPM dataset processed by the new model is 33.6490, and the structural similarity (SSIM) value is 0.9174, which also achieves good results on the Piglet dataset with different doses. The results also show that the method removes noise and artifacts while effectively preserving CT images' architecture and grain information.

COVID-19 dynamics and immune response: Linking within-host and between-host dynamics

The global impact of COVID-19 has led to the development of numerous mathematical models to understand and control the pandemic. However, these models have not fully captured how the disease's dynamics are influenced by both within-host and between-host factors. To address this, a new mathematical model is proposed that links these dynamics and incorporates immune response. The model is compartmentalized with a fractional derivative in the sense of Caputo-Fabrizio, and its properties are studied to show a unique solution. Parameter estimation is carried out by fitting real-life data, and sensitivity analysis is conducted using various methods. The model is then numerically implemented to demonstrate how the dynamics within infected hosts drive human-to-human transmission, and various intervention strategies are compared based on the percentage of averted deaths. The simulations suggest that a combination of medication to boost the immune system, prevent infected cells from producing the virus, and adherence to COVID-19 protocols is necessary to control the spread of the virus since no single intervention strategy is sufficient.

Scale-dependent and driving relationships between spatial features and carbon storage and sequestration in an urban park, in Zhengzhou, China

Research indicates that urban ecosystems can store large amounts of carbon. However, few studies have examined how the spatial features of park greenspace affect its carbon-carrying capacity, and how those effects vary with the spatial scale. Lidar point clouds and remote sensing images were extracted for the 196 ha green space in the China Green Expo to study carbon storage and sequestration in parks. Full subset regression, stepwise regression, HP analysis, and structural equation modeling were used to examine the scale dependency and the driving relationship between carbon storage and carbon sequestration in parks. The results show that the optimal statistical sample diameters for carbon density and carbon sequestration density in parks are 100 m. Under the influence of impermeable surfaces and water bodies, the statistical values of carbon density were minimized when the sample plot diameter was 700 m. Biodiversity and forest structure are the main drivers of carbon density, with the influence of water bodies being more prominent on a larger scale. Texture characteristics explain more carbon density than the vegetation index, and RVI could better explain the variation of carbon sequestration than NDVI. This study explores scaled changes in carbon density, carbon sequestration density in parks, and their driving relationships, which can aid in developing carbon sequestration strategies based on parks.

Controlled formation of multiple core-shell structures in metal-organic frame materials for efficient microwave absorption

The design of metal-organic frameworks (MOF) derived composites with multiple loss mechanisms and multi-scale micro/nano structures is an important research direction of microwave absorbing materials. Herein, multi-scale bayberry-like Ni-MOF@N-doped carbon composites (Ni-MOF@NC) are obtained by a MOF assisted strategy. By utilizing the special structure of MOF and regulating its composition, the effective improvement of Ni-MOF@NC's microwave absorption performance has been achieved. The nanostructure on the surface of core-shell Ni-MOF@NC can be regulated and N doping on carbon skeleton by adjusting the annealing temperature. The optimal reflection loss of Ni-MOF@NC is -69.6 dB at 3 mm, and the widest effective absorption bandwidth is 6.8 GHz. This excellent performance can be attributed to the strong interface polarization caused by multiple core-shell structures, the defect and dipole polarization caused by N doping, and the magnetic loss caused by Ni. Meanwhile, the coupling of magnetic and dielectric properties enhances the impedance matching of Ni-MOF@NC. The work proposes a particular idea of designing and synthesizing an applicable microwave absorption material that possesses excellent microwave absorption performance and promising application potential.

Northern spotted owl nesting habitat under high potential wildfire threats along the California Coastal Redwood Forest

Large and severe wildfires, exacerbated by climate change and human behavior, are occurring more frequently in many forests across the western United States. While wildfire is a natural part of most terrestrial ecosystems, rapidly changing fire regimes have the potential to alter habitat beyond the adaptive capabilities of species. Spatial assessments of wildfire risks to species habitat may allow managers to pinpoint locations for management activities. To illustrate this, we spatially assessed wildfire risk within habitat that supports the nesting activity of the federally threatened northern spotted owl (Strix occidentalis caurina) in the California redwood coast ecoregion. To accomplish this, we built a scale-optimized ensemble nesting habitat suitability model and identified habitat with the highest wildfire hazard potential. Percent canopy cover at 100-m scale, slope at 400-m scale, and January precipitation at 800-m scale were the most influential environmental covariates for predicting northern spotted owl nesting habitat. Nearly 60 % of nesting habitat was predicted to be at high or very high (>1986 index value) wildfire risks. We identified three areas in the Maple Creek Area of Humboldt County, Jackson State Demonstration Forest in Mendocino County, and Point Reyes National Seashore in Marin County, California with a high concentration of nesting habitat that are at a very high risk of experiencing high severity wildfires. We recommend these areas be targeted for future research to understand the impact of wildfire on northern spotted owl as well as management attention.

Lightweight multi-scale classification of chest radiographs via size-specific batch normalization

BACKGROUND AND OBJECTIVE

Convolutional neural networks are widely used to detect radiological findings in chest radiographs. Standard architectures are optimized for images of relatively small size (for example, 224 × 224 pixels), which suffices for most application domains. However, in medical imaging, larger inputs are often necessary to analyze disease patterns. A single scan can display multiple types of radiological findings varying greatly in size, and most models do not explicitly account for this. For a given network, whose layers have fixed-size receptive fields, smaller input images result in coarser features, which better characterize larger objects in an image. In contrast, larger inputs result in finer grained features, beneficial for the analysis of smaller objects. By compromising to a single resolution, existing frameworks fail to acknowledge that the ideal input size will not necessarily be the same for classifying every pathology of a scan. The goal of our work is to address this shortcoming by proposing a lightweight framework for multi-scale classification of chest radiographs, where finer and coarser features are combined in a parameter-efficient fashion.

METHODS

We experiment on CheXpert, a large chest X-ray database. A lightweight multi-resolution (224 × 224, 448 × 448 and 896 × 896 pixels) network is developed based on a Densenet-121 model where batch normalization layers are replaced with the proposed size-specific batch normalization. Each input size undergoes batch normalization with dedicated scale and shift parameters, while the remaining parameters are shared across sizes. Additional external validation of the proposed approach is performed on the VinDr-CXR data set.

RESULTS

The proposed approach (AUC 83.27±0.17, 7.1M parameters) outperforms standard single-scale models (AUC 81.76±0.18, 82.62±0.11 and 82.39±0.13 for input sizes 224 × 224, 448 × 448 and 896 × 896, respectively, 6.9M parameters). It also achieves a performance similar to an ensemble of one individual model per scale (AUC 83.27±0.11, 20.9M parameters), while relying on significantly fewer parameters. The model leverages features of different granularities, resulting in a more accurate classification of all findings, regardless of their size, highlighting the advantages of this approach.

CONCLUSIONS

Different chest X-ray findings are better classified at different scales. Our study shows that multi-scale features can be obtained with nearly no additional parameters, boosting performance.

TT-Net: Tensorized Transformer Network for 3D medical image segmentation

Accurate segmentation of organs, tissues and lesions is essential for computer-assisted diagnosis. Previous works have achieved success in the field of automatic segmentation. However, there exists two limitations. (1) They are remain challenged by complex conditions, such as segmentation target is variable in location, size and shape, especially for different imaging modalities. (2) Existing transformer-based networks suffer from a high parametric complexity. To solve these limitations, we propose a new Tensorized Transformer Network (TT-Net). In this paper, (1) Multi-scale transformer with layers-fusion is proposed to faithfully capture context interaction information. (2) Cross Shared Attention (CSA) module that based on pHash similarity fusion (pSF) is well-designed to extract the global multi-variate dependency features. (3) Tensorized Self-Attention (TSA) module is proposed to deal with the large number of parameters, which can also be easily embedded into other models. In addition, TT-Net gains a good explainability through visualizing the transformer layers. The proposed method is evaluated on three widely accepted public datasets and one clinical dataset, which contains different imaging modalities. Comprehensive results show that TT-Net outperforms other state-of-the-art methods for the four different segmentation tasks. Besides, the compression module which can be easily embedded into other transformer-based methods achieves lower computation with comparable segmentation performance.

Effects of land use and slope on water quality at multi-spatial scales: a case study of the Weihe River Basin

Exploring the impact of land use and slope on basin water quality can effectively contribute to the protection of the latter at the landscape level. This research concentrates on the Weihe River Basin (WRB). Water samples were collected from 40 sites within the WRB in April and October 2021. A quantitative analysis of the relationship between integrated landscape pattern (land use type, landscape configuration, slope) and basin water quality at the sub-basin, riparian zone, and river scales was conducted based on multiple linear regression analysis (MLR) and redundancy analysis (RDA). The correlation between water quality variables and land use was higher in the dry season than in the wet season. The riparian scale was the best spatial scale model to explain the relationship between land use and water quality. Agricultural and urban lands had a strong correlation with water quality, which was most affected by land use area and morphological indicators. In addition, the greater the area and aggregation of forest land and grassland, the better the water quality, while urban land presented larger areas with poorer water quality. The influence of steeper slopes on water quality was more remarkable than that of plains at the sub-basin scale, while the impact of flatter areas was greater at the riparian zone scale. The results indicated the importance of multiple time-space scales to reveal the complex relationship between land use and water quality. We suggest that watershed water quality management should focus on multi-scale landscape planning measures.

A novel multi-attention, multi-scale 3D deep network for coronary artery segmentation

Automatic segmentation of coronary arteries provides vital assistance to enable accurate and efficient diagnosis and evaluation of coronary artery disease (CAD). However, the task of coronary artery segmentation (CAS) remains highly challenging due to the large-scale variations exhibited by coronary arteries, their complicated anatomical structures and morphologies, as well as the low contrast between vessels and their background. To comprehensively tackle these challenges, we propose a novel multi-attention, multi-scale 3D deep network for CAS, which we call CAS-Net. Specifically, we first propose an attention-guided feature fusion (AGFF) module to efficiently fuse adjacent hierarchical features in the encoding and decoding stages to capture more effectively latent semantic information. Then, we propose a scale-aware feature enhancement (SAFE) module, aiming to dynamically adjust the receptive fields to extract more expressive features effectively, thereby enhancing the feature representation capability of the network. Furthermore, we employ the multi-scale feature aggregation (MSFA) module to learn a more distinctive semantic representation for refining the vessel maps. In addition, considering that the limited training data annotated with a quality golden standard are also a significant factor restricting the development of CAS, we construct a new dataset containing 119 cases consisting of coronary computed tomographic angiography (CCTA) volumes and annotated coronary arteries. Extensive experiments on our self-collected dataset and three publicly available datasets demonstrate that the proposed method has good segmentation performance and generalization ability, outperforming multiple state-of-the-art algorithms on various metrics. Compared with U-Net3D, the proposed method significantly improves the Dice similarity coefficient (DSC) by at least 4% on each dataset, due to the synergistic effect among the three core modules, AGFF, SAFE, and MSFA. Our implementation is released at https://github.com/Cassie-CV/CAS-Net.

MsGoF: Breast lesion classification on ultrasound images by multi-scale gradational-order fusion framework

BACKGROUND AND OBJECTIVE

Predicting the malignant potential of breast lesions based on breast ultrasound (BUS) images is a crucial component of computer-aided diagnosis system for breast cancers. However, since breast lesions in BUS images generally have various shapes with relatively low contrast and present complex textures, it still remains challenging to accurately identify the malignant potential of breast lesions.

METHODS

In this paper, we propose a multi-scale gradational-order fusion framework to make full advantages of multi-scale representations incorporating with gradational-order characteristics of BUS images for breast lesions classification. Specifically, we first construct a spatial context aggregation module to generate multi-scale context representations from the original BUS images. Subsequently, multi-scale representations are efficiently fused in feature fusion block that is armed with special fusion strategies to comprehensively capture morphological characteristics of breast lesions. To better characterize complex textures and enhance non-linear modeling capability, we further propose isotropous gradational-order feature module in the feature fusion block to learn and combine multi-order representations. Finally, these multi-scale gradational-order representations are utilized to perform prediction for the malignant potential of breast lesions.

RESULTS

The proposed model was evaluated on three open datasets by using 5-fold cross-validation. The experimental results (Accuracy: 85.32%, Sensitivity: 85.24%, Specificity: 88.57%, AUC: 90.63% on dataset A; Accuracy: 76.48%, Sensitivity: 72.45%, Specificity: 80.42%, AUC: 78.98% on dataset B) demonstrate that the proposed method achieves the promising performance when compared with other deep learning-based methods in BUS classification task.

CONCLUSIONS

The proposed method has demonstrated a promising potential to predict malignant potential of breast lesion using ultrasound image in an end-to-end manner.

Prototype-guided multi-scale domain adaptation for Alzheimer's disease detection

Alzheimer's disease (AD) is the most common form of dementia and there is no effective treatment currently. Using artificial intelligence technology to assist the diagnosis and intervention as early as possible is of great significance to delay the development of AD. Structural Magnetic Resonance Imaging (sMRI) has shown great practical values on computer-aided AD diagnosis. Affected by data from different sources or acquisition domains in realistic scenarios, MRI data often suffer from domain shift problem. In this paper, we propose a deep Prototype-Guided Multi-Scale Domain Adaptation (PMDA) framework to handle MRI data with domain shift problem, and realize automatic auxiliary diagnosis of AD, Mild Cognitive Impairment (MCI) and Cognitively Normal (CN). PMDA is composed of three modules: (1) MRI multi-scale feature extraction module combines the advantages of 3D convolution and self-attention to effectively extract multi-scale features in high-dimensional space, (2) Prototype Maximum Density Divergence (Pro-MDD) module adopts prototype learning to constrain the feature outlier samples in a mini-batch when MDD is used to align source domain and target domain, and (3) Adversarial Domain Adaptation module is applied to achieve global feature alignment of the source domain and target domain and co-training two distinctive discriminators to mitigate the over-fitting issue. Experiments have been performed on 3T and 1.5T sMRI with domain shift in ADNI dataset. The experimental results demonstrated that the proposed framework PMDA outperforms supervised learning methods and several state-of-the-art domain adaptation methods and achieves a superior accuracy of 92.11%, 76.01% and 82.37% on AD vs. CN, AD vs. MCI, and MCI vs. CN tasks, respectively.

Quaternion attention multi-scale widening network for endoscopy image super-resolution

OBJECTIVE

In the field of endoscopic imaging, Super-Resolution (SR) plays an important role in Manufactured Diagnosis, physicians and machine Automatic Diagnosis. Although many recent studies have been performed, by using deep convolutional neural networks on endoscopic Super-Resolution, most of the methods have large parameters, which limits their practical application. In addition, almost all of these methods treat each channel equally based on the real-valued domain, without considering the difference among the different channels. Our objective is to design a super-resolution model named Quaternion Attention Multi-scale Widening Network (QAMWN) for endoscopy images to address the above problem.

APPROACH

QAMWN contains a stacked Quaternion Attention Multi-Scale Widening Block (QAMWB), that composed of Multi-Scale Feature Widening Aggregation Module (MFWAM) and Quaternion Residual Channel Attention (QRCA). The MFWAM adopts multi-scale architecture with step-wise widening on feature channels for better feature extraction; and in QRCA, quaternion is introduced to construct Residual Channel Attention Mechanism, which obtains adaptively scales features by considering compact cross channel interactions in the hyper-complex domain.

MAIN RESULTS

To verify the efficacy of our method, it is performed on two public endoscopic datasets, CVC ClinicDB and Kvasir dataset. The experimental results show that our proposed method can achieve a better trade-off in model size and performance. More importantly, the proposed QAMWN outperforms previous state-of-the-art methods in both metrics and visualization.

SIGNIFICANCE

We propose a lightweight super-resolution network for endoscopy and achieves better performance with fewer parameters, which helps in clinical diagnosis of endoscopy.

Super-resolution reconstruction based on Gaussian transform and attention mechanism

Image super-resolution reconstruction can reconstruct low resolution blurred images in the same scene into high-resolution images. Combined with multi-scale Gaussian difference transform, attention mechanism and feedback mechanism are introduced to construct a new super-resolution reconstruction network. Three improvements are made. Firstly, its multi-scale Gaussian difference transform can strengthen the details of low resolution blurred images. Secondly, it introduces the attention mechanism and increases the network depth to better express the high-frequency features. Finally, pixel loss function and texture loss function are used together, focusing on the learning of structure and texture respectively. The experimental results show that this method is superior to the existing methods in quantitative and qualitative indexes, and promotes the recovery of high-frequency detail information.

Effect of multiple spatial scale characterization of land use on water quality

Land use in uplands is an important factor affecting water quality in its respective catchment, and its influences at the different spatial scales and configurations warrant further investigation. Here, we selected 26 catchments in the upper Han River (China) and sampled the surface water at the outlet of each catchment in four seasons during 2019. Multivariate statistics were used to identify the relationships between land use characteristics in uplands and water quality in river system. The results indicated that chemical oxygen demand (COD_Mn); pH; dissolved oxygen; electrical conductivity; nutrient, i.e., NH₄⁺-N, NO₃^--N; and dissolved phosphorus (DP) in rivers displayed significant seasonal variations. Stepwise regression revealed that landscape metrics such as patch density, landscape shape index, and splitting index were important factors influencing water quality in rivers regardless of their spatiality and seasonality. Urban was the most frequently chosen land-use type in the best prediction models, and forest area showed a negative correlation with water quality parameters in most cases for example, DP. Overall, the influence of land use on river water quality was slightly stronger at reach scale than at catchment and riparian scales. Also, nutrients (i.e., NH₄⁺-N, NO₃^--N, and DP) in rivers were primarily impacted by the land use characteristic at catchment and riparian scales. Our results suggested that multi-scale explorations would help to achieve a fully understanding on the impacts of land use on river water quality.

Advances and perspectives on mass transfer and enzymatic hydrolysis in the enzyme-mediated lignocellulosic biorefinery: A review

Enzymatic hydrolysis is a critical process for the cellulase-mediated lignocellulosic biorefinery to produce sugar syrups that can be converted into a whole range of biofuels and biochemicals. Such a process operating at high-solid loadings (i.e., scarcely any free water or roughly ≥ 15% solids, w/w) is considered more economically feasible, as it can generate a high sugar concentration at low operation and capital costs. However, this approach remains restricted and incurs "high-solid effects", ultimately causing the lower hydrolysis yields with increasing solid loadings. The lack of available water leads to a highly viscous system with impaired mixing that exhibits strong transfer resistance and reaction limitation imposed on enzyme action. Evidently, high-solid enzymatic hydrolysis involves multi-scale mass transfer and multi-phase enzyme reaction, and thus requires a synergistic perspective of transfer and biotransformation to assess the interactions among water, biomass components, and cellulase enzymes. Porous particle characteristics of biomass and its interface properties determine the water form and distribution state surrounding the particles, which are summarized in this review aiming to identify the water-driven multi-scale/multi-phase bioprocesses. Further aided by the cognition of rheological behavior of biomass slurry, solute transfer theories, and enzyme kinetics, the coupling effects of flow-transfer-reaction are revealed under high-solid conditions. Based on the above basic features, this review lucidly explains the causes of high-solid hydrolysis hindrances, highlights the mismatched issues between transfer and reaction, and more importantly, presents the advanced strategies for transfer and reaction enhancements from the viewpoint of process optimization, reactor design, as well as enzyme/auxiliary additive customization.

Industrial gearbox fault diagnosis based on multi-scale convolutional neural networks and thermal imaging

Infrared thermal technology plays a vital role in the health condition monitoring of gearbox. In the traditional infrared thermal technology-based methods, Gaussian pyramid is applied as the feature extraction approach, which has disadvantages of noise influence and information missing. Focus on such disadvantages, an improved multi-scale decomposition method combined with convolutional neural network is proposed to extract the fault features of the multi-scale infrared images in this paper. It can enlarge the data length at large scales, and thus reduce the fluctuations of feature values and reserve the fault information. The effectiveness of the proposed method is validated using the experiment infrared data of one industrial gearbox. Results demonstrate that our proposed method has the best performance comparing with five methods.

Multi-scale variability of hydrothermal regime based on wavelet analysis - The middle reaches of the Yangtze River, China

Water temperature is a major driver of riverine ecosystems and has an extremely significant impact on them. Understanding the multi-scale water temperature dynamics in a river basin is critical to analyze the water temperature status of rivers. In this study, the intra-annual and inter-annual time series of water temperature (WT) at Yichang station in the middle reaches of the Yangtze River over the past 62 years was analyzed using complex Morlet wavelet functions to reveal the complex structure of water temperature variation at multiple time scales. The ecological impact of water temperature changes on the reproduction of the "Four Major Chinese Carp" under the influence of the Three Gorges Dam (TGD). The results showed that the water temperature at Yichang Station has a multi-level time scale structure, with an increasing trend at the inter-annual scale from 1956 to 2017, but different variations at the seasonal scale, and the water temperature cycles at both the inter-annual and seasonal scales have time scale variations of about 8-14 years and 4-7 years, with obvious characteristics of WT variation stages. The inter-annual and summer scales will have low WT in 2017-2022 and high WT in 2023-2027, while the other seasonal scales will have high WT in the next few years, either in the short or medium term. The correlation between air temperature and WT is the most significant among the three drivers of air temperature, flow and rainfall, and the correlation between WT and the air temperature is the most significant in winter scale under the influence of the Three Gorges Dam construction. Since the completion of TGD in 2003, the summer drainage temperature has decreased and the breeding period of the "Four Major Chinese Carp" has been shortened by 30-40 days compared to that before the construction of TGD. The results of this study can be used as a basis for further exploration of the formation mechanism of river water temperature and provide a scientific basis for river ecological protection.

Cross-scale Attention Guided Multi-instance Learning for Crohn's Disease Diagnosis with Pathological Images

Multi-instance learning (MIL) is widely used in the computer-aided interpretation of pathological Whole Slide Images (WSIs) to solve the lack of pixel-wise or patch-wise annotations. Often, this approach directly applies "natural image driven" MIL algorithms which overlook the multi-scale (i.e. pyramidal) nature of WSIs. Off-the-shelf MIL algorithms are typically deployed on a single-scale of WSIs (e.g., 20× magnification), while human pathologists usually aggregate the global and local patterns in a multi-scale manner (e.g., by zooming in and out between different magnifications). In this study, we propose a novel cross-scale attention mechanism to explicitly aggregate inter-scale interactions into a single MIL network for Crohn's Disease (CD), which is a form of inflammatory bowel disease. The contribution of this paper is two-fold: (1) a cross-scale attention mechanism is proposed to aggregate features from different resolutions with multi-scale interaction; and (2) differential multi-scale attention visualizations are generated to localize explainable lesion patterns. By training ~250,000 H&E-stained Ascending Colon (AC) patches from 20 CD patient and 30 healthy control samples at different scales, our approach achieved a superior Area under the Curve (AUC) score of 0.8924 compared with baseline models. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.

Digital models in biotechnology: Towards multi-scale integration and implementation

Industrial biotechnology encompasses a large area of multi-scale and multi-disciplinary research activities. With the recent megatrend of digitalization sweeping across all industries, there is an increased focus in the biotechnology industry on developing, integrating and applying digital models to improve all aspects of industrial biotechnology. Given the rapid development of this field, we systematically classify the state-of-art modelling concepts applied at different scales in industrial biotechnology and critically discuss their current usage, advantages and limitations. Further, we critically analyzed current strategies to couple cell models with computational fluid dynamics to study the performance of industrial microorganisms in large-scale bioprocesses, which is of crucial importance for the bio-based production industries. One of the most challenging aspects in this context is gathering intracellular data under industrially relevant conditions. Towards comprehensive models, we discuss how different scale-down concepts combined with appropriate analytical tools can capture intracellular states of single cells. We finally illustrated how the efforts could be used to develop digitals models suitable for both cell factory design and process optimization at industrial scales in the future.

MultiR-Net: A Novel Joint Learning Network for COVID-19 segmentation and classification

The outbreak of COVID-19 has caused a severe shortage of healthcare resources. Ground Glass Opacity (GGO) and consolidation of chest CT scans have been an essential basis for imaging diagnosis since 2020. The similarity of imaging features between COVID-19 and other pneumonia makes it challenging to distinguish between them and affects radiologists' diagnosis. Recently, deep learning in COVID-19 has been mainly divided into disease classification and lesion segmentation, yet little work has focused on the feature correlation between the two tasks. To address these issues, in this study, we propose MultiR-Net, a 3D deep learning model for combined COVID-19 classification and lesion segmentation, to achieve real-time and interpretable COVID-19 chest CT diagnosis. Precisely, the proposed network consists of two subnets: a multi-scale feature fusion UNet-like subnet for lesion segmentation and a classification subnet for disease diagnosis. The features between the two subnets are fused by the reverse attention mechanism and the iterable training strategy. Meanwhile, we proposed a loss function to enhance the interaction between the two subnets. Individual metrics can not wholly reflect network effectiveness. Thus we quantify the segmentation results with various evaluation metrics such as average surface distance, volume Dice, and test on the dataset. We employ a dataset containing 275 3D CT scans for classifying COVID-19, Community-acquired Pneumonia (CAP), and healthy people and segmented lesions in pneumonia patients. We split the dataset into 70% and 30% for training and testing. Extensive experiments showed that our multi-task model framework obtained an average recall of 93.323%, an average precision of 94.005% on the classification test set, and a 69.95% Volume Dice score on the segmentation test set of our dataset.

DMPNet: densely connected multi-scale pyramid networks for crowd counting

Crowd counting has been widely studied by deep learning in recent years. However, due to scale variation caused by perspective distortion, crowd counting is still a challenging task. In this paper, we propose a Densely Connected Multi-scale Pyramid Network (DMPNet) for count estimation and the generation of high-quality density maps. The key component of our network is the Multi-scale Pyramid Network (MPN), which can extract multi-scale features of the crowd effectively while keeping the resolution of the input feature map and the number of channels unchanged. To increase the information transfer between the network layer, we used dense connections to connect multiple MPNs. In addition, we also designed a novel loss function, which can help our model achieve better convergence. To evaluate our method, we conducted extensive experiments on three challenging benchmark crowd counting datasets. Experimental results show that compared with the state-of-the-art algorithms, DMPNet performs well in both parameters and results. The code is available at: https://github.com/lpfworld/DMPNet.

Towards bi-directional skip connections in encoder-decoder architectures and beyond

U-Net, as an encoder-decoder architecture with forward skip connections, has achieved promising results in various medical image analysis tasks. Many recent approaches have also extended U-Net with more complex building blocks, which typically increase the number of network parameters considerably. Such complexity makes the inference stage highly inefficient for clinical applications. Towards an effective yet economic segmentation network design, in this work, we propose backward skip connections that bring decoded features back to the encoder. Our design can be jointly adopted with forward skip connections in any encoder-decoder architecture forming a recurrence structure without introducing extra parameters. With the backward skip connections, we propose a U-Net based network family, namely Bi-directional O-shape networks, which set new benchmarks on multiple public medical imaging segmentation datasets. On the other hand, with the most plain architecture (BiO-Net), network computations inevitably increase along with the pre-set recurrence time. We have thus studied the deficiency bottleneck of such recurrent design and propose a novel two-phase Neural Architecture Search (NAS) algorithm, namely BiX-NAS, to search for the best multi-scale bi-directional skip connections. The ineffective skip connections are then discarded to reduce computational costs and speed up network inference. The finally searched BiX-Net yields the least network complexity and outperforms other state-of-the-art counterparts by large margins. We evaluate our methods on both 2D and 3D segmentation tasks in a total of six datasets. Extensive ablation studies have also been conducted to provide a comprehensive analysis for our proposed methods.

Two-stage streaming keyword detection and localization with multi-scale depthwise temporal convolution

A keyword spotting (KWS) system running on smart devices should accurately detect the appearances and predict the locations of predefined keywords from audio streams, with small footprint and high efficiency. To this end, this paper proposes a new two-stage KWS method which combines a novel multi-scale depthwise temporal convolution (MDTC) feature extractor and a two-stage keyword detection and localization module. The MDTC feature extractor learns multi-scale feature representation efficiently with dilated depthwise temporal convolution, modeling both the temporal context and the speech rate variation. We use a region proposal network (RPN) as the first-stage KWS. At each frame, we design multiple time regions, which all take the current frame as the end position but have different start positions. These time regions (or formally anchors) are used to indicate rough location candidates of keyword. With frame level features from the MDTC feature extractor as inputs, RPN learns to propose keyword region proposals based on the designed anchors. To alleviate the keyword/non-keyword class imbalance problem, we specifically introduce a hard example mining algorithm to select effective negative anchors in RPN training. The keyword region proposals from the first-stage RPN contain keyword location information which is subsequently used to explicitly extract keyword related sequential features to train the second-stage KWS. The second-stage system learns to classify and transform region proposal to keyword IDs and ground-truth keyword region respectively. Experiments on the Google Speech Command dataset show that the proposed MDTC feature extractor surpasses several competitive feature extractors with a new state-of-the-art command classification error rate of 1.74%. With the MDTC feature extractor, we further conduct wake-up word (WuW) detection and localization experiments on a commercial WuW dataset. Compared to a strong baseline, our proposed two-stage method achieves relatively 27-32% better false rejection rate at one false alarm per hour, while for keyword localization, the two-stage approach achieves more than 0.95 mean intersection-over-union ratio, which is clearly better than the one-stage RPN method.

The asymmetric and multi-scale volatility correlation between global oil price and economic policy uncertainty of China

With the monthly data of WTI oil price and economic policy uncertainty (EPU) of China from January 2000 to August 2020, this paper detailedly investigates the asymmetric volatility correlations between two types of EPU of China and global oil price in different time scales. The empirical results demonstrate that the volatility correlation between EPU of China and West Texas Intermediate (WTI) oil price is mainly reflected in the monetary policy uncertainty (MPU), while that of fiscal policy uncertainty (FPU) is much weaker. Specifically speaking, the volatility correlation between MPU of China and downward WTI oil price is significantly negative in the short-middle term (4-8 months) and changes to positive in the middle-long term (8-16 months), while that of upward WTI oil price is only significantly positive in the long term (16-32 months). Our findings provide a deeper understanding of the oil price-EPU correlation in China, and can be valuable guidance for diversified market participants such as government policy-makers and global investors.

Impacts of worldwide individual non-pharmaceutical interventions on COVID-19 transmission across waves and space

Governments worldwide have rapidly deployed non-pharmaceutical interventions (NPIs) to mitigate the COVID-19 pandemic. However, the effect of these individual NPI measures across space and time has yet to be sufficiently assessed, especially with the increase of policy fatigue and the urge for NPI relaxation in the vaccination era. Using the decay ratio in the suppression of COVID-19 infections and multi-source big data, we investigated the changing performance of different NPIs across waves from global and regional levels (in 133 countries) to national and subnational (in the United States of America [USA]) scales before the implementation of mass vaccination. The synergistic effectiveness of all NPIs for reducing COVID-19 infections declined along waves, from 95.4% in the first wave to 56.0% in the third wave recently at the global level and similarly from 83.3% to 58.7% at the USA national level, while it had fluctuating performance across waves on regional and subnational scales. Regardless of geographical scale, gathering restrictions and facial coverings played significant roles in epidemic mitigation before the vaccine rollout. Our findings have important implications for continued tailoring and implementation of NPI strategies, together with vaccination, to mitigate future COVID-19 waves, caused by new variants, and other emerging respiratory infectious diseases.

Colon tissue image segmentation with MWSI-NET

Developments in deep learning have resulted in computer-aided diagnosis for many types of cancer. Previously, pathologists manually performed the labeling work in the analysis of colon tissues, which is both time-consuming and labor-intensive. Results are easily affected by subjective conditions. Therefore, it is beneficial to identify the cancerous regions of colon cancer with the assistance of computer-aided technology. Pathological images are often difficult to process due to their irregularity, similarity between cancerous and non-cancerous tissues and large size. We propose a multi-scale perceptual field fusion structure based on a dilated convolutional network. Using this model, a structure of dilated convolution kernels with different aspect ratios is inserted, which can process cancerous regions of different sizes and generate larger receptive fields. Thus, the model can fuse detailed information at different scales for better semantic segmentation. Two different attention mechanisms are adopted to highlight the cancerous areas. A large, open-source dataset was used to verify improved efficacy when compared to previously disclosed methods.

Eco-efficiency in China's Loess Plateau Region and its influencing factors: a data envelopment analysis from both static and dynamic perspectives

China's Loess Plateau Region (LPR) plays a significant role in national ecological security and development. Due to the advantage that relates environment with economy, eco-efficiency has become an important indicator of sustainable analysis. Using cross-level panel data for the period 2006-2017, this paper studied LPR's static eco-efficiency and dynamic performance through a combined application of DEA super-efficient slack-based measure and Malmquist Productivity Index at multi-scales. LPR's eco-efficiency was found to experience a slight increase during the study period. The value decreased roughly from east to west, with high eco-efficiency mainly distributed in provincial cities and resource-based cities. The decomposition of the Malmquist Index indicated that technological change contributed most to the improvement of eco-efficiency in the LPR. Besides, this paper explained the variations of eco-efficiency based on the integrated input-output indicators and TOBIT regression model. Economic scale, population density, government regulation, technical innovation, and openness degree were identified as positive influencing factors, while the structure of the industry and land-use intensity were found to have negative impacts on eco-efficiency. Resource-based cities were found to have stronger potentials for eco-efficiency improvement than non-resource-based cities. This paper revealed the characteristics of LPR's eco-efficiency from three perspectives: a spatiotemporal perspective, a macro-meso-micro perspective, and a static-dynamic perspective. The findings of this study hold important implications for policy makers.

Multi-scale semi-supervised clustering of brain images: Deriving disease subtypes

Disease heterogeneity is a significant obstacle to understanding pathological processes and delivering precision diagnostics and treatment. Clustering methods have gained popularity for stratifying patients into subpopulations (i.e., subtypes) of brain diseases using imaging data. However, unsupervised clustering approaches are often confounded by anatomical and functional variations not related to a disease or pathology of interest. Semi-supervised clustering techniques have been proposed to overcome this and, therefore, capture disease-specific patterns more effectively. An additional limitation of both unsupervised and semi-supervised conventional machine learning methods is that they typically model, learn and infer from data using a basis of feature sets pre-defined at a fixed anatomical or functional scale (e.g., atlas-based regions of interest). Herein we propose a novel method, "Multi-scAle heteroGeneity analysIs and Clustering" (MAGIC), to depict the multi-scale presentation of disease heterogeneity, which builds on a previously proposed semi-supervised clustering method, HYDRA. It derives multi-scale and clinically interpretable feature representations and exploits a double-cyclic optimization procedure to effectively drive identification of inter-scale-consistent disease subtypes. More importantly, to understand the conditions under which the clustering model can estimate true heterogeneity related to diseases, we conducted extensive and systematic semi-simulated experiments to evaluate the proposed method on a sizeable healthy control sample from the UK Biobank (N = 4403). We then applied MAGIC to imaging data from Alzheimer's disease (ADNI, N = 1728) and schizophrenia (PHENOM, N = 1166) patients to demonstrate its potential and challenges in dissecting the neuroanatomical heterogeneity of common brain diseases. Taken together, we aim to provide guidance regarding when such analyses can succeed or should be taken with caution. The code of the proposed method is publicly available at https://github.com/anbai106/MAGIC.

MBANet: Multi-branch aware network for kidney ultrasound images segmentation

Due to the influence of kidney morphology, heterogeneous structure and image quality, segmenting kidney in ultrasound images is challenging. To alleviate this challenge, we proposed a novel deep neural network architecture, namely Multi-branch Aware Network (MBANet), to segment kidney accurately and robustly. MBANet mainly consists of multi-scale feature pyramid (MSFP), multi-branch encoders (MBE) and master decoder. The design of MSFP can make the network more accessible to different kinds of class details at different scales. The information exchange between MBE can reduce the loss of feature information and improve the segmentation accuracy of the network. In addition, we designed a multi-scale fusion block (MFBlock) in the MBE to further extract and fuse more refined multi-scale image information. In order to further improve the robustness of MBANet, this paper also designed a step-by-step training mechanism. We validated the proposed approach and compared to several state-of-the-art approaches on the same kidney ultrasound datasets using six quantitative metrics. The results of our method on the six indicators of pixel accuracy (PA), intersection over union (IoU), precision, recall, specificity and F1-score (F1) are 98.83%, 92.38%, 97.10%, 95.03%, 99.46% and 0.9601, respectively. Compared with the comparison method, the average values on the six indicators are improved by about 2%. The evaluation results and segmentation results demonstrate that the proposed approach achieves the best overall performance on kidney ultrasound images segmentation.

Impact of COVID-19 outbreak on multi-scale asymmetric spillovers between food and oil prices

This paper analyzes the time-frequency spillover effects between food and crude oil markets, two particularly important commodity markets, under the impact of the pandemic. Using the BK frequency domain spillover index and the rolling window method, we explore the spillover effects between the food and crude oil markets under the influence of COVID-19, and compare the changes of spillover effects in each market before and during the pandemic. Based the network connectedness method and the Bayesian structural time series method, we further reveal the changes of the pairwise spillover effects between markets on different time scales. Our study shows that the food-oil market system has the strongest spillover effect in the short term, and the spillovers during the pandemic are significantly weaker than that under the financial crisis. In addition, the pandemic has significantly increased the impact of corn on the crude oil market, but reduced its spillovers on soybeans and rice. Finally, during the COVID-19 period, the wheat market is likely to receive more spillovers from other markets, particularly corn and soybeans. These findings are of great significance for market participants with different horizons to understand the spillover effects of food and oil markets under the impact of the pandemic and to avoid the risk transmission across markets or assets.