Adaptive Relation-Aware Network for zero-shot classification

Supervised learning-based image classification in computer vision relies on visual samples containing a large amount of labeled information. Considering that it is labor-intensive to collect and label images and construct datasets manually, Zero-Shot Learning (ZSL) achieves knowledge transfer from seen categories to unseen categories by mining auxiliary information, which reduces the dependence on labeled image samples and is one of the current research hotspots in computer vision. However, most ZSL methods fail to properly measure the relationships between classes, or do not consider the differences and similarities between classes at all. In this paper, we propose Adaptive Relation-Aware Network (ARAN), a novel ZSL approach that incorporates the improved triplet loss from deep metric learning into a VAE-based generative model, which helps to model inter-class and intra-class relationships for different classes in ZSL datasets and generate an arbitrary amount of high-quality visual features containing more discriminative information. Moreover, we validate the effectiveness and superior performance of our ARAN through experimental evaluations under ZSL and more practical GZSL settings on three popular datasets AWA2, CUB, and SUN.

Three-dimensional pseudo-continuous arterial spin-labelled perfusion imaging for diagnosing upper cervical lymph node metastasis in patients with nasopharyngeal carcinoma: a whole-node histogram analysis

AIM

To evaluate whole-node histogram parameters of blood flow (BF) maps derived from three-dimensional pseudo-continuous arterial spin-labelled (3D pCASL) imaging in discriminating metastatic from benign upper cervical lymph nodes (UCLNs) for nasopharyngeal carcinoma (NPC) patients.

MATERIALS AND METHODS

Eighty NPC patients with a total of 170 histologically confirmed UCLNs (67 benign and 103 metastatic) were included retrospectively. Pre-treatment 3D pCASL imaging was performed and whole-node histogram analysis was then applied. Histogram parameters and morphological features, such as minimum axis diameter (MinAD), maximum axis diameter (MaxAD), and location of UCLNs, were assessed and compared between benign and metastatic lesions. Predictors were identified and further applied to establish a combined model by multivariate logistic regression in predicting the probability of metastatic UCLNs. Receiver operating characteristic (ROC) curves were used to analyse the diagnostic performance.

RESULTS

Metastatic UCLNs had larger MinAD and MinAD/MaxAD ratio, greater energy and entropy values, and higher incidence of level II (upper jugular group), but lower BF10th value than benign nodes (all p<0.05). MinAD, BF10th, energy, and entropy were validated as independent predictors in diagnosing metastatic UCLNs. The combined model yielded an area under the curve (AUC) of 0.932, accuracy of 84.42 %, sensitivity of 80.6 %, and specificity of 90.29 %.

CONCLUSIONS

Whole-node histogram analysis on BF maps is a feasible tool to differentiate metastatic from benign UCLNs in NPC patients, and the combined model can further improve the diagnostic efficacy.

Effect of Chemical Composition on the Thermoplastic Formability and Nanoindentation of Ti-Based Bulk Metallic Glasses

A series of Ti41Zr25Be34-xNix (x = 4, 6, 8, 10 at.%) and Ti41Zr25Be34-xCux (x = 4, 6, 8 at.%) bulk metallic glasses were investigated to examine the influence of Ni and Cu content on the viscosity, thermoplastic formability, and nanoindentation of Ti-based bulk metallic glasses. The results demonstrate that Ti41Zr25Be30Ni4 and Ti41Zr25Be26Cu8 amorphous alloys have superior thermoplastic formability among the Ti41Zr25Be34-xNix and Ti41Zr25Be34-xCux amorphous alloys due to their low viscosity in the supercooled liquid region and wider supercooled liquid region. The hardness and modulus exhibit obvious variations with increasing Ni and Cu content in Ti-based bulk metallic glasses, which can be attributed to alterations in atomic density. Optimal amounts of Ni and Cu in Ti-based bulk metallic glasses enhance thermoplastic formability and mechanical properties. The influence of Ni and Cu content on the hardness of Ti-based bulk metallic glasses is discussed from the perspective of the mean atomic distance.

GNDAN: Graph Navigated Dual Attention Network for Zero-Shot Learning

Zero-shot learning (ZSL) tackles the unseen class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Typically, to guarantee desirable knowledge transfer, a direct embedding is adopted for associating the visual and semantic domains in ZSL. However, most existing ZSL methods focus on learning the embedding from implicit global features or image regions to the semantic space. Thus, they fail to: 1) exploit the appearance relationship priors between various local regions in a single image, which corresponds to the semantic information and 2) learn cooperative global and local features jointly for discriminative feature representations. In this article, we propose the novel graph navigated dual attention network (GNDAN) for ZSL to address these drawbacks. GNDAN employs a region-guided attention network (RAN) and a region-guided graph attention network (RGAT) to jointly learn a discriminative local embedding and incorporate global context for exploiting explicit global embeddings under the guidance of a graph. Specifically, RAN uses soft spatial attention to discover discriminative regions for generating local embeddings. Meanwhile, RGAT employs an attribute-based attention to obtain attribute-based region features, where each attribute focuses on the most relevant image regions. Motivated by the graph neural network (GNN), which is beneficial for structural relationship representations, RGAT further leverages a graph attention network to exploit the relationships between the attribute-based region features for explicit global embedding representations. Based on the self-calibration mechanism, the joint visual embedding learned is matched with the semantic embedding to form the final prediction. Extensive experiments on three benchmark datasets demonstrate that the proposed GNDAN achieves superior performances to the state-of-the-art methods. Our code and trained models are available at https://github.com/shiming-chen/GNDAN.

Deep Metric Learning Based on Meta-Mining Strategy With Semiglobal Information

Recently, deep metric learning (DML) has achieved great success. Some existing DML methods propose adaptive sample mining strategies, which learn to weight the samples, leading to interesting performance. However, these methods suffer from a small memory (e.g., one training batch), limiting their efficacy. In this work, we introduce a data-driven method, meta-mining strategy with semiglobal information (MMSI), to apply meta-learning to learn to weight samples during the whole training, leading to an adaptive mining strategy. To introduce richer information than one training batch only, we elaborately take advantage of the validation set of meta-learning by implicitly adding additional validation sample information to training. Furthermore, motivated by the latest self-supervised learning, we introduce a dictionary (memory) that maintains very large and diverse information. Together with the validation set, this dictionary presents much richer information to the training, leading to promising performance. In addition, we propose a new theoretical framework that can formulate pairwise and tripletwise metric learning loss functions in a unified framework. This framework brings new insights to society and facilitates us to generalize our MMSI to many existing DML methods. We conduct extensive experiments on three public datasets, CUB200-2011, Cars-196, and Stanford Online Products (SOP). Results show that our method can achieve the state of the art or very competitive performance. Our source codes have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/MMSI.

Measurements of All-Particle Energy Spectrum and Mean Logarithmic Mass of Cosmic Rays from 0.3 to 30 PeV with LHAASO-KM2A

We present the measurements of all-particle energy spectrum and mean logarithmic mass of cosmic rays in the energy range of 0.3-30 PeV using data collected from LHAASO-KM2A between September 2021 and December 2022, which is based on a nearly composition-independent energy reconstruction method, achieving unprecedented accuracy. Our analysis reveals the position of the knee at 3.67±0.05±0.15  PeV. Below the knee, the spectral index is found to be -2.7413±0.0004±0.0050, while above the knee, it is -3.128±0.005±0.027, with the sharpness of the transition measured with a statistical error of 2%. The mean logarithmic mass of cosmic rays is almost heavier than helium in the whole measured energy range. It decreases from 1.7 at 0.3 PeV to 1.3 at 3 PeV, representing a 24% decline following a power law with an index of -0.1200±0.0003±0.0341. This is equivalent to an increase in abundance of light components. Above the knee, the mean logarithmic mass exhibits a power law trend towards heavier components, which is reversal to the behavior observed in the all-particle energy spectrum. Additionally, the knee position and the change in power-law index are approximately the same. These findings suggest that the knee observed in the all-particle spectrum corresponds to the knee of the light component, rather than the medium-heavy components.

Latent Semantic Consensus for Deterministic Geometric Model Fitting

Estimating reliable geometric model parameters from the data with severe outliers is a fundamental and important task in computer vision. This paper attempts to sample high-quality subsets and select model instances to estimate parameters in the multi-structural data. To address this, we propose an effective method called Latent Semantic Consensus (LSC). The principle of LSC is to preserve the latent semantic consensus in both data points and model hypotheses. Specifically, LSC formulates the model fitting problem into two latent semantic spaces based on data points and model hypotheses, respectively. Then, LSC explores the distributions of points in the two latent semantic spaces, to remove outliers, generate high-quality model hypotheses, and effectively estimate model instances. Finally, LSC is able to provide consistent and reliable solutions within only a few milliseconds for general multi-structural model fitting, due to its deterministic fitting nature and efficiency. Compared with several state-of-the-art model fitting methods, our LSC achieves significant superiority for the performance of both accuracy and speed on synthetic data and real images. The code will be available at https://github.com/guobaoxiao/LSC.

Weakly-Supervised RGBD Video Object Segmentation

Depth information opens up new opportunities for video object segmentation (VOS) to be more accurate and robust in complex scenes. However, the RGBD VOS task is largely unexplored due to the expensive collection of RGBD data and time-consuming annotation of segmentation. In this work, we first introduce a new benchmark for RGBD VOS, named DepthVOS, which contains 350 videos (over 55k frames in total) annotated with masks and bounding boxes. We futher propose a novel, strong baseline model - Fused Color-Depth Network (FusedCDNet), which can be trained solely under the supervision of bounding boxes, while being used to generate masks with a bounding box guideline only in the first frame. Thereby, the model possesses three major advantages: a weakly-supervised training strategy to overcome the high-cost annotation, a cross-modal fusion module to handle complex scenes, and weakly-supervised inference to promote ease of use. Extensive experiments demonstrate that our proposed method performs on par with top fully-supervised algorithms. We will open-source our project on https://github.com/yjybuaa/depthvos/ to facilitate the development of RGBD VOS.

[Preliminary study on automatic quantification and grading of leopard spots fundus based on deep learning technology]

Objective: To achieve automatic segmentation, quantification, and grading of different regions of leopard spots fundus (FT) using deep learning technology. The analysis includes exploring the correlation between novel quantitative indicators, leopard spot fundus grades, and various systemic and ocular parameters. Methods: This was a cross-sectional study. The data were sourced from the Beijing Eye Study, a population-based longitudinal study. In 2001, a group of individuals aged 40 and above were surveyed in five urban communities in Haidian District and three rural communities in Daxing District of Beijing. A follow-up was conducted in 2011. This study included individuals aged 50 and above who participated in the second 5-year follow-up in 2011, considering only the data from the right eye. Color fundus images centered on the macula of the right eye were input into the leopard spot segmentation model and macular detection network. Using the macular center as the origin, with inner circle diameters of 1 mm, 3 mm, and outer circle diameter of 6 mm, fine segmentation of the fundus was achieved. This allowed the calculation of the leopard spot density (FTD) and leopard spot grade for each region. Further analyses of the differences in ocular and systemic parameters among different regions' FTD and leopard spot grades were conducted. The participants were categorized into three refractive types based on equivalent spherical power (SE): myopia (SE<-0.25 D), emmetropia (-0.25 D≤SE≤0.25 D), and hyperopia (SE>0.25 D). Based on axial length, the participants were divided into groups with axial length<24 mm, 24-26 mm, and>26 mm for the analysis of different types of FTD. Statistical analyses were performed using one-way analysis of variance, Kruskal-Wallis test, Bonferroni test, and Spearman correlation analysis. Results: The study included 3 369 participants (3 369 eyes) with an average age of (63.9±10.6) years; among them, 1 886 were female (56.0%) and 1, 483 were male (64.0%). The overall FTD for all eyes was 0.060 (0.016, 0.163); inner circle FTD was 0.000 (0.000, 0.025); middle circle FTD was 0.030 (0.000, 0.130); outer circle FTD was 0.055 (0.009, 0.171). The results of the univariate analysis indicated that FTD in various regions was correlated with axial length (overall: r=0.38, P<0.001; inner circle: r=0.31, P<0.001; middle circle: r=0.36, P<0.001; outer circle: r=0.39, P<0.001), subfoveal choroidal thickness (SFCT) (overall: r=-0.69, P<0.001; inner circle: r=-0.57, P<0.001; middle circle: r=-0.68, P<0.001; outer circle: r=-0.72, P<0.001), age (overall: r=0.34, P<0.001; inner circle: r=0.30, P<0.001; middle circle: r=0.31, P<0.001; outer circle: r=0.35, P<0.001), gender (overall: r=-0.11, P<0.001; inner circle: r=-0.04, P<0.001; middle circle: r=-0.07, P<0.001; outer circle: r=-0.11, P<0.001), SE (overall: r=-0.20; P<0.001; inner circle: r=-0.19, P<0.001; middle circle: r=-0.20, P<0.001; outer circle: r=-0.20, P<0.001), uncorrected visual acuity (overall: r=-0.18, P<0.001; inner circle: r=-0.26, P<0.001; middle circle: r=-0.24, P<0.001; outer circle: r=-0.22, P<0.001), and body mass index (BMI) (overall: r=-0.11, P<0.001; inner circle: r=-0.13, P<0.001; middle circle: r=-0.14, P<0.001; outer circle: r=-0.13, P<0.001). Further multivariate analysis results indicated that different region FTD was correlated with axial length (overall: β=0.020, P<0.001; inner circle: β=-0.022, P<0.001; middle circle: β=0.027, P<0.001; outer circle: β=0.022, P<0.001), SFCT (overall: β=-0.001, P<0.001; inner circle: β=-0.001, P<0.001; middle circle: β=-0.001, P<0.001; outer circle: β=-0.001, P<0.001), and age (overall: β=0.002, P<0.001; inner circle: β=0.001, P<0.001; middle circle: β=0.002, P<0.001; outer circle: β=0.002, P<0.001). The distribution of overall (H=56.76, P<0.001), inner circle (H=72.22, P<0.001), middle circle (H=75.83, P<0.001), and outer circle (H=70.34, P<0.001) FTD differed significantly among different refractive types. The distribution of overall (H=373.15, P<0.001), inner circle (H=367.67, P<0.001), middle circle (H=389.14, P<0.001), and outer circle (H=386.89, P<0.001) FTD differed significantly among different axial length groups. Furthermore, comparing various levels of FTD with systemic and ocular parameters, significant differences were found in axial length (F=142.85, P<0.001) and SFCT (F=530.46, P<0.001). Conclusions: The use of deep learning technology enables automatic segmentation and quantification of different regions of theFT, as well as preliminary grading. Different region FTD is significantly correlated with axial length, SFCT, and age. Individuals with older age, myopia, and longer axial length tend to have higher FTD and more advanced FT grades.

[Emphasizing the timing and procedure selection for vitrectomy in pathological myopic traction maculopathy]

Myopic maculopathy is the primary cause of irreversible visual impairment in patients with pathologic myopia, and myopic traction maculopathy often requires vitrectomy for treatment. Myopic traction maculopathy encompasses epiretinal membrane, foveoschisis, macular hole, and macular hole-related retinal detachment. It is recommended to perform vitrectomy combined with inner limiting membrane peeling for Type II epiretinal membrane, foveal-sparing inner limiting membrane peeling for foveoschisis, inverted inner limiting membrane flap technique for macular hole, and vitrectomy combined with macular buckle for refractory macular hole-related retinal detachment. Myopic traction maculopathy is a chronically progressive condition, and surgeons need to accurately determine the timing of surgery and choose appropriate procedures to maximize the benefits for patients.

Demonstration of real-time positron emission tomography biology-guided radiotherapy delivery to targets

BACKGROUND

Biology-guided radiotherapy (BgRT) is a novel technology that uses positron emission tomography (PET) data to direct radiotherapy delivery in real-time. BgRT enables the precise delivery of radiation doses based on the PET signals emanating from PET-avid tumors on the fly. In this way, BgRT uniquely utilizes radiotracer uptake as a biological beacon for controlling and adjusting dose delivery in real-time to account for target motion.

PURPOSE

To demonstrate using real-time PET for BgRT delivery on the RefleXion X1 radiotherapy machine. The X1 radiotherapy machine is a rotating ring-gantry radiotherapy system that generates a nominal 6MV photon beam, PET, and computed tomography (CT) components. The system utilizes emitted photons from PET-avid targets to deliver effective radiation beamlets or pulses to the tumor in real-time.

METHODS

This study demonstrated a real-time PET BgRT delivery experiment under three scenarios. These scenarios included BgRT delivering to (S1 ) a static target in a homogeneous and heterogeneous environment, (S2 ) a static target with a hot avoidance structure and partial PET-avid target, and (S3 ) a moving target. The first step was to create stereotactic body radiotherapy (SBRT) and BgRT plans (offline PET data supported) using RefleXion's custom-built treatment planning system (TPS). Additionally, to create a BgRT plan using PET-guided delivery, the targets were filled with 18F-Fluorodeoxyglucose (FDG), which represents a tumor/target, that is, PET-avid. The background materials were created in the insert with homogeneous water medium (for S1 ) and heterogeneous water with styrofoam mesh medium. A heterogeneous background medium simulated soft tissue surrounding the tumor. The treatment plan was then delivered to the experimental setups using a pre-commercial version of the X1 machine. As a final step, the dosimetric accuracy for S1 and S2 was assessed using the ArcCheck analysis tool-the gamma criteria of 3%/3 mm. For S3 , the delivery dose was quantified using EBT-XD radiochromic film. The accuracy criteria were based on coverage, where 100% of the clinical target volume (CTV) receives at least 97% of the prescription dose, and the maximum dose in the CTV was ≤130% of the maximum planned dose (97 % ≤ CTV ≤ 130%).

RESULTS

For the S1, both SBRT and BgRT deliveries had gamma pass rates greater than 95% (SBRT range: 96.9%-100%, BgRT range: 95.2%-98.9%), while in S2 , the gamma pass rate was 98% for SBRT and between 95.2% and 98.9% for BgRT plan delivering. For S3 , both SBRT and BgRT motion deliveries met CTV dose coverage requirements, with BgRT plans delivering a very high dose to the target. The CTV dose ranges were (a) SBRT:100.4%-120.4%, and (b) BgRT: 121.3%-139.9%.

CONCLUSIONS

This phantom-based study demonstrated that PET signals from PET-avid tumors can be utilized to direct real-time dose delivery to the tumor accurately, which is comparable to the dosimetric accuracy of SBRT. Furthermore, BgRT delivered a PET-signal controlled dose to the moving target, equivalent to the dose distribution to the static target. A future study will compare the performance of BgRT with conventional image-guided radiotherapy.

Persistence of antibodies 5 years after hepatitis B vaccination in preterm birth children: A retrospective cohort study using real-world data

Previous studies did not provide substantial evidence for long-term immune persistence after the hepatitis B vaccine (HepB) in preterm birth (PTB) children. Consequently, there is ongoing controversy surrounding the booster immunization strategy for these children. Therefore, we conducted a retrospective cohort study to evaluate the disparities in immune persistence between PTB children and full-term children. A total of 1027 participants were enrolled in this study, including 505 PTB children in the exposure group and 522 full-term children in the control group. The negative rate of hepatitis B surface antibody (HBsAb) in the PTB group was significantly lower than that in the control group (47.9% vs. 41.4%, p = .035). The risk of HBsAb-negative in the exposure group was 1.5 times higher than that in the control group (adjusted odds ratio [aOR] = 1.5, 95% confidence interval [CI]: 1.1-2.0). The geometric mean concentration (GMC) of HBsAb was much lower for participants in the exposure group compared to participants in the control group (9.3 vs. 12.4 mIU/mL, p = .029). Subgroup analysis showed that the very preterm infants (gestational age <32 weeks) and the preterm low birth weight infants (birth weight <2000 g) had relatively low GMC levels of 3.2 mIU/mL (95% CI: 0.9-11.1) and 7.9 mIU/mL (95% CI: 4.2-14.8), respectively. Our findings demonstrated that PTB had a significant impact on the long-term persistence of HBsAb after HepB vaccination. The very preterm infants (gestational age <32 weeks) and the preterm low birth weight infants (birth weight <2000 g) may be special populations that should be given priority for HepB booster vaccination.

MetaKernel: Learning Variational Random Features With Limited Labels

Few-shot learning deals with the fundamental and challenging problem of learning from a few annotated samples, while being able to generalize well on new tasks. The crux of few-shot learning is to extract prior knowledge from related tasks to enable fast adaptation to a new task with a limited amount of data. In this paper, we propose meta-learning kernels with random Fourier features for few-shot learning, we call MetaKernel. Specifically, we propose learning variational random features in a data-driven manner to obtain task-specific kernels by leveraging the shared knowledge provided by related tasks in a meta-learning setting. We treat the random feature basis as the latent variable, which is estimated by variational inference. The shared knowledge from related tasks is incorporated into a context inference of the posterior, which we achieve via a long-short term memory module. To establish more expressive kernels, we deploy conditional normalizing flows based on coupling layers to achieve a richer posterior distribution over random Fourier bases. The resultant kernels are more informative and discriminative, which further improves the few-shot learning. To evaluate our method, we conduct extensive experiments on both few-shot image classification and regression tasks. A thorough ablation study demonstrates that the effectiveness of each introduced component in our method. The benchmark results on fourteen datasets demonstrate MetaKernel consistently delivers at least comparable and often better performance than state-of-the-art alternatives.

Zero-Shot Learning With Attentive Region Embedding and Enhanced Semantics

The performance of zero-shot learning (ZSL) can be improved progressively by learning better features and generating pseudosamples for unseen classes. Existing ZSL works typically learn feature extractors and generators independently, which may shift the unseen samples away from their real distribution and suffers from the domain bias problem. In this article, to tackle this challenge, we propose a variational autoencoder (VAE)-based framework, that is, joint Attentive Region Embedding with Enhanced Semantics (AREES), which is tailored to advance the zero-shot recognition. Specifically, AREES is end-to-end trainable and consists of three network branches: 1) attentive region embedding is used to learn the semantic-guided visual features by the attention mechanism (AM); 2) a decomposition structure and a semantic pivot regularization are used to extract enhanced semantics; and 3) a multimodal VAE (mVAE) with the cross-reconstruction loss and the distribution alignment loss is used to obtain a shared latent embedding space of visual features and semantics. Finally, features' extraction and features' generation are optimized together in AREES to address the domain shift problem to a large extent. The comprehensive evaluations on six benchmarks, including the ImageNet, demonstrate the superiority of the proposed model over its state-of-the-art counterparts.

Exploring Separable Attention for Multi-Contrast MR Image Super-Resolution

Super-resolving the magnetic resonance (MR) image of a target contrast under the guidance of the corresponding auxiliary contrast, which provides additional anatomical information, is a new and effective solution for fast MR imaging. However, current multi-contrast super-resolution (SR) methods tend to concatenate different contrasts directly, ignoring their relationships in different clues, e.g., in the high-and low-intensity regions. In this study, we propose a separable attention network (comprising high-intensity priority (HP) attention and low-intensity separation (LS) attention), named SANet. Our SANet could explore the areas of high-and low-intensity regions in the "forward" and "reverse" directions with the help of the auxiliary contrast while learning clearer anatomical structure and edge information for the SR of a target-contrast MR image. SANet provides three appealing benefits: First, it is the first model to explore a separable attention mechanism that uses the auxiliary contrast to predict the high-and low-intensity regions, diverting more attention to refining any uncertain details between these regions and correcting the fine areas in the reconstructed results. Second, a multistage integration module is proposed to learn the response of multi-contrast fusion at multiple stages, get the dependency between the fused representations, and boost their representation ability. Third, extensive experiments with various state-of-the-art multi-contrast SR methods on fastMRI and clinical in vivo datasets demonstrate the superiority of our model. The code is released at https://github.com/chunmeifeng/SANet.

SRRM4-mediated REST to REST4 dysregulation promotes tumor growth and neural adaptation in breast cancer leading to brain metastasis

BACKGROUND

Effective control of brain metastasis remains an urgent clinical need due a limited understanding of the mechanisms driving it. Although the gain of neuro-adaptive attributes in breast-to-brain metastases (BBMs) has been described, the mechanisms that govern this neural acclimation and the resulting brain metastasis competency are poorly understood. Herein, we define the role of neural-specific splicing factor Serine/Arginine Repetitive Matrix Protein 4 (SRRM4) in regulating microenvironmental adaptation and brain metastasis colonization in breast cancer cells.

METHODS

Utilizing pure neuronal cultures and brain-naive and patient-derived BM tumor cells, along with in vivo tumor modeling, we surveyed the early induction of mediators of neural acclimation in tumor cells.

RESULTS

When SRRM4 is overexpressed in systemic breast cancer cells, there is enhanced BBM leading to poorer overall survival in vivo. Concomitantly, SRRM4 knockdown expression does not provide any advantage in central nervous system metastasis. In addition, reducing SRRM4 expression in breast cancer cells slows down proliferation and increases resistance to chemotherapy. Conversely, when SRRM4/REST4 levels are elevated, tumor cell growth is maintained even in nutrient-deprived conditions. In neuronal coculture, decreasing SRRM4 expression in breast cancer cells impairs their ability to adapt to the brain microenvironment, while increasing SRRM4/RE-1 Silencing Transcription Factor (REST4) levels leads to greater expression of neurotransmitter and synaptic signaling mediators and a significant colonization advantage.

CONCLUSIONS

Collectively, our findings identify SRRM4 as a regulator of brain metastasis colonization, and a potential therapeutic target in breast cancer.

Few-Shot Medical Image Segmentation via Generating Multiple Representative Descriptors

Automatic medical image segmentation has witnessed significant development with the success of large models on massive datasets. However, acquiring and annotating vast medical image datasets often proves to be impractical due to the time consumption, specialized expertise requirements, and compliance with patient privacy standards, etc. As a result, Few-shot Medical Image Segmentation (FSMIS) has become an increasingly compelling research direction. Conventional FSMIS methods usually learn prototypes from support images and apply nearest-neighbor searching to segment the query images. However, only a single prototype cannot well represent the distribution of each class, thus leading to restricted performance. To address this problem, we propose to Generate Multiple Representative Descriptors (GMRD), which can comprehensively represent the commonality within the corresponding class distribution. In addition, we design a Multiple Affinity Maps based Prediction (MAMP) module to fuse the multiple affinity maps generated by the aforementioned descriptors. Furthermore, to address intra-class variation and enhance the representativeness of descriptors, we introduce two novel losses. Notably, our model is structured as a dual-path design to achieve a balance between foreground and background differences in medical images. Extensive experiments on four publicly available medical image datasets demonstrate that our method outperforms the state-of-the-art methods, and the detailed analysis also verifies the effectiveness of our designed module.

Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generation

We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for challenging graph-constrained architectural layout generation tasks. The proposed graph-Transformer-based generator includes a novel graph Transformer encoder that combines graph convolutions and self-attentions in a Transformer to model both local and global interactions across connected and non-connected graph nodes. Specifically, the proposed connected node attention (CNA) and non-connected node attention (NNA) aim to capture the global relations across connected nodes and non-connected nodes in the input graph, respectively. The proposed graph modeling block (GMB) aims to exploit local vertex interactions based on a house layout topology. Moreover, we propose a new node classification-based discriminator to preserve the high-level semantic and discriminative node features for different house components. To maintain the relative spatial relationships between ground truth and predicted graphs, we also propose a novel graph-based cycle-consistency loss. Finally, we propose a novel self-guided pre-training method for graph representation learning. This approach involves simultaneous masking of nodes and edges at an elevated mask ratio (i.e., 40%) and their subsequent reconstruction using an asymmetric graph-centric autoencoder architecture. This method markedly improves the model's learning proficiency and expediency. Experiments on three challenging graph-constrained architectural layout generation tasks (i.e., house layout generation, house roof generation, and building layout generation) with three public datasets demonstrate the effectiveness of the proposed method in terms of objective quantitative scores and subjective visual realism. New state-of-the-art results are established by large margins on these three tasks.

Unsupervised spatially embedded deep representation of spatial transcriptomics

Optimal integration of transcriptomics data and associated spatial information is essential towards fully exploiting spatial transcriptomics to dissect tissue heterogeneity and map out inter-cellular communications. We present SEDR, which uses a deep autoencoder coupled with a masked self-supervised learning mechanism to construct a low-dimensional latent representation of gene expression, which is then simultaneously embedded with the corresponding spatial information through a variational graph autoencoder. SEDR achieved higher clustering performance on manually annotated 10 × Visium datasets and better scalability on high-resolution spatial transcriptomics datasets than existing methods. Additionally, we show SEDR's ability to impute and denoise gene expression (URL: https://github.com/JinmiaoChenLab/SEDR/ ).

A Bayesian Federated Learning Framework With Online Laplace Approximation

Federated learning (FL) allows multiple clients to collaboratively learn a globally shared model through cycles of model aggregation and local model training, without the need to share data. Most existing FL methods train local models separately on different clients, and then simply average their parameters to obtain a centralized model on the server side. However, these approaches generally suffer from large aggregation errors and severe local forgetting, which are particularly bad in heterogeneous data settings. To tackle these issues, in this paper, we propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side. On the server side, a multivariate Gaussian product mechanism is employed to construct and maximize a global posterior, largely reducing the aggregation errors induced by large discrepancies between local models. On the client side, a prior loss that uses the global posterior probabilistic parameters delivered from the server is designed to guide the local training. Binding such learning constraints from other clients enables our method to mitigate local forgetting. Finally, we achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.

Generative Multi-Label Zero-Shot Learning

Multi-label zero-shot learning strives to classify images into multiple unseen categories for which no data is available during training. The test samples can additionally contain seen categories in the generalized variant. Existing approaches rely on learning either shared or label-specific attention from the seen classes. Nevertheless, computing reliable attention maps for unseen classes during inference in a multi-label setting is still a challenge. In contrast, state-of-the-art single-label generative adversarial network (GAN) based approaches learn to directly synthesize the class-specific visual features from the corresponding class attribute embeddings. However, synthesizing multi-label features from GANs is still unexplored in the context of zero-shot setting. When multiple objects occur jointly in a single image, a critical question is how to effectively fuse multi-class information. In this work, we introduce different fusion approaches at the attribute-level, feature-level and cross-level (across attribute and feature-levels) for synthesizing multi-label features from their corresponding multi-label class embeddings. To the best of our knowledge, our work is the first to tackle the problem of multi-label feature synthesis in the (generalized) zero-shot setting. Our cross-level fusion-based generative approach outperforms the state-of-the-art on three zero-shot benchmarks: NUS-WIDE, Open Images and MS COCO. Furthermore, we show the generalization capabilities of our fusion approach in the zero-shot detection task on MS COCO, achieving favorable performance against existing methods.

Leveraging Balanced Semantic Embedding for Generative Zero-Shot Learning

Generative (generalized) zero-shot learning [(G)ZSL] models aim to synthesize unseen class features by using only seen class feature and attribute pairs as training data. However, the generated fake unseen features tend to be dominated by the seen class features and thus classified as seen classes, which can lead to inferior performances under zero-shot learning (ZSL), and unbalanced results under generalized ZSL (GZSL). To address this challenge, we tailor a novel balanced semantic embedding generative network (BSeGN), which incorporates balanced semantic embedding learning into generative learning scenarios in the pursuit of unbiased GZSL. Specifically, we first design a feature-to-semantic embedding module (FEM) to distinguish real seen and fake unseen features collaboratively with the generator in an online manner. We introduce the bidirectional contrastive and balance losses for the FEM learning, which can guarantee a balanced prediction for the interdomain features. In turn, the updated FEM can boost the learning of the generator. Next, we propose a multilevel feature integration module (mFIM) from the cycle-consistency branch of BSeGN, which can mitigate the domain bias through feature enhancement. To the best of our knowledge, this is the first work to explore embedding and generative learning jointly within the field of ZSL. Extensive evaluations on four benchmarks demonstrate the superiority of BSeGN over its state-of-the-art counterparts.

Dynamic Unary Convolution in Transformers

It is uncertain whether the power of transformer architectures can complement existing convolutional neural networks. A few recent attempts have combined convolution with transformer design through a range of structures in series, where the main contribution of this paper is to explore a parallel design approach. While previous transformed-based approaches need to segment the image into patch-wise tokens, we observe that the multi-head self-attention conducted on convolutional features is mainly sensitive to global correlations and that the performance degrades when these correlations are not exhibited. We propose two parallel modules along with multi-head self-attention to enhance the transformer. For local information, a dynamic local enhancement module leverages convolution to dynamically and explicitly enhance positive local patches and suppress the response to less informative ones. For mid-level structure, a novel unary co-occurrence excitation module utilizes convolution to actively search the local co-occurrence between patches. The parallel-designed Dynamic Unary Convolution in Transformer (DUCT) blocks are aggregated into a deep architecture, which is comprehensively evaluated across essential computer vision tasks in image-based classification, segmentation, retrieval and density estimation. Both qualitative and quantitative results show our parallel convolutional-transformer approach with dynamic and unary convolution outperforms existing series-designed structures.

TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning

Zero-shot learning (ZSL) tackles the novel class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Semantic knowledge is typically represented by attribute descriptions shared between different classes, which act as strong priors for localizing object attributes that represent discriminative region features, enabling significant and sufficient visual-semantic interaction for advancing ZSL. Existing attention-based models have struggled to learn inferior region features in a single image by solely using unidirectional attention, which ignore the transferable and discriminative attribute localization of visual features for representing the key semantic knowledge for effective knowledge transfer in ZSL. In this paper, we propose a cross attribute-guided Transformer network, termed TransZero++, to refine visual features and learn accurate attribute localization for key semantic knowledge representations in ZSL. Specifically, TransZero++ employs an attribute → visual Transformer sub-net (AVT) and a visual → attribute Transformer sub-net (VAT) to learn attribute-based visual features and visual-based attribute features, respectively. By further introducing feature-level and prediction-level semantical collaborative losses, the two attribute-guided transformers teach each other to learn semantic-augmented visual embeddings for key semantic knowledge representations via semantical collaborative learning. Finally, the semantic-augmented visual embeddings learned by AVT and VAT are fused to conduct desirable visual-semantic interaction cooperated with class semantic vectors for ZSL classification. Extensive experiments show that TransZero++ achieves the new state-of-the-art results on three golden ZSL benchmarks and on the large-scale ImageNet dataset. The project website is available at: https://shiming-chen.github.io/TransZero-pp/TransZero-pp.html.

Automatic Detection of Tooth-Gingiva Trim Lines on Dental Surfaces

Detecting the tooth-gingiva trim line from a dental surface plays a critical role in dental treatment planning and aligner 3D printing. Existing methods treat this task as a segmentation problem, which is resolved with geometric deep learning based mesh segmentation techniques. However, these methods can only provide indirect results (i.e., segmented teeth) and suffer from unsatisfactory accuracy due to the incapability of making full use of high-resolution dental surfaces. To this end, we propose a two-stage geometric deep learning framework for automatically detecting tooth-gingiva trim lines from dental surfaces. Our framework consists of a trim line proposal network (TLP-Net) for predicting an initial trim line from the low-resolution dental surface as well as a trim line refinement network (TLR-Net) for refining the initial trim line with the information from the high-resolution dental surface. Specifically, our TLP-Net predicts the initial trim line by fusing the multi-scale features from a U-Net with a proposed residual multi-scale attention fusion module. Moreover, we propose feature bridge modules and a trim line loss to further improve the accuracy. The resulting trim line is then fed to our TLR-Net, which is a deep-based LDDMM model with the high-resolution dental surface as input. In addition, dense connections are incorporated into TLR-Net for improved performance. Our framework provides an automatic solution to trim line detection by making full use of raw high-resolution dental surfaces. Extensive experiments on a clinical dental surface dataset demonstrate that our TLP-Net and TLR-Net are superior trim line detection methods and outperform cutting-edge methods in both qualitative and quantitative evaluations.

The Importance of Expert Knowledge for Automatic Modulation Open Set Recognition

Automatic modulation classification (AMC) is an important technology for the monitoring, management, and control of communication systems. In recent years, machine learning approaches are becoming popular to improve the effectiveness of AMC for radio signals. However, the automatic modulation open-set recognition (AMOSR) scheme that aims to identify the known modulation types and recognize the unknown modulation signals is not well studied. Therefore, in this paper, we propose a novel multi-modal marginal prototype framework for radio frequency (RF) signals (MMPRF) to improve AMOSR performance. First, MMPRF addresses the problem of simultaneous recognition of closed and open sets by partitioning the feature space in the way of one versus other and marginal restrictions. Second, we exploit the wireless signal domain knowledge to extract a series of signal-related features to enhance the AMOSR capability. In addition, we propose a GAN-based unknown sample generation strategy to allow the model to understand the unknown world. Finally, we conduct extensive experiments on several publicly available radio modulation data, and experimental results show that our proposed MMPRF outperforms the state-of-the-art AMOSR methods.

Distilling a Powerful Student Model via Online Knowledge Distillation

Existing online knowledge distillation approaches either adopt the student with the best performance or construct an ensemble model for better holistic performance. However, the former strategy ignores other students' information, while the latter increases the computational complexity during deployment. In this article, we propose a novel method for online knowledge distillation, termed feature fusion and self-distillation (FFSD), which comprises two key components: FFSD, toward solving the above problems in a unified framework. Different from previous works, where all students are treated equally, the proposed FFSD splits them into a leader student set and a common student set. Then, the feature fusion module converts the concatenation of feature maps from all common students into a fused feature map. The fused representation is used to assist the learning of the leader student. To enable the leader student to absorb more diverse information, we design an enhancement strategy to increase the diversity among students. Besides, a self-distillation module is adopted to convert the feature map of deeper layers into a shallower one. Then, the shallower layers are encouraged to mimic the transformed feature maps of the deeper layers, which helps the students to generalize better. After training, we simply adopt the leader student, which achieves superior performance, over the common students, without increasing the storage or inference cost. Extensive experiments on CIFAR-100 and ImageNet demonstrate the superiority of our FFSD over existing works. The code is available at https://github.com/SJLeo/FFSD.

Fish Rhbdd3 positively regulates IFN response through RIG-I signaling pathway

Rhomboid domain-containing protein 3 (Rhbdd3) is a member of the rhomboid family, which can modulate the innate immune response in mammals. Nonetheless, the function and regulatory mechanism of fish Rhbdd3 during viral infection have not been characterized. In this study, Rhbdd3 was firstly cloned from common carp (Cyprinus carpio) and nominated as CcRhbdd3. Phylogenetically characterization showed that CcRhbdd3 shared a relatively long evolutionary distance with its mammalian homologs. In vivo experiment demonstrated that spring viraemia of carp virus (SVCV) infection promoted the expression of CcRhbdd3 in the liver, spleen, kidney and muscle tissues. Furthermore, overexpression of CcRhbdd3 significantly inhibited SVCV propagation, whereas knockdown of CcRhbdd3 markedly promoted SVCV replication in susceptible cells. RNA-seq and following validation showed that CcRhbdd3 overexpression upregulated the expression of several RIG-I signaling related genes, including TRIM25, TRAF2, MDA5, LGP2, IFN1, IFN3, RIG-I, IRF3 and ISG15. Moreover, CcRhbdd3 promoted the expression of NF-κB, a central immune regulator. Subcellular localization experiments showed that CcRhbdd3 was primarily distributed in the cytoplasm and co-localized with Rab5 in the early endosomes. Truncation experiments further demonstrated that the C-terminus containing the ubiquitin-binding associated domain, was crucial for both the subcellular localization and antiviral activity of CcRhbdd3. The findings in this study provide new insight into the host antiviral mechanism against aquatic RNA virus infection, and will facilitate the development of therapeutic strategies for the infection of SVCV.

Pruning Networks With Cross-Layer Ranking & k-Reciprocal Nearest Filters

This article focuses on filter-level network pruning. A novel pruning method, termed CLR-RNF, is proposed. We first reveal a "long-tail" pruning problem in magnitude-based weight pruning methods and then propose a computation-aware measurement for individual weight importance, followed by a cross-layer ranking (CLR) of weights to identify and remove the bottom-ranked weights. Consequently, the per-layer sparsity makes up the pruned network structure in our filter pruning. Then, we introduce a recommendation-based filter selection scheme where each filter recommends a group of its closest filters. To pick the preserved filters from these recommended groups, we further devise a k -reciprocal nearest filter (RNF) selection scheme where the selected filters fall into the intersection of these recommended groups. Both our pruned network structure and the filter selection are nonlearning processes, which, thus, significantly reduces the pruning complexity and differentiates our method from existing works. We conduct image classification on CIFAR-10 and ImageNet to demonstrate the superiority of our CLR-RNF over the state-of-the-arts. For example, on CIFAR-10, CLR-RNF removes 74.1% FLOPs and 95.0% parameters from VGGNet-16 with even 0.3% accuracy improvements. On ImageNet, it removes 70.2% FLOPs and 64.8% parameters from ResNet-50 with only 1.7% top-five accuracy drops. Our project is available at https://github.com/lmbxmu/CLR-RNF.

Measurement of Ultra-High-Energy Diffuse Gamma-Ray Emission of the Galactic Plane from 10 TeV to 1 PeV with LHAASO-KM2A

The diffuse Galactic γ-ray emission, mainly produced via interactions between cosmic rays and the interstellar medium and/or radiation field, is a very important probe of the distribution, propagation, and interaction of cosmic rays in the Milky Way. In this Letter, we report the measurements of diffuse γ rays from the Galactic plane between 10 TeV and 1 PeV energies, with the square kilometer array of the Large High Altitude Air Shower Observatory (LHAASO). Diffuse emissions from the inner (15°10  TeV). The energy spectrum in the inner Galaxy regions can be described by a power-law function with an index of -2.99±0.04, which is different from the curved spectrum as expected from hadronic interactions between locally measured cosmic rays and the line-of-sight integrated gas content. Furthermore, the measured flux is higher by a factor of ∼3 than the prediction. A similar spectrum with an index of -2.99±0.07 is found in the outer Galaxy region, and the absolute flux for 10≲E≲60  TeV is again higher than the prediction for hadronic cosmic ray interactions. The latitude distributions of the diffuse emission are consistent with the gas distribution, while the longitude distributions show clear deviation from the gas distribution. The LHAASO measurements imply that either additional emission sources exist or cosmic ray intensities have spatial variations.

Evaluation of Treatment Interruptions and Recovery during Biology-Guided Radiotherapy Delivery

PURPOSE/OBJECTIVE(S)

A Biology-guided Radiotherapy (BgRT) based device is designed to use Positron Emission Tomography (PET) signals to achieve tracked dose delivery. The goal of this study is to investigate the dose delivery accuracy in case of interruption during BgRT treatment, and resumption in a separate treatment session for a multi-target delivery, as the PET activity continues to decay.

MATERIALS/METHODS

A custom-built large anthropomorphic phantom (LAP) including a 26 mm spherical target with 3D independent motion and two 22 mm spherical targets with 1D sinusoidal motion embedded in water was used. All three targets were filled with FGD in an 8:1 target to background uptake ratio (41.52 kBq/ml in target and 5.19 kBq/ml in background). During BgRT delivery, the treatment was intentionally paused during delivery to the second target and the current treatment session was ended to generate a partial fraction. Then the partial fraction was continued in a new session, where the CT scan localization and PET pre-scan were repeated using the existing PET activity present in the phantom. The newly acquired PET pre-scan, was then used to determine if sufficient PET counts were present to resume treatment delivery. The interruption and recovery algorithm is designed to calculate the fluence that needs to be delivered to the remaining targets as well as the residual fluence to be given to the targets that have already received partial dose prior to the interruption. Once the new fluence is recomputed, the treatment is resumed. The delivered doses were captured using radiochromic film (EBT-XD) inserted in the target as well as post-treatment dose calculations based on the delivered beamlet sequence to evaluate the results in terms of dosimetric coverage and margin loss. The margin loss is calculated as the maximum difference between the distance from the Clinical Target Volume (CTV) contour to the 97% isodose contour in the treatment plan and the on the film. The dosimetric coverage is defined as the percentage of voxels within the CTV that lies within 97% and 130% of the prescribed dose.

RESULTS

As shown in the table below, a margin loss of less than 3 mm for all targets and 100% CTV coverage was achieved. After treatment interruptions, the PET safety evaluation based on the PET pre-scan helped to determine whether the treatment could be continued on the same day using the same injected PET activity (an NTS value ≧ 2 and AC value ≧ 5 kBq/ml).

CONCLUSION

This study demonstrated that the BgRT system is able to deliver the prescribed dose to all targets with independent motion, even when an interruption and resumption occurs during treatment. In case such an interruption if the remaining PET activity satisfies the BgRT safety evaluation, the treatment can continue to deliver the remainder of the BgRT doses.

Intrafraction Dosimetric Evaluation of Biology-Guided Radiotherapy to a Target Under Respiratory Motion

PURPOSE/OBJECTIVE(S)

To evaluate the reproducibility and variability of biology-guided radiotherapy (BgRT) treatments using a large anthropomorphic phantom modeling the motion amplitude of a lung tumor.

MATERIALS/METHODS

RefleXion X1 is equipped with two opposing 90 degrees PET detector arcs to capture the radionuclide emissions and direct the 6MV Linac to treat the lesions in real time. A custom-built phantom filled with a liquid [¹⁸F]Fluorodeoxyglucose (FDG) solution was used. Fillable target and OAR structures were 3D printed and attached to motion stages. The GTV = CTV was matched to the spherical 22 mm diameter target, and the PTV was a 5 mm expansion from the CTV volume. The Biology Tracking Zone (BTZ) was generated after adding 5 mm margin to the motion extent of the CTV. The OAR was a large C-shape annulus (emulating a heart) that was approximately 3 cm from the target. The 3D independent motion trajectory of the target was designed to mimic lung motion: range of +5.8 mm to -4.9 mm in LR, range of +14.4 mm to -11.3 mm in SI, and range of +5.2 mm to -5.1 mm in AP directions. The OAR motion waveform used a 1D sinusoidal pattern with a 5 mm amplitude in SI direction. The target and the OAR were filled with 40 kBq/mL while the background had 5 kBq/mL FDG. A BgRT Modeling (imaging-only) PET acquisition was performed using RefleXion X1 and used to generate a 4-fraction BgRT treatment plan prescribing 10 Gy/fraction to PTV. For each delivery, target, OAR and background were filled with the same FDG concentrations as in the BgRT Modeling PET planning scan. Dosimetry to the target and OAR were both measured using an ion-chamber (Exradin A14SL) and film in the coronal plane through the center of the GTV for all 4 fractions.

RESULTS

The mean activity concentration within the (BTZ) was 7.4 ± 0.8 kBq/mL. The calculated signal-to-noise ratio metric (Normalized Target Signal) within the BTZ was 4.0 ± 0.3. Total treatment times were all less than 35 minutes (34.3 ± 0.2). Prescription dose coverage to the CTV for all 4 fractions was 100%. Ion chamber measurements in the CTV were -1.6 ± 1.3% relative to the planned dose over the active area of the ion-chamber. Minimum and maximum doses to the CTV, measured on film, were -7.7 ± 2.2% and 1.3 ± 1.4%, calculated relative to the planned dose distribution, respectively. The OAR maximum point dose measured on film was -8.7 ± 2.9%, calculated relative to the maximum OAR dose predicted on the bounded dose-volume histogram.

CONCLUSION

Based on this initial study, accurate and reproducible dosimetry can be achieved for targets under respiratory motion using biology-guided radiotherapy over the course of a complete course of treatment. Further studies are needed to evaluate the intrafraction dosimetry of BgRT delivery under various motion models and tumor sizes.

Multimodal Transformer for Accelerated MR Imaging

Accelerated multi-modal magnetic resonance (MR) imaging is a new and effective solution for fast MR imaging, providing superior performance in restoring the target modality from its undersampled counterpart with guidance from an auxiliary modality. However, existing works simply combine the auxiliary modality as prior information, lacking in-depth investigations on the potential mechanisms for fusing different modalities. Further, they usually rely on the convolutional neural networks (CNNs), which is limited by the intrinsic locality in capturing the long-distance dependency. To this end, we propose a multi-modal transformer (MTrans), which is capable of transferring multi-scale features from the target modality to the auxiliary modality, for accelerated MR imaging. To capture deep multi-modal information, our MTrans utilizes an improved multi-head attention mechanism, named cross attention module, which absorbs features from the auxiliary modality that contribute to the target modality. Our framework provides three appealing benefits: (i) Our MTrans use an improved transformers for multi-modal MR imaging, affording more global information compared with existing CNN-based methods. (ii) A new cross attention module is proposed to exploit the useful information in each modality at different scales. The small patch in the target modality aims to keep more fine details, the large patch in the auxiliary modality aims to obtain high-level context features from the larger region and supplement the target modality effectively. (iii) We evaluate MTrans with various accelerated multi-modal MR imaging tasks, e.g., MR image reconstruction and super-resolution, where MTrans outperforms state-of-the-art methods on fastMRI and real-world clinical datasets.

Dosimetric Accuracy of Multi-Target Biology-Guided Radiotherapy Treatments in a Single Session

PURPOSE/OBJECTIVE(S)

We present the first dosimetric measurements of single session, multi-target BgRT deliveries using a clinically realistic motion phantom on a research-only version of the RefleXion X1 system.

MATERIALS/METHODS

A custom-made anthropomorphic phantom of a human torso with embedded fillable targets mimicking 18F-FDG-avid lesions was used. From the three embedded spherical targets, Target 1 was 26 mm in diameter coupled with a 3D independent respiratory motion with 22 mm range, whereas Target 2 and 3 were 22 mm in diameter and moved with a 1D 5 mm maximum sinusoidal motion. The 18F-FDG concentration in the background cavity of the phantom was 5 kBq/ml, and the targets were loaded with 10:1, 8:1 and 6:1 contrast relative to the background for Targets 1, 2, 3, respectively. Spherical structures were contoured as GTVs (CTV = GTV) and a 5 mm margin was added to create PTVs. Motion extent of the tumors were captured to create biological tracking zones for each target. Treatment plans were generated using a research version of the Reflexion treatment planning software to deliver 8 Gy/fx to the PTVs. The treatment delivery was repeated 2 times, and each time the phantom was refilled according to the plan. PET image evaluation metrics for each of the three targets were also recorded. Target dosimetry was measured using a combination of radiographic film and ion chamber. The maximum distance between the 97% prescription isodose line from the plan and the film measurements was used to characterize the dosimetric accuracy of the tracked deliveries. CTV and PTV min, max, and mean doses measured on film were also recorded for each target.

RESULTS

Treatment plans were successfully created with 100% prescription dose coverage to each target loaded with different FDG ratios. Total treatment times for the single-plan, three-target deliveries were less than 80 minutes. PET evaluation metrics at imaging-only and pre-scan, and planning and film dosimetry to the GTV and PTV for each of the three targets is shown in table below (mean ± standard deviation of both deliveries). The CTV dose coverage was maintained for all targets. The shrinkage distance of the 97% prescription dose isodose line on the film plane for all three targets was less than 3 mm for both tests, and ranged from -0.4 to -2.34 mm.

CONCLUSION

These results demonstrate that high tracking accuracy and dosimetric accuracy can be achieved in single session, multi-target deliveries over a range of target-to-background 18F-FDG concentrations and target motion patterns.

Treatment Plan Creation and Delivery with and without BgRT for Static and Motion Trajectories

PURPOSE/OBJECTIVE(S)

In this work we try to validate the motion tracking capabilities of BgRT for periodic and step motion trajectories. SBRT plans that matches the corresponding BgRT plans are created and delivered to the same phantom with and without motion and results are evaluated. Using BgRT based SBRT plans eliminates any user bias and creates SBRT plans that would represent treatment delivery scenarios that could have happened if the PET guided BgRT was not present for that treatment.

MATERIALS/METHODS

To validate SBRT plans that matches the BgRT plans, we used three different types of motion patterns (1) static, (2) lung tumor motion and (3) one-centimeter step-shift. The lung tumor motion (∼25 mm in IEC-Y, ∼7 mm in IEC-X and ∼ 10 mm in IEC-Z) was used as it represents a continuous motion of the target for the entire length of the study while the step-shift case corresponds to the patient or tumor shifting between the localization CT and the start of treatment. First, a 10 Gy per fraction BgRT plan was created for each of the three experiments based on the corresponding PET image. Then, the BgRT plans were delivered to the corresponding targets with and without motion and results are evaluated. To perform a comparative study that assess the performance of BgRT and traditional SBRT (planning and delivery methods), the exact same plan fluence of BgRT plan for each experiment was used to create the corresponding SBRT plans. The newly created SBRT plans were delivered to the corresponding phantom experiments and were compared against BgRT delivery in terms of dose coverage and target margin loss using radiochromic film that moves with the target. The margin loss was calculated as the difference between the distance from the CTV contour to the 97% isodose contour in the treatment plan and the CTV contour to the 97% isodose contour on the film. Dosimetric coverage was on the other hand calculated as the percentage of the voxels within the CTV that lies within 97% and 130% of the prescribed dose.

RESULTS

The results showed that the margin loss for BgRT is less than 3 mm, while for the SBRT plans were more than 3 mm when target motion is present. The dosimetric coverage for BgRT was 100% for all three cases, however less than 100% for the SBRT cases with motion. Table showing margin loss for the various experiments for a prescription dose of 10 Gy.

CONCLUSION

The results shows that BgRT is capable of tracking the tumor motion and delivering the prescribed dose to the moving target.

Characterization of Biology-Guided Radiotherapy Accuracy as a Function of PET Tracer Uptake

PURPOSE/OBJECTIVE(S)

To characterize the tracking capability and dosimetric accuracy of biology-guided radiotherapy (BgRT) under clinically relevant PET tracer uptake scenarios relative to the background.

MATERIALS/METHODS

A custom-made anthropomorphic phantom filled with a liquid 18F-FDG solution including two embedded fillable 22 mm diameter spherical structures mimicking GTV (= CTV) and OAR was coupled to motion stages to create an independent 3D respiratory motion with 22 mm maximum range for target and a 5 mm 1D sinusoidal motion in the OAR. The biology-tracking zone (BTZ) was generated by adding 5 mm margin to the motion extent. The three BgRT scenarios studied were representative of tumors with good (8:1), borderline (4:1) and undesired (2:1) PET biodistributions compared to background. The clinical safety limit of BgRT uses Activity Concentration within the BTZ (AC ≥ 5 kBq/ml) and Normalized Target Signal as a contrast metric (NTS ≧ 2.7 for planning and ≧ 2 for delivery). The BgRT deliveries were repeated 3 times with radiochromic film and integrated ion chamber capturing the target and OAR doses. Tracked dosimetry was assessed using a margin-loss calculation defined as the maximum linear difference in distance between the planned and delivered 97% prescription iso-dose lines.

RESULTS

The imaging-only PET images used to create BgRT plans had an AC of 7.0, 5.3, and 1.6 kBq/ml with an NTS of 6.8, 5.3, and 1.8 for 8:1, 4:1, and 2:1 concentrations, respectively. Qualitatively, the target was not visible on the planning PET images 2:1 loading scenario. At delivery, the mean pre-scan activity concentrations were 6.8, 4.7, and 3.7 kBq/ml with corresponding mean NTS of 3.7, 2.6, 1.5 for 8:1, 4:1 and 2:1 deliveries. The pre-scan values of AC or NTS did not satisfy the clinical system safety limits for 4:1 and 2:1 ratio experiments, but the engineering software allowed for the delivery to capture the resulting doses. The deliveries showed a prescription dose coverage to the CTV of 100% for the 8:1 and 4:1 cases, but 88% for the 2:1 case. When compared to the planned dose values, the delivered minimum doses were -7.6%, -8.6% and -10.9%, whereas the maximum dose differences in CTV were 1.2%, 0% and -4.8% of the planned dose distributions of the 8:1, 4:1 and 2:1 cases, respectively. Calculated margin losses were -2.3, -3.8, and -5.5 mm, for the 8:1, 4:1, and 2:1 cases, respectively. The maximum OAR doses were less than the maximum doses predicted on the bounded DVH curves for all scenarios.

CONCLUSION

With sufficient tracer uptake in the target, BgRT can deliver tracked dosimetry for targets with a large respiratory motion profile. Both the good BgRT candidate and borderline cases produced clinically acceptable delivered doses, even though the borderline case was flagged by the clinical system safety checks. As expected, the delivered BgRT dose distributions were suboptimal with reduced tumor over background PET contrast.

Normalization Techniques in Training DNNs: Methodology, Analysis and Application

Normalization techniques are essential for accelerating the training and improving the generalization of deep neural networks (DNNs), and have successfully been used in various applications. This paper reviews and comments on the past, present and future of normalization methods in the context of DNN training. We provide a unified picture of the main motivation behind different approaches from the perspective of optimization, and present a taxonomy for understanding the similarities and differences between them. Specifically, we decompose the pipeline of the most representative normalizing activation methods into three components: the normalization area partitioning, normalization operation and normalization representation recovery. In doing so, we provide insight for designing new normalization technique. Finally, we discuss the current progress in understanding normalization methods, and provide a comprehensive review of the applications of normalization for particular tasks, in which it can effectively solve the key issues.

Comparison of Cold-Sprayed Coatings of Copper-Based Composite Deposited on AZ31B Magnesium Alloy and 6061 T6 Aluminum Alloy Substrates

Copper-coated graphite and copper mixture powders were deposited on AZ31B magnesium alloy and 6061 T6 aluminum alloy substrates under different process parameters by a solid-state cold spray technique. The microstructure of the copper-coated graphite and copper composite coatings was visually examined using photographs taken with an optical microscope and a scanning electron microscope. The surface roughness of the coatings was investigated with a 3D profilometer. The thickness of the coatings was determined through the analysis of the microstructure images, while the adhesion of the coatings was characterized using the scratch test method. The results indicate that the surface roughness of the coatings sprayed on the two different substrates gradually decreases as gas temperature and gas pressure increase. Additionally, the thickness and adhesion of the coatings deposited on the two different substrates both increase with an increase in gas temperature and gas pressure. Comparing the surface roughness, thickness, and adhesion of the coatings deposited on the two different substrates, the surface roughness and adhesion of the coatings on the soft substrate are greater than those of the coatings on the hard substrate, while the thickness of the coatings is not obviously affected by the hardness of the substrate. Furthermore, it is noteworthy that the surface roughness, thickness, and adhesion of the copper-coated graphite and copper composite coatings sprayed on the two different substrates exhibit a distinct linear relationship with particle velocity.

Adaptive Siamese Tracking With a Compact Latent Network

In this article, we provide an intuitive viewing to simplify the Siamese-based trackers by converting the tracking task to a classification. Under this viewing, we perform an in-depth analysis for them through visual simulations and real tracking examples, and find that the failure cases in some challenging situations can be regarded as the issue of missing decisive samples in offline training. Since the samples in the initial (first) frame contain rich sequence-specific information, we can regard them as the decisive samples to represent the whole sequence. To quickly adapt the base model to new scenes, a compact latent network is presented via fully using these decisive samples. Specifically, we present a statistics-based compact latent feature for fast adjustment by efficiently extracting the sequence-specific information. Furthermore, a new diverse sample mining strategy is designed for training to further improve the discrimination ability of the proposed compact latent network. Finally, a conditional updating strategy is proposed to efficiently update the basic models to handle scene variation during the tracking phase. To evaluate the generalization ability and effectiveness and of our method, we apply it to adjust three classical Siamese-based trackers, namely SiamRPN++, SiamFC, and SiamBAN. Extensive experimental results on six recent datasets demonstrate that all three adjusted trackers obtain the superior performance in terms of the accuracy, while having high running speed.

IC9600: A Benchmark Dataset for Automatic Image Complexity Assessment

Image complexity (IC) is an essential visual perception for human beings to understand an image. However, explicitly evaluating the IC is challenging, and has long been overlooked since, on the one hand, the evaluation of IC is relatively subjective due to its dependence on human perception, and on the other hand, the IC is semantic-dependent while real-world images are diverse. To facilitate the research of IC assessment in this deep learning era, we built the first, to our best knowledge, large-scale IC dataset with 9,600 well-annotated images. The images are of diverse areas such as abstract, paintings and real-world scenes, each of which is elaborately annotated by 17 human contributors. Powered by this high-quality dataset, we further provide a base model to predict the IC scores and estimate the complexity density maps in a weakly supervised way. The model is verified to be effective, and correlates well with human perception (with the Pearson correlation coefficient being 0.949). Last but not the least, we have empirically validated that the exploration of IC can provide auxiliary information and boost the performance of a wide range of computer vision tasks. The dataset and source code can be found at https://github.com/tinglyfeng/IC9600.

[Long-term mortality risk of valvular heart disease adults over 35 years old in Chinese communities]

Objective: To investigate the risk and influencing factors of long-term mortality of valvular heart disease (VHD) adults aged 35 years and over in Chinese communities. Methods: A cohort study was carried out. The data of the subjects who underwent echocardiography were collected from the Chinese Hypertension Survey between 2012 and 2015 and survival outcomes were followed up between 2018 and 2019. Kaplan-Meier survival curves were plotted and compared using log-rank test. Cox proportional hazards models were used to analyze the influence of VHD on mortality. Results: During an average follow-up time of (4.6±0.9) years, a total of 23 237 participants (10 881 males and 12 356 females) were pooled into the final analysis from 5 eastern, 5 central, and 4 western provinces, cities and autonomous regions in China, with a mean age of (56.9±13.2) years. Among the included participants, 1 004 had VHD (467 males and 537 females), with a mean age was of (68.1±12.6) years. In the Kaplan-Meier analysis, participants with VHD had a significantly increased risk of all-cause mortality (log-rank χ²=351.82, P<0.001) and cardiovascular mortality (log-rank χ²=284.14, P<0.001) compared with those without VHD. Multivariate Cox regression analysis showed that compared with those without VHD, the participants with rheumatic VHD had a 45% increased risk of all-cause mortality (HR=1.45, 95%CI: 1.12-1.89) and degenerative VHD increased the risk of cardiovascular mortality by 69% (HR=1.69, 95%CI: 1.19-2.38). The risk factors of cardiovascular mortality for VHD were age 55 years and over (55-<75 years: HR=4.93, 95%CI: 1.17-20.85;≥75 years: HR=11.92, 95%CI: 2.85-49.80) and diabetes mellitus (HR=1.71, 95%CI: 1.00-2.93). Conclusions: VHD is a risk factor of all-cause mortality and cardiovascular mortality among adults aged 35 years and over. Age 55 years and over and diabetes mellitus are adverse prognostic factors for patients with VHD.

A tera-electron volt afterglow from a narrow jet in an extremely bright gamma-ray burst

Some gamma-ray bursts (GRBs) have a tera-electron volt (TeV) afterglow, but the early onset of this has not been observed. We report observations with the Large High Altitude Air Shower Observatory of the bright GRB 221009A, which serendipitously occurred within the instrument field of view. More than 64,000 photons >0.2 TeV were detected within the first 3000 seconds. The TeV flux began several minutes after the GRB trigger, then rose to a peak about 10 seconds later. This was followed by a decay phase, which became more rapid ~650 seconds after the peak. We interpret the emission using a model of a relativistic jet with half-opening angle ~0.8°. This is consistent with the core of a structured jet and could explain the high isotropic energy of this GRB.

A Discriminative Cross-Aligned Variational Autoencoder for Zero-Shot Learning

Zero-shot learning (ZSL) aims to classify unseen samples based on the relationship between the learned visual features and semantic features. Traditional ZSL methods typically capture the underlying multimodal data structures by learning an embedding function between the visual space and the semantic space with the Euclidean metric. However, these models suffer from the hubness problem and domain bias problem, which leads to unsatisfactory performance, especially in the generalized ZSL (GZSL) task. To tackle such a problem, we formulate a discriminative cross-aligned variational autoencoder (DCA-VAE) for ZSL. The proposed model effectively utilizes a modified cross-modal-alignment variational autoencoder (VAE) to transform both visual features and semantic features obtained by the discriminative cosine metric into latent features. The key to our method is that we collect principal discriminative information from visual and semantic features to construct latent features which contain the discriminative multimodal information associated with unseen samples. Finally, the proposed model DCA-VAE is validated on six benchmarks including the large dataset ImageNet, and several experimental results demonstrate the superiority of DCA-VAE over most existing embedding or generative ZSL models on the standard ZSL and the more realistic GZSL tasks.

Learning Multi-Attention Context Graph for Group-Based Re-Identification

Learning to re-identify or retrieve a group of people across non-overlapped camera systems has important applications in video surveillance. However, most existing methods focus on (single) person re-identification (re-id), ignoring the fact that people often walk in groups in real scenarios. In this work, we take a step further and consider employing context information for identifying groups of people, i.e., group re-id. On the one hand, group re-id is more challenging than single person re-id, since it requires both a robust modeling of local individual person appearance (with different illumination conditions, pose/viewpoint variations, and occlusions), as well as full awareness of global group structures (with group layout and group member variations). On the other hand, we believe that person re-id can be greatly enhanced by incorporating additional visual context from neighboring group members, a task which we formulate as group-aware (single) person re-id. In this paper, we propose a novel unified framework based on graph neural networks to simultaneously address the above two group-based re-id tasks, i.e., group re-id and group-aware person re-id. Specifically, we construct a context graph with group members as its nodes to exploit dependencies among different people. A multi-level attention mechanism is developed to formulate both intra-group and inter-group context, with an additional self-attention module for robust graph-level representations by attentively aggregating node-level features. The proposed model can be directly generalized to tackle group-aware person re-id using node-level representations. Meanwhile, to facilitate the deployment of deep learning models on these tasks, we build a new group re-id dataset which contains more than 3.8K images with 1.5K annotated groups, an order of magnitude larger than existing group re-id datasets. Extensive experiments on the novel dataset as well as three existing datasets clearly demonstrate the effectiveness of the proposed framework for both group-based re-id tasks.

Nitric acid oxidation and urea modification of carbon fibres as biofilm carriers

AbstractCarbon fibres (CF) are commonly used as carriers in biofilm-based wastewater treatment. The surface properties of the CF are herein modified using a combination of nitric acid oxidation and urea to optimise the carrier to immobilise bacterial cells. The capacity of the CF carriers to immobilise bacterial cells and activated sludge is evaluated using bacterial cell adhesion and sludge immobilisation tests. The total interaction energy profiles between the CF supports and bacterial cells were calculated according to the Derjaguin-Landau-Verwey-Overbeek (DLVO) theory to explain the mechanism by which these modifications enhance this immobilisation capacity. CF-U has a high capacity for immobilising bacterial cells and activated sludge (3.7 g-sludge/g-CF supports) owing to its low total interaction energy. Nitric acid oxidation reduced the diiodomethane contact angle of CF from 55.1° to 38.5°, which reduced the Lifshitz-van der Waals interaction energy, while urea modification further increased the zeta potential of CF from 12.8 mV to -0.7 mV, thereby reducing the electrostatic interaction energy. Experiments and DLVO theory both determined that a combination of nitric acid oxidation and urea modification significantly enhanced the ability of CF to immobilise microorganisms.

Automatic Schelling Point Detection From Meshes

Mesh Schelling points explain how humans focus on specific regions of a 3D object. They have a large number of important applications in computer graphics and provide valuable information for perceptual psychology studies. However, detecting mesh Schelling points is time-consuming and expensive since the existing techniques are mostly based on participant observation studies. To overcome these limitations, we propose to employ powerful deep learning techniques to detect mesh Schelling points in an automatic manner, free from participant observation studies. Specifically, we utilize the mesh convolution and pooling operations to extract informative features from mesh objects, and then predict the 3D heat map of Schelling points in an end-to-end manner. In addition, we propose a Deep Schelling Network (DS-Net) to automatically detect the Schelling points, including a multi-scale fusion component and a novel region-specific loss function to improve our network for a better regression of heat maps. To the best of our knowledge, DS-Net is the first deep neural network for detecting Schelling points from 3D meshes. We evaluate DS-Net on a mesh Schelling point dataset obtained from participant observation studies. The experimental results demonstrate that DS-Net is capable of detecting mesh Schelling points effectively and outperforms various state-of-the-art mesh saliency methods and deep learning models, both qualitatively and quantitatively.

Learning from multiple annotators for medical image segmentation

Supervised machine learning methods have been widely developed for segmentation tasks in recent years. However, the quality of labels has high impact on the predictive performance of these algorithms. This issue is particularly acute in the medical image domain, where both the cost of annotation and the inter-observer variability are high. Different human experts contribute estimates of the "actual" segmentation labels in a typical label acquisition process, influenced by their personal biases and competency levels. The performance of automatic segmentation algorithms is limited when these noisy labels are used as the expert consensus label. In this work, we use two coupled CNNs to jointly learn, from purely noisy observations alone, the reliability of individual annotators and the expert consensus label distributions. The separation of the two is achieved by maximally describing the annotator's "unreliable behavior" (we call it "maximally unreliable") while achieving high fidelity with the noisy training data. We first create a toy segmentation dataset using MNIST and investigate the properties of the proposed algorithm. We then use three public medical imaging segmentation datasets to demonstrate our method's efficacy, including both simulated (where necessary) and real-world annotations: 1) ISBI2015 (multiple-sclerosis lesions); 2) BraTS (brain tumors); 3) LIDC-IDRI (lung abnormalities). Finally, we create a real-world multiple sclerosis lesion dataset (QSMSC at UCL: Queen Square Multiple Sclerosis Center at UCL, UK) with manual segmentations from 4 different annotators (3 radiologists with different level skills and 1 expert to generate the expert consensus label). In all datasets, our method consistently outperforms competing methods and relevant baselines, especially when the number of annotations is small and the amount of disagreement is large. The studies also reveal that the system is capable of capturing the complicated spatial characteristics of annotators' mistakes.

SiMaN: Sign-to-Magnitude Network Binarization

Binary neural networks (BNNs) have attracted broad research interest due to their efficient storage and computational ability. Nevertheless, a significant challenge of BNNs lies in handling discrete constraints while ensuring bit entropy maximization, which typically makes their weight optimization very difficult. Existing methods relax the learning using the sign function, which simply encodes positive weights into +1s, and -1s otherwise. Alternatively, we formulate an angle alignment objective to constrain the weight binarization to {0,+1} to solve the challenge. In this article, we show that our weight binarization provides an analytical solution by encoding high-magnitude weights into +1s, and 0s otherwise. Therefore, a high-quality discrete solution is established in a computationally efficient manner without the sign function. We prove that the learned weights of binarized networks roughly follow a Laplacian distribution that does not allow entropy maximization, and further demonstrate that it can be effectively solved by simply removing the l₂ regularization during network training. Our method, dubbed sign-to-magnitude network binarization (SiMaN), is evaluated on CIFAR-10 and ImageNet, demonstrating its superiority over the sign-based state-of-the-arts. Our source code, experimental settings, training logs and binary models are available at https://github.com/lmbxmu/SiMaN.

Semantic-Aware Dynamic Generation Networks for Few-Shot Human-Object Interaction Recognition

Recognizing human-object interaction (HOI) aims at inferring various relationships between actions and objects. Although great progress in HOI has been made, the long-tail problem and combinatorial explosion problem are still practical challenges. To this end, we formulate HOI as a few-shot task to tackle both challenges and design a novel dynamic generation method to address this task. The proposed approach is called semantic-aware dynamic generation networks (SADG-Nets). Specifically, SADG-Net first assigns semantic-aware task representations for different batches of data, which further generates dynamic parameters. It obtains the features that highlight intercategory discriminability and intracategory commonality adaptively. In addition, we also design a dual semantic-aware encoder module (DSAE-Module), that is, verb-aware and noun-aware branches, to yield both action and object prototypes of HOI for each task space, which generalizes to novel combinations by transferring similarities among interactions. Extensive experimental results on two benchmark datasets, that is, humans interacting with common objects (HICO)-FS and trento universal HOI (TUHOI)-FS, illustrate that our SADG-Net achieves superior performance over state-of-the-art approaches, which proves its impressive effectiveness on few-shot HOI recognition.