Document-level relation extraction via graph transformer networks and temporal convolutional networks

PIPER: A logic-driven deep contrastive optimization pipeline for event temporal reasoning

Event temporal relation extraction is an important task for information extraction. The existing methods usually rely on feature engineering and require post-process to achieve optimization, though inconsistent optimization may occur in the post-process module and main neural network due to their independence. Recently, a few works start to incorporate the temporal logic rules into the neural network and achieve joint optimization. However, these methods still suffer from two shortcomings: (1) Although the joint optimization is applied, the differences between rules are neglected in the unified design of rule losses and further the interpretability and flexibility of the design of model are reduced. (2) Because of lacking abundant syntactic connections between events and rule-match features, the performance of the model may be suppressed by the inefficient interaction in training between features and rules. To tackle these issues, this paper proposes PIPER, a logic-driven deep contrastive optimization pipeline for event temporal reasoning. Specifically, we apply joint optimization (including multi-stage and single-stage joint paradigms) by combining independent rule losses (i.e., flexibility) to make PIPER more interpretable. Also, by proposing a hierarchical graph distillation network to obtain more abundant syntactic information, the designed rule-match features can effectively aid in the interaction between low-level features and high-level rules during training. The final experiments on TB-Dense and MATRES demonstrate that the proposed model can achieve competitive performance compared with the recent advances.

Vision-Language Model for Visual Question Answering in Medical Imagery

In the clinical and healthcare domains, medical images play a critical role. A mature medical visual question answering system (VQA) can improve diagnosis by answering clinical questions presented with a medical image. Despite its enormous potential in the healthcare industry and services, this technology is still in its infancy and is far from practical use. This paper introduces an approach based on a transformer encoder-decoder architecture. Specifically, we extract image features using the vision transformer (ViT) model, and we embed the question using a textual encoder transformer. Then, we concatenate the resulting visual and textual representations and feed them into a multi-modal decoder for generating the answer in an autoregressive way. In the experiments, we validate the proposed model on two VQA datasets for radiology images termed VQA-RAD and PathVQA. The model shows promising results compared to existing solutions. It yields closed and open accuracies of 84.99% and 72.97%, respectively, for VQA-RAD, and 83.86% and 62.37%, respectively, for PathVQA. Other metrics such as the BLUE score showing the alignment between the predicted and true answer sentences are also reported.