Hierarchical Graph Convolutional Networks for Structured Long Document Classification

TeKo: Text-Rich Graph Neural Networks With External Knowledge

Graph neural networks (GNNs) have gained great prevalence in tackling various analytical tasks on graph-structured data (i.e., networks). Typical GNNs and their variants adopt a message-passing principle that obtains network representations by the attribute propagates along network topology, which however ignores the rich textual semantics (e.g., local word-sequence) that exist in numerous real-world networks. Existing methods for text-rich networks integrate textual semantics by mainly using internal information such as topics or phrases/words, which often suffer from an inability to comprehensively mine the textual semantics, limiting the reciprocal guidance between network structure and textual semantics. To address these problems, we present a novel text-rich GNN with external knowledge (TeKo), in order to make full use of both structural and textual information within text-rich networks. Specifically, we first present a flexible heterogeneous semantic network that integrates high-quality entities as well as interactions among documents and entities. We then introduce two types of external knowledge, that is, structured triplets and unstructured entity descriptions, to gain a deeper insight into textual semantics. Furthermore, we devise a reciprocal convolutional mechanism for the constructed heterogeneous semantic network, enabling network structure and textual semantics to collaboratively enhance each other and learn high-level network representations. Extensive experiments illustrate that TeKo achieves state-of-the-art performance on a variety of text-rich networks as well as a large-scale e-commerce searching dataset.

Multi-modal long document classification based on Hierarchical Prompt and Multi-modal Transformer

In the realm of long document classification (LDC), previous research has predominantly focused on modeling unimodal texts, overlooking the potential of multi-modal documents incorporating images. To address this gap, we introduce an innovative approach for multi-modal long document classification based on the Hierarchical Prompt and Multi-modal Transformer (HPMT). The proposed HPMT method facilitates multi-modal interactions at both the section and sentence levels, enabling a comprehensive capture of hierarchical structural features and complex multi-modal associations of long documents. Specifically, a Multi-scale Multi-modal Transformer (MsMMT) is tailored to capture the multi-granularity correlations between sentences and images. This is achieved through the incorporation of multi-scale convolutional kernels on sentence features, enhancing the model's ability to discern intricate patterns. Furthermore, to facilitate cross-level information interaction and promote learning of specific features at different levels, we introduce a Hierarchical Prompt (HierPrompt) block. This block incorporates section-level prompts and sentence-level prompts, both derived from a global prompt via distinct projection networks. Extensive experiments are conducted on four challenging multi-modal long document datasets. The results conclusively demonstrate the superiority of our proposed method, showcasing its performance advantages over existing techniques.