Zhang Q, Li J, Ye Q, Lin Y, Chen X, Fu YG. DWSSA: Alleviating
over-smoothness for deep Graph Neural Networks.
Neural Netw 2024;
174:106228. [PMID:
38461705 DOI:
10.1016/j.neunet.2024.106228]
[Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 01/15/2024] [Accepted: 03/05/2024] [Indexed: 03/12/2024]
Abstract
Graph Neural Networks (GNNs) have demonstrated great potential in achieving outstanding performance in various graph-related tasks, e.g., graph classification and link prediction. However, most of them suffer from the following issue: shallow networks capture very limited knowledge. Prior works design deep GNNs with more layers to solve the issue, which however introduces a new challenge, i.e., the infamous over-smoothness. Graph representation over emphasizes node features but only considers the static graph structure with a uniform weight are the key reasons for the over-smoothness issue. To alleviate the issue, this paper proposes a Dynamic Weighting Strategy (DWS) for addressing over-smoothness. We first employ Fuzzy C-Means (FCM) to cluster all nodes into several groups and get each node's fuzzy assignment, based on which a novel metric function is devised for dynamically adjusting the aggregation weights. This dynamic weighting strategy not only enables the intra-cluster interactions, but also inter-cluster aggregations, which well addresses undifferentiated aggregation caused by uniform weights. Based on DWS, we further design a Structure Augmentation (SA) step for addressing the issue of underutilizing the graph structure, where some potentially meaningful connections (i.e., edges) are added to the original graph structure via a parallelable KNN algorithm. In general, the optimized Dynamic Weighting Strategy with Structure Augmentation (DWSSA) alleviates over-smoothness by reducing noisy aggregations and utilizing topological knowledge. Extensive experiments on eleven homophilous or heterophilous graph benchmarks demonstrate the effectiveness of our proposed method DWSSA in alleviating over-smoothness and enhancing deep GNNs performance.
Collapse