The core idea of parameter sharing is to use the same parameters in all the AGGREGATE and UPDATE functions in the GNN. Generally, this approach is most effective in GNNs with more than six layers, and it is often used in conjunction with gated update functions.

University of Michigan - Ann Arbor

In the context of GNNs, many of the standard regularization approaches are known to work well, including L2 regularization, dropout, and layer normalization. However, there are also regularization strategies that are somewhat specific to the GNN setting.

Regularization in GNN

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

Parameter Sharing

In edge dropout, we randomly remove (or mask) edges in the adjacency matrix during training, with the intuition that this will make the GNN less prone to overfitting and more robust to noise in the adjacency matrix. This approach has been particularly successful in the application of GNNs to knowledge graphs, and it was an essential technique used in the original graph attention network (GAT) work.

Learn Before

Related