Some early work of GNN rely solely on convolution operations. The idea of this class of GNN is to define full model as a combination or stacking of multiple convolution layers and nonlinearities. For example, Defferrard et al. defined convolutions based on a spectral convolution based on the eigenvalues of the Laplacian, and defined the spectral filter using Chebyshev polynomials. 

University of Notre Dame

There’re several types of GNN inspired by convolutions operations and their conenctions:

1)	GNN That Use Purely Convolutional Approaches

2)	Graph Convolutional Networks (GCN)

3)	GNN Without Message Passing


Convolution-Inspired GNNs

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

GNN That Use Purely Convolutional Approaches

The Graph Convolutional Network (GCN) model, introduced by Kipf and Welling, achieves powerful graph representations by stacking simple convolutional layers and utilizing a normalized variant of the adjacency matrix with self-loops. The layer-wise propagation rule is defined as:

$$\mathbf{H}^{(k)}=\sigma(\tilde{\mathbf{A}} \mathbf{H}^{(k-1)} \mathbf{W}^{(k)})$$

where $$\tilde{\mathbf{A}} = (\mathbf{D}+\mathbf{I})^{-\frac{1}{2}} (\mathbf{I}+\mathbf{A}) (\mathbf{D}+\mathbf{I})^{-\frac{1}{2}}$$.

Graph Convolutional Network (GCN)

Several recent Graph Neural Network (GNN) models propose removing message passing to simplify the network. These models can be generally defined as: $$\mathbf{Z} = MLP_{\theta}(f(\mathbf{A}) MLP_{\phi}(\mathbf{X}) )$$ where $$MLP$$ denotes a dense neural network and $$f$$ is some deterministic function. For example, Wu defines $$f$$ as: $$f(\mathbf{A}) = \tilde{\mathbf{A}}^k$$. The intuition is that trainable parameters are not required in the convolution layer. Rather, a dense layer can be applied at the start and end of the network, and a deterministic convolutional layer in the middle can be used to incorporate the graph structure. These models have been proven to outperform parameterized message passing models on many classification benchmarks.

Learn Before

Related