For any GNN model, the encoder can be modified to be a graph level variational encoder by adding a pooling layer, i.e.,

$\mu_{\mathbf{z}_G}=POOL_{\mu}(GNN_{\mu} (\mathbf{A}, \mathbf{X}))$,
$log \sigma_{\mathbf{z}_G}=POOL_{\sigma}(GNN_{\sigma} (\mathbf{A}, \mathbf{X}))$

Where $POOL: \mathbb{R}^{|V|\times d}\rightarrow \mathbb{R}^d$. Here we use two different GNNs to parameterize the mean and variance of posterior distribution, and we define a posterior for each single graph instead of each single node.


Encoder For Graph-Level Latents

The goal of graph level variational decoder is to define a posterior given a graph level latent embedding.  The original graph VAE proposed to combine a multi-layer perceptron (MLP) and Bernoulli distribution assumption to obtain the posterior:

$p_{\theta}(G | \mathbf{z}_G) = \prod_{(u,v)\in \mathcal{V}} \tilde{A}[u,v] A[u,v]+(1-\tilde{A}[u,v]) (1-A[u,v])$

Where $A$ is adjacency matrix and $\tilde{A}=\sigma(MLP(\mathbf{z}_G))$ is the predicted matrix of edge probabilities. The overall log-likelihood objective is equivalent to a set of independent binary cross-entropy loss.


Decoder For Graph-Level Latents

The node level VGAE can be modified to work with graph level representation by modifying the encoder and decoder with graph level representaions. 

University of Notre Dame

VAEs model aims to train a probabilistic decoder model pθ(A|Z), from which we can sample realistic graphs (i.e., adjacency matrices) Aˆ ∼ pθ(A|Z) by conditioning on a latent variable Z. In a probabilistic sense, we aim to learn a conditional distribution over adjacency matrices (with the distribution being conditioned on some latent variable).

Variational Autoencoder Approaches (VAEs)

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

In more formal and mathematical detail, to build a VAE for graphs we must
specify some key components.

Key Components of the VAE model

Given the key components and a set of training graphs {G1, .., Gn}, we can
train a VAE model by minimizing the evidence likelihood lower bound (ELBO), as shown below. The basic idea is that we seek to maximize the reconstruction ability of our decoder—i.e., the likelihood term Eqθ(Z|Gi) [pθ(Gi|Z)]—while minimizing the KL-divergence between our posterior latent distribution qθ(Z|Gi) and the prior p(Z).

 The Evidence Likelihood Lower Bound (ELBO)

Variational Graph Autoencoders (VGAE) Based On Graph-Level Latent Representations

We can adapt the VAE model to graphs by encoding and decoding graphs based on node embeddings and latent representations of graphs. More specifically, the encoder generates latent representations for each node in the graph and the decoder takes pairs of embeddings as input and uses these embeddings to predict the likelihood of an edge occurring between the two given nodes.

Learn Before

Related

Learn After