Given adjacency matrix $A$ and node features $X$, use two separate GNNs to generate mean ($\mu_Z$) and variance ($\sigma_Z$) parameters.

- $\mu_Z =$ GNN$_{\mu}$(**A**, **X**)
- log$\sigma_Z$ = GNN$_\sigma$(**A**, **X**)

where $\mu_Z$ and log$\sigma_Z$ are |V| x d-dimensional matrices that specify the mean and variance embeddings for each node in input graph, respectively.

Once we have $\mu_Z$ and log$\sigma_Z$, we sample a set of latent node embeddings:

**$Z$** = $\epsilon \circ exp(log(\sigma_Z)) + \mu_Z$ 

Encoder for Node-level latents

Given a matrix of sampled node embeddings $Z \in \mathbb{R}^{|V|×d}$, the goal of the decoder model is to predict the likelihood of all the edges in the graph. More formally, the decoder must specify $p_\theta$(**A**|**Z**), which is the posterior probability of the adjacency matrix given the node embeddings.

Decoder for Node-level latents

The basic VGAE model is quite limited in its generative abilities, mainly due to its decoder. The decoder has no parameters, so the model is not able to generate non-trivial graph structures without a training graph as input.

Limitations of basic VGAE models

We can adapt the VAE model to graphs by encoding and decoding graphs based on node embeddings and latent representations of graphs. More specifically, the encoder generates latent representations for each node in the graph and the decoder takes pairs of embeddings as input and uses these embeddings to predict the likelihood of an edge occurring between the two given nodes.

Carleton College

VAEs model aims to train a probabilistic decoder model pθ(A|Z), from which we can sample realistic graphs (i.e., adjacency matrices) Aˆ ∼ pθ(A|Z) by conditioning on a latent variable Z. In a probabilistic sense, we aim to learn a conditional distribution over adjacency matrices (with the distribution being conditioned on some latent variable).

Variational Autoencoder Approaches (VAEs)

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

In more formal and mathematical detail, to build a VAE for graphs we must
specify some key components.

Key Components of the VAE model

Given the key components and a set of training graphs {G1, .., Gn}, we can
train a VAE model by minimizing the evidence likelihood lower bound (ELBO), as shown below. The basic idea is that we seek to maximize the reconstruction ability of our decoder—i.e., the likelihood term Eqθ(Z|Gi) [pθ(Gi|Z)]—while minimizing the KL-divergence between our posterior latent distribution qθ(Z|Gi) and the prior p(Z).

 The Evidence Likelihood Lower Bound (ELBO)

The node level VGAE can be modified to work with graph level representation by modifying the encoder and decoder with graph level representaions. 

Learn Before

Related

Learn After