 In the case of graphs, the probabilistic encoder model takes a graph G as input. From this input, qφ then defines a distribution qφ(Z|G) over latent representations. We need to specify the latent conditional distribution as Z ∼ N (µφ(G), σ(φ(G)), where µφ and σφ are neural networks that generate the mean and variance parameters for a normal distribution, from which we sample latent embeddings Z.

A Probabilistic Encoder Model qφ

The decoder takes a latent representation Z as input and uses this input to specify a conditional distribution over graphs. 

A Probabilistic Decoder Model pθ

A prior distribution p(Z) over the latent space. We will adopt the standard Gaussian prior Z ∼ N (0, 1), which is commonly used for VAEs.

A Prior Distribution p(Z)

In more formal and mathematical detail, to build a VAE for graphs we must
specify some key components.

Hamilton College

VAEs model aims to train a probabilistic decoder model pθ(A|Z), from which we can sample realistic graphs (i.e., adjacency matrices) Aˆ ∼ pθ(A|Z) by conditioning on a latent variable Z. In a probabilistic sense, we aim to learn a conditional distribution over adjacency matrices (with the distribution being conditioned on some latent variable).

Variational Autoencoder Approaches (VAEs)

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

Key Components of the VAE model

Given the key components and a set of training graphs {G1, .., Gn}, we can
train a VAE model by minimizing the evidence likelihood lower bound (ELBO), as shown below. The basic idea is that we seek to maximize the reconstruction ability of our decoder—i.e., the likelihood term Eqθ(Z|Gi) [pθ(Gi|Z)]—while minimizing the KL-divergence between our posterior latent distribution qθ(Z|Gi) and the prior p(Z).

 The Evidence Likelihood Lower Bound (ELBO)

The node level VGAE can be modified to work with graph level representation by modifying the encoder and decoder with graph level representaions. 

Variational Graph Autoencoders (VGAE) Based On Graph-Level Latent Representations

We can adapt the VAE model to graphs by encoding and decoding graphs based on node embeddings and latent representations of graphs. More specifically, the encoder generates latent representations for each node in the graph and the decoder takes pairs of embeddings as input and uses these embeddings to predict the likelihood of an edge occurring between the two given nodes.

Learn Before

Related

Learn After