The sampled latent representations encode enough information to allow
our decoder to reconstruct the input. The first goal ensures that we learn to decode meaningful graphs from the encoded latent representations, when we have training graphs as input.

Enough Information for ELBO

The latent distribution is as close as possible to the prior. The second goal acts as a regularizer and ensures that we can decode meaningful graphs even when we sample latent representations from the prior p(Z). This second goal is critically important if we want to generate new graphs after training: we can generate new graphs by sampling from the prior and feeding these latent embeddings to the decoder, and this process will only work if this second goal is satisfied.

Closer Distribution to the Prior for ELBO

The key intuition of ELBO is that we want to generate a distribution over latent representations so that the two (conflicting) goals are satisfied.

Hamilton College

Given the key components and a set of training graphs {G1, .., Gn}, we can
train a VAE model by minimizing the evidence likelihood lower bound (ELBO), as shown below. The basic idea is that we seek to maximize the reconstruction ability of our decoder—i.e., the likelihood term Eqθ(Z|Gi) [pθ(Gi|Z)]—while minimizing the KL-divergence between our posterior latent distribution qθ(Z|Gi) and the prior p(Z).

 The Evidence Likelihood Lower Bound (ELBO)

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Learn Before

Related

Learn After