$p_\theta$(*G*|**Z**) corresponds to the binary cross-entropy loss over the edge probabilities when we assume independence between edges:

$p_\theta$(*G*|**Z**)  = $\prod_{(u,v) \in V^2} p_\theta(A[u,v]=1|z_u,z_v)$

Computing the reconstruction loss in the Decoder for Node-level latents

Given a matrix of sampled node embeddings $Z \in \mathbb{R}^{|V|×d}$, the goal of the decoder model is to predict the likelihood of all the edges in the graph. More formally, the decoder must specify $p_\theta$(**A**|**Z**), which is the posterior probability of the adjacency matrix given the node embeddings.

Carleton College

We can adapt the VAE model to graphs by encoding and decoding graphs based on node embeddings and latent representations of graphs. More specifically, the encoder generates latent representations for each node in the graph and the decoder takes pairs of embeddings as input and uses these embeddings to predict the likelihood of an edge occurring between the two given nodes.

Variational Graph Autoencoders (VGAE) Based On Node-Level Latent Representations

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

Given adjacency matrix $A$ and node features $X$, use two separate GNNs to generate mean ($\mu_Z$) and variance ($\sigma_Z$) parameters.

- $\mu_Z =$ GNN$_{\mu}$(**A**, **X**)
- log$\sigma_Z$ = GNN$_\sigma$(**A**, **X**)

where $\mu_Z$ and log$\sigma_Z$ are |V| x d-dimensional matrices that specify the mean and variance embeddings for each node in input graph, respectively.

Once we have $\mu_Z$ and log$\sigma_Z$, we sample a set of latent node embeddings:

**$Z$** = $\epsilon \circ exp(log(\sigma_Z)) + \mu_Z$ 

Encoder for Node-level latents

Decoder for Node-level latents

The basic VGAE model is quite limited in its generative abilities, mainly due to its decoder. The decoder has no parameters, so the model is not able to generate non-trivial graph structures without a training graph as input.

Learn Before

Related

Learn After