$$p_\theta(\mathbf{A}|\mathbf{Z})$$ corresponds to the binary cross-entropy loss over the edge probabilities when we assume independence between edges: $$p_\theta(\mathbf{A}|\mathbf{Z}) = \prod_{(u,v) \in V^2} p_\theta(A[u,v]=1|z_u,z_v)$$

Reconstruction Loss Formula for Node-Level Latent Decoders in VGAEs

Given a matrix of sampled node embeddings $$Z \in \mathbb{R}^{|V| \times d}$$, the goal of the decoder model in a Variational Graph Autoencoder (VGAE) is to predict the likelihood of all the edges in the graph. More formally, the decoder must specify $$p_\theta(\mathbf{A}|\mathbf{Z})$$, which is the posterior probability of the adjacency matrix given the node embeddings.

Carleton College

Google

We can adapt the VAE model to graphs by encoding and decoding graphs based on node embeddings and latent representations of graphs. More specifically, the encoder generates latent representations for each node in the graph and the decoder takes pairs of embeddings as input and uses these embeddings to predict the likelihood of an edge occurring between the two given nodes.

Variational Graph Autoencoders (VGAE) Based On Node-Level Latent Representations

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

The basic VGAE model is quite limited in its generative abilities, mainly due to its decoder. The decoder has no parameters, so the model is not able to generate non-trivial graph structures without a training graph as input.

Limitations of basic VGAE models

Given an adjacency matrix $$\mathbf{A}$$ and node features $$\mathbf{X}$$, an encoder for node-level latent representations in a Variational Graph Autoencoder (VGAE) uses two separate Graph Neural Networks (GNNs) to generate mean ($$\mu_Z$$) and variance ($$\sigma_Z$$) parameters:

$$\mu_Z = \text{GNN}_{\mu}(\mathbf{A}, \mathbf{X})$$

$$\log\sigma_Z = \text{GNN}_\sigma(\mathbf{A}, \mathbf{X})$$

where $$\mu_Z$$ and $$\log\sigma_Z$$ are $$|V| \times d$$-dimensional matrices specifying the mean and variance embeddings for each node in the input graph, respectively. After computing $$\mu_Z$$ and $$\log\sigma_Z$$, a set of latent node embeddings is sampled using the reparameterization trick:

$$Z = \epsilon \circ \exp(\log(\sigma_Z)) + \mu_Z$$

Learn Before

Related

Learn After