Graph Neural Networks (GNNs) can be theoretically motivated by their connections to variational inference in probabilistic graphical models (PGMs). From this probabilistic perspective, the embeddings $z_{u}, \forall u \in V$ for each node are viewed as latent variables to be inferred. Assuming the graph structure and the input node features $X$ are observed, the goal is to infer the underlying latent variables that can explain this observed data. Consequently, the message passing operations underlying GNNs can be viewed as a neural network analogue of message passing algorithms commonly used for variational inference to infer distributions over these latent variables.