In the first step of neural message passing, at each iteration $$k$$ of the Graph Neural Network (GNN), the $$\text{AGGREGATE}$$ function takes as input the set of embeddings of the nodes in node $$u$$'s graph neighborhood $$N(u)$$ and generates a message $$m^{(k)}_{N(u)}$$ based on this aggregated neighborhood information.

Carleton College

Google

Neural Message Passing can be broken down into three steps

Steps of Neural Message Passing

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

In the second step of neural message passing, the update function $$\text{UPDATE}$$ combines the aggregated message $$m^{(k)}_{N(u)}$$ with the previous embedding $$h^{(k-1)}_u$$ of node $$u$$ to generate the updated embedding $$h^{(k)}_u$$ for the current iteration $$k$$. The initial embeddings at $$k = 0$$ are set to the input features for all nodes, such that $$h^{(0)}_u = \mathbf{x}_u$$ for all $$u \in V$$.

Neural Message Passing: Update Step

Neural Message Passing: Aggregation Step

After running $$K$$ iterations of the GNN message passing, we can use the output of the final layer to define the embeddings for each node, i.e., $$z_u = h^{(K)}_u, \forall u \in V$$.

Learn Before

Related