This approach is based on the theory of permutation invariant neural networks. For example, an aggregation function with the following form is a universal set function approximator:

$m_{N(u)}=MLP_{\theta}(\sum_{v\in N(u)}MLP_{\phi}(h_{v}))$ Any permutation-invariant function that maps a set of embeddings to a single embedding can be approximated to an arbitrary accuracy by a model following this equation.

Set pooling approaches based on the above equation often lead to small increases in performance, though they also introduce an increased risk of overfitting, depending on the depth of the MLPs used.

If set pooling is used, it is common to use MLPs that have only a single hidden layer, since these models are sufficient to satisfy the theory, but are not so over parameterized so as to risk catastrophic overfitting.