Transformers use contextual information from the knowledge graph to generate the embeddings for entities and relations.

San Jose State University

Various architecture models are used to capture the interaction between entities and relations.

 - Linear/Bilinear Models
 - Factorization Models
 - Neural Networks
 - Convolutional Neural Networks
 - Recurrent Neural Networks
 - Transformers
 - Graph Neural Networks (GNNs)


Encoding Models

A Survey on Knowledge Graphs: Representation, Acquisition and Applications

Convolutional networks, also known as convolutional neural networks, or CNNs, are a specialized kind of neural network for processing data that has a known grid-like topology. Examples include time-series data, which can be thought of as a 1-D grid taking samples at regular time intervals, and image data, which can be thought of as a 2-D grid of pixels. 

The network employs a mathematical operation called convolution. Convolution is a specialized kind of linear operation. Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers.

The CNN gets its power from the following concepts:
- Sparse connectivity
- Parameter sharing

Convolutional Neural Network (CNN)

GNNs use knowledge graphs to generate the entity and relation embeddings.

Encoding Models using Graph Neural Networks (GNNs)

Various neural networks feed entities or relations or both to deep neural networks and computes a semantic matching score.

Encoding Models using Neural Networks

Recurrent Neural Networks (RNNs) can be used as encoding models to capture long-term dependencies within knowledge graphs, generating semantic embeddings that represent the interactions between entities and relations.

Encoding Models using Recurrent Neural Networks

Encoding Models using Transformers

Factorization methods formulated KRL models as three-way tensor $\mathcal{X}$ decomposition.

Encoding Models using Factorisation Models

Linear/Bilinear Models encode the interaction of entities and relations using linear/bilinear operations.

Learn Before

Related