Convolutional networks, also known as convolutional neural networks, or CNNs, are a specialized kind of neural network for processing data that has a known grid-like topology. Examples include time-series data, which can be thought of as a 1-D grid taking samples at regular time intervals, and image data, which can be thought of as a 2-D grid of pixels. 

The network employs a mathematical operation called convolution. Convolution is a specialized kind of linear operation. Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers.

The CNN gets its power from the following concepts:
- Sparse connectivity
- Parameter sharing

Convolutional Neural Network (CNN)

GNNs use knowledge graphs to generate the entity and relation embeddings.

Encoding Models using Graph Neural Networks (GNNs)

Various neural networks feed entities or relations or both to deep neural networks and computes a semantic matching score.

Encoding Models using Neural Networks

Recurrent Neural Networks (RNNs) can be used as encoding models to capture long-term dependencies within knowledge graphs, generating semantic embeddings that represent the interactions between entities and relations.

Encoding Models using Recurrent Neural Networks

Transformers use contextual information from the knowledge graph to generate the embeddings for entities and relations.

Encoding Models using Transformers

Factorization methods formulated KRL models as three-way tensor $\mathcal{X}$ decomposition.

Encoding Models using Factorisation Models

Linear/Bilinear Models encode the interaction of entities and relations using linear/bilinear operations.

Encoding Models using Linear/Bilinear Models

Various architecture models are used to capture the interaction between entities and relations.

 - Linear/Bilinear Models
 - Factorization Models
 - Neural Networks
 - Convolutional Neural Networks
 - Recurrent Neural Networks
 - Transformers
 - Graph Neural Networks (GNNs)


San Jose State University

New York University

Knowledge representation learning(KRL) maps entities and relations onto a low dimensional vector space by capturing its semantic meaning. KRL is also called multi-relation learning or statistical relational learning. The recent advancement in distributed representation learning can be categorized into four scopes

 - Representation Space 
 - Scoring Function 
 - Encoding Models 
 - Embedding with Auxiliary Information


Knowledge Representation Learning /Knowledge Graph Embedding

A Survey on Knowledge Graphs: Representation, Acquisition and Applications

KRL model will answer the following four questions

1. ” Which representation space to choose?”
2. ”How to measure the plausibility of triplets in a specific space?”
3. “Which encoding model to use for modeling relational interaction ?”
4. ”Whether to utilize auxiliary information?”


KRL Model 

The space where entities and relations are represented after analysis. Representation space is also called embedding space. The embedding space should satisfy three conditions differentiability, calculation possibility, and definability of a scoring function. The representation space can be categorized into

 - Point-Wise Space
 - Complex Vector Space
 - Gaussian Distribution
 - Manifold and Group:


Representation Space 

Scoring function is used to measure the credibility of facts, it is also referred to as energy function There are two types of scoring functions to measure the plausibility of facts.

 - Distance-based Scoring Function
 - Semantic Matching


Scoring Function

Encoding Models

In this form of knowledge representation extra information like text descriptions, type constraints, visual information ,relational paths along with knowledge graphs are used to learn embeddings. The external information needed can be categorized into

 - Textual Description
 - Type Information
 - Visual Information
 - Uncertain Information


Learn Before

Related

Learn After