A scored inference system may compute a score for each possible output but still need an approximate search algorithm to find a high-scoring output, because exhaustive enumeration can be far too large. These approximate algorithms are not guaranteed to find the output that maximizes the score.

Approximate Search in Scored Inference

In a distance-based scoring function for knowledge graphs, the plausibility of a fact is measured by the distance between entities. This relationship is typically expressed as $$h + r \sim t$$, where $$h$$ represents the head entity, $$r$$ represents the relation, and $$t$$ represents the tail entity.

Distance-Based Scoring Function

In a similarity-based (or semantic matching) scoring function for knowledge graphs, the plausibility of a fact is measured by matching the latent semantics of entities and relations. This relationship is typically expressed using a bilinear formulation, such as $$h^T M_r \sim t^T$$, where $$h$$ and $$t$$ represent the head and tail entity vectors, and $$M_r$$ represents the relation matrix.

Semantic Matching Scoring Function

A scoring function can assign a score to each possible output for a given input, and an inference system can then search for the output that maximizes that score. A common AI design pattern is to learn an approximate scoring function and then apply an approximate maximization algorithm.

Google

San Jose State University

Knowledge representation learning(KRL) maps entities and relations onto a low dimensional vector space by capturing its semantic meaning. KRL is also called multi-relation learning or statistical relational learning. The recent advancement in distributed representation learning can be categorized into four scopes

 - Representation Space 
 - Scoring Function 
 - Encoding Models 
 - Embedding with Auxiliary Information


Knowledge Representation Learning /Knowledge Graph Embedding

A Survey on Knowledge Graphs: Representation, Acquisition and Applications

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

KRL model will answer the following four questions

1. ” Which representation space to choose?”
2. ”How to measure the plausibility of triplets in a specific space?”
3. “Which encoding model to use for modeling relational interaction ?”
4. ”Whether to utilize auxiliary information?”


KRL Model 

The space where entities and relations are represented after analysis. Representation space is also called embedding space. The embedding space should satisfy three conditions differentiability, calculation possibility, and definability of a scoring function. The representation space can be categorized into

 - Point-Wise Space
 - Complex Vector Space
 - Gaussian Distribution
 - Manifold and Group:


Representation Space 

Scoring Function

Various architecture models are used to capture the interaction between entities and relations.

 - Linear/Bilinear Models
 - Factorization Models
 - Neural Networks
 - Convolutional Neural Networks
 - Recurrent Neural Networks
 - Transformers
 - Graph Neural Networks (GNNs)


Encoding Models

In this form of knowledge representation extra information like text descriptions, type constraints, visual information ,relational paths along with knowledge graphs are used to learn embeddings. The external information needed can be categorized into

 - Textual Description
 - Type Information
 - Visual Information
 - Uncertain Information


Learn Before

Related

Learn After