A binary relation is defined as a set of ordered pairs of objects where some common "relation" holds true across all ordered pairs in the set.

For example, the relation "is greater than" across the set $\{1, 2, 3\}$ would give $\mathbb{S}=\{(3, 1), (2, 1), (3, 2)\}$.

Binary Relations

A relational database is a database that explicitly stores information on relations between entities that are significant in our model. While regular training data allows the model to infer and learn relationships, a relational database explicitly states the relations significant for the problem at hand.

A knowledge base is any database intended to convey "commonsense" knowledge that humans normally know from day-to-day experiences for a deep learning model. Significant knowledge bases include *Freebase*, *OpenCyc*, *WordNet*, *Wikibase*, and *GeneOntology*. 

Relational Databases and Knowledge Bases

The most common practical application of knowledge representation and relational learning is **link prediction**. 

Link prediction is the act of attempting to use a given knowledge base to predict missing relations in the knowledge base. It is a form of generaliation to predict new facts based on old ones.

Link Prediction

On the frontier of deep learning applications, research is being done on the representation of knowledge in relational manners to learn reasoning.

University of Connecticut

In this node, we describe how to use deep learning to solve applications in computer vision, speech recognition, natural language processing, and other areas of commercial interest. 

Applications of Deep Learning

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Its crucial that the time and memory cost of running inference in a ML model is lower than the time and memory cost of training. Model compression is a strategy for reducing the cost of inference. Its main idea is to replace the original, expensive model with a smaller model that requires less memory and runtime to store and evaluate. This strategy can be used when size of the original model needs to prevent overfitting. 

Model Compression

 -> Building systems that have dynamic structure in the graph is a strategy for accelerating data-processing systems. These systems dynamically determine neural network should be run on an input. Conditional Computation refers to the form of dynamic structure inside neural networks. 

-> A strategy for speeding up an inference in a classifier is to employ a cascade of classifiers. This strategy may be used to detect the presence of a rare object.

-> Since decision tree determines which of its subtrees should be evaluated for each input, its an example of dynamic structure. Switch, where a hidden unit can receive input from diﬀerent units depending on the contex, is also a dynamic structure. 

-> One disadvantage of using dynamically structured systems is the reduced degree of parallelism that results from the system following diﬀerent code branches for diﬀerent inputs. 

Learn Before

Related

Learn After