In order to capture the entity information in text, position embeddings are introduced to specify the relative distances between words and entities.

Hamilton College

Neural relation extraction (NRE) models introduce neural networks to automatically extract semantic features from text. Compared with SRE models,
NRE methods can effectively capture textual information and generalize to wider range of data.

Neural Relation Extraction Models

Han, X. (n.d.). More Data, More Relations, More Context and More Openness:
A Review and Outlook for Relation Extraction *, Retrieved June 4, 2022, from https://arxiv.org/pdf/2004.03186.pdf

More Relations, More Context and More Openness:
A Review and Outlook for Relation Extraction

It learns compositional representations for sentences recursively.

Recursive Neural Networks

It can better handle long sequential data.

Recurrent Neural Networks (RNNs)

GNNs are a general framework for defining deep neural networks on graph data. The goal of GNNs is to generate representations of nodes that actually depend on the structure of the graph, as well as any feature information available.

Graph Neural Networks (GNNs)

It utilizes attention mechanism to aggregate global relational information.

Attention-based Neural Networks

Different from SRE models, NRE mainly utilizes word embeddings and position embeddings instead of hand-craft features as inputs. Word embeddings are the most used input representations in NLP, which encode the semantic meaning of
words into vectors.

Word Embeddings

Position Embeddings

Convolutional networks, also known as convolutional neural networks, or CNNs, are a specialized kind of neural network for processing data that has a known grid-like topology. Examples include time-series data, which can be thought of as a 1-D grid taking samples at regular time intervals, and image data, which can be thought of as a 2-D grid of pixels. 

The network employs a mathematical operation called convolution. Convolution is a specialized kind of linear operation. Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers.

The CNN gets its power from the following concepts:
- Sparse connectivity
- Parameter sharing

Learn Before

Related