For most current applications, graph neural networks are used in one of three tasks: node classification, graph classification, or relation prediction.

Applications and Loss Functions

It is important to research on how we can make graph neural networks more powerful than WL algorithm because there are some research that indicates that models such as message-passing graph neural networks are no more powerful than the WL algorithm.

Graph Neural Networks and WL algorithm

Graph neural networks capture the internal graph structures of natural language, like syntactic and semantic parse trees.

University of California, Santa Cruz

Google

The categorization of texts into organized groups, which is usually implemented by supervised learning.
Applications: sentiment analysis, spam detection, news categorization, topic analysis, and more.

Text Classification

Minaee, Shervin et al. Deep Learning Based Text Classification: A Comprehensive Review. Retrieved May 22, 2022, from https://arxiv.org/pdf/2004.03705.pdf

Deep Learning Based Text Classification: A Comprehensive Review

One of the earliest linear embedding models, trained on 200,000 words using less than 1 million parameters.

Latent Semantic Analysis (LSA)

Graph Neural Networks

A text classification category that views text as a "bag of words." This is the simplest model for text classification, but is still very successful. 

Steps:
1) Learn the vector representation for each word using an embedding model
2) Take vector sum or average of embeddings as the text representation
3) Pass through one or more feed-forward layers, known as Multi-Layer Perceptrons (MLPs)
4) Perform classification on final layer's representation using a classifier

Feed-Forward Neural Networks

Siamese Neural Networks are a special case of text classification designed for text matching.

Siamese Neural Networks

Linear classifiers use a linear combination of inputs to predict the category.

Linear Classifiers

Sentiment analysis is used to refer to the task of automatically determining the whether a piece of text is positive, negative, or neutral. It can also determine someone's attitude towards a particular target or topic.

Sentiment Analysis

Text-pair classification extends standard classification methods to process two distinct texts simultaneously. When provided with a pair of texts composed of tokens $$x_1 \dots x_m$$ and $$y_1 \dots y_n$$, the two sequences are concatenated into a single combined sequence. The total length of this unified sequence is $$len$$, calculated as $$len = n + m + 2$$ to account for necessary special tokens. A classification label is subsequently predicted for the entire sequence by utilizing the aggregated representation vector, specifically $$\mathbf{h}_{\mathrm{cls}}$$. This overarching framework addresses multiple NLP challenges, including semantic equivalence judgement (assessing if two texts share identical meanings), text entailment judgement (evaluating if a hypothesis logically stems from a premise), grounded commonsense inference (gauging the likelihood of an event given its context), and question-answering inference (verifying if an answer correctly matches a question).

Text-Pair Classification

In text classification, a specific category or label is assigned to a piece of text. For example, in the context of sentiment analysis, the sentence 'We had a delightful dinner together.' would be assigned the label 'Positive'.

Learn Before

Related

Learn After