Social media contents are a combination of text,audio,image and video ,so the latter three have a significant role in increasing the accuracy of hate speech classifiers when used as a feature while training.

San Jose State University

Features from the data used for a classification model always have a greater impact on how the model works. Supervised learning is used in hate speech classification models.
The various features extracted from the text are

 - Simple Surface Features
 - Word Generalization
 - Sentiment Analysis
 - Lexical Resources
 - Linguistic Features
 - Knowledge-Based Features
 - Meta-Information
 - Multimodal Information




Textual Features extracted for Hate Speech Detection using NLP

A Survey on Hate Speech Detection using Natural Language Processing by Anna Schmidt  and Michael Wiegand

https://aclanthology.org/W17-1101.pdf

A Survey on Hate Speech Detection using Natural Language Processing

Surface level features in text classification tasks are 

 - Bag of words
 - Unigrams( Word level)
 - Large n-grams(Word Level)
 - N-grams combined with other features 
 - Character n-grams
 - Frequency of URL mentions and punctuations
 - Comment and token length 
 - Capitalization
 - Words not in English dictionaries 
 - Number of non-alpha numeric characters present in tokens 


Simple Surface Features Needed for Text Classification Tasks

 - Words are clustered and cluster id is given to each word.
 - Cluster id along with other features are used to do text classification.

 - Brown clustering algorithm is an example, this algorithm produces hard clusters 
 which means each word belongs to a specific cluster.

 - Neural networks introduced distributed word representations also called word embeddings which serve the purpose of word generalization


Word Generalization for Hate Speech Classification

 - Hate speech always reflects negative sentiment so we can say hate speech is correlated  with sentiment analysis.

 - Sentiment analysis is used as an additional classification.Some researchers followed a multistep approach in which sentiment analysis was used to check for negative polarity statements followed by hate speech detection.

 - Some approaches use sentiment( analysis) information as a feature for the classification task.


Sentiment Analysis Feature Generation for Hate Speech Classification

 - Hate speech sentences or passages contain words that particularly represent slurs or insults so this lexicon can be regarded as a feature for classification models.

 - The web can be used as a source to curate a word list.
 - Specialized word list can be created by taking the words that mostly occur in slurs, slang languages and words used to target a specific group of people.

 - Wordlist is generated for hate speech classification tasks.
 - Lexicon features are not used as stand alone features for classification problems.


Lexical Resource Feature for Hate Speech Detection

 - Combining n-gram features with POS-information-enriched tokens, but classifier performance was not increased significantly.
 - Deep syntactic information( long distance relation between words in a sentence).

 - Dependency relationships in a sentence can be used as a feature which increases performance of the classification tasks.
 - Dependency relations are selected manually or using Bayesian Logistic Regression
Some researchers have used an offensive score which is calculated based on  the number of times an offensive word repeats in a sentence or passage
 - In an approach called Smokey system researcher uses specific linguistic and syntactic features to detect hate speech. In addition to this, praise rules and politeness rules are implemented to detect co-occurence of good words.
 - For example imperative statements like imperative statements (e.g. Get lost!,Get a life!)
the co occurrence of the pronoun you modified by noun phrases (as in you bozos)



Linguistic Features For Hate Speech Detection

Contextual information as a feature for hate speech detection.

Knowledge-Based Features for Hate Speech Detection

Since most of the late speech comes from social media platforms ,it is very easy to retrieve meta information about the hate speech utterances. Meta-information extracted are 

 - Background information of a user 
 - Number of profane words in the history of a user
 - Gender of the user 
 - Number of posts by the user.
 - Number of replies to a post 
 - Average number of total replies to a follower 
 - Geographical origin 




Learn Before

Related