 - Consider the example “Put on a wig and lipstick and be who you really are..” this is directed towards a boy,one could
infer that this is a remark to malign the gender identity of the boy being addressed

 - The example utterance above is from Formspring.

 - These kinds of utterances reiterate the need for world knowledge in addition to language features to successfully classify hate speech.


Need for Knowledge-Based Features 

 - An approach is to employ automatic reasoning over world knowledge focusing on anti-LGBT hate speech.
 - In this approach they use ConceptNet which is a general purpose ontology
This ConceptNet  is supplemented with contents extracted from social media network Formspring.This augmented knowledge base is called BullySpace.

 - The concepts expressed in the knowledge base are used for comparing concepts expressed in a passage.

 - The extracted concepts from the user passage are compared with four canonical concepts namely positive and negative valence and the two genders, male and female.

 - The similarity score between the extracted and canonical concept determine whether an utterance is an example of  hate speech.


Approaches used in Knowledge-Based Feature  utilizing hate speech detector.

Contextual information as a feature for hate speech detection.

San Jose State University

Features from the data used for a classification model always have a greater impact on how the model works. Supervised learning is used in hate speech classification models.
The various features extracted from the text are

 - Simple Surface Features
 - Word Generalization
 - Sentiment Analysis
 - Lexical Resources
 - Linguistic Features
 - Knowledge-Based Features
 - Meta-Information
 - Multimodal Information




Textual Features extracted for Hate Speech Detection using NLP

A Survey on Hate Speech Detection using Natural Language Processing by Anna Schmidt  and Michael Wiegand

https://aclanthology.org/W17-1101.pdf

A Survey on Hate Speech Detection using Natural Language Processing

Surface level features in text classification tasks are 

 - Bag of words
 - Unigrams( Word level)
 - Large n-grams(Word Level)
 - N-grams combined with other features 
 - Character n-grams
 - Frequency of URL mentions and punctuations
 - Comment and token length 
 - Capitalization
 - Words not in English dictionaries 
 - Number of non-alpha numeric characters present in tokens 


Simple Surface Features Needed for Text Classification Tasks

 - Words are clustered and cluster id is given to each word.
 - Cluster id along with other features are used to do text classification.

 - Brown clustering algorithm is an example, this algorithm produces hard clusters 
 which means each word belongs to a specific cluster.

 - Neural networks introduced distributed word representations also called word embeddings which serve the purpose of word generalization


Word Generalization for Hate Speech Classification

 - Hate speech always reflects negative sentiment so we can say hate speech is correlated  with sentiment analysis.

 - Sentiment analysis is used as an additional classification.Some researchers followed a multistep approach in which sentiment analysis was used to check for negative polarity statements followed by hate speech detection.

 - Some approaches use sentiment( analysis) information as a feature for the classification task.


Sentiment Analysis Feature Generation for Hate Speech Classification

 - Hate speech sentences or passages contain words that particularly represent slurs or insults so this lexicon can be regarded as a feature for classification models.

 - The web can be used as a source to curate a word list.
 - Specialized word list can be created by taking the words that mostly occur in slurs, slang languages and words used to target a specific group of people.

 - Wordlist is generated for hate speech classification tasks.
 - Lexicon features are not used as stand alone features for classification problems.


Lexical Resource Feature for Hate Speech Detection

 - Combining n-gram features with POS-information-enriched tokens, but classifier performance was not increased significantly.
 - Deep syntactic information( long distance relation between words in a sentence).

 - Dependency relationships in a sentence can be used as a feature which increases performance of the classification tasks.
 - Dependency relations are selected manually or using Bayesian Logistic Regression
Some researchers have used an offensive score which is calculated based on  the number of times an offensive word repeats in a sentence or passage
 - In an approach called Smokey system researcher uses specific linguistic and syntactic features to detect hate speech. In addition to this, praise rules and politeness rules are implemented to detect co-occurence of good words.
 - For example imperative statements like imperative statements (e.g. Get lost!,Get a life!)
the co occurrence of the pronoun you modified by noun phrases (as in you bozos)



Linguistic Features For Hate Speech Detection

Knowledge-Based Features for Hate Speech Detection

Since most of the late speech comes from social media platforms ,it is very easy to retrieve meta information about the hate speech utterances. Meta-information extracted are 

 - Background information of a user 
 - Number of profane words in the history of a user
 - Gender of the user 
 - Number of posts by the user.
 - Number of replies to a post 
 - Average number of total replies to a follower 
 - Geographical origin 




Meta-Information Feature Extraction for Hate Speech Detection


Social media contents are a combination of text,audio,image and video ,so the latter three have a significant role in increasing the accuracy of hate speech classifiers when used as a feature while training.

Learn Before

Related

Learn After