A system allocates resources (jobs or credit) unfairly to different groups. For example algorithms that use embeddings as part of a search for hiring potential programmers or doctors might thus incorrectly downweight documents with women’s names.

University of Michigan - Ann Arbor

In addition to their ability to learn word meaning from text, embeddings, alas, also reproduce the implicit biases and stereotypes that were latent in the text. As the prior section just showed, embeddings can roughly model relational similarity: ‘queen’ as the closest word to ‘king’ -‘man’ + ‘woman’ implies the analogy man:woman::king:queen.

Bias in Word Embeddings

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

Allocational harm

Embeddings don’t just reflect the statistics of their input, but also amplify bias; gendered terms become more gendered in embedding space than they bias amplification were in the input text statistics, and biases are more exaggerated than in actual labor employment statistics.

Bias amplification

A harm caused by representational harm a system demeaning or even ignoring some social groups. For example African-American names like ‘Leroy’ and ‘Shaniqua’ had a higher GloVe cosine with unpleasant words while European-American names (‘Brad’, ‘Greg’, ‘Courtney’) had a higher cosine with pleasant words.

Representational harm

Researches are try to debias by developing a transformation of the embedding space that removes gender stereotypes but preserves definitional gender or changing the training procedure. However, although debiasing these sorts of debiasing may reduce bias in embeddings, they do not eliminate it, and this remains an open problem.

Learn Before

Related