Some AI software purports to help employers with streamlining their hiring practices — including using AI to evaluate video interviews by analyzing speech patterns, tone of voice, facial movements, and body language of employee candidates. This has the risk of significantly discriminating against people with many different types of disability, such as blindness, deafness, facial paralyzation from a stroke, speech disorders, neurodivergence, and etc.

University of Michigan - Ann Arbor

Though machine learning has become more ubiquitous, discussions on AI bias tend to center around gender and race, and not disability. However, there tends to be lower representation for disabled individuals in datasets due to selective disclosure of disability. Even when disability information is being intentionally collected, some may choose not to reveal their disability — likely in fear of social or financial consequences in doing so. 

The Difficulty of Disability Fairness in Relation to AI

Whittaker, M., Alper, M., Bennett, C. L., Hendren, S., Kaziunas, L., Mills, M., ... & West, S. M. (2019). Disability, bias, and AI. AI Now Institute, 8.

Disability, Bias, and AI

Many AI “solutions” play into the medical model, in that they are primarily focused on “fixing” individuals who fall outside of the medical norm. AI designed to “help deaf people take part in spoken language” or “help autistic people with emoting” only serve to bring people closer to the able-bodied or neurotypical norm. Though some may seek these types of AI treatments, many of these said treatments would rather focus on fitting people into an already biased norm instead of assisting them as they are. 

AI and the Medical Model

If AI were to be used to diagnose neurodivergence, such as autism, it would be far less reliable for those who fall outside of the normative framework — white and assigned male at birth. Because machine learning models are trained on already existing data, they would merely replicate and perpetuate biased diagnostic metrics. If there were no trained human psychiatrist present to assist with the diagnosis, this could lead to a higher number of false negative diagnoses for autistic individuals across different races, classes, genders, and ethnicities. 

Historical Bias Impacting AI Diagnosis for Autism

Machine learning models are trained on data. So anything that falls outside of that data is considered an “outlier”. When a researcher tested an AI model for autonomous vehicles, she showed herself footage of a friend who liked to roll backwards on a wheelchair. These models then chose to run over the friend. On later, more sophisticated models that were “trained to recognize people in wheelchairs”, the models ran over her friend with “greater confidence”, because they assumed that all wheelchair users would always be moving forward rather than backwards. 

Disability as an AI Outlier

AI already has a pre-existing bias for the white, Western, and middle-class society. So, AI models that are intended as vision assistance for those who are blind and low vision may not recognize common household objects for people who are lower-income and live in non-Western countries. 

AI Recognition for Blind and Low Vision

AI Implications on Disability Employment

While there is a wealth of research into how AI-based assistive technologies can assist people with disabilities, there is far less research into the biases that AI has against disability compared to AI biases against gender and race. The lack of academic research into the topic of AI disability bias can have far reaching consequences as AI and AI "solutions" perpetuate and apply the medical model theory of disability.

Lack of Research Into Disability Bias

By design, many AI models are a 'black box', meaning that even those who created the AI models don't know exactly how or why they create the outputs that they do. A study found that AI models that were trained to act maliciously under certain conditions could not have these malicious behaviors removed even after using standard safety training techniques. 

Potentially, AI models with biased data may not be able to remove the biased outputs unless they start over completely from scratch without biased data. However, if biased data is inserted into the AI model, that could continuously poison the model's output and cause difficulty in removing that bias from future results. 

Learn Before

Related