The Presence of Covert Racism in Language Models

A recent study conducted by a team of AI researchers has shed light on the covert racism present in popular Language Models (LLMs). The team, comprised of professionals from the Allen Institute for AI, Stanford University, and the University of Chicago, discovered that these LLMs exhibit bias against individuals who speak African American English (AAE). This finding has significant implications for the use of LLMs in various applications, such as job screening and police reporting.

In their research, the team trained multiple LLMs on samples of AAE text and presented them with questions about the user. The results were alarming, as the LLMs consistently provided negative adjectives such as “dirty,” “lazy,” “stupid,” and “ignorant” when responding to questions phrased in AAE. In contrast, questions written in standard English elicited positive adjectives to describe the user. This disparity highlights the presence of covert racism in the responses generated by LLMs.

Covert racism in text manifests through negative stereotypes that are subtly embedded in the language. When individuals are perceived to be of African American descent, descriptions of them may carry undertones of prejudice. Terms like “lazy,” “dirty,” and “obnoxious” are often used to characterize them, while individuals of white descent are portrayed in a more favorable light with terms like “ambitious,” “clean,” and “friendly.”

Implications and Challenges

The findings of this study raise concerns about the use of LLMs in sensitive applications like job screening and law enforcement. The presence of covert racism in the responses generated by these models can lead to biased outcomes and perpetuate discriminatory practices. While efforts have been made to filter out overtly racist responses, addressing covert racism poses a significant challenge due to its subtle nature.

The prevalence of covert racism in LLMs underscores the need for more extensive efforts to eliminate bias from their responses. As these models continue to play a crucial role in various domains, it is imperative to address and rectify the underlying biases that influence their behavior. Moving forward, it is essential for researchers and developers to collaborate in developing more inclusive and unbiased AI systems to ensure fair and equitable outcomes for all individuals.