Examining the Biases of AI in Ranking Resumes

As the recruitment process becomes increasingly automated, the use of artificial intelligence tools such as OpenAI’s ChatGPT in screening resumes has raised concerns about bias and discrimination. University of Washington graduate student Kate Glazko discovered that recruiters were using AI to summarize resumes and rank candidates, including those with disabilities. This raised questions about how AI systems perceive disability-related credentials in resumes and how it impacts the ranking process.

In a recent study conducted by UW researchers, it was revealed that AI systems like ChatGPT consistently rated resumes with disability-related honors lower than those without such credentials. The system’s explanation for such rankings often reflected biased perceptions of disabled individuals. For example, a resume with an autism leadership award was deemed to have “less emphasis on leadership roles,” perpetuating the stereotype that individuals with autism are not effective leaders.

To mitigate these biases, researchers attempted to customize ChatGPT with written instructions to avoid ableist tendencies. The results showed a reduction in bias for most disability types tested, with improvements in the ranking of resumes implying disabilities such as deafness, blindness, cerebral palsy, autism, and general disability. However, only three out of the six disabilities ranked higher than resumes without any mention of disability, highlighting the need for ongoing efforts to address ableism in AI algorithms.

The study’s lead author, Kate Glazko, emphasized the significance of these findings for disabled job seekers who must navigate the decision of whether to disclose disability-related credentials in their resumes. The use of AI in resume screening poses challenges for individuals with disabilities, as biased algorithms may overshadow their qualifications and achievements in the job application process.

Researchers explored the potential of training AI systems like GPT-4 to be less biased by customizing them with written instructions aimed at promoting disability justice and diversity, equity, and inclusion principles. While the trained chatbot exhibited improvements in ranking enhanced CVs, there were limitations in reducing biases consistently across all disability types. This underscores the complexity of addressing algorithmic bias in AI systems and the need for ongoing research and development in this area.

The study points to the need for continued research to document and address biases in AI systems used for hiring practices. It suggests testing other AI systems, examining the intersection of bias against disabilities with other identity attributes such as gender and race, and exploring further customization to reduce biases more effectively. Additionally, it highlights the importance of making AI technology more equitable and fair for all individuals, including those with disabilities.

The study sheds light on the challenges and implications of using AI in resume screening, particularly for individuals with disabilities. By uncovering biases in algorithmic decision-making processes, researchers are paving the way for more inclusive and fair recruitment practices. Ongoing efforts to address bias in AI systems are essential to ensure equal opportunities for all job seekers, regardless of their background or identity.