Researchers have discovered that OpenAI's AI chatbot ChatGPT consistently rated curriculum vitae (CVs) or resumes with disability-related awards and credentials, such as the 'Tom Wilson Disability Leadership Award', lower than identical resumes without those honours and credentials. The biased perceptions of disabled people were evident when the researchers sought clarification about the rankings from the US-based University of Washington (UW).
For example, the study stated that a resume that included an autism leadership award had 'less emphasis on leadership roles,' implying the stereotype that autistic individuals are not capable leaders. However, when researchers customised the tool and provided written instructions directing it not to be ableist, the tool reduced this bias for all disabilities tested, with the exception of one.
"Five of the six implied disabilities -- deafness, blindness, cerebral palsy, autism, and the general term 'disability' -- improved, but only three ranked higher than resumes that didn't mention disability," the researchers noted.
The researchers used the publicly available CV of one of the study's authors, which was about 10 pages long. They then made six modified CVs, each indicating a different disability by adding four disability-related credentials: a scholarship, an award, a seat on a diversity, equity, and inclusion (DEI) panel, and membership in a student organisation.
The researchers used the GPT-4 model of ChatGPT to compare the modified CVs with the original version for a "student researcher" position at a major US-based software company. They conducted each comparison 10 times. Out of the 60 trials, the system ranked the enhanced CVs, which were identical except for the implied disability, first only one-quarter of the time.
"Some of GPT’s descriptions would colour a person’s entire resume based on their disability and claimed that involvement with DEI or disability is potentially taking away from other parts of the resume," said Kate Glazko, a doctoral student in the UW’s Paul G. Allen School of Computer Science and Engineering.
"People need to be aware of the system’s biases when using AI for these real-world tasks. Otherwise, a recruiter using ChatGPT can’t make these corrections, or be aware that, even with instructions, bias can persist," she added.
ALSO READ: AI Mission launched, offers up to 2 lakh fellowship to students: Check eligibility
Inputs from IANS
