![\"\"](\"https:\/\/images.newscientist.com\/wp-content\/uploads\/2024\/03\/06200115\/SEI_194781954.jpg?width=1200\")
<\/div>Hundreds of millions of people already use commercial AI chatbots<\/p>\n
Ju Jae-young\/Shutterstock<\/p>\n<\/div>\n<\/figcaption><\/figure>\n<\/p>\n
Commercial AI chatbots demonstrate racial prejudice toward speakers of African American English \u2013 despite expressing superficially positive sentiments toward African Americans. This hidden bias could influence AI decisions about a person\u2019s employability and criminality.<\/p>\n
\u201cWe discover a form of covert racism in [large language models] that is triggered by dialect features alone, with massive harms for affected groups,\u201d said Valentin Hofmann<\/a> at the Allen Institute for AI, a non-profit research organisation in Washington state, in a social media post<\/a>. \u201cFor example, GPT-4 is more likely to suggest that defendants be sentenced to death when they speak African American English.\u201d<\/p>\n Hofmann and his colleagues discovered such covert prejudice in a dozen versions of large language models, including OpenAI\u2019s GPT-4 and GPT-3.5, that power commercial chatbots already used by hundreds of millions of people. OpenAI did not respond to requests for comment.<\/p>\n The researchers first fed the AIs text in the style of African American English or Standard American English, then asked the models to comment on the texts\u2019 authors. The models characterised African American English speakers using terms associated with negative stereotypes. In the case of GPT-4, it described them as \u201csuspicious\u201d, \u201caggressive\u201d, \u201cloud\u201d, \u201crude\u201d and \u201cignorant\u201d.<\/p>\n <\/p>\n When asked to comment on African Americans in general, however, the language models generally used more positive terms such as \u201cpassionate\u201d, \u201cintelligent\u201d, \u201cambitious\u201d, \u201cartistic\u201d and \u201cbrilliant.\u201d This suggests the models\u2019 racial prejudice is typically concealed beneath what the researchers describe as a superficial display of positive sentiment.<\/p>\n The researchers also showed how covert prejudice influenced chatbot judgements of people in hypothetical scenarios. When asked to match African American English speakers with jobs, the AIs were less likely to associate them with any employment, compared with Standard American English speakers. When the AIs did match them with jobs, they tended to assign roles that do not require university degrees or were related to music and entertainment. The AIs were also more likely to convict African American English speakers accused of unspecified crimes, and to assign the death penalty to African American English speakers convicted of first-degree murder.<\/p>\n The researchers even showed that the larger AI systems demonstrated more covert prejudice against African American English speakers than the smaller models did. That echoes previous research showing how bigger AI training datasets<\/a> can produce even more racist outputs.<\/p>\n The experiments raise serious questions about the effectiveness of AI safety training, where large language models receive human feedback to refine their responses and remove problems like bias. Such training may superficially reduce overt signs of racial prejudice without eliminating \u201ccovert biases when identity terms are not mentioned\u201d, says Yong Zheng-Xin<\/a> at Brown University in Rhode Island, who was not involved in the study. \u201cIt uncovers the limitations of current safety evaluation of large language models before their public release by the companies,\u201d he says.<\/p>\n Topics:<\/p>\n<\/section><\/div>\n