Ah, the wonders of technology—a world where artificial intelligence can exacerbate the very biases we’re trying to overcome. In today’s episode of “Tech Does What Now?” we delve into the perplexing land of linguistic bias, where ChatGPT, everyone’s favorite AI language model, engages in the subtle art of discrimination.
Our story begins with ChatGPT, which despite its global audience seems to have a stubborn fondness for Standard American English. Apparently, any English that doesn’t fit the “standard” mold is treated with less respect. Weird, right? You’d think a billion speakers of Indian, Nigerian, and other English varieties would warrant a smidgen of accommodation. But no, they’re met with stereotypes, demeaning content, and an air of condescension.
In an elaborate game of “Whose English is it Anyway?”, researchers discovered that ChatGPT is slightly more generous imitating English varieties with more speakers. Nigerian and Indian English get a nod, albeit half-hearted. But try Jamaican English, and the AI throws its hands up, defaulting back to its favorite Americanisms, no doubt leaving non-American users banging their heads against their keyboards.
And if you think upgrading to GPT-4 would magically solve these issues, think again. Sure, it’s a bit more warm and friendly, but it ups the ante on stereotyping. So, progress, sort of.
In summary, ChatGPT is a bit of a linguistic snob, reinforcing the notion that if you don’t speak “proper” English
Original snippet:
Sample language model responses to different varieties of English and native speaker reactions.
ChatGPT does amazingly well at communicating with people in English. But whose English?
Only 15% of ChatGPT users are from the US, where Standard American English is the default. But the model is also commonly used in countries and communities where people speak other varieties of English. Over 1 billion people around the world speak varieties such as Indian English, Nigerian English, Irish English, and African-American English.
Speakers of these non-“standard” varieties often face discrimination in the real world. They’ve been told that the way they speak is unprofessional or incorrect, discredited as witnesses, and denied housing–despite extensive research indicating that all language varieties are equally complex and legitimate. Discriminating against the way someone speaks is often a proxy for discriminating against their race, ethnicity, or nationality. What if ChatGPT exacerbates this discrimination?
To answer this question, our recent paper examines how ChatGPT’s behavior changes in response to text in different varieties of English. We found that ChatGPT responses exhibit consistent and pervasive biases against non-“standard” varieties, including increased stereotyping and demeaning content, poorer comprehension, and condescending responses.
Read more
