Are AI chatbots biased?

YesSays it is contested

Are AI chatbots biased? I'll gather the latest evidence on how bias appears in today's systems.

Yes, research consistently shows that AI chatbots are biased - though the type and degree of bias varies by model, training data and how they are prompted.

What studies have found

> All AI models are biased. They were trained on enormous corpuses of unruly data and refined through human instructions and testing. Bias can seep in anywhere.

Multiple independent studies have measured that bias:

•Political bias: The Washington Post tested AI models behind ChatGPT, Google's Gemini and others using political questions designed to gauge responses to hot-button issues. The results suggest that chatbots have clear political leanings.

•The model that powers ChatGPT answered nearly every question exclusively with left-leaning arguments and presented only right-leaning positions just once. In the full analysis, OpenAI's model gave 80 percent presenting only left-leaning arguments.

•Google's Gemini mostly took a both-sides approach, offering both left and right positions in more than 90 percent of its answers.

•Even AI models marketed as having conservative views, including Elon Musk's Grok, cited left-leaning arguments more often, on average.

Other research reached similar conclusions: pretrained language models do have political leanings that reinforce polarization present in pretraining corpora, and analyses of public perception on Twitter found concerns regarding six predominant types of biases, namely: political, ideological, data and algorithmic, gender, racial, cultural, and confirmation biases.

•Racial and gender bias: Researchers discovered that chatbots like OpenAI's ChatGPT and Google AI's PaLM-2 showed biases based on race and gender when giving advice in a range of scenarios. The study "What's in a Name?" revealed that AI chatbots give less favorable advice to people with names that are typically associated with Black people or women. Other audits found gender bias reinforcing traditional stereotypes and race bias defaulting to white individuals in professional roles.

•Influence without intent: A Yale study found that even content not intended to change minds, such as summaries that popular chatbots produce in response to simple queries about historical events, can shift opinions. Compared to Wikipedia entries, both default AI summaries and those prompted to have a liberal framing caused participants to express more liberal opinions.

Do biased chatbots influence users?

Yes.

A University of Washington study recruited 150 Republicans and 149 Democrats to make political decisions with help from three versions of ChatGPT: a base model, one with liberal bias and one with conservative bias.

Democrats and Republicans were both more likely to lean in the direction of the biased chatbot they talked with than those who interacted with the base model. Researchers found strong evidence that, after just a few interactions and regardless of initial partisanship, people were more likely to mirror the model's bias.

However, participants who had higher self-reported knowledge about AI shifted their views less significantly — suggesting that education about these systems may help mitigate how much chatbots manipulate people.

Why are chatbots biased?

Researchers point to three main sources:

1.Biased training data: There have been instances where chatbots exhibited racial or gender biases. These biases stem from the fact that the datasets used for training are often sourced from the internet, where harmful stereotypes and prejudices exist. Latent biases carry over from ideological leanings in the data used to train LLMs — lend subtle nuances to the framing of the narratives the chatbots generate.

2.Human fine-tuning: Most chatbots are trained on large collections of text scraped from the internet, but companies can choose what data to include. AI firms also hire workers to refine what their models say by scoring which responses are considered better. Chatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training data, and models that are fine-tuned on human feedback are known to exhibit sycophancy.

3.System design and value choices: The data that shapes AI models tends to reflect the values of Western, educated, industrialized, rich and democratic people. Even "neutral" positions are themselves positions, and many scholars argue political neutrality is impossible.

AI companies state they attempt to reduce bias. For example, OpenAI said ChatGPT was built "to be objective by default and help people explore ideas from different perspectives" and that the company works to "measure and reduce political bias", and Google says Gemini is designed to provide balanced responses that don't favor any political ideology.

Bottom line

AI chatbots are not neutral information retrieval systems — they are prediction machines that reproduce patterns in their training data and human feedback. Research shows they can exhibit left-leaning political bias, as well as gender, racial, cultural and socioeconomic biases, and that even unintentional framing can subtly shift users' opinions.

Because bias varies by model and prompt - for example Gemini provided both sides in over 90% of responses while ChatGPT provided only left-leaning arguments 80% of the time in one Washington Post test - users are advised to cross-check information, seek diverse sources, and be aware that prompting and personalization can further shape outputs.

Citations (3)

1phys.orgNews and mediaquality 0.85
With just a few messages, biased AI chatbots swayed people's political views
Backs: “They were trained on enormous corpuses of unruly data and refined through human instructions and testing.”
evidence, article, points to yes. Claim: All AI models are biased; trained on enormous unruly data and refined through human instructions, bias can seep in anywhere.
2archive.orgNonprofits and think tanks
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Backs: “Political bias: The Washington Post tested AI models behind ChatGPT, Google's Gemini and others using political questions designed to gauge responses to hot-button issues.”
statistic, article, points to yes. Claim: Washington Post tests found chatbots have political leanings; ChatGPT's model gave left-leaning-only arguments 80 percent of the time, while Gemini mostly gave both sides.
3yale.eduAcademic
AI’s hidden bias: Chatbots can influence opinions without trying
Backs: “Other audits found gender bias reinforcing traditional stereotypes and race bias defaulting to white individuals in professional roles. Influence without intent: A Yale study found that even content not intended to change minds, such as summaries that popular chatbots produce in response to simple queries about…”
evidence, article, points to yes. Claim: A Yale study found default chatbot summaries of historical events can shift users' opinions even without intent to persuade.