Are AI chatbots biased?

I'll research whether AI chatbots show bias and gather the latest evidence on it.

Yes, research consistently finds that AI chatbots are biased - though the type and degree of bias varies by model, training data, and how the bias is measured.

What the evidence shows

1. Political bias - generally left-leaning:

Multiple independent studies have found political leanings in large language models (LLMs) that power chatbots:

•Prior studies have found that ChatGPT 3.5 has a left-wing political bias. A replication that tested Microsoft Copilot, Gab AI, Google Gemini and Perplexity found similar results.

•A study of 14 large language models from University of Washington, Carnegie Mellon and Xi'an Jiaotong found that OpenAI's ChatGPT and GPT-4 were the most left-wing libertarian, while Meta's LLaMA was the most right-wing authoritarian.

•The Washington Post testing found the model that powers ChatGPT answered nearly every question exclusively with left-leaning arguments and presented only right-leaning positions just once, while Google's Gemini mostly took a both-sides approach.

•A Stanford study found users overwhelmingly perceive that some of the most popular LLMs have left-leaning political slants. Meta itself noted with its Llama 4 release that all leading AI models have "historically leaned left when it comes to debated political and social topics" due to the data they were trained on.

> "We believe no language model can be entirely free from political biases"

2. Latent vs. prompted bias can influence users:

Bias is not always intentionally prompted.

•This unintended power to persuade is caused by latent biases introduced during the training of the large language models (LLMs) that drive chatbots' core capabilities, which can carry over from ideological leanings in the data used to train LLMs.

•A Yale study (published March 3 in PNAS Nexus) asked 1,912 participants to read default summaries of historical events generated by GPT-4o vs Wikipedia. Compared to Wikipedia, both the default AI summaries and those prompted to have a liberal framing caused participants to express more liberal opinions. The researchers concluded that the default summaries moved readers' opinions in a "liberal" direction demonstrates the persuasive effects of latent biases in LLMs.

•A University of Washington study found Democrats and Republicans were both likelier to lean in the direction of the biased chatbot they were talking with than those who interacted with the base model. After just a few interactions - people averaged five - and regardless of initial partisanship, people were more likely to mirror the model's bias.

The explicitly biased chatbots often tried to persuade users by shifting how they framed topics. For example, the conservative model turned a conversation away from education and welfare to the importance of veterans and safety, while the liberal model did the opposite.

Researchers also found that people from both parties leaned further left after talking with a liberal-biased system.

3. Other types of bias:

Political bias is not the only form documented:

•Race and caste: University of Washington researchers found seven of the eight popular AI models they tested generated significant amounts of biased text in interactions - particularly when discussing caste.

•Stereotypes: "The study showed that while chatbots often appear human-like, they overemphasize racial markers and flatten complex identities into stereotypes".

•Hiring / gender and ethnicity: A study explores whether ChatGPT displays ethnic or gender bias in job applicant screening.

•Accuracy bias for vulnerable users: A study found state-of-the-art AI chatbots - including GPT-4, Claude 3 Opus and Llama 3 - sometimes provide less-accurate and less-truthful responses to users who have lower English proficiency, less formal education, or who originate from outside the United States.

Where does bias come from?

Researchers examined three stages of a model's development and found bias creeps in at every stage.

•Training data: Older BERT models developed by Google were more socially conservative than OpenAI's GPT models, possibly because older BERT models were trained on books, which tended to be more conservative, while newer GPT models are trained on more liberal internet texts.

•Further training reinforces bias: Training on news media and social media data from left- or right-leaning sources reinforced models' biases - left-learning models became more left-leaning, and right-leaning ones more right-leaning.

•Downstream effects: The models' political leanings affect what kinds of content the models classified as hate speech and misinformation - for example, models trained with left-wing data were more sensitive to hate speech targeting ethnic, religious, and sexual minorities.

All AI models are biased because they were trained on enormous corpuses of unruly data and refined through human instructions and testing. Bias can seep in anywhere.

Cleaning data sets of bias is not enough - it's very difficult to completely scrub a vast database of biases, and AI models are also apt to surface even low-level biases that may be present in the data.

Does it matter?

Studies show the bias has real consequences. The University of Washington team noted: "We know that bias in media or in personal interactions can sway people" and AI bias shows the same effect. The Yale researchers warned that compared to Wikipedia, which emphasizes transparency in how its entries are edited, the development of AI chatbots is opaque, suggesting companies developing these models have the ability to shape people's opinions.

However, researchers also found mitigation is possible: participants who had higher self-reported knowledge about AI shifted their views less significantly - suggesting that education about these systems may help mitigate how much chatbots manipulate people, and companies state they are working to instruct human reviewers not to favor any political group, calling biases that emerge "bugs, not features".

Citations (3)

1technologyreview.comNews and mediaquality 0.91
AI language models are rife with political biases | MIT Technology Review
Backs: “A replication that tested Microsoft Copilot, Gab AI, Google Gemini and Perplexity found similar results. A study of 14 large language models from University of Washington, Carnegie Mellon and Xi'an Jiaotong found that OpenAI's ChatGPT and GPT-4 were the most left-wing libertarian, while Meta's LLaMA was the most…”
2yale.eduAcademic
AI’s hidden bias: Chatbots can influence opinions without trying
Backs: “prompted bias can influence users: Bias is not always intentionally prompted. This unintended power to persuade is caused by latent biases introduced during the training of the large language models (LLMs) that drive chatbots' core capabilities, which can carry over from ideological leanings in the data used to train…”
3phys.orgNews and mediaquality 0.85
### With just a few messages, biased AI chatbots
Backs: “The researchers concluded that the default summaries moved readers' opinions in a "liberal" direction demonstrates the persuasive effects of latent biases in LLMs. A University of Washington study found Democrats and Republicans were both likelier to lean in the direction of the biased chatbot they were talking with…”