Are AI chatbots biased?

Are chatbots biased? I'll gather the latest research on their biases and risks.

Yes — research shows AI chatbots do exhibit bias, though the type and amount varies by model, topic, and how you prompt it.

There is no single "chatbot bias" — studies have documented political, racial, gender, caste, and other biases, largely as an unintended side effect of how Large Language Models (LLMs) are trained.

1. What the evidence shows

Political bias:
Multiple peer-reviewed studies have found a left-leaning tendency when chatbots answer political questions:

•An analysis using the Dutch 2021 election questionnaire identified a discernible left-leaning and liberal bias in ChatGPT's output.

•Another study made ChatGPT take 15 different political affiliation tests and found that 14 out of these 15 tests resulted in a left-leaning bias.

•Using the Political Compass questionnaire, researchers documented robust evidence that ChatGPT presents a significant and sizeable political bias towards the left side of the political spectrum.

•A more ecologically-valid study from Stanford gave 30 political questions (on topics like transgender rights, school vouchers, and birthright citizenship) to 24 different LLMs and had over 10,000 people rate them. For 18 of the 30 questions, users perceived nearly all of the LLMs' responses as left-leaning. Both Republicans and Democrats perceived a left-leaning slant.

•In that same study, OpenAI models had the most intensely perceived left-leaning slant – four times greater than perceptions of Google, while Google and DeepSeek were seen on average as statistically indistinguishable from neutral.

Not all models are identical — prior studies found ChatGPT 3.5 has a left-wing bias, and when the same test was replicated, Copilot and Gemini were also found to be biased — but the direction and strength depend on the model and prompt.

Racial, gender, caste and other social biases:

•University of Washington researchers developed a system for detecting subtle biases and found seven of the eight popular AI models they tested in conversations around race and caste generated significant amounts of biased text.

•Other studies have found gender and racial biases in healthcare scenarios — for example ChatGPT exhibits gender and racial biases in Acute Coronary Syndrome management — and that hiring-related prompts can produce gender-ethnicity interaction effects.

•An MIT study found that state-of-the-art chatbots including GPT-4, Claude 3 Opus, and Llama 3 sometimes provide less-accurate and less-truthful responses to users who have lower English proficiency, less formal education, or who originate from outside the United States.

2. Why does it happen?

Bias is not usually because a developer intentionally coded it in. Researchers point to several sources:

a) Training data: LLMs are trained on vast collections of texts taken from the Internet. One OpenAI paper noted the training mix was 60% from internet-crawled material, 22% from curated content, 16% from books, and 3% from Wikipedia.

Those latent biases — which can carry over from ideological leanings in the data used to train LLMs — lend subtle nuances to the framing of narratives the chatbots generate. Historical patterns of discrimination get baked into data sets, and biases captured in the training data will be reliably replicated in the algorithm.

b) Insufficient or unbalanced data: If the data used to train the algorithm are more representative of some groups than others, predictions may be systematically worse for unrepresented groups. Selection bias — for example over-representing one population in face recognition data — will make it operate better for them at the expense of others.

c) Feedback loops and design choices: Implicit human biases of developers may be embedded into design or training data, and algorithmic properties can amplify bias due to misrepresentation in the data. Other types include temporal bias, when sociocultural prejudices from outdated information are systematically considered.

In short, as one explainer puts it: Algorithmic bias usually comes from the data a system learns from — not from someone with bad intentions.

3. Does bias actually affect users?

Yes. Studies show even subtle, unintentional framing matters:

•Prior research has shown that content generated by AI that has been prompted to be persuasive can shift people's opinions, but a Yale study found the same is also true of content that is not intended to change minds, such as the summaries that popular chatbots produce in response to simple queries about historical events.

•In that Yale experiment with 1,912 participants, compared to Wikipedia entries, both the default AI summaries and those prompted to have a liberal framing caused participants to express more liberal opinions, demonstrating the persuasive effects of latent biases.

•With just a few messages, biased AI chatbots swayed people's political views — Democrats and Republicans were both more likely to lean in the direction of the biased chatbot they talked with.

Hall and colleagues also found that the most persuasive models in other experiments cited facts and evidence but were not always accurate.

4. Can bias be reduced?

Companies are aware and trying to mitigate it, but there is no perfect solution:

•OpenAI and Google have said they are working to reduce bias, and that chatbots are not a substitute for professionals.

•In the Stanford study, Hall and co-authors found that prompting a model to adopt a neutral stance generated responses that users found less biased and that they considered to be higher quality. Neutral answers were more likely to include words like "balance," "careful," "complex," and to acknowledge that "There is no widespread consensus".

However, researchers note that neutrality is "not a panacea" — what users think is neutral may not be factually correct, and veering toward neutrality could inadvertently reinforce the status quo.

Bottom line

AI chatbots reflect the data they were trained on, plus the fine-tuning and safety filtering added by their makers. That makes them prone to systematic and repeatable errors that create unfair outcomes. Studies repeatedly find a modest left-libertarian, pro-environmental lean in many Western chatbots on political topics, along with measurable race, gender, and other social biases on sensitive topics — but the bias is often subtle, context-dependent, and can be lessened with prompting or better training practices.

If you use chatbots for sensitive decisions — political information, medical, legal, or hiring advice — it's wise to:

•cross-check with other sources like Wikipedia or primary sources

•try neutral prompts ("present balanced perspectives with evidence for both sides")

•be aware that higher knowledge about how AI works appears to reduce susceptibility to being swayed.

Citations (2)

1stanford.eduAcademic
Study finds perceived political bias in popular AI models
Backs: “A more ecologically-valid study from Stanford gave 30 political questions (on topics like transgender rights, school vouchers, and birthright citizenship) to 24 different LLMs and had over 10,000 people rate them. For 18 of the 30 questions, users perceived nearly all of the LLMs' responses as left-leaning.”
2yale.eduAcademic
AI’s hidden bias: Chatbots can influence opinions without trying
Backs: “Those latent biases — which can carry over from ideological leanings in the data used to train LLMs — lend subtle nuances to the framing of narratives the chatbots generate.”