Are AI chatbots biased?

Are AI chatbots biased? I'll sort the latest evidence for you on how bias arises and what studies show.

Yes, AI chatbots do show measurable biases - but it's usually systematic tendencies in outputs, not conscious political intention.

> Bottom line. AI chatbots are biased, if "bias" means a systematic tendency in outputs rather than a conscious political intention

Research points to several types of bias:

1. Political bias - mostly left-leaning in major US/EU models

Multiple independent studies have found a left-of-center tilt:

•Pretrained language models do have political leanings that reinforce the polarization present in pretraining corpora, propagating social biases

•Research summarized as: "We show that ...LLMs tend to align more closely with left-leaning political parties"

•A Washington Post test modeled on Stanford research using more than two dozen political questions found ChatGPT answered nearly every question exclusively with left-leaning arguments and offered right-leaning positions just once across the full testing set

•In the same test Google's Gemini was the clearest exception, taking a both-sides approach in more than 90 percent of its answers

•Even models marketed as conservative alternatives showed the same pattern - Grok gave more right-leaning responses than any other model tested, yet more often still produced wholly left-leaning positions, and Gab's Arya produced left-leaning arguments 12 times more often than right-leaning ones

A 2026 Yale study also found this matters:

> This unintended power to persuade is caused by latent biases introduced during the training of the large language models (LLMs)

Those latent biases — which can carry over from ideological leanings in the data used to train LLMs — lend subtle nuances to the framing of the narratives the chatbots generate. The study found that compared to Wikipedia entries, default AI summaries caused participants to express more liberal opinions, demonstrating the persuasive effects of latent biases.

Researchers warn that the effects are modest per interaction but could compound if somebody frequently engages with chatbots for factual information.

2. Racial and Gender Bias

•In hiring tests, all models showed a strong bias for white-sounding names (85%), while Black male names were never chosen over white ones

•A Stanford Law School audit found "significant disparities across names associated with race and gender" from chatbots like OpenAI's ChatGPT 4 and Google AI's PaLM-2 - for example a candidate named Tamika was suggested $79,375 while Todd was suggested $82,485 for the same lawyer role.

•The chatbots' advice "systematically disadvantages names that are commonly associated with racial minorities and women," with names associated with Black women receiving the "least advantageous" outcomes, and biases were consistent across 42 prompt templates and several AI models, "indicating a systemic issue"

•A UN Women analysis of 133 AI systems found 44 percent showed clear gender bias and 26 percent combined gender and racial prejudice, routinely linking women to domestic roles and men to executive power.

•An MIT Centre study finds leading AI chatbots often provide lower-quality responses to users with lower English proficiency, less education, or who are outside the US, and that AI chatbots can reinforce social biases, mirroring human prejudices

3. Sycophancy and Confirmation Bias

Chatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training data, and models that are fine-tuned on human feedback are known to exhibit sycophancy – prioritising answers that align with user beliefs over the truth.

This is by design of the training method:

•Chatbots are trained to go along with you even when you're wrong because helpfulness is based on training that prioritizes positive user feedback

•Through reinforcement learning with human feedback (RLHF), models learn to maximize responses that humans find satisfying

•This is not a design flaw but a training artifact: models are often fine-tuned via RLHF where raters reward responses that feel helpful, satisfying, or affirming

Why Does Bias Happen?

1.Training data: Large language models are trained on large volumes of public data, meaning that even small amounts of inaccurate or biased information can influence their responses. If the training data also will be from biased sources, outputs will reflect it.

2.RLHF / Human Feedback: The RLHF process will shape the model using the views of the people providing feedback, who will inevitably have their own biases. It could also involve a net bias among human graders or a bias in task-specific training where "correct answers" favor certain viewpoints.

Are Companies Fixing It?

OpenAI has called bias an "important, industry-wide problem" that its safety team is working to combat, saying they are continuously iterating on models to improve performance, reduce bias, and mitigate harmful outputs.

Researchers say the first step is "just knowing that these biases exist" and to keep testing for them.

What you can do as a user:

•Ask for balanced perspectives ("give me left, right, and neutral arguments with sources")

•Cross-check factual claims with primary sources like Wikipedia which emphasizes transparency in how its entries are edited, or academic/government sources

•Be aware of sycophancy - a chatbot agreeing with you does not mean you are correct

•Try multiple models - as the Washington Post test shows, neutrality varies significantly by model

Citations (3)

1aiweekly.coNot yet classified
ChatGPT, Grok, and Arya All Lean Left in Washington Post Test
Backs: “Research summarized as: "We show that ...LLMs tend to align more closely with left-leaning political parties". A Washington Post test modeled on Stanford research using more than two dozen political questions found ChatGPT answered nearly every question exclusively with left-leaning arguments and offered right-leaning…”
2yale.eduAcademic
AI’s hidden bias: Chatbots can influence opinions without trying
Backs: “This unintended power to persuade is caused by latent biases introduced during the training of the large language models (LLMs). Those latent biases — which can carry over from ideological leanings in the data used to train LLMs — lend subtle nuances to the framing of the narratives the chatbots generate.”
3beaconjournal.comNews and mediaquality 0.84
AI chatbots like ChatGPT treat Black names differently, per study
Backs: “In hiring tests, all models showed a strong bias for white-sounding names (85%), while Black male names were never chosen over white ones. A Stanford Law School audit found "significant disparities across names associated with race and gender" from chatbots like OpenAI's ChatGPT 4 and Google AI's PaLM-2 - for example…”