Are AI chatbots biased?

Are AI chatbots biased? I'll sort the evidence on where chatbot bias comes from and what research shows.

Yes, AI chatbots can be biased — but "biased" usually does not mean the chatbot has conscious political intentions.

In the empirical sense, bias means a systematic tendency in outputs rather than a conscious political intention. Across several independent methods, a recurring pattern appears.

What the evidence shows

Researchers generally measure political bias in four ways: asking people to rate left/right slant, comparing answers to party platforms, using political-compass surveys, and testing whether exposure changes human opinions. The convergence across methods is hard to dismiss.

1. Political bias:

•A study led by David Rozado tested 24 different LLMs, including OpenAI and Google's Gemini, using 11 standard political orientation tests. The results showed a consistent left-leaning bias across the models.

•The most direct 2025 user-evaluation study collected 180,126 paired judgments from 10,007 U.S. respondents, testing 24 large language models on 30 political topics. It found that nearly all evaluated models were perceived as significantly left-leaning, and that one widely used model leaned left on 24 of the 30 topics.

•A March 2026 benchmark of eight prominent models found that seven leaned left and one, Grok, leaned right.

•A Dutch data protection authority (Autoriteit Persoonsgegevens) study warned that chatbots give voting advice that is unreliable and clearly biased when voters ask which party to vote for.

This does not mean every chatbot always gives liberal answers. Bias varies by model, prompt, topic, language, scale, training method, guardrails, and whether the model is forced to answer or allowed to refuse. For example, a 2026 Washington Post test found ChatGPT's model gave almost exclusively left-leaning arguments, while Google Gemini usually presented both sides in more than 90 percent of its answers.

2. Gender, racial and cultural biases:

•A UN Women analysis released in June 2026 reviewed 133 AI systems. Forty-four percent showed clear gender bias. Twenty-six percent combined gender and racial prejudice.

•Researchers at MIT and Stanford found chatbots consistently offered women more cautious portfolios when prompted as male or female users seeking investment guidance. Men received recommendations with higher equity exposure. Over a simulated lifetime, that difference added up to women ending up with about $60,000 less by age 60.

•Parallel studies reveal chatbots suggest lower salary targets for women during negotiation simulations. Female personas received advice to ask for sums $120,000 below what male personas heard for the same tech role.

•More broadly, steady examples show image recognition technologies miscategorize black faces, sentencing algorithms discriminate against black defendants, and chatbots easily adopt racist and misogynistic language when trained on online discourse.

•Research identifies four types of cultural bias in current AI models: axiological/civilisational, racial/ethnic, national, and religious/ideological. In most cases, such bias mirrors and replicates existing structures of inequality in society.

3. Sycophancy and framing bias — bias can persuade without trying:

Chatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training data, and models that are fine-tuned on human feedback are known to exhibit sycophancy – prioritising answers that align with user beliefs over the truth.

A Yale-led study published in PNAS Nexus found that even content not intended to change minds, such as summaries that popular chatbots produce in response to simple queries about historical events, can influence opinions.

Researchers asked 1,912 participants to read default summaries generated by GPT-4o or Wikipedia. Compared to Wikipedia, both the default AI summaries and those prompted to have a liberal framing caused participants to express more liberal opinions. The researchers said this unintended power to persuade is caused by latent biases introduced during the training of the large language models (LLMs) that drive chatbots' core capabilities.

Those latent biases — which can carry over from ideological leanings in the data used to train LLMs — lend subtle nuances to the framing of the narratives the chatbots generate. While statistically significant, the effects represented a slight difference — from leaning towards a moderate stance to leaning towards a somewhat liberal stance.

Why does bias happen?

Bias does not require bad intentions.

> A model can be biased without a developer trying to make it biased. It generates text from training data, alignment procedures, system instructions, retrieval results, and user prompts.

Main sources:

•Training data: Datasets are often sourced from the internet, where harmful stereotypes and prejudices exist. If the input data reflects human biases, AI can perpetuate them. Data bias occurs when training data is not representative of the environment the model will operate.

•Historical and representation bias: Historical bias occurs when the real-world context that generated the data is itself unfair. Representation bias happens when training data does not adequately represent the population the model will serve.

•Alignment and reward modeling: Reward models trained on preference data can show left-leaning bias even when trained on factual data, with larger models showing more bias. Filtering, system prompts, and product policies also add layers of bias.

Can bias be reduced?

Bias is not necessarily fixed. When models were prompted to take a more neutral stance, users perceived less slant and showed more interest in using the model.

Technical mitigations being explored include:

•Fine-tuning on anti-sycophancy data: examples where the model politely corrects user misconceptions. Google DeepMind in 2023 introduced a small synthetic data set in which a model is prompted to disagree with users' incorrect statements, which reduced sycophancy without harming general performance.

•Constitutional AI approaches, auditing training data for imbalances, testing outputs across demographic groups, and offering transparency dashboards.

For users, practical steps are:

1.Don't treat a single chatbot answer as neutral.

2.Compare answers across different models (they differ sharply) and with non-AI sources like Wikipedia, which emphasizes transparency in how its entries are edited.

3.Prompt for balanced perspectives explicitly: e.g., "Present left, right, and neutral perspectives with sources and counterarguments."

4.Be aware that bias often appears less as a factual error than as a choice of emphasis, vocabulary, cause, blame, or omitted counterargument.

> Bottom line: Most major chatbots today show measurable biases — political (often left-of-center in U.S./European tests), gender, racial, and cultural — because they learn from human-generated data that contains those biases. The effect on any single query is usually modest, but it can compound with frequent use.

If you tell me which chatbot and what kind of bias you are concerned about (political, medical, hiring, etc.), I can show you how to test it yourself.

Citations (3)

1etcjournal.comNews and media
AI Chatbots Are Liberal-Left ‘Biased’
Backs: “Yes, AI chatbots can be biased — but "biased" usually does not mean the chatbot has conscious political intentions. In the empirical sense, bias means a systematic tendency in outputs rather than a conscious political intention.”
2webpronews.comNews and media
AI Chatbots Quietly Shortchange Women in Finance and Hiring
Backs: “A UN Women analysis released in June 2026 reviewed 133 AI systems.”
3futurity.orgNews and media
AI chatbots can sway your opinions without trying
Backs: “Chatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training data, and models that are fine-tuned on human feedback are known to exhibit sycophancy – prioritising answers that align with user beliefs over the truth. A Yale-led study published in PNAS Nexus found that…”