Are AI chatbots biased?

Are AI chatbots biased? I'll sort the evidence on types of bias and what recent research finds.

Yes, evidence from multiple recent studies shows AI chatbots are biased — though the type and degree of bias varies by model, task and prompt.

What the research finds

1. Political / ideological bias

Researchers consistently find a left-leaning or center-left tilt in prominent models:

•A Washington Post analysis found OpenAI's models answering hot-button questions with left-leaning arguments in 80% of cases, while other models were less one-sided

•A University of Copenhagen analysis found popular chatbots such as ChatGPT and Gemini "are not neutral and tend to favor certain political parties when asked who users should vote for"

•In that Danish test, "all of the tested chatbots place themselves at the center or to the left of center on the political spectrum"

Bias can be subtle — not saying "vote for X" but emphasizing certain arguments or recommending some parties disproportionately.

Studies also show this bias can change users' views:

•Yale researchers found that latent biases introduced during training of large language models (LLMs) lend "subtle nuances to the framing of the narratives the chatbots generate"

•"We show that querying an AI chatbot to obtain historical facts can influence people's opinions even when the information provided is accurate and nobody has prompted the tool to try to persuade you"

•Compared to Wikipedia entries, both default AI summaries and those prompted to have liberal framing caused participants to express more liberal opinions

Other work shows politically biased models were "more effective than political advertisements at nudging both Democrats and Republicans to support" opposing candidates, and that effects occurred "regardless of a person's own political party and after only a few conversational exchanges".

2. Racial and gender bias

•A Stanford Law School audit of GPT-4, GPT-3.5 and Google's PaLM-2 found "significant disparities across names associated with race and gender" — for example a chatbot suggesting a salary of $79,375 for a name like Tamika versus $82,485 for a name like Todd

•Overall, the chatbots' advice "systematically disadvantages names that are commonly associated with racial minorities and women," with names associated with Black women receiving the "least advantageous" outcomes

•Biases were "consistent across 42 prompt templates and several AI models, 'indicating a systemic issue.'"

Similar patterns appear in other domains:

•In hiring, image-generation and text tests, models produced white CEOs even when gender varied, while lower-status roles were assigned based on dialect

•In healthcare, specifying patients as female, African American or Hispanic led to decreased guideline-recommended management in one ChatGPT-3.5 test

•In finance, researchers found guidance "could be dispensing biased advice based on race or gender"

A 2026 MIT Centre for Constructive Communication study also found chatbots "can reinforce social biases, mirroring human prejudices" and often provide lower-quality responses to users with lower English proficiency, less education, or outside the US.

3. Other documented biases

•University College London researchers found ChatGPT and other chatbots give "flawed or biased moral advice, especially when users rely on them for decision-making," often displaying a "bias for inaction" and a tendency to answer "no" regardless of context

•Health audits found approximately half of answers to be misinformation in some tests

Why do chatbots become biased?

Bias is not usually intentional, but arises from how models are built:

•Biased or incomplete training data: Researchers attribute hallucinations and bias to "biased or incomplete training data". Chatbots are "trained on large volumes of public data, meaning that even small amounts of inaccurate or biased information can influence their responses".

•Internet-sourced stereotypes: Datasets "are often sourced from the internet, where harmful stereotypes and prejudices exist".

•Latent biases carried over: Ideological leanings in training data carry over into LLMs and shape framing.

•Algorithmic and data bias together: "Biases can be hidden in both algorithms and training data for each modality".

•Human-feedback tuning: Models fine-tuned on human feedback are known to exhibit "sycophancy – prioritising answers that align with user beliefs over the truth".

OpenAI has described bias as an "important, industry-wide problem" that its safety team is working to combat and said it is "continuously iterating on models to improve performance, reduce bias, and mitigate harmful outputs", but researchers note guardrails do not always work.

Does this mean chatbots are unusable?

Not necessarily, but experts advise caution:

•Bias is often modest and subtle — shifting framing rather than overtly false propaganda — but effects could compound if someone frequently relies on chatbots for facts.

•Transparency is limited: most commercial models do not disclose precise training data, making objectivity hard to evaluate, and users cannot see why a recommendation was made.

•Mitigation strategies being explored include more diverse training data, algorithm audits, dataset expansion, and continuous monitoring.

The best practice is to treat chatbots as helpful but imperfect tools: cross-check important information (especially on politics, health, finance, hiring), vary prompts, avoid revealing personal preferences if you want more neutral answers, and consult authoritative sources like peer-reviewed research, official statistics, or human experts for high-stakes decisions.

Citations (3)

1phys.orgNews and mediaquality 0.85
Chatbots show political bias and steer voters toward some parties, analysis finds
Backs: “A Washington Post analysis found OpenAI's models answering hot-button questions with left-leaning arguments in 80% of cases, while other models were less one-sided. A University of Copenhagen analysis found popular chatbots such as ChatGPT and Gemini "are not neutral and tend to favor certain political parties when…”
2yale.eduAcademic
AI’s hidden bias: Chatbots can influence opinions without trying
Backs: “Bias can be subtle — not saying "vote for X" but emphasizing certain arguments or recommending some parties disproportionately. Studies also show this bias can change users' views: Yale researchers found that latent biases introduced during training of large language models (LLMs) lend "subtle nuances to the framing…”
3beaconjournal.comNews and mediaquality 0.84
AI chatbots like ChatGPT treat Black names differently, per study
Backs: “Other work shows politically biased models were "more effective than political advertisements at nudging both Democrats and Republicans to support" opposing candidates, and that effects occurred "regardless of a person's own political party and after only a few conversational exchanges". 2. Racial and gender bias. A…”