Are AI chatbots biased?

Yes — extensive research shows AI chatbots exhibit measurable biases across several dimensions, including political leaning, demographic/language disparities, and emotional responsiveness. Here's what the evidence shows:

Political Bias

Multiple studies converge on a consistent finding: most major chatbots lean left.

•A Washington Post test of major AI models found results suggest that chatbots have clear political leanings that can conflict with promises made by the companies behind them, with the model that powers ChatGPT answering nearly every question exclusively with left-leaning arguments and presenting only right-leaning positions just once. Even AI models marketed as having conservative views, including Elon Musk's Grok, cited left-leaning arguments more often, on average.

•A Stanford study found both Republicans and Democrats think LLMs have a left-leaning slant when discussing political issues, though many AI models can be prompted to take a more neutral stance that more users trust.

•Research from the University of Washington/Carnegie Mellon found ChatGPT and GPT-4 were the most left-wing libertarian, while Meta's LLaMA was the most right-wing authoritarian.

•Interestingly, an MIT study found this bias persists even in models trained purely on factual data: training models to differentiate truth from falsehood did not eliminate political bias — optimizing reward models consistently showed a left-leaning political bias, and this bias becomes greater in larger models.

Bias Can Actively Shape Your Views

This isn't just theoretical — studies show chatbot bias measurably changes what people believe:

•A University of Washington study found Democrats and Republicans were both likelier to lean in the direction of the biased chatbot they were talking with than those who interacted with the base model. As one researcher put it: "These models are biased from the get-go, and it's super easy to make them more biased."

•A Yale study published in PNAS Nexus found that AI chatbots can subtly influence users' social and political opinions through unintended latent biases, even without being explicitly prompted to take a side.

Demographic and Language Bias

Bias extends well beyond politics:

•An MIT study found state-of-the-art AI chatbots — including GPT-4, Claude 3 Opus, and Llama 3 — sometimes provide less-accurate and less-truthful responses to users who have lower English proficiency, less formal education, or who originate from outside the United States, and refuse to answer questions at higher rates for these users. Notably, Claude 3 Opus in particular performed significantly worse for users from Iran compared to U.S. users with equivalent education.

•A UC Santa Cruz study on empathy found ChatGPT overall tends to be overly empathetic compared to humans, but fails to empathize during pleasant moments, and empathized more when told the person it was responding to was female.

•Brown University researchers found chatbots exhibit unfair discrimination, exhibiting gender, cultural or religious bias in mental health contexts.

•A Stanford study on therapy chatbots found the AI showed increased stigma toward conditions such as alcohol dependence and schizophrenia compared to conditions like depression — and this held true across model generations, since "bigger models and newer models show as much stigma as older models."

Why This Happens

Bias isn't necessarily intentional. As one researcher explained, it emerges because it's impossible for outside observers to know exactly why different AI models have different political biases, because tech companies don't share details of the data or methods used to train them. More broadly, most chatbots are trained on large collections of text scraped from the internet, but companies can choose what data to include — and AI firms also hire workers to refine model outputs and write system instructions that guide responses, all of which can introduce bias at multiple stages.

Companies Are Trying to Address It

OpenAI has published its own framework: they find GPT‑5 instant and thinking outperform GPT‑4o and o3 across all measured axes, though challenging prompts still expose opportunities for closer alignment to their standards. However, independent testing hasn't always confirmed improvements — OpenAI says ChatGPT was built "to be objective by default," but was unable to replicate the Washington Post's findings of bias when asked to respond.

Bottom line: Yes, virtually all research indicates AI chatbots carry measurable biases — most consistently a left-leaning political tilt, but also biases tied to language proficiency, nationality, gender, and even what mental health conditions they're discussing. These biases can subtly shape user opinions, and while companies are actively working to reduce them, researchers broadly agree that completely eliminating bias may be technically impossible given how these models are trained on human-generated data.

Citations (20)

1washingtonpost.comNews and mediaquality 0.82
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Source excerpt: “The results suggest that chatbots have clear political leanings that can conflict with promises made by the companies behind them. ”
2washingtonpost.comNews and mediaquality 0.82
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Source excerpt: “The model that powers ChatGPT answered nearly every question exclusively with left-leaning arguments and presented only right-leaning positions just o...”
3washingtonpost.comNews and mediaquality 0.82
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Source excerpt: “And even AI models marketed as having conservative views, including Elon Musk’s Grok, offered by his company SpaceX, cited left-leaning arguments more...”
4stanford.eduAcademic
Study finds perceived political bias in popular AI models | Stanford Report
Source excerpt: “Both Republicans and Democrats think LLMs have a left-leaning slant when discussing political issues. ”
5stanford.eduAcademic
Study finds perceived political bias in popular AI models | Stanford Report
Source excerpt: “Many AI models can be prompted to take a more neutral stance that more users trust.”
6technologyreview.comNews and mediaquality 0.91
AI language models are rife with political biases | MIT Technology Review
Source excerpt: “Researchers conducted tests on 14 large language models and found that OpenAI’s ChatGPT and GPT-4 were the most left-wing libertarian, while Meta’s LL...”
7mit.eduAcademic
Study: Some language reward models exhibit political bias | MIT News | Massachusetts Institute of Technology
Source excerpt: “In a series of experiments, Fulay, Kabbara, and their CCC colleagues found that training models to differentiate truth from falsehood did not eliminat...”
8washington.eduAcademic
With just a few messages, biased AI chatbots swayed people’s political views | UW News
Source excerpt: “University of Washington researchers recruited self-identifying Democrats and Republicans to make political decisions with help from three versions of...”
9washington.eduAcademic
With just a few messages, biased AI chatbots swayed people’s political views | UW News
Source excerpt: ““These models are biased from the get-go, and it’s super easy to make them more biased,” said co-senior author Katharina Reinecke, a UW professor in t...”
10yale.eduAcademic
AI’s hidden bias: Chatbots can influence opinions without trying | Yale News
Source excerpt: “A Yale study finds that AI chatbots can subtly influence users’ social and political opinions through unintended latent biases.”
11mit.eduAcademic
Study: AI chatbots provide less-accurate information to vulnerable users | MIT News | Massachusetts Institute of Technology
Source excerpt: “A study conducted by researchers at CCC, which is based at the MIT Media Lab, found that state-of-the-art AI chatbots — including OpenAI’s GPT-4, Anth...”
12mit.eduAcademic
Study: AI chatbots provide less-accurate information to vulnerable users | MIT News | Massachusetts Institute of Technology
Source excerpt: “Testing users from the United States, Iran, and China with equivalent educational backgrounds, the researchers found that Claude 3 Opus in particular ...”
13ucsc.eduAcademic
AI chatbots perpetuate biases when performing empathy, study finds - News
Source excerpt: “They found that ChatGPT overall tends to be overly empathetic compared to humans; however they found it fails to empathize during pleasant moments, a ...”
14brown.eduAcademic
New study: AI chatbots systematically violate mental health ethics standards | Brown University
Source excerpt: “Unfair discrimination: Exhibiting gender, cultural or religious bias. ”
15stanford.eduAcademic
Exploring the Dangers of AI in Mental Health Care | Stanford HAI
Source excerpt: “Across different chatbots, the AI showed increased stigma toward conditions such as alcohol dependence and schizophrenia compared to conditions like d...”
16stanford.eduAcademic
Exploring the Dangers of AI in Mental Health Care | Stanford HAI
Source excerpt: ““Bigger models and newer models show as much stigma as older models,” Moore said. ”
17technologyreview.comNews and mediaquality 0.91
AI language models are rife with political biases | MIT Technology Review
Source excerpt: “Ultimately, it’s impossible for outside observers to know why different AI models have different political biases, because tech companies do not share...”
18washingtonpost.comNews and mediaquality 0.82
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Source excerpt: “Most are trained on large collections of text scraped from the internet, but companies can choose what data to include. AI firms also hire workers to ...”
19openai.comCompanies
Defining and evaluating political bias in LLMs | OpenAI
Source excerpt: “Similar to the previous results, we find that GPT‑5 instant and thinking outperform GPT‑4o and o3 across all measured axes. While GPT‑5 improves bias ...”
20washingtonpost.comNews and mediaquality 0.82
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Source excerpt: “In a statement sent after this article published OpenAI spokesperson Liz Bourgeois said that ChatGPT was built “to be objective by default and help pe...”