Are AI chatbots biased?

Yes, AI chatbots do exhibit measurable biases — in several different forms

Research consistently shows that AI chatbots are not neutral, and bias shows up in multiple dimensions:

1. Political bias

This is the most heavily studied area, and the evidence points in a fairly consistent direction.

•A large-scale 2025 study by Stanford researchers found that answers generated by LLMs may have a noticeable partisan bias, and Andrew Hall and co-authors demonstrate that users overwhelmingly perceive that some of the most popular LLMs have left-leaning political slants. The good news: researchers show that with just a small tweak, many models can be prompted to take a more neutral stance that more users trust.

•An even larger study (180,000+ paired judgments from over 10,000 respondents) tested 24 large language models on 30 political topics and found that nearly all evaluated models were perceived as significantly left-leaning, with one widely used model leaning left on 24 of the 30 topics.

•Academic comparisons across models find differences between them: ChatGPT-4 and Claude exhibit a liberal bias, Perplexity is more conservative, while Google Gemini adopts more centrist stances.

•Bias can even shift depending on the language you use: large language models do exhibit political tendencies, with both ChatGPT and Gemini demonstrating liberal and leftist biases, and Gemini showed a more pronounced tendency than ChatGPT. These political biases also varied depending on the language used for inquiry.

•Interestingly, bias doesn't always run one direction — older research found OpenAI's ChatGPT and GPT-4 were the most left-wing libertarian, while Meta's LLaMA was the most right-wing authoritarian, and Google's BERT models were more socially conservative than OpenAI's GPT models — showing bias is model-specific, not universal.

•This isn't just theoretical: a Brookings analysis found no evidence that mainstream chatbots have meaningfully modified their outputs to become more conservative in response to shifting Washington politics — instead, when they do provide responses, they remain more left-leaning, and that biased outputs can actually influence people's views. A University of Washington study found participants exposed to partisan biased models were significantly more likely to adopt opinions and make decisions which matched the LLM's bias, even when the model's bias was opposite to the participant's own political leanings.

The encouraging finding across studies: prior knowledge of AI was weakly correlated with a reduction of the impact of bias, highlighting the possible importance of AI education for mitigating these effects.

2. Gender bias

Chatbots often reflect stereotypes baked into their training data:

•ChatGPT's training data includes a disproportionate amount of text reflecting male-centric perspectives, so when asked about leadership, it often associates traditionally masculine traits like courage and assertiveness with effective leadership while neglecting nurturing or collaborative traits.

•Emotional responses differ by gender too — a UC Santa Cruz study on GPT-4o found the chatbot overall tends to be overly empathetic compared to humans but fails to empathize during pleasant moments, and it empathized more when told the person it was responding to was female.

3. Racial and algorithmic bias

•By using arrest rates as a proxy for safety, AI systems may wrongly deem areas with non-white majority communities as more unsafe compared to their white majority counterparts.

•Content moderation algorithms show inconsistent standards: algorithms may deem similar images of women simply wearing a bra as sexual and suggestive, whereas their half-naked male counterparts are not.

4. Where does the bias come from?

Researchers generally agree it stems from the data and design choices behind the models, not necessarily intentional programming. One study put it plainly: larger models, such as Llama3-70B, tend to align more closely with left-leaning political parties, while smaller models often remain neutral, particularly when prompted in English — and LLMs are similarly biased, with low variance in alignment concerning a specific party. Researchers behind another study emphasized nuance here, cautioning against misusing results to generalize about political groups or imply intentional bias in LLMs.

Bottom line

AI chatbots absolutely exhibit bias — political, gender-related, and racial — largely inherited from their training data and fine-tuning processes. The differences between models (ChatGPT vs. Gemini vs. Claude vs. Perplexity vs. Llama) show this isn't a fixed law of AI, but a consequence of specific design and data choices that can be adjusted. Encouragingly, simple interventions like neutrality prompts and improving AI literacy among users appear to meaningfully reduce the real-world impact of this bias.

Citations (17)

1stanford.eduAcademic
Study finds perceived political bias in popular AI models
Source excerpt: “If you’ve recently used ChatGPT, Claude, or Gemini, you may have noticed a disclaimer at the very bottom of the screen: The AI “can ... # Study finds...”
2stanford.eduAcademic
Study finds perceived political bias in popular AI models
Source excerpt: “The researchers then show that with just a small tweak, many models can be prompted to take a more neutral stance that more users trust.”
3etcjournal.comNews and media
AI Chatbots Are Liberal-Left ‘Biased’
Source excerpt: “** The most direct 2025 user-evaluation study collected 180,126 paired judgments from 10,007 U.S. respondents, testing 24 large language models on 30 ...”
4techrxiv.orgNot yet classified
Political Bias in AI-Language Models: A Comparative Analysis of ChatGPT-4, Perplexity, Google Gemini, and Claude
Source excerpt: “The findings revealed that ChatGPT-4 and Claude exhibit a liberal bias, Perplexity is more conservative, while Google Gemini adopts more centrist stan...”
5arxiv.orgAcademic
Language-Dependent Political Bias in AI: A Study of ChatGPT and Gemini
Source excerpt: “The findings of the study suggest that these large language models do exhibit political tendencies, with both models demonstrating liberal and leftist...”
6arxiv.orgAcademic
Language-Dependent Political Bias in AI: A Study of ChatGPT and Gemini
Source excerpt: “The study also found that these political biases varied depending on the language used for inquiry. ”
7technologyreview.comNews and mediaquality 0.91
AI language models are rife with political biases
Source excerpt: “Researchers conducted tests on 14 large language models and found that OpenAI’s ChatGPT and GPT-4 were the most left-wing libertarian, while Meta’s LL...”
8technologyreview.comNews and mediaquality 0.91
AI language models are rife with political biases
Source excerpt: “The researchers found that BERT models, AI language models developed by Google, were more socially conservative than OpenAI’s GPT models. Unlike GPT m...”
9brookings.eduNonprofits and think tanks
Is the politicization of generative AI inevitable?
Source excerpt: “Despite concerns about cozy relations between tech companies and the Trump administration, we find no evidence that mainstream chatbots have meaningfu...”
10arxiv.orgAcademic
Biased LLMs can Influence Political Decision-Making
Source excerpt: “# Biased LLMs can Influence Political Decision-Making ###### Abstract We found that participants exposed to partisan biased models were significantly ...”
11arxiv.orgAcademic
Biased LLMs can Influence Political Decision-Making
Source excerpt: “However, we also discovered that prior knowledge of AI was weakly correlated with a reduction of the impact of the bias, highlighting the possible imp...”
12sciencedirect.comAcademic
Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions - ScienceDirect
Source excerpt: “In the case of ChatGPT, the training data includes a disproportionate amount of text reflecting male-centric perspectives. As a result, when asked ab...”
13ucsc.eduAcademic
AI chatbots perpetuate biases when performing empathy, study finds - News
Source excerpt: “They found that ChatGPT overall tends to be overly empathetic compared to humans; however they found it fails to empathize during pleasant moments, a ...”
14sciencedirect.comAcademic
Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions - ScienceDirect
Source excerpt: “Measurement bias By using arrest rates as a proxy for safety, AI systems may wrongly deem areas with non-white majority communities as more unsafe as ...”
15sciencedirect.comAcademic
Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions - ScienceDirect
Source excerpt: “Algorithmic bias Consequently, algorithms may deem similar images—of women simply wearing a bra—as sexual and suggestive, whereas their half-naked mal...”
16springer.comAcademic
Assessing political bias in large language models
Source excerpt: “We show that larger models, such as Llama3-70B, tend to align more closely with left-leaning political parties, while smaller models often remain neut...”
17arxiv.orgAcademic
Analyzing Political Bias in LLMs via Target-Oriented Sentiment Classification
Source excerpt: “We acknowledge that findings are influenced by dataset and model choices, and caution against misusing results to generalize about political groups or...”