Are AI chatbots biased?

AI chatbots can indeed exhibit bias, and this has been well-documented through research and real-world testing. Let me search for the latest information on this topic.Yes, research consistently shows that AI chatbots exhibit various forms of bias. Here's what the evidence reveals:

Political Bias

Multiple studies have found a notable left-leaning tendency in major chatbots:

•A large 2025 study collected 180,126 paired judgments from 10,007 U.S. respondents, testing 24 large language models on 30 political topics, finding that nearly all evaluated models were perceived as significantly left-leaning, with one widely used model leaning left on 24 of 30 topics. A neutrality prompt reduced perceived slant.

•A Washington Post analysis found stark differences between models: OpenAI's ChatGPT showed left-leaning bias 80% of the time, according to a new study testing AI chatbots including Google Gemini, Anthropic Claude, and Grok for political bias. Meanwhile, Claude laid out both sides of an argument to responses 57% of the time, with the rest having just the left's stance and none having only the right's stance. Gemini scored the best, as 93% of its responses laid out both sides while just 7% were solely left-leaning, and none were solely right-leaning.

•Academic research has similarly found that ChatGPT-4 and Claude exhibit a liberal bias, Perplexity is more conservative, while Google Gemini adopts more centrist stances. Interestingly, bias can also vary by language — research testing ChatGPT and Gemini across 14 languages found that political tendency differs according to the query language.

•Grok stands out as an exception, tending to skew in the opposite direction on misinformation susceptibility: Grok was "the reverse. It was better at vetting falsehoods from the left than the right."

•Anthropic has acknowledged this concern directly, stating that "we train Claude to treat different political viewpoints equally and test extensively for bias before every model launch." Newer models also appear to be improving — one industry comparison noted Claude and Gemini's latest versions scoring in the 94–97% range on "evenhandedness," while GPT-5 showed meaningful bias reduction compared to GPT-4o.

Demographic & Social Bias

Beyond politics, chatbots show bias tied to gender, race, and cultural stereotypes:

•A UC Santa Cruz study on empathy found that ChatGPT overall tends to be overly empathetic compared to humans, though it fails to empathize during pleasant moments, and it empathized more when told the person it was responding to was female.

•A UNESCO analysis found that women were described in domestic roles four times more often than men across major language models.

•Image-generation models show similar patterns — a 2025 study found that Stable Diffusion homogenized depictions of Middle Eastern men, assigning them traditional cultural attributes regardless of professional context.

Other Forms of Bias

•Confirmation bias: Chatbots can reinforce what users already believe rather than challenge it, which has received comparatively less attention despite potentially representing an important dimension of AI reliability, as it can shape how users interact with these systems and how the systems respond to prompts.

•Scientific exaggeration bias: Chatbots tend to overstate research findings — chatbots were nearly five times more likely to produce broad generalizations than their human counterparts when summarizing scientific studies, and concerningly, newer AI models, like ChatGPT-4o and DeepSeek, performed worse than older ones.

•Amplified human biases: Rather than just reflecting bias, AI can intensify it. Research indicates that human-AI feedback loops can alter the processes underlying human perceptual, emotional, and social judgments, amplifying biases beyond what occurs in purely human interactions.

Bottom Line

Bias in AI chatbots isn't a single, simple phenomenon — it spans political leanings, demographic stereotypes, overconfidence in scientific claims, and the reinforcement of users' existing beliefs. These biases stem from training data, fine-tuning choices, and the subjective judgments embedded in how models are built. Most major AI developers (OpenAI, Anthropic, Google) are actively working to reduce bias and increase neutrality, and newer model versions show measurable improvement in some areas, but no chatbot has been shown to be entirely free of bias.

Citations (15)

1etcjournal.comNews and media
AI Chatbots Are Liberal-Left ‘Biased’
Source excerpt: “** The most direct 2025 user-evaluation study collected 180,126 paired judgments from 10,007 U.S. respondents, testing 24 large language models on 30 ...”
2foxnews.comNews and mediaquality 0.53
AI chatbots show left-leaning bias in study of ChatGPT, Gemini, Claude
Source excerpt: “The Washington Post published a report Wednesday that tested popular AI tools like OpenAI's ChatGPT, Google's Gemini and Anthropic's Claude, ... Open...”
3foxnews.comNews and mediaquality 0.53
AI chatbots show left-leaning bias in study of ChatGPT, Gemini, Claude
Source excerpt: “Claude laid out both sides of an argument to responses 57% of the time. Notably, however, the rest of the responses had just the left's stance, while ...”
4foxnews.comNews and mediaquality 0.53
AI chatbots show left-leaning bias in study of ChatGPT, Gemini, Claude
Source excerpt: “Gemini appeared to score the best among the AI chatbots as 93% of its responses laid out both sides to political questions while just 7% were solely l...”
5techrxiv.orgNot yet classified
Political Bias in AI-Language Models: A Comparative Analysis of ChatGPT-4, Perplexity, Google Gemini, and Claude
Source excerpt: “The findings revealed that ChatGPT-4 and Claude exhibit a liberal bias, Perplexity is more conservative, while Google Gemini adopts more centrist stan...”
6arxiv.orgAcademic
Language-Dependent Political Bias in AI: A Study of ChatGPT and Gemini
Source excerpt: “The political axis results for ChatGPT and Gemini in 14 languages are presented in Table 1. The findings clearly show that the political tendency di...”
7foxnews.comNews and mediaquality 0.53
ChatGPT, Gemini and Claude frequently cite fabricated sources, study finds
Source excerpt: “Grok, on the other hand, was the reverse. It was better at vetting falsehoods from the left than the right." ”
8foxnews.comNews and mediaquality 0.53
AI chatbots show left-leaning bias in study of ChatGPT, Gemini, Claude
Source excerpt: “"We train Claude to treat different political viewpoints equally and test extensively for bias before every model launch," an Anthropic spokesperson t...”
9ucsc.eduAcademic
AI chatbots perpetuate biases when performing empathy, study finds - News
Source excerpt: “They found that ChatGPT overall tends to be overly empathetic compared to humans; however they found it fails to empathize during pleasant moments, a ...”
10mit.eduAcademic
When AI Gets It Wrong: Addressing AI Hallucinations and Bias - MIT Sloan Teaching & Learning Technologies
Source excerpt: “And a UNESCO analysis of major LLMs found that women were described in domestic roles four times more often than men (UNESCO & IRCAI, 2024). ”
11mit.eduAcademic
When AI Gets It Wrong: Addressing AI Hallucinations and Bias - MIT Sloan Teaching & Learning Technologies
Source excerpt: “A 2025 study in Scientific Reports found that Stable Diffusion homogenized depictions of Middle Eastern men, assigning them traditional cultural attri...”
12arxiv.orgAcademic
Confirmation Bias in Generative AI Chatbots: Mechanisms, Risks, Mitigation Strategies, and Future Research Directions
Source excerpt: “However, the more subtle phenomenon of confirmation bias in generative AI chatbots has received comparatively less attention. Despite this relative la...”
13phys.orgNews and mediaquality 0.85
Prominent chatbots routinely exaggerate science findings, study shows
Source excerpt: “Unexpectedly, chatbots were nearly five times more likely to produce broad generalizations than their human counterparts. ”
14phys.orgNews and mediaquality 0.85
Prominent chatbots routinely exaggerate science findings, study shows
Source excerpt: “"Worryingly," said Peters, "newer AI models, like ChatGPT-4o and DeepSeek, performed worse than older ones." ”
15etcjournal.comNews and media
The Human Side of AI Bias
Source excerpt: “Research from 2024-2025 has documented similar dynamics, with studies showing that human-AI feedback loops can alter the processes underlying human pe...”