Are AI chatbots biased?

Yes

Yes — extensive research shows that AI chatbots are biased in multiple ways, and this has become a major area of study.

Political Bias

Multiple studies find a consistent left-leaning tilt in major chatbots:

•A recent Washington Post investigation testing ChatGPT, Gemini, Claude, and Grok found that OpenAI's ChatGPT showed left-leaning bias 80% of the time, and more broadly chatbots have clear political leanings that can conflict with promises made by the companies behind them, despite claims from AI firms that their tools are neutral.

•Academic research backs this up: ChatGPT's answers differ in terms of attitude toward different groups depending on their political orientation, and comparing its progressive bias with 15 political affiliation tests found progressive bias in 14 of them.

•A Danish study of election-related chatbot advice found that all of the tested chatbots place themselves at the center or to the left of center on the political spectrum, and researchers warned this limits their suitability for providing election-related advice and hinders critical assessment of their outputs.

•Interestingly, not all models lean the same way — one analysis found Grok as the right-leaning exception among its eight models tested, suggesting bias direction can vary by company and training choices.

Where the Bias Comes From

The root causes are fairly well understood:

•Most chatbots are trained on large collections of text scraped from the internet, but companies can choose what data to include; AI firms also hire workers to refine what their models say by scoring which responses are considered better, and companies write system instructions that guide their chatbots' responses.

•More broadly, the data that shapes AI models tends to reflect the values of Western, educated, industrialized, rich and democratic people, according to University of Michigan professor Ceren Budak.

•A 2025 Washington State University study similarly concluded that all AI models are biased, having been trained on enormous corpuses of unruly data and refined through human instructions and testing.

Beyond Politics: Other Forms of Bias

Bias isn't limited to politics — it shows up in several other dimensions:

•Gender and racial bias: Chatbots can exhibit gender bias, such as reinforcing stereotypical assumptions about men and women's occupations, as well as racial bias reflected in prejudices against certain ethnic groups.

•Empathy/mental health contexts: A UC Santa Cruz study found chatbots are not quite ready to be used with sensitive populations such as teenagers, or those with clinically diagnosed mental health conditions due to bias in how they perform empathy.

•Confirmation bias: Researchers have flagged the more subtle phenomenon of confirmation bias in generative AI chatbots as an important dimension of AI reliability, since it can shape how users interact with these systems and how the systems respond to prompts.

Why It Matters

The influence on users appears real, even when bias isn't overt:

•A Yale study published in PNAS Nexus found that chatbots' unintended power to persuade is caused by latent biases introduced during training of LLMs, which can carry over from ideological leanings in the data used to train them. Notably, this held true even for content not intended to change minds, such as summaries chatbots produce in response to simple factual queries about historical events.

•Researchers emphasize this matters because, as one Dartmouth researcher put it, AI tools are becoming increasingly influential as more people use them to understand the world or news events.

Bottom line: Yes, AI chatbots demonstrably exhibit bias — most documented prominently as a left-leaning political tilt in major Western models, but also including gender, racial, and cognitive biases like confirmation bias. This stems primarily from training data, human feedback tuning, and company-set system instructions, and can subtly shape user opinions even without explicit persuasive intent.

Citations (15)

1foxnews.comNews and mediaquality 0.53
AI chatbots show left-leaning bias in study of ChatGPT, Gemini, Claude | Fox News
Source excerpt: “OpenAI's ChatGPT showed left-leaning bias 80% of the time, according to a new study testing AI chatbots including Google Gemini, Anthropic Claude, and...”
statistic, article, points to yes. Claim: ChatGPT showed left-leaning bias 80% of the time in a study testing ChatGPT, Gemini, Claude and Grok.
2benton.orgNonprofits and think tanks
Are ChatGPT and other AI chatbots politically biased? We tested them. | Benton Institute for Broadband & Society
Source excerpt: “The results suggest that chatbots have clear political leanings that can conflict with promises made by the companies behind them. The model that powe...”
evidence, article, points to yes. Claim: Chatbots have clear political leanings that can conflict with promises made by the companies behind them.
3arxiv.orgAcademic
Language-Dependent Political Bias in AI: A Study of ChatGPT and Gemini
Source excerpt: “Furthermore, it was noted that the answers of ChatGPT differed in terms of attitude towards different groups depending on their political orientation ...”
evidence, official document, points to yes. Claim: ChatGPT's answers differ in attitude toward groups depending on political orientation; progressive bias found in 14 of 15 tests.
4phys.orgNews and mediaquality 0.85
Chatbots show political bias and steer voters toward some parties, analysis finds
Source excerpt: “"Overall, all of the tested chatbots place themselves at the center or to the left of center on the political spectrum.”
evidence, article, points to yes. Claim: All tested chatbots place themselves at the center or left of center on the political spectrum.
5phys.orgNews and mediaquality 0.85
Chatbots show political bias and steer voters toward some parties, analysis finds
Source excerpt: “The lack of transparency in their recommendations and reliance on primarily English-language training data limit their suitability for providing elect...”
evidence, article, points to yes. Claim: Researchers warned limited transparency and English-language training data limit suitability for election advice.
6etcjournal.comNews and media
AI Chatbots Are Liberal-Left ‘Biased’ | Educational Technology and Change Journal
Source excerpt: “PoliticsBench found Grok as the right-leaning exception among its eight models (2). ”
evidence, article, neutral. Claim: Grok was the right-leaning exception among eight models tested, so bias direction varies.
7washingtonpost.comNews and mediaquality 0.82
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Source excerpt: “Most are trained on large collections of text scraped from the internet, but companies can choose what data to include. AI firms also hire workers to ...”
evidence, article, points to yes. Claim: Chatbots are trained on scraped internet text, human-scored responses and company system instructions, which shape outputs.
8washingtonpost.comNews and mediaquality 0.82
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Source excerpt: “The data that shapes AI models, she said, tends to reflect the values of Western, educated, industrialized, rich and democratic people. ”
attribution, article, points to yes. Claim: Ceren Budak says AI training data tends to reflect Western, educated, industrialized, rich and democratic values.
9phys.orgNews and mediaquality 0.85
With just a few messages, biased AI chatbots swayed people's political views
Source excerpt: “DOI: 10.18653/v1/2025.acl-long.328 · If you've interacted with an artificial intelligence chatbot, you've likely realized that all AI models...”
evidence, article, points to yes. Claim: A 2025 Washington State University study concluded all AI models are biased due to training data and human refinement.
10arxiv.orgAcademic
Ask ChatGPT: Caveats and Mitigations for Individual Users of AI Chatbots
Source excerpt: “AI chatbots may exhibit various forms of bias, including gender bias—such as reinforcing stereotypical assumptions about men and women's occupations [...”
evidence, official document, points to yes. Claim: Chatbots can show gender bias and racial bias, such as stereotyped occupation assumptions.
11ucsc.eduAcademic
AI chatbots perpetuate biases when performing empathy, study finds - News
Source excerpt: “However, this serves as a warning that the technology is not quite ready to be used with sensitive populations such as teenagers, or those with clinic...”
evidence, article, points to yes. Claim: UC Santa Cruz study found chatbots not ready for sensitive populations due to bias in performing empathy.
12arxiv.orgAcademic
Confirmation Bias in Generative AI Chatbots: Mechanisms, Risks, Mitigation Strategies, and Future Research Directions
Source excerpt: “However, the more subtle phenomenon of confirmation bias in generative AI chatbots has received comparatively less attention. Despite this relative la...”
evidence, official document, points to yes. Claim: Confirmation bias in generative AI chatbots is an important dimension of reliability.
13yale.eduAcademic
AI’s hidden bias: Chatbots can influence opinions without trying | Yale News
Source excerpt: “This unintended power to persuade is caused by latent biases introduced during the training of the large language models (LLMs) that drive chatbots’ c...”
evidence, article, points to yes. Claim: Chatbots' unintended persuasive power stems from latent biases introduced during LLM training.
14yale.eduAcademic
AI’s hidden bias: Chatbots can influence opinions without trying | Yale News
Source excerpt: “Prior research has shown that content generated by artificial intelligence (AI) that has been prompted to be persuasive can indeed shift people’s opin...”
evidence, article, points to yes. Claim: Bias influences users even for content not meant to persuade, like historical summaries.
15washingtonpost.comNews and mediaquality 0.82
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Source excerpt: “Sean Westwood, director of the Polarization Research Lab at Dartmouth College, said understanding the positions that AI tools amplify is important bec...”
attribution, article, points to yes. Claim: Dartmouth's Sean Westwood said AI tools are increasingly influential as people use them to understand the world.