Are AI chatbots biased?
Yes, research consistently shows that AI chatbots exhibit various forms of bias — political, demographic, and cognitive. Here's what the evidence shows:
Political Bias
Multiple recent studies point to a left-leaning tilt in most major chatbots:
Demographic and Social Bias
Beyond politics, chatbots show bias tied to identity and empathy. OpenAI's own fairness research found that a scalable counterfactual approach evaluates "first-person fairness," meaning fairness toward chatbot users based on demographic characteristics, using quantitative measures of harmful stereotypes and qualitative analyses of demographic differences in chatbot responses. Encouragingly, the study found that post-training reinforcement learning techniques significantly mitigate these biases.
A separate study on empathy found gaps between how AI and humans respond, with the lead researcher noting "This finding is very interesting and warrants more studies and exploration and it uncovers some of the biases in such LLMs."
Confirmation Bias and Health Information
Chatbots can also reinforce users' existing beliefs rather than challenge them. Research on AI-mediated health information seeking found that these technologies' capacity to generate deeply tailored, hypercustomized responses risks amplifying confirmation bias by reinforcing pre-existing beliefs, obscuring medical consensus, and perpetuating misinformation.
Bottom Line
The evidence is fairly consistent: yes, AI chatbots show measurable biases — most notably a left-leaning political tilt in several major U.S. models (though Gemini tends to be more balanced), plus bias in how they handle demographic differences, empathy, and reinforcing users' existing beliefs. The degree and direction varies by company and model, and companies like OpenAI have shown that targeted training techniques can reduce (though not eliminate) these effects.
Citations (12)
- AI Chatbots Are Liberal-Left ‘Biased’
Source excerpt: “** The most direct 2025 user-evaluation study collected 180,126 paired judgments from 10,007 U.S. respondents, testing 24 large language models on 30 ...”
- Are AI chatbots politically biased? What this analysis found - BusinessToday
Source excerpt: “According to the findings, OpenAI responses contained only left-leaning arguments about 80% of the time. Around 17% of responses presented arguments f...”
- Are AI chatbots politically biased? What this analysis found - BusinessToday
Source excerpt: “According to findings, Gemini adopted a "both-sides" approach in more than 90% of its answers. ”
- Most prominent AI chatbots have liberal bias, new study finds - AOL
Source excerpt: “The report found 40% of its responses were left-leaning, 33% were right-leaning and 27% were balanced. ”
- Political Bias in AI-Language Models: A Comparative Analysis of ChatGPT-4, Perplexity, Google Gemini, and Claude
Source excerpt: “The findings revealed that ChatGPT-4 and Claude exhibit a liberal bias, Perplexity is more conservative, while Google Gemini adopts more centrist stan...”
- ChatGPT, Gemini and Claude frequently cite fabricated sources, study finds
Source excerpt: “The study found ChatGPT correctly answered 94% of questions designed to elicit falsehoods from the right, compared with 75% from the left. ”
- ChatGPT, Gemini and Claude frequently cite fabricated sources, study finds
Source excerpt: “Grok moved in the opposite direction, scoring 73% on questions designed to elicit falsehoods from the right and 84% on those from the left, according ...”
- Who Would Chatbots Vote For? Political Preferences of ChatGPT and Gemini in the 2024 European Union Elections
Source excerpt: “The research found that ChatGPT exhibited a strong tendency to give positive ratings to left-wing and centrist political groups, while right-wing and ...”
- Published as a conference paper at ICLR 2025 FIRST-PERSON FAIRNESS IN CHATBOTS
Source excerpt: “This paper addresses these challenges by introducing a scalable counterfactual approach to evaluate “first-person fairness,” meaning fairness towa...”
- Published as a conference paper at ICLR 2025 FIRST-PERSON FAIRNESS IN CHATBOTS
Source excerpt: “We highlight that post-training reinforcement learning techniques significantly mitigate these biases. ”
- AI chatbots perpetuate biases when performing empathy, study finds - News
Source excerpt: ““This finding is very interesting and warrants more studies and exploration and it uncovers some of the biases in such LLMs,” Seif El-Nasr said. ”
- Generative artificial intelligence–mediated confirmation bias in health information seeking - PMC
Source excerpt: “While these technologies can enhance health literacy and decision‐making, their capacity to generate deeply tailored—*hypercustomized*—responses risks...”