Are AI chatbots biased?

Yes — AI chatbots exhibit several well-documented forms of bias

Research over the past few years has consistently found that AI chatbots are not neutral, and bias shows up in multiple distinct ways:

1. Political/ideological bias

This is the most heavily studied area. A large 2025 study by Stanford researchers using 180,126 paired judgments from over 10,000 U.S. respondents found that the most direct 2025 user-evaluation study collected 180,126 paired judgments from 10,007 U.S. respondents, testing 24 large language models on 30 political topics, finding that nearly all evaluated models were perceived as significantly left-leaning, and that one widely used model leaned left on 24 of the 30 topics. Notably, Andrew Hall, a professor of political economy at Stanford Graduate School of Business, and two co-authors demonstrate that users overwhelmingly perceive that some of the most popular LLMs have left-leaning political slants.

This isn't limited to the U.S. or English-language models. A German study found larger models, such as Llama3-70B, tend to align more closely with left-leaning political parties, while smaller models often remain neutral, particularly when prompted in English, and interestingly, these findings suggest a left-leaning bias inherent in all models studied. Research on parliamentary voting records across multiple countries found that state-of-the-art LLMs consistently display left-leaning or centrist tendencies alongside clear negative bias toward right-conservative parties across languages.

A comparative study across major chatbots found that ChatGPT-4 and Claude exhibit a liberal bias, Perplexity is more conservative, while Google Gemini adopts more centrist stances based on their training data sets — showing bias isn't uniform across all systems.

However, measuring this bias is tricky. One study found that LLMs do not consistently reveal their political biases in response to standard questions, with their political orientations being elusive, easily influenced by subtle changes in phrasing and context. The good news: the Stanford team found with just a small tweak, many models can be prompted to take a more neutral stance that more users trust.

2. Demographic bias (race, gender, etc.)

OpenAI itself studied this in a major 2025 paper, introducing a method to evaluate a scalable counterfactual approach to evaluate "first-person fairness," meaning fairness toward chatbot users based on demographic characteristics, employing a Language Model as a Research Assistant to yield quantitative measures of harmful stereotypes and qualitative analyses of demographic differences in chatbot responses, applied across six language models across millions of interactions, covering sixty-six tasks in nine domains and spanning two genders and four races. Importantly, they found that post-training reinforcement learning techniques significantly mitigate these biases — meaning bias can be reduced through deliberate training interventions.

3. Emotional/empathy bias

A UC Santa Cruz study on GPT-4o found gaps exist between how AI chatbots perform empathy compared to humans, suggesting bias can creep into even seemingly neutral emotional support contexts.

4. Bias in sensitive domains like mental health

Stanford researchers found that AI therapy chatbots powered by large language models can introduce biases and failures that could result in dangerous consequences, particularly around stigma toward certain mental health conditions.

5. Confirmation bias and influence on users

Beyond containing bias themselves, chatbots can amplify users' existing beliefs. Research on "confirmation bias in generative AI chatbots" examines how confirmation bias, as understood in cognitive psychology, might manifest in the particular setting of AI-generated conversations. Separately, UC San Diego researchers presented work in late 2025 showing chatbot bias can meaningfully shape human decision-making, aiming to reduce the risk of systemic bias in decision-making across media, education and public policy caused by content alteration from LLMs.

Why does this happen?

Bias largely stems from training data (which reflects existing societal biases), the design choices made during fine-tuning/RLHF, and model scale — one study even found that LLM characteristics, including release date, model scale, and region of origin affect political bias, with results indicating political biases evolve with model scale and release date, and are also influenced by regional factors.

Bottom line

Bias in AI chatbots is real, measurable, and multi-dimensional — spanning political leanings, demographic stereotypes, emotional responses, and domain-specific blind spots (like mental health). It's not necessarily intentional, but it is a consistent finding across many independent research teams using different methodologies. The encouraging part is that bias can be reduced through techniques like reinforcement learning fine-tuning and careful prompting, though complete neutrality remains elusive.