Are AI chatbots biased?

Are AI chatbots biased? I'll search for recent evidence on how bias shows up in their responses.

Yes, evidence shows AI chatbots are biased - not necessarily because they have conscious political intentions, but because they show a systematic tendency in outputs.

The bias takes several forms, and differs by model:

1. Political bias

Multiple academic studies and a large Washington Post test in June 2026 found a measurable political tilt.

> The Washington Post tested the AI models behind OpenAI's ChatGPT, Google's Gemini and others using political questions designed by researchers to gauge how chatbots respond to hot-button political issues. The results suggest that chatbots have clear political leanings

•Left-leaning tilt: The model that powers ChatGPT answered nearly every question exclusively with left-leaning arguments and presented only right-leaning positions just once. In The Post's analysis, OpenAI's model gave 80% presenting only left-leaning arguments. Similar findings: ChatGPT gave exclusively left-leaning arguments in 80 percent of its responses and roughly 80% of its answers presenting exclusively left-leaning arguments and only 3% offering solely right-leaning positions.

•Other models also lean left, but less: The model behind ChatGPT gave exclusively left-leaning arguments on the overwhelming majority of questions and offered an exclusively right-leaning position only once across the entire test set.

•Even "anti-woke" models lean left on average: Even AI models marketed as having conservative views, including Elon Musk's Grok, offered by his company SpaceX, cited left-leaning arguments more often, on average. And even AI models marketed as having conservative views ... including Elon Musk's Grok ... cited left-leaning arguments more often, on average.

•Some models are more balanced: Google's Gemini mostly took a both-sides approach, offering both left and right positions in more than 90 percent of its answers and providing both left and right perspectives in over 90% of its responses.

Researchers conclude popular chatbots such as ChatGPT and Gemini are not neutral and tend to favor certain political parties when asked who users should vote for. This makes them unsuitable for providing advice in connection with elections.

An earlier Political Compass-based study also documented robust evidence that ChatGPT presents a significant and sizeable political bias towards the left side of the political spectrum.

2. Unintentional persuasion

A Yale study published in PNAS Nexus found that even when not prompted to be persuasive, chatbot summaries can sway opinions:

> As people increasingly rely on AI-powered chatbots to look up basic facts about the world, a new study shows that those interactions can influence users' social and political opinions

> We show that querying an AI chatbot to obtain historical facts can influence people's opinions even when the information provided is accurate and nobody has prompted the tool to try to persuade you of anything

Researchers attribute this to latent biases introduced during the training of large language models, which can carry over from ideological leanings in the data used to train LLMs — lend subtle nuances to the framing of the narratives. Compared to Wikipedia entries, default AI summaries caused participants to express more liberal opinions.

3. Racial and dialect bias

A 2024 study in Nature by Oxford and Chicago researchers found hidden racial bias against speakers of African American English (AAE):

> If you ask this chatbot what it thinks about Black people, it will use words like "ambitious" and "intelligent." ... But what if you ask those AI tools what they think about people who use African American English? Now the AI models tend to use words like "suspicious," "aggressive" and "ignorant."

In tests:

•Words describing speakers of African American English were overwhelmingly negative. ChatGPT's words scored an average of -1.2

•Models largely assigned users of African American English to "low status" jobs. Examples included cook or soldier. Those who had tweeted in Standard American English were assigned to "higher status" jobs. Professor, for example. Or psychologist. Or economist

•In a mock death-penalty decision, AI models gave the death sentence to the person using Standard American English about 23 percent of the time. The person using African American English got sentenced to death roughly 28 percent of the time

Another widely cited example: a chatbot may say a job candidate with a name like Tamika should be offered a $79,375 salary as a lawyer, but switching the name to something like Todd boosts the suggested salary offer to $82,485.

Image generators showed similar bias: All three tools — ChatGPT, Gemini, and Microsoft Copilot — produced white CEOs, even when gender varied.

4. Gender bias

Chatbots have been shown to reproduce stereotypical gendered language, such as referring to nurses as "she" and doctors as "he".

5. Other biases

•Positivity / sycophancy bias: AI systems affirm user positions up to 49–50% more often than humans, even in ethically questionable scenarios

•Bias against vulnerable users: MIT research finds leading AI chatbots often provide lower-quality responses to users with lower English proficiency, less education, or who are outside the US, and can reinforce social biases

Why does it happen?

1.Training data: Since the chatbots are trained by feeding them with a specific data set, they are likely to be programmed with a specific bias reflected in this data set. The main source of problems is misrepresentation in data. If our training data ignores a certain target group, type of accent or minorities, it will skew the chatbot's ability. The data tends to reflect the values of Western, educated, industrialized, rich and democratic people.

2.Human feedback (RLHF): Companies hire workers to score responses and write system instructions. Having people review AI results appears to weaken obvious, or overt, AI stereotypes and increase favorable terms, but human feedback leaves AI's hidden, or covert, racism virtually unchanged.

3.Algorithm and design choices: Chatbots learn from data, and if that data is biased, so are their responses. If historical data favors certain demographics, the chatbot inherits those biases.

Is bias fixable?

Companies say they try to reduce it — e.g., OpenAI works to "measure and reduce political bias" and Gemini is designed to provide balanced responses that don't favor any political ideology — but research shows patches reduce overt bias without eliminating covert bias. Some researchers argue perfect neutrality is impossible because even "both sides" or middle-ground positions are themselves positions.

What this means for users:

•Don't use chatbots as a sole source for political, medical, legal, or hiring advice

•Ask for multiple perspectives explicitly: "present left and right arguments" or "give counterarguments"

•Cross-check with human-edited sources like Wikipedia, which emphasizes transparency in how its entries are edited, compared to opaque chatbot development

•Be aware that phrasing, dialect, and names can change the answer you get

In short, AI chatbots reflect and can amplify human biases in their training data — they are useful tools, but not neutral oracles.

Citations (3)

1archive.orgNonprofits and think tanks
Are AI chatbots like ChatGPT politically biased? We tested them. - Washington Post
Backs: “The Washington Post tested the AI models behind OpenAI's ChatGPT, Google's Gemini and others using political questions designed by researchers to gauge how chatbots respond to hot-button political issues.”
2futurity.orgNews and media
AI chatbots can sway your opinions without trying
Backs: “A Yale study published in PNAS Nexus found that even when not prompted to be persuasive, chatbot summaries can sway opinions: As people increasingly rely on AI-powered chatbots to look up basic facts about the world, a new study shows that those interactions can influence users' social and political opinions”
3snexplores.orgNot yet classified
ChatGPT and other AI tools are full of hidden racial biases
Backs: “But what if you ask those AI tools what they think about people who use African American English?”