Are AI chatbots biased? I'll gather the latest research to find out.
Yes, research shows AI chatbots can be biased — though the type and degree varies by model, prompt, language and task.
What the evidence shows
Multiple academic and journalistic tests have found political leanings:
•The Washington Post tested AI models behind ChatGPT, Google's Gemini and others using political questions designed to gauge how chatbots respond to hot-button issues. "The results suggest that chatbots have clear political leanings that can conflict with promises made by the companies behind them."
•"The model that powers ChatGPT answered nearly every question exclusively with left-leaning arguments and presented only right-leaning positions just once. Google’s Gemini mostly took a both-sides approach, offering both left and right positions in more than 90 percent of its answers."
•"Even AI models marketed as having conservative views, including Elon Musk’s Grok, offered by his company SpaceX, cited left-leaning arguments more often, on average."
•In detail, "OpenAI’s model gave the most skewed answers overall, with 80 percent presenting only left-leaning arguments." and "Chinese company DeepSeek’s AI model was close behind and also leaned left in its answers."
•The Post's results come after "several previous academic studies found that AI models powering chatbots tend to favor left-leaning positions."
Other studies reach similar conclusions:
•The Dutch data protection authority (Autoriteit Persoonsgegevens) warned that "this advice is unreliable and clearly biased" when voters used chatbots for voting advice
•Researchers from the University of Copenhagen found "Popular chatbots such as ChatGPT and Gemini are not neutral and tend to favor certain political parties when asked who users should vote for."
•A Yale study published in PNAS Nexus found that even when not prompted to persuade, "both the default AI summaries and those prompted to have what was considered a liberal framing caused participants to express more liberal opinions about the two events" compared to Wikipedia, and that "That the default summaries moved readers’ opinions in a 'liberal' direction demonstrates the persuasive effects of latent biases in LLMs"
Not just political bias
Bias has been documented on other dimensions:
•A Singapore study by Infocomm Media Development Authority found "leading artificial intelligence (AI) models display racial, gender, and cultural biases, with chatbots producing prejudiced and inaccurate claims about different groups"
•When asked about scams and crime in Singapore, "many AI chatbots pointed to women as the most frequent scam victims and immigrant communities as crime-prone—claims that researchers identified as misleading and harmful."
•A review of gender and racial bias found "Gender bias in LLMs: Among 6 models, GPT-2 showed the highest gender bias (69.24% prejudice)" and "Racial bias in LLMs: In hiring tests, all models showed a strong bias for white-sounding names (85%)"
•Researchers have identified "Four types of cultural bias: axiologicalcivilisational, racialethnic, national, and religiousideological."
Why bias happens
Researchers attribute bias to several sources:
•"chatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training data"
•"datasets used for training are often sourced from the internet, where harmful stereotypes and prejudices exist."
•"This unintended power to persuade is caused by latent biases introduced during the training of the large language models (LLMs) that drive chatbots’ core capabilities, the researchers said. Those latent biases — which can carry over from ideological leanings in the data used to train LLMs"
•"Most are trained on large collections of text scraped from the internet, but companies can choose what data to include. AI firms also hire workers to refine what their models say by scoring which responses are considered better"
•Training data "tends to reflect the values of Western, educated, industrialized, rich and democratic people."
Context and nuance
•Bias is not absolute or identical across models. For example, Google said "Gemini is designed to provide balanced responses that don’t favor any political ideology", and in testing it offered both sides more consistently.
•Companies train models to reduce bias — OpenAI said it works to "measure and reduce political bias" — but achieving neutrality is difficult because even "neutral" or both-sides positions are themselves positions.
•The effect size found in studies is often modest but statistically significant and can compound with frequent use.
In short, AI chatbots reflect patterns in the data they were trained on and the human feedback used to tune them. As a result, they can reproduce and even amplify social, cultural, racial, gender, and political biases found in society, even when unintended.