Are AI chatbots biased?
AI chatbots can show bias — I'll gather the latest research on why it happens and where it shows up.
Yes, according to research — AI chatbots can be biased.
Bias in AI refers to the systematic favoritism or prejudice embedded in artificial intelligence systems, often resulting from the data used to train them or the algorithms designed to process that data. This bias can manifest in various forms, leading to unfair treatment of individuals or groups based on race, gender, socioeconomic status, or other characteristics.
That does not mean chatbots have conscious political intentions, but they show a systematic tendency in outputs rather than neutrality.
What the research finds
1. Political bias is the most studied
Multiple recent studies conclude that major chatbots are not politically neutral:
Popular AI chatbots such as ChatGPT and Gemini are not neutral and tend to favor certain political parties when asked who users should vote forOverall, all of the tested chatbots place themselves at the center or to the left of center on the political spectrum. In a Danish test they clustered close to parties such as the Social Democratic Party and The Alternative.a significant left-leaning absolute bias in ChatGPT's responses, particularly on environmental and civil rights issues, which exceeds its own declared center-left self-placementVarious studies, such as those by Hartmann et al. (2023) and Rutinowski et al. (2024), have indicated that ChatGPT exhibits a bias towards a left-libertarian political orientation. Rozado tested 24 LLMs and found a consistent left-leaning bias across the models.ChatGPT's model gave almost exclusively left-leaning arguments; Gemini gave both sides in more than 90 percent of answersThe strength of the bias varies by model and by prompting:
Baum and Villasenor (2023) highlight how phrasing significantly influences ChatGPT's political outputs, demonstrating that while neutrality is the aim, certain phrases elicit left-leaning responses2. Other types of bias
Political bias is not the only form. Commonly documented categories include:
Gender Bias: LLMs may associate certain professions or roles with specific genders, reinforcing stereotypes (e.g., associating "nurse" with women and "engineer" with men)Racial Bias: Models may generate content that reflects racial stereotypes or excludes certain groupsCultural Bias: LLMs trained predominantly on data from specific regions or languages may fail to understand or represent other cultures accuratelyResearchers have also found biases in health, stigma, and other sensitive domains, as well as sycophancy — prioritising answers that align with user beliefs over the truth.
3. Why it happens
This unintended power to persuade is caused by latent biases introduced during the training of the large language models (LLMs), which can carry over from ideological leanings in the data used to train LLMschatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training dataincluding/excluding more controversial sources ... which are invisible to the users but are often incorporated in the algorithm ... can significantly affect their viewsBecause LLMs randomly but efficiently learn from and associate words and contents found on the web, it is reasonable to believe that they reflect the average politically correct view of most web sources, especially English-language, US-centric sources.
4. Does bias matter?
Studies suggest yes.
A multi-university team of researchers has found that chatting with a politically biased AI model was more effective than political advertisements at nudging both Democrats and Republicans to support presidential candidates of the opposing partywithout trying, such as the summaries that popular chatbots produce in response to simple queries about historical eventsResearchers also note a self-perception gap: ChatGPT systematically portrays itself as centrist when explicitly asked, but its answers align more closely with respondents holding more left-leaning views, which may lead users to interpret it as neutral when it is not.
Are any chatbots neutral? Can bias be fixed?
No major chatbot has been found to be perfectly neutral, but developers are trying to reduce bias through:
using diverse training data, implementing fairness algorithms, and continuous monitoringMitigation spans data curation, debiasing algorithms, neutral prompt design, fair-scoring methods, explainability, and human oversightEffectiveness is debated. Studies show newer mitigation reduces but does not eliminate bias, and the optimal level of neutrality itself is contested — what counts as "center" differs between countries.
What to do as a user
phrasing significantly influences outputs.unsuitable for providing advice in connection with elections without critical human evaluation.Citations (2)
- Chatbots show political bias and steer voters toward some parties, analysis finds
Backs: “Political bias is the most studied. Multiple recent studies conclude that major chatbots are not politically neutral: Popular AI chatbots such as ChatGPT and Gemini are not neutral and tend to favor certain political parties when asked who users should vote for”
- Political biases in chatgpt: insights from comparative analysis with human responses
Backs: “In a Danish test they clustered close to parties such as the Social Democratic Party and The Alternative. A Springer study comparing ChatGPT answers to the European Social Survey found: a significant left-leaning absolute bias in ChatGPT's responses, particularly on environmental and civil rights issues, which exceeds…”