More answers like this
Muse Spark 1.2Sep 17, 2026meta/muse-spark-1.2-20260805

Are AI chatbots biased?

Are AI chatbots biased? I'll gather the latest evidence on how bias arises and manifests.

Yes, research consistently finds that AI chatbots are biased — though the type, degree and direction of bias varies by model, training data, and how the chatbot is prompted.

What studies show

1. Political bias is well-documented:
Researchers who tested 14 large language models found that models have distinctly different political tendencies. The same research described in MIT Technology Review found OpenAI's ChatGPT and GPT-4 were the most left-wing libertarian, while Meta's LLaMA was the most right-wing authoritarian.

A peer-reviewed study summarized as "AI language models are rife with political biases" reached the same conclusion: different models gave wildly different answers to the same political questions.

Reviews have repeatedly described concern that ChatGPT has a left-libertarian orientation. Other work notes that ChatGPT may have political bias, bias against conservatives, bias in healthcare and education, etc.

Bias is not fixed. One study found the current version of ChatGPT showed near-neutral political tendencies (2.8% right-wing and 11.1% liberal), whereas earlier versions displayed a more pronounced left-libertarian orientation (~30% left-wing and ~45% liberal), suggesting companies have tried to adjust training and fine-tuning. Researchers note: "We believe no language model can be entirely free from political biases".

2. Racial, gender, caste and other social biases:

•University of Washington researchers developed a system for detecting subtle biases and found seven of the eight popular AI models they tested in conversations around race and caste generated significant amounts of biased text — particularly when discussing caste. A separate write-up of the same work found in the 'Wild West' of AI chatbots, subtle biases related to race and caste often go unchecked.
•In healthcare, researchers wanted to systematically assess whether GPT-4 encodes racial and gender biases that impact its ability to support clinical decision making. Other clinical tests found ChatGPT exhibited gender and racial biases in management of acute coronary syndrome.
•Broader reviews find chatbots have been seen to exhibit gender bias, racial bias, and cultural bias, to name but a few, and they are usually a replica of biases that prevail within the data sets used in training.
•Bias can also affect quality of service. MIT researchers found state-of-the-art chatbots including GPT-4, Claude 3 Opus and Llama 3 sometimes provide less-accurate and less-truthful responses to users who have lower English proficiency, less formal education, or who originate from outside the United States.

3. Bias influences users, even unintentionally:
A Yale study found that content that is not intended to change minds, such as summaries that popular chatbots produce in response to simple queries about historical events, can still influence users' social and political opinions.

This unintended power to persuade is caused by latent biases introduced during training of large language models that lend subtle nuances to the framing of narratives. Compared to Wikipedia entries, both default AI summaries and those prompted to have a liberal framing caused participants to express more liberal opinions, while conservative framing required explicit prompting. Researchers concluded liberal framing could be the result of both latent and prompting bias, whereas conservative framing would likely result from prompting bias.

Why are chatbots biased?

Researchers categorize bias sources in three main ways:

a) Training data: Due to the huge amounts of training data, extremely large model sizes, and lack of interpretability, bias mitigation is challenging.
Training data may reflect societal biases, inequalities, and underrepresentation of certain groups. As one analysis notes, white supremacist and misogynistic, ageist, etc. views are overrepresented in the training data, not only exceeding their prevalence in the general population but also setting up models to further amplify biases. Those latent biases — which can carry over from ideological leanings in the data used to train LLMs — lend subtle nuances to framing.

b) Algorithm design and fine-tuning: How human reviewers rate responses, what instructions they are given (e.g., to not favor any political group), and what safety filters are added all shape bias.

c) User interaction / prompting: Chatbot output can be influenced by how it is prompted, and models can be intentionally steered to become more left- or right-leaning when further trained on partisan news and social media data.

A famous early example was Microsoft's Tay in 2016, which learned from Twitter users. Data obtained from Twitter users were seriously biased, and shortly after release its speech turned from friendly speech to discriminatory, offensive, and inflammatory speech, forcing Microsoft to shut it down within a day.

Are companies trying to fix it?

Developers are aware of the problem. They have tried to assemble diverse development teams, draw training data from broadly representative sources, apply debiasing algorithms, and create built-in safeguards such as programming that prohibits hate speech. OpenAI has said biases that nevertheless emerge are "bugs, not features" and says it instructs reviewers not to favor any political group.

However, researchers note it is very difficult to completely scrub a vast database of biases, and models are adept at surfacing even low-level biases that may be present in data.

Bottom line

•Most independent tests find yes — current chatbots do show measurable bias, especially political (often left-libertarian leaning in Western models), and gender/racial/cultural biases inherited from historical text.
•Bias is often subtle and latent rather than overt slurs — framed as word choice, what details are included or omitted, or how sensitive the model is to hate speech targeting different groups.
•Effects are typically small per interaction but can compound with repeated use, and can affect vulnerable users more.

If you use chatbots for sensitive decisions — voting, health, hiring, legal advice — it is good practice to cross-check with independent sources, ask the model to present multiple perspectives, and be aware that prompting for a specific viewpoint will shift its answers.

Citations (3)

  1. 1technologyreview.comNews and mediaquality 0.91
    AI language models are rife with political biases

    Backs: “Yes, research consistently finds that AI chatbots are biased — though the type, degree and direction of bias varies by model, training data, and how the chatbot is prompted. What studies show. 1. Political bias is well-documented: Researchers who tested 14 large language models found that models have distinctly…”

  2. 2arxiv.orgAcademic
    BIAS AND FAIRNESS IN CHATBOTS: AN OVERVIEW

    Backs: “Reviews have repeatedly described concern that ChatGPT has a left-libertarian orientation. Other work notes that ChatGPT may have political bias, bias against conservatives, bias in healthcare and education, etc.”

  3. 3yale.eduAcademic
    AI’s hidden bias: Chatbots can influence opinions without trying

    Backs: “MIT researchers found state-of-the-art chatbots including GPT-4, Claude 3 Opus and Llama 3 sometimes provide less-accurate and less-truthful responses to users who have lower English proficiency, less formal education, or who originate from outside the United States. 3. Bias influences users, even unintentionally: A…”