I'll look into whether AI chatbots show bias and what the latest research says.
Yes, research consistently finds that AI chatbots can be biased - though the type and degree of bias varies by model, topic, and how the question is asked.
What studies show
•Systematic political lean: A Washington Post investigation tested the AI models behind OpenAI's ChatGPT, Google's Gemini and others using political questions designed to gauge responses to hot-button issues and found chatbots have clear political leanings. The model that powers ChatGPT answered nearly every question exclusively with left-leaning arguments and presented only right-leaning positions just once. Google's Gemini mostly took a both-sides approach, offering both left and right positions in more than 90 percent of its answers. And even AI models marketed as having conservative views, including Elon Musk's Grok, offered left-leaning arguments more often, on average.
•Unintentional persuasion: A Yale study published in PNAS Nexus found that querying an AI chatbot to obtain historical facts can influence people's opinions even when nobody has prompted the tool to try to persuade you. Compared to Wikipedia entries, both the default AI summaries and those prompted to have a liberal framing caused participants to express more liberal opinions about two historical events. The researchers said that demonstrates the persuasive effects of latent biases in LLMs.
•Status quo bias: A comprehensive study by researchers at the University of Waterloo found large language models suffer from a pronounced "status quo bias," consistently recommending existing, high-emission choices. Evaluating 11 prominent AI models across nearly 55,000 tested prompts, the team found that chatbots endorsed existing policies twice as often as proposed climate reforms.
•Other demographic and quality biases:
•AI chatbots can exhibit biases similar to those of their human creators, manifesting as gender bias, racial bias, and even bias based on socio-economic status.
•Researchers found that AI chatbots often provide recommendations that are inconsistent across generative AI platforms and may vary by sociodemographic groups, differing by gender and race of the person in the hypothetical scenarios.
•MIT Centre for Constructive Communication research finds that AI chatbots can reinforce social biases, mirroring human prejudices and often provide lower-quality responses to users with lower English proficiency, less education, or who are outside the US.
•History questions can regularly produce muddy, biased, or wrong answers.
Why does it happen?
Bias does not usually mean developers intentionally programmed a viewpoint. Researchers point to three main sources:
1.Training data: AI chatbots learn from the data they are trained on, which can inadvertently perpetuate existing biases present in the data. For example, if a chatbot is trained on a dataset that contains biased language or stereotypes, it is likely to replicate those biases. If datasets are skewed or unrepresentative, the AI learning from them may produce biased results.
2.Internet-scale data: Biases stem from the fact that the datasets used for training are often sourced from the internet, where harmful stereotypes and prejudices exist. Bias arises when training data lacks diversity and minority or underserved groups may receive inaccurate recommendations.
3.Latent and fine-tuning biases: This unintended power to persuade is caused by latent biases introduced during the training of the large language models that drive chatbots' core capabilities, which can carry over from ideological leanings in the data used to train LLMs. Researchers also note chatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training data, and models that are fine-tuned on human feedback are known to exhibit sycophancy – prioritising answers that align with user beliefs over the truth.
Does this mean chatbots are always biased?
Not equally or intentionally:
•Some models are more balanced than others. Gemini's both-sides approach in more than 90% of answers is an example.
•Bias is often subtle — the Yale study noted the effects represent slight differences — from leaning towards a moderate stance to leaning towards a somewhat liberal stance — but could compound if somebody frequently engages with chatbots for factual information.
•Companies are aware of the issue. Addressing these biases requires careful data curation, algorithmic transparency, and ongoing monitoring and evaluation of chatbot behavior.
What you can do as a user
1.Cross-check important factual, political, medical, or financial answers with independent sources like Wikipedia, primary documents, or professional advice.
2.Ask for balanced perspectives explicitly, e.g., "present left and right arguments" or "provide sources for and against."
3.Try more than one model — advice can differ not only by the chosen chatbot but also by the framing of the prompt.
4.Be aware of sycophancy — chatbots may agree with you to be helpful rather than to be accurate.
In short, AI chatbots are not neutral arbiters. They reflect patterns in human data and human feedback, which means they can inherit and amplify human biases — political, social, racial, gender, and even a preference for the status quo.