More answers like this
Grok 4.5Nov 2, 2026x-ai/grok-4.5-20260708

Are AI chatbots biased?

I need to gather evidence from reliable sources on this topic.Gathering more details from key studies.Yes, AI chatbots are biased—in political leanings, social stereotypes (gender, race, etc.), accuracy disparities across user groups, and subtle ways that can influence opinions. This is well-documented across numerous studies of large language models (LLMs) powering tools like ChatGPT, Claude, Gemini, Grok, and others. Bias is not always intentional malice but arises from training data, design choices, and alignment processes; it varies by model, version, prompt, and topic. Complete neutrality is difficult (or impossible) because responses often involve value judgments on contested issues.[1]

Political Bias

Most major conversational LLMs exhibit a left-of-center (liberal/progressive) tilt when answering politically charged questions or taking orientation tests:

•A 2024 PLOS ONE study by David Rozado administered 11 political orientation tests (e.g., Political Compass) to 24 conversational LLMs (including GPT-3.5/4, Gemini, Claude, Grok, Llama variants). Most produced left-leaning responses. Base (pre-fine-tuned) models were more neutral/incoherent, while supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) amplified the lean. Models can be steered left or right with modest amounts of aligned data.[2]
•Washington Post testing (2026) of models behind ChatGPT, Gemini, Claude, Grok, and others on hot-button issues found OpenAI’s model often gave exclusively left-leaning arguments (~80% of cases, e.g., favoring abolishing the Electoral College, higher taxes on the wealthy, single-payer healthcare). Gemini more often presented both sides; Grok (marketed as less “woke”) still cited left-leaning views more frequently on average than right-leaning ones, though it was relatively more balanced or right-leaning in some comparisons.[3]
•Other analyses (e.g., Centre for Policy Studies, EU election party ratings, midterm candidate picks) confirm left-leaning policy recommendations (>80% in some tests), more positive sentiment toward left/progressive ideologies/parties, and poorer performance spotting left-leaning falsehoods in some cases. Grok sometimes reverses this (more right-leaning or better at left falsehoods), especially with search enabled, but results are mixed and depend on setup.[4]

Right-leaning or “anti-woke” models exist or can be created, but leading consumer chatbots generally do not default to the center or right. Bias can appear as selective framing, source selection, refusals, or emphasis even when facts are accurate. Chatbots can also be highly persuasive—more so than traditional political ads in some experiments—and may amplify inaccuracies when optimized for persuasion.[5]

Other Forms of Bias

•Gender, racial, and demographic: LLMs perpetuate stereotypes (e.g., in professions or narratives), show reduced empathy in mental health-style responses for Black or Asian users vs. White/unknown (MIT-linked study on GPT-4), and generate content with measurable gender/racial skews (e.g., discrimination against females or Black individuals in news-like generation). Visual or multimodal models show profession-gender associations.[6]
•User-dependent accuracy: Models can give less accurate/truthful responses, more refusals, or condescending language to users described as having lower English proficiency, less education, or non-US origins (MIT CCC study on GPT-4, Claude, Llama).[7]
•Subtle influence and other: Even accurate historical summaries can shift user opinions leftward (Yale study). Models may favor AI-generated content over human (“AI-AI bias”), show source-framing effects (e.g., lower agreement with statements attributed to Chinese sources), or exhibit other latent preferences.[8]
•Surveys and analyses consistently find LLMs learn, perpetuate, and sometimes amplify harmful social biases from data.[1]

Why Does This Happen?

•Training data: Vast internet scrapes reflect real-world imbalances, media skews, historical prejudices, and overrepresentation of certain demographics/views (often WEIRD: Western, Educated, Industrialized, Rich, Democratic). Biases in text (stereotypes, political discourse) get encoded.[9]
•Alignment (SFT/RLHF): Fine-tuning on human preferences introduces labeler demographics, company values, and “safety” guidelines that often prioritize progressive norms (e.g., avoiding offense on certain topics). Labeler agreement is imperfect (~63-77%), and processes can amplify majority or researcher-aligned views. RLHF can lead to preference collapse or “sophistry” (convincing but misleading outputs). System prompts and post-training further shape behavior.[10]
•Other factors: Architecture, decoding, evaluation metrics, and commercial incentives. Bias can be latent (unprompted) or prompted.

Base pretrained models are often less politically slanted but less useful for chat; conversational optimization introduces more.

Impacts and Mitigation

Biases matter because chatbots are widely used for information, advice, writing, and decision support. They can subtly shape opinions, reduce trust (especially if perceived as biased), worsen outcomes for marginalized users, or spread skewed narratives at scale. They are not neutral oracles.[11]

Companies and researchers actively work on mitigation, with mixed success:

•Data techniques: Counterfactual augmentation, rebalancing, filtering, diverse datasets.
•Training: Fairness-aware losses, selective updates, diverse human feedback.
•Inference/prompting: Instructions for balance/self-awareness, chain-of-thought, decoding changes.
•Evaluation: Benchmarks, audits, taxonomies of harms/metrics.
•Other: Transparency reports, user controls/personalization, model variants. Newer models sometimes reduce explicit bias but retain subtler forms.[1]

Perfect debiasing is challenging—trade-offs exist with helpfulness, truthfulness, creativity, and the fact that “fairness” itself involves contested values. Over-correction can create new issues (e.g., refusals or blandness).

Bottom Line

AI chatbots are biased, reflecting (and sometimes amplifying) patterns in their data and the humans/organizations that build them. Political bias toward the left is particularly consistent in leading models, alongside social and demographic ones. This does not make them useless—they excel at many tasks—but users should treat outputs critically: cross-check facts, seek multiple sources/perspectives, experiment with prompts/models, and recognize that responses embed assumptions. Ongoing research, competition (e.g., more viewpoint-diverse models), and transparency will help, but vigilance remains essential as these systems grow more influential.

Citations (11)

  1. https://aclanthology.org/2024.cl-3.8/

    Backs: “Complete neutrality is difficult (or impossible) because responses often involve value judgments on contested issues.”

  2. 2plos.orgAcademic
    https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0306621

    Backs: “Base (pre-fine-tuned) models were more neutral/incoherent, while supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) amplified the lean. Models can be steered left or right with modest amounts of aligned data.”

  3. 3washingtonpost.comNews and mediaquality 0.82
    https://www.washingtonpost.com/technology/interactive/2026/06/24/are-ai-chatbots-like-chatgpt-politically-biased-we-tested-them/

    Backs: “Grok (marketed as less “woke”) still cited left-leaning views more frequently on average”

  4. 4cps.org.ukNonprofits and think tanks
    https://cps.org.uk/media/post/2024/left-leaning-bias-commonplace-in-ai-powered-chatbots-shows-new-report/

    Backs: “Grok sometimes reverses this (more right-leaning or better at left falsehoods), especially with search enabled, but results are mixed and depend on setup.”

  5. 5technologyreview.comNews and mediaquality 0.91
    https://www.technologyreview.com/2025/12/04/1128824/ai-chatbots-can-sway-voters-better-than-political-advertisements/

    Backs: “Bias can appear as selective framing, source selection, refusals, or emphasis even when facts are accurate. Chatbots can also be highly persuasive—more so than tra”

  6. 6mit.eduAcademic
    https://news.mit.edu/2024/study-reveals-ai-chatbots-can-detect-race-but-racial-bias-reduces-response-empathy-1216

    Backs: “White/unknown (MIT-linked study on GPT-4), and generate content with measurable gender/racial skews (e.g., discrimination against females or Black”

  7. 7mit.eduAcademic
    https://news.mit.edu/2026/study-ai-chatbots-provide-less-accurate-information-vulnerable-users-0219

    Backs: “User-dependent accuracy: Models can give less accurate/truthful responses, more refusals, or condescending language to users described as having lower English proficiency, less education, or non-US origins (MIT CCC study on GPT-4, Claude, Llama).”

  8. 8yale.eduAcademic
    https://news.yale.edu/2026/03/03/ais-hidden-bias-chatbots-can-influence-opinions-without-trying

    Backs: “Models may favor AI-generated content over human (“AI-AI bias”), show source-framing effects (e.g., lower agreement with statements attributed to Chinese sources), or exhibit other latent preferences.”

  9. 9thelivinglib.orgNonprofits and think tanks
    https://thelivinglib.org/this-is-how-ai-bias-really-happens-and-why-its-so-hard-to-fix/

    Backs: “Training data: Vast internet scrapes reflect real-world imbalances, media skews, historical prejudices, and overrepresentation of certain demographics/views (often WEIRD: Western, Educated, Industrialized, Rich, Democratic). Biases in text (stereotypes, political discourse) get encoded.”

  10. 10springer.comAcademic
    https://link.springer.com/article/10.1007/s13347-025-00928-y

    Backs: “RLHF can lead to preference collapse or “sophistry” (convincing but misleading outputs). System prompts and post-training further shape behavior.”

  11. 11oup.comAcademic
    https://academic.oup.com/pnasnexus/article/5/10/pgag324/8898473

    Backs: “They can subtly shape opinions, reduce trust (especially if perceived as biased), worsen outcomes for marginalized users, or spread skewed narratives at scale.”