Is open-source AI safer or riskier than closed/proprietary AI?

Is Open-Source AI Safer or Riskier Than Closed AI?

There's no simple answer—the debate genuinely cuts both ways, and the framing "open equals dangerous, closed equals safe" is considered overly simplistic by many researchers. A recent paper on open-weight AI governance notes that "the dominant policy debate over open-weight AI models has consolidated around a binary framing: open equals dangerous, closed equals safe," and argues this framing oversimplifies the issue. Here's a breakdown of the actual tradeoffs:

The Case for Closed AI Being Safer

Centralized control and patchability. Fully closed models' weights and code are proprietary, for internal use only, meaning external actors are not able to misuse the model and flaws are less likely to proliferate downstream and can be fixed once discovered. However, with closed models it is also harder for external developers to discover misuse risks, flaws, and use the model for wider innovation and research.

Irreversibility problem. One of the strongest arguments against open-weight release is that once weights are released, they cannot be recalled, revoked, or updated—a genuine containment problem.

Empirical safety gaps. Recent red-teaming research found notable differences in practice: open-weight models lag significantly behind proprietary models in safety, with one open-weight 120B-parameter model achieving 70.0% direct attack success rate and 32.8% indirect attack success rate. This highlights a persistent gap between open-weight and proprietary frontier models in safety alignment—while open-weight models can match proprietary ones on raw task capability, the safety training that enables refusal of harmful instructions remains a key differentiator.

Guardrail removal. The main argument for restricting open-weight models is that releasing model weights allows malicious actors to remove safety guardrails through fine-tuning—a concern with empirical grounding, as researchers demonstrated that safety alignment can be compromised by fine-tuning with as few as ten adversarially designed examples, at a cost of less than $0.20. The UK AI Security Institute confirmed that current anti-tampering techniques remain insufficient: safeguards can be removed with just a few dozen training examples.

The Case for Open AI Being Safer (or at Least Not Riskier)

The vulnerability isn't unique to open models. Critically, that same fine-tuning attack is not exclusive to open-weight models—researchers demonstrated the attack on both open-weight models (Llama-2) and closed API models (GPT-3.5 Turbo), showing the fine-tuning risk extends to any model for which fine-tuning access is provided, which includes most commercial APIs.

Closed models aren't invulnerable either. Industry advocates for open weights counter that the same containment logic applies to closed models that get breached or jailbroken, arguing the answer to both problems is broader defensive capability, not narrower access.

Transparency enables safety research. Openness has real safety benefits: it drives innovation, enables AI safety/security research, enables flexible AI adoption, and spreads benefits and access to AI. More eyes on a model's weights and architecture can surface vulnerabilities that closed systems hide.

Democratization and sovereignty. Open weights let nations and organizations without GPUs download smaller versions of models and adapt them locally, partially bypassing a hardware barrier that is otherwise insurmountable—reducing concentration of power in a handful of companies, which some view as its own safety concern.

Trust boundaries differ, not disappear. With closed/API models, weights never get released and everything runs on the vendor's servers—so your primary risk becomes a trust-boundary one: data leaving your control and reliance on the vendor. Open weights shift risk from "trusting a vendor" to "risk of misuse by bad actors," rather than eliminating risk altogether.

The Honest Uncertainty

Even AI safety researchers admit the empirical picture is unsettled: it is unclear how much counterfactual risk open-weight models pose compared to closed-weight models, and the research community emphasizes gathering more information through additional research and analysis of impacts across the ecosystem. Researchers also caution against both 'openness washing' and 'safety washing'—i.e., don't assume either side's marketing reflects reality.

Notably, experts themselves are divided: a poll of AI experts is split on whether to ban future open-source AI models that could be used to create weapons of mass destruction.

Bottom Line

Most serious analyses—including recent ones from AI safety institutes—now reject the binary and argue a positive future with AI will involve a balance of proprietary and open-weight model development, with risk depending heavily on which specific model, what safeguards accompany it, and how it's deployed, rather than on the open/closed label alone.