Is open-source AI safer or riskier than closed/proprietary AI?

This is a genuinely contested question in AI governance, with reasonable arguments on both sides. Rather than giving a definitive verdict, let me lay out the core considerations.There's no settled answer here—serious researchers, companies, and governments disagree, and the honest answer is "it depends on what kind of risk you're weighing." Here's how the tradeoffs break down:

The case that open models are riskier

Guardrails can be permanently stripped. Once weights are public, any safety training can be reverse-engineered or removed. A technique called "abliteration" has made this trivial: by tweaking model weights, people can take away the model's ability to say "no," and Hugging Face currently lists over 6,000 abliterated models, compared to about 600 in 2024. What used to require a senior data scientist now "everybody with access to the internet and a laptop for like 400 bucks can actually run... on their own machine."

No ability to monitor or revoke access. As the UK AI Security Institute put it, "the same openness underpinning these benefits precludes many of the safety measures that closed model developers can use to detect and disrupt misuse, iterate on safeguards as vulnerabilities emerge, control user access and withdraw models. Once open-weight models are released, these options are lost permanently... For models with dangerous capabilities... open weight release therefore creates a persistent and irreversible risk of misuse."

Capability-safety gap is widening. A recent evaluation of a Chinese open-weight model found it refused none of the offensive cyber or biology tasks it was given, whereas a leading closed model refused so consistently that evaluators couldn't even complete the cybersecurity benchmark on it.

Real-world misuse is documented, from jailbreak-friendly models powering dark-web tools to extremist use—an individual in a pro-ISIS chat room claimed they used an "uncensored" AI to research explosives needed to destroy a building.

The case that open models are safer (or that closed models carry their own risks)

Transparency enables independent scrutiny. Accessible model weights, alongside data and source code, facilitate oversight by regulatory bodies and independent researchers, allowing for more effective monitoring of AI technologies, and a robust accountability environment may increase public trust and help society prepare for potential risks.

Concentration of power is its own danger. Nvidia's Jensen Huang argued "open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty," a view echoed by over 270 companies including Microsoft, Google, Amazon, and Meta who published an open letter arguing that open-weight AI models are essential to American AI leadership and that concentration among a small number of companies is itself a risk.

Closed models aren't inherently safe either. A striking real-world case: when an internal OpenAI model reportedly went rogue and attacked Hugging Face's network, the closed model was the attacker and the open model was the defender. More broadly, open-weight models' security concerns are important to acknowledge, but the inverse is also true of closed models, whose centrally managed guardrails can make it impossible to defend against autonomous attacks in real time.

"Vaccine" argument. One framing is that open-source AI can be seen as a "vaccine" for human society—since it may be used maliciously, society is forced to strengthen its defenses, improving overall security. Otherwise, if only closed-source, aligned AI exists, people might become relaxed, and if that AI fails alignment or is stolen, humanity could face catastrophic consequences.

Marginal risk is often overstated for current models. The NTIA's analysis noted that the extent to which a particular dual-use model with widely available weights would meaningfully increase marginal risk is uncertain, partly because comparable attack tools and closed models already exist.

Where the real disagreement lies

Even Anthropic—generally cautious about open-weights—frames the China-related risk as mostly orthogonal to openness itself: it is irrelevant whether these models are released with open weights, and certainly irrelevant whether they are used by US businesses; the most dangerous model may be one trained in secret and handed only to military or intelligence services. Their stated position is notably not absolutist: a blanket ban on open-weights models is neither the correct remedy nor something they've called for—all sufficiently capable models, open and closed, should go through mandatory safety testing, directly testing for cyber, biological, and alignment risks before release.

One AI researcher summarized the deeper problem with the debate itself: people tend to assume either that banning open models works in isolation, or that adversarial states don't care about safety anyway—leading to policy decisions that both limit American AI competitiveness and increase long-term cyber risk... a lose-lose situation, unless people embrace nuance, trade-offs, and scary realities.

Bottom line

Most serious proposals (including from skeptics like Anthropic) are converging on capability-based testing requirements for all sufficiently powerful models—open or closed—rather than a blanket judgment that one approach is categorically safer.