Is open-source AI safer or riskier than closed/proprietary AI?

Both sidesSays it is contestedSources conflict

The Bloomberg Law piece has an Anthropic executive saying open models carry more risk, while the Linnk piece argues open-sourcing benefits from transparency and its benefits outweigh its risks.

This safety debate has strong arguments on both sides — I'll gather recent research to compare open and closed AI risks.

Is open-source AI safer or riskier than closed/proprietary AI? The consensus among researchers, companies and regulators is: neither is automatically safer — they have different risk profiles, and the answer depends on what type of risk you are measuring.

1. Arguments that open-source / open-weights AI is safer

Transparency and auditability:

•Open-source approaches make agentic AI safer by enabling transparency, leveraging collective intelligence, preventing monopolistic control, and fostering collaborative safety research.

•Andrew Ng has argued that open-weight models are, in some respects, safer than closed ones — because outside researchers can inspect and verify what they’re doing.

•Much as transparency strengthens the security of open-source software (OSS), allowing for public study and verification can help secure open foundation models.

•Moonshot AI contends that releasing model weights openly gives users, researchers, and governments the ability to inspect, audit, and customize AI systems, and that this transparency is inherently safer than trusting a handful of corporations to police themselves behind closed doors.

•Moonshot AI, a Beijing-based startup, has claimed that China's open-source AI development strategy is fundamentally safer than the closed approaches of major US companies.

Security through transparency vs. obscurity:

•Proprietary systems are often a 'security through obscurity' model, whereas open source is 'security through transparency'—vulnerabilities are often found and patched faster by a global community than by a single vendor.

•The principle that proprietary code is safer because it is hidden, is increasingly recognized as a flawed approach. Open source code, conversely, benefits from "security through transparency".

•Conversely, the open-source community argues that "security through obscurity" is a fallacy. They contend that by keeping models closed, the industry is ignoring systemic risks that only a global community of researchers can identify and fix.

Democratization and resilience:

•On the positive side, open models offer more flexibility, transparency, and accessibility, advancing research and fostering innovation.

•Open models can be downloaded and modified, offering advantages in cost, privacy, research transparency, and broader cyber-defense.

•From this perspective, open-source AI is actually safer because it eliminates the single point of failure and prevents a small group of engineers from having unilateral control over the world's most powerful cognitive tools.

Closed models are not immune:

•Closed models can be compromised via hacking, leaks, reverse engineering or duplication.

•Available evidence does not show that open models are riskier than closed models or information already available through standard research techniques such as online searching.

•Past months have made it clear that safety remains a gnarly problem for closed models. According to a recent risk report from Anthropic, several extensive hacking operations have been undertaken with the assistance of Claude.

2. Arguments that open-source AI is riskier

Irreversibility and loss of control:

•Open models also present safety and security risks, such as the potential for malicious use and the inability to roll back or update models once released.

•Neither approach is automatically safer. Open weights enable outside testing and transparency, but they can also be modified and redistributed without the original developer’s control.

•While open models face challenges regarding guardrail removal, the benefits of transparency and distributed accountability outweigh the risks of security through obscurity.

Easier misuse and safeguard stripping:

•Anthropic argues that freely available models like GLM-5.3 pose serious cyberattack risks because they lack centralized safeguards and can be retrained to bypass safety controls.

•Researchers have shown it is surprisingly easy to retrain an open model to drop its refusal training.

•Closed providers can filter prompts and revoke accounts, which open-weight providers cannot do after download.

Industry and public perception:

•Anthropic PBC’s top sales executive outside the US argued that open alternatives to its AI models carry “more risk” of cybersecurity incidents or other dangers.

•“We need to be really careful with open-source on safety,” Guillaume Princen, Anthropic’s head of international, told Bloomberg TV. “I can’t talk for open models. But by definition, there’s more risk with them.”

•Survey respondents consider open source AI tools to be riskier than proprietary AI for most types of AI risks. The most relevant AI risks cited include cybersecurity (62 percent of respondents), regulatory compliance (54 percent), and intellectual property (50 percent).

3. What does the empirical evidence say?

The most relevant test case is biosecurity / biorisk uplift — whether AI helps someone create a bioweapon:

•The RAND Corporation’s 2024 study is the most rigorous empirical assessment of AI uplift to date. Twelve teams of three researchers each spent 80 hours developing bioweapon attack plans—half using AI, half using only the internet.

•No statistically significant uplift. All plans scored between “untenable” and “problematic.” AI-assisted plans were statistically indistinguishable from internet-only plans.

However, results depend on methodology:

•Wet lab vs. in silico gap. The two completed wet lab studies (Active Site, LANL) found no statistically significant uplift on their primary endpoints, while in silico studies have found minimal (OpenAI/Gryphon, RAND, Meta) to substantial (Scale AI/SecureBio, Anthropic) uplift on knowledge and planning tasks.

•A 2026 Scale AI / SecureBio study found novices with AI models were 4.16x more accurate than controls. AI model groups exceeded expert baselines on 3/4 benchmarks. 89.6% of participants reported no difficulty overcoming safeguards.

This illustrates why nuance matters: for wet-lab execution (actually growing a virus in a lab), AI has not yet shown statistically significant uplift; for knowledge and planning tasks on a computer, uplift can be large, and safeguards are often easy to bypass.

A related nuance: most models called "open source AI" are not truly open source. The Open Source Initiative definition requires software, weights, and training data. As Stanford Law notes, providing all the data is probably functionally impossible for most of the models that anybody cares about using, so most "open source" models are actually open-weights. You can get the weights, but what would you do with them? You can’t look at them and figure how they work, really.

4. Bottom line: A tradeoff

•Closed AI = Centralized control: Easier to patch, monitor, filter prompts, revoke access, and enforce policy, but vulnerable to insider failure, lack of auditability, concentrations of power, single points of failure, and "security through obscurity."

•Open AI = Distributed transparency: More auditable, customizable, privacy-preserving (can run locally), cheaper, and benefits from many eyes finding bugs faster, but irreversible once released, easily stripped of safeguards, and shifts the security burden to the downstream user or enterprise.

Because of this, many researchers now advocate for responsible open sourcing rather than a binary choice: enhancing data transparency and provenance, improving open evaluation and benchmarking, conducting multilevel security audits, comparing with closed source models, and studying broader societal impact, linked to greater transparency across the entire model pipeline.

> In short: If your threat model is accidental bugs, bias, and concentration of power, open tends to be safer. If your threat model is deliberate malicious use by actors who will intentionally remove safeguards, open is riskier because you cannot recall the model.

Citations (4)

1linnk.aiNot yet classified
Near to Mid-term Risks and Opportunities of Open Source Generative AI
Backs: “They contend that by keeping models closed, the industry is ignoring systemic risks that only a global community of researchers can identify and fix. Democratization and resilience: On the positive side, open models offer more flexibility, transparency, and accessibility, advancing research and fostering innovation.”
evidence, article, points to yes. Claim: Open models offer flexibility, transparency and accessibility, and responsible open sourcing is advocated; transparency and distributed accountability benefits are said to outweigh the risks of security through obscurity.
2bloomberglaw.comNews and media
Anthropic Executive Says Open-Source AI Models Are Higher Risk
Backs: “Closed providers can filter prompts and revoke accounts, which open-weight providers cannot do after download. Industry and public perception: Anthropic PBC’s top sales executive outside the US argued that open alternatives to its AI models carry “more risk” of cybersecurity incidents or other dangers.”
attribution, article, points to no. Claim: Anthropic's head of international argued that open alternatives to its models carry more risk of cybersecurity incidents and other dangers.
3securebio.orgNot yet classified
Uplift Studies – SecureBio
Backs: “Twelve teams of three researchers each spent 80 hours developing bioweapon attack plans—half using AI, half using only the internet. No statistically significant uplift. All plans scored between “untenable” and “problematic.” AI-assisted plans were statistically indistinguishable from internet-only plans.”
evidence, data page, neutral. Claim: RAND's 2024 study found no statistically significant AI uplift in bioweapon attack planning, though other in silico studies found larger uplift.
4stanford.eduAcademic
Why Open Source AI Is Having Its Moment, and What Legal Leaders Need to Know
Backs: “A related nuance: most models called "open source AI" are not truly open source. The Open Source Initiative definition requires software, weights, and training data. As Stanford Law notes, providing all the data is probably functionally impossible for most of the models that anybody cares about using, so most "open…”
attribution, other, neutral. Claim: Stanford Law notes most so-called open source models are really open-weights, since providing all the training data is probably functionally impossible.