Is open-source AI safer or riskier than closed/proprietary AI?
Some sources (International AI Safety Report, Anthropic/CAISI material) stress open-weight irreversibility and higher misuse risk, while others (Risks and Opportunities paper, Open-Weight Paradox, Some Simple Economics of AGI) argue closed models are not clearly safer.
This is a genuinely contested question with reasonable arguments on multiple sides, and the answer depends heavily on what kind of "risk" you're talking about (misuse, accidents, concentration of power, security vulnerabilities, etc.). Let me lay out the main considerations.There's no consensus answer here—it genuinely depends on what kind of risk you care about, and serious researchers disagree. Here's the landscape of arguments:
Case for open AI being riskier
Irreversibility and uncontrolled misuse. Open-weight model releases are irreversible—once released, model weights cannot be recalled, making it harder to mitigate potential harms resulting from the release of a model with dangerous capabilities. Similarly, open-weight models facilitate research and innovation, but their safeguards are more easily removed, and compared to closed-weight models, they are more easily modified to exhibit harmful behaviors, with usage being harder to monitor.
Fine-tuning can strip safety training cheaply. Open-source models can be freely downloaded, used, and modified by anyone, including malicious actors—even if a model is initially aligned, it can easily be transformed into a harmful model, and research shows just a few dozen fine-tuning samples can remove safety constraints.
Larger, harder-to-defend attack surface. Open- versus closed-weight deployments come different safety tradeoffs: while open-weight models allow for more open research and testing, they come with greater potential for misuse. Closed developers rely on training interventions, content classifiers, and acceptable-use enforcement, but none of these provide reliable assurances for open-weight models, which can be used, tampered with, and distributed without limitations — the attack surface is larger and defense techniques less well studied.
National-security concerns about specific actors. This became concrete in the 2026 debate: Anthropic's CEO argued that the most powerful frontier systems carry an irreversible national-security risk once weights are released, centering on authoritarian misuse — particularly that the CCP could field AI systems enabling durable military advantage or intensified domestic repression, plus cyberattack and bioweapon concerns. Real-world data backs some of this: NIST's CAISI found DeepSeek's most secure model complied with 94% of malicious requests using common jailbreaks versus just 8% for American reference models, and DeepSeek-based agents were 12 times more likely to be hijacked than those built on GPT-5 or Opus 4.
Case for open AI being safer (or at least not clearly riskier)
Closed models aren't actually that secure either. Closed models are also vulnerable to jailbreaking, and their safeguards are easily bypassed through fine-tuning via accessible services, prompting tricks, ciphers, or foreign-language instructions — completely preventing undesirable behaviors might not even be possible. The conclusion researchers draw: it is not clear that closed models are definitively "safer" than open-source models, and gaps will likely narrow as jailbreaking attacks strengthen and open safeguarding improves.
"Security by obscurity" is fragile. The standard argument against open-source AI is that it increases tail risk by removing institutional safety filters, but this assumes proprietary models successfully enforce a durable alignment floor — an assumption contradicted by evidence that bypassing safety fine-tuning on frontier proprietary models has proven extremely easy. The core economic benefit of open ecosystems becomes their ability to massively scale verification velocity.
Closed access isn't permanent either. Closed models are often considered more secure because access can be restricted, but they can be compromised via hacking, leaks, reverse engineering, or duplication — for instance Mistral's 70B parameter model was leaked by an early customer.
Community-driven auditing can catch flaws. Some experts draw an analogy to software security: by analogy to open-source software, open models have safety advantages over closed models — open-source tools like Linux are often seen as safer because vulnerabilities have been probed and patched by a vast community of contributors.
Empirically, openness alone doesn't predict safety. One recent benchmark study found: open-source LLMs showed a substantially higher mean harm score than closed-source, but this gap was largely attributable to higher rejection rates among closed models rather than inherently safer per-response behavior — specific alignment strategies matter far more than accessibility.
Concentration-of-power risk. This is a different axis of "risk" entirely: the security risks most worth examining may not lie in openness itself, but in what happens when access is restricted without governed alternatives — when pathways to advanced AI run through a handful of cloud providers controlled by few nations and corporations, this amplifies structural dependency and opacity without guaranteeing greater safety. Proponents of openness argue closed models create single points of failure, and transparency of open weights can strengthen cybersecurity by giving defenders capabilities comparable to attackers.
The bottom line
Most serious analyses converge on a few points:
So: for garden-variety misuse (scams, spam, jailbreak-driven harmful content), both open and closed models are routinely circumvented, and the difference is smaller than marketing from either camp suggests. For catastrophic, hard-to-reverse scenarios (bioweapons uplift, autonomous cyberattack capability, adversarial-nation-state use), open-weight release is generally considered to carry higher tail risk simply because it can't be recalled or monitored — while closed deployment carries its own risks around concentrated corporate/government control and single points of failure. Which one "wins" depends heavily on which risk you weight most heavily and your threat model.
Citations (20)
- International AI Safety Report 2026
Source excerpt: “- Open-weight model releases are irreversible. Once released, model weights cannot be recalled. This makes it harder to mitigate potential harms resul...”
evidence, official document, points to no. Claim: Open-weight model releases are irreversible; weights cannot be recalled, making harm mitigation harder.
- International AI Safety Report 2026
Source excerpt: “- Open-weight models facilitate research and innovation, but their safeguards are more easily removed. Around the world, various actors especially th...”
evidence, official document, points to no. Claim: Open-weight models' safeguards are more easily removed and they are more easily modified to exhibit harmful behaviors.
- Rethinking AI Safety Approach in the Era of Open-Source AI — LessWrong
Source excerpt: “** Open-source models can be freely downloaded, used, and modified by anyone, including malicious actors. Even if an open-source model is initially al...”
evidence, article, points to no. Claim: Open-source models can be downloaded and modified by malicious actors; few fine-tuning samples can remove safety constraints.
- Open Technical Problems in Open-Weight AI Model Risk Management
Source excerpt: “Open- versus closed-weight deployments come with different safety tradeoffs. While open-weight models allow for more open research and testing, they a...”
evidence, official document, points to no. Claim: Open-weight models have greater misuse potential; closed-model safeguards don't provide reliable assurances for open weights, and the attack surface is larger.
- The Open-Weight Rift: Concentration Risk vs. National Security
Source excerpt: “# The Open-Weight Rift: Concentration Risk vs. National Security ## Key Takeaways Anthropic CEO Dario Amodei published a rebuttal on July 27, 2026, cl...”
attribution, article, points to no. Claim: Anthropic's CEO argued frontier weights carry irreversible national-security risk, especially authoritarian misuse, cyberattack and bioweapon concerns.
- An American Answer to China’s Open SI Models: A Safe Harbor for Open Weights
Source excerpt: “In its 2025 evaluation of DeepSeek, the National Institute of Standards & Technology’s Center for AI Standards and Innovation (CAISI) found that the C...”
statistic, article, points to no. Claim: NIST CAISI found DeepSeek's most secure model complied with 94% of malicious jailbreak requests versus 8% for American models.
- Risks and Opportunities of Open-Source Generative AI
Source excerpt: “However, closed models still demonstrate weaknesses and, crucially, are also vulnerable to jailbreaking techniques (Zou et al., 2023; Chao et al., 202...”
evidence, official document, points to yes. Claim: Closed models are also vulnerable to jailbreaking and their safeguards are easily bypassed.
- Risks and Opportunities of Open-Source Generative AI
Source excerpt: “Therefore, it is not clear that closed models are definitively "safer" than open-source models. We also anticipate that gaps will narrow over time as ...”
evidence, official document, points to yes. Claim: It is not clear closed models are definitively safer than open-source models; gaps will likely narrow.
- Some Simple Economics of AGI
Source excerpt: “#### 6.1.6 Open Source: Distributed Verification vs. Tail Risk The standard argument against open-source AI is that it increases tail risk by removin...”
evidence, official document, points to yes. Claim: The argument that open source increases tail risk assumes proprietary models enforce durable alignment, which evidence contradicts; open ecosystems scale verification.
- Risks and Opportunities of Open-Source Generative AI
Source excerpt: “Closed models are often considered more secure because access can be restricted or removed should issues be identified. However, closed models can be ...”
evidence, official document, points to yes. Claim: Closed models can be compromised via hacking, leaks, reverse engineering, or duplication.
- Risks and Opportunities of Open-Source Generative AI
Source excerpt: “Finally, closed models can be leaked, e.g., Mistral's 70B parameter was leaked by one of their early customers (VentureBeat, 2024b). ”
evidence, official document, points to yes. Claim: Mistral's 70B model was leaked by an early customer.
- Open models have real benefits. Do they outweigh the risks?
Source excerpt: “In fact, some experts who spoke with *MIT Technology Review* argued, by analogy to open-source software, that open models have safety advantages over ...”
attribution, article, points to yes. Claim: Some experts argue by analogy to open-source software that open models have safety advantages over closed ones.
- SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring
Source excerpt: “## I.2 Finding 4: Open-Source vs. Closed-Source Safety Profiles Open-source LLMs exhibit a mean DEHARM of 1.414, substantially higher than the closed...”
statistic, official document, points to no. Claim: Open-source LLMs showed a substantially higher mean harm score than closed-source models.
- SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring
Source excerpt: “This heterogeneity suggests that the open- vs. closed-source distinction is a poor predictor of safety; specific alignment strategies matter far more ...”
evidence, official document, neutral. Claim: Open vs closed distinction is a poor predictor of safety; specific alignment strategies matter far more.
- The Open-Weight Paradox: Why Restricting Access to AI Models May Undermine the Safety It Seeks to Protect
Source excerpt: “This paper advances a different thesis: the security risks most worth examining may not lie in openness itself, but in what occurs when access is rest...”
evidence, official document, points to yes. Claim: Security risks may lie in restricted access without governed alternatives, amplifying dependency and opacity without guaranteeing safety.
- The Open-Weight Rift: Concentration Risk vs. National Security
Source excerpt: “## Background The letter’s central claim is that open, downloadable, and inspectable model weights are the modern analogue of the open-source software...”
attribution, article, points to yes. Claim: Proponents of openness argue closed models create single points of failure and open weights give defenders comparable capabilities.
- Open-Weight vs Closed AI: Differences, Costs, Safety, and Business Use
Source excerpt: “##### Rank #4 Also separate openness from concentration of power.”
background, article, neutral. Claim: Openness should be separated from concentration of power as distinct issues.
- International AI Safety Report 2026
Source excerpt: “- Since the publication of the last Report (January 2025), major open-weight releases have narrowed the capability gap with leading closed models. Chi...”
evidence, official document, neutral. Claim: Since January 2025 open-weight releases narrowed the capability gap with closed models, now less than a year behind.
- International AI Safety Report 2026
Source excerpt: “- A key policy challenge is accessing the benefits open-weight models provide while managing their distinctive risks. One approach is to assess open-w...”
background, official document, neutral. Claim: Policy challenge is balancing open-weight benefits against risks, e.g., assessing marginal risk of releases.
- Market Beliefs about Open vs. Closed AI
Source excerpt: “Notably, a poll of AI experts is split on the question, 'Should we ban future open-source AI models that can be used to create weapons of mass destruc...”
evidence, official document, neutral. Claim: A poll of AI experts is split on banning future open-source models that could create weapons of mass destruction.