Is open-source AI safer or riskier than closed/proprietary AI?

Both sidesSays it is contestedSources conflict

Some sources (arXiv 2412.13821, AI Frontiers, AAF) say open-weight models are more vulnerable to misuse, while others (arXiv 2405.08597, Brookings, arXiv 2604.17413) say closed models are not clearly safer and openness can improve security.

This is a genuinely contested debate in AI policy with substantive arguments on both sides. Let me search for the latest thinking on this.There's no single right answer here—the debate is genuinely contested among researchers, companies, and policymakers, and the answer often depends on what kind of risk you're weighing (misuse, concentration of power, security vulnerabilities, geopolitics) and what capability level of model you're talking about. Here's a breakdown of the main considerations on each side:

Arguments that open-source/open-weight AI is riskier

•Removable safeguards: Once weights are released, anyone can fine-tune away safety training. Researchers note that open-weight models have been shown to be extremely vulnerable to adversarial detuning, allowing malicious actors to both disable safeguards and elicit dangerous capabilities.

•Irreversibility: Unlike closed models, the open equivalent of a closed model will be more susceptible to unauthorized misuse or modification, and can be deployed and misused with limited visibility from the original developer. You can't recall a leaked/released model.

•Uplift for bad actors: Groups like the American Action Forum warn as open-source systems become widely available, they could pose safety risks by increasing the chances of bad actors using these systems for harmful purposes including learning how to create bioweapons and carry out cyberattacks.

•Real-world harms already observed: One study found models with openly available parameters are a key factor and key driver behind the creation of AIG-NCII and AIG-CSAM, though this does not imply open-weight models are net harmful.

•Geopolitical concerns: U.S. policy discussions worry about technological leadership by distributing access to cutting-edge models—and the associated technological power—to foreign adversaries.

Arguments that open-source/open-weight AI is safer (or at least not clearly riskier)

•Closed models aren't inherently safe either: A widely-cited analysis concludes closed models still demonstrate weaknesses and are also vulnerable to jailbreaking techniques; safeguards are easily bypassed through fine-tuning, prompting tricks, cyphers, or other languages, and completely preventing undesirable behaviors might not even be possible. Therefore it is not clear that closed models are definitively "safer" than open-source models.

•Closed models can leak or be stolen too: Closed models are often considered more secure because access can be restricted, but they can be compromised via hacking, leaks, reverse engineering or duplication—for example, Mistral's 70B parameter model was leaked by an early customer.

•Transparency helps defense: Brookings argues open systems can actually be the foundation for building more secure systems, as they are open to scrutiny by anyone and easier to evaluate. Similarly, cybersecurity researchers note openness can allow companies to become aware of flaws in their products and make quick improvements at a much faster pace.

•Enables independent safety research: Open source AI models have enabled important AI safety research by allowing smaller safety research teams that lack the resources of major AI companies to run critical experiments.

•Concentration-of-power risk: Some argue the real danger is over-reliance on a few closed providers. Brookings warns if American AI is dominated by a small handful of closed model providers, the result will be an overconcentration of power, raising concerns including regulatory capture and impeded market entry. A 2026 paper goes further, arguing the security risks most worth examining may not lie in openness itself, but in what occurs when access is restricted without governed alternatives—when advanced AI capability runs through a handful of cloud providers, that arrangement amplifies structural dependency and opacity without guaranteeing greater safety.

•Even Anthropic (a closed-model lab) complicates the "closed=safe" framing: Anthropic's CEO has written that the identity of the safest model isn't just about open vs. closed—it is irrelevant whether these models are released with open weights; the most dangerous model may be one that is trained in secret and handed only to a hostile military or intelligence service.

•Evidence gap: Marginal risk from open models is often uncertain. NTIA notes the marginal uplift in capabilities for cyber attackers that widely available weights introduce may be the most significant risk, but the extent to which a particular model would meaningfully increase marginal risk is therefore uncertain.

The honest bottom line

Most serious analyses land on "it depends":

1.On misuse/bioweapons/cyberattacks: Open models likely carry somewhat higher marginal risk once truly frontier-level dangerous capabilities exist, because safeguards can be stripped and access can't be revoked. But for current models, evidence of catastrophic uplift is thin, and closed models face similar jailbreak vulnerabilities.

2.On systemic/structural risk: Closed models concentrate power, create single points of failure (leaks, hacks), and reduce independent oversight — which some argue is the bigger long-term danger.

3.Policy consensus is elusive: Even top-level reports admit uncertainty — the 2025 International AI Safety Report noted wide disagreement among experts about the likelihood of losing control over advanced AI systems, and found "no scientific consensus" on the risk of AI being used to manipulate public opinion.

Practical takeaway: rather than "open is safer" or "closed is safer" as a blanket rule, most experts now favor a capability-based approach — the risk profile depends heavily on how powerful/dangerous a specific model's capabilities are (e.g., bio/cyber uplift), not simply its licensing model. Today's mid-tier open models pose modest marginal risk; the debate intensifies sharply as models approach frontier capabilities in high-stakes domains.

Citations (18)

1arxiv.orgAcademic
Towards Responsible Governing AI Proliferation
Source excerpt: “First, while closed source models themselves are not currently as well secured as would be desirable, open-weight models have been shown to be extreme...”
evidence, official document, points to no. Claim: Open-weight models have been shown to be extremely vulnerable to adversarial detuning, letting malicious actors disable safeguards and elicit dangerous capabilities.
2ai-frontiers.orgNonprofits and think tanks
Precaution Shouldn't Keep Open-Source AI Behind the Frontier | AI Frontiers
Source excerpt: “All else equal, the open equivalent of a closed model will be more susceptible to unauthorized misuse or modification. Open models can be deployed and...”
evidence, article, points to no. Claim: The open equivalent of a closed model is more susceptible to unauthorized misuse or modification and can be deployed with limited developer visibility.
3americanactionforum.orgNonprofits and think tanks
Open-Source AI: The Debate That Could Redefine AI Innovation - AAF
Source excerpt: “As open-source systems become widely available, they could pose safety risks by increasing the chances of bad actors using these systems for harmful p...”
attribution, article, points to no. Claim: American Action Forum warns widely available open-source systems could let bad actors cause harm, including bioweapons and cyberattacks.
4arxiv.orgAcademic
Video Deepfake Abuse: How Company Choices Predictably Shape Misuse Patterns
Source excerpt: “#### **5.3** Weighing Competing Considerations Navigating the benefits and risks of open-weight models: As we have discussed, models with openly avai...”
evidence, official document, points to no. Claim: Models with openly available parameters are a key driver behind creation of AIG-NCII and AIG-CSAM.
5arxiv.orgAcademic
Video Deepfake Abuse: How Company Choices Predictably Shape Misuse Patterns
Source excerpt: “However, this does not imply open-weight models are net harmful. ”
evidence, official document, neutral. Claim: This does not imply open-weight models are net harmful.
6ntia.govGovernment
Geopolitical Considerations | National Telecommunications and Information Administration
Source excerpt: “technological leadership by distributing access to cutting-edge models—and the associated technological power—to foreign adversaries. ”
evidence, official document, points to no. Claim: U.S. policy worries about distributing cutting-edge model access and technological power to foreign adversaries.
7arxiv.orgAcademic
Risks and Opportunities of Open-Source Generative AI
Source excerpt: “However, closed models still demonstrate weaknesses and, crucially, are also vulnerable to jailbreaking techniques (Zou et al., 2023; Chao et al., 202...”
evidence, official document, points to yes. Claim: Closed models still show weaknesses and are vulnerable to jailbreaking; safeguards are easily bypassed.
8arxiv.orgAcademic
Risks and Opportunities of Open-Source Generative AI
Source excerpt: “Therefore, it is not clear that closed models are definitively "safer" than open-source models. ”
evidence, official document, points to yes. Claim: It is not clear that closed models are definitively safer than open-source models.
9arxiv.orgAcademic
Risks and Opportunities of Open-Source Generative AI
Source excerpt: “Closed models are often considered more secure because access can be restricted or removed should issues be identified. However, closed models can be ...”
evidence, official document, points to yes. Claim: Closed models can be compromised via hacking, leaks, reverse engineering or duplication despite restricted access.
10arxiv.orgAcademic
Risks and Opportunities of Open-Source Generative AI
Source excerpt: “Finally, closed models can be leaked, e.g., Mistral's 70B parameter was leaked by one of their early customers (VentureBeat, 2024b).”
evidence, official document, points to yes. Claim: Mistral's 70B parameter model was leaked by an early customer.
11brookings.eduNonprofits and think tanks
Why open-weight models are crucial for American AI leadership | Brookings
Source excerpt: “But open systems can actually be the foundation for building more secure systems, as they are open to scrutiny by anyone and easier to evaluate. ”
attribution, article, points to yes. Claim: Brookings argues open systems can be the foundation for more secure systems, being open to scrutiny and easier to evaluate.
12arxiv.orgAcademic
Generative AI in Cybersecurity
Source excerpt: “If many people can get their hands on AI models and can interact with them without any barriers, that can allow companies to become aware of flaws in ...”
evidence, official document, points to yes. Claim: Openness lets companies become aware of flaws in their products and fix them faster.
13arxiv.orgAcademic
Generative AI in Cybersecurity
Source excerpt: “Open source AI models have enabled important AI safety research by allowing smaller safety research teams that lack the resources of major AI companie...”
evidence, official document, points to yes. Claim: Open source models enable safety research by smaller teams lacking major AI companies' resources.
14brookings.eduNonprofits and think tanks
Why open-weight models are crucial for American AI leadership | Brookings
Source excerpt: “If American AI is dominated by a small handful of closed model providers, the result will be an overconcentration of power. That raises many concerns,...”
attribution, article, points to yes. Claim: Brookings warns domination by a few closed providers causes overconcentration of power, regulatory capture and impeded market entry.
15arxiv.orgAcademic
The Open-Weight Paradox: Why Restricting Access to AI Models May Undermine the Safety It Seeks to Protect
Source excerpt: “This paper advances a different thesis: the security risks most worth examining may not lie in openness itself, but in what occurs when access is rest...”
attribution, official document, points to yes. Claim: A 2026 paper argues security risks lie in restricted access without governed alternatives, not in openness itself.
16anthropic.comCompanies
Our position on open-weights models \ Anthropic
Source excerpt: “It is irrelevant whether these models are released with open weights, and certainly irrelevant whether they are used by US businesses. In fact, the mo...”
attribution, article, neutral. Claim: Anthropic's CEO says open vs closed weights is irrelevant; the most dangerous model may be trained in secret and given to a hostile military.
17ntia.govGovernment
Public Safety | National Telecommunications and Information Administration
Source excerpt: “In the near term, the marginal uplift in capabilities for cyber attackers that widely available weights introduce to social engineering and phishing u...”
evidence, official document, neutral. Claim: NTIA notes marginal cyber uplift from widely available weights may be significant but is uncertain for any particular model.
18ai-frontiers.orgNonprofits and think tanks
Precaution Shouldn't Keep Open-Source AI Behind the Frontier | AI Frontiers
Source excerpt: “The 2025 International AI Safety Report noted wide disagreement among experts about the likelihood of losing control over advanced AI systems. It like...”
attribution, article, neutral. Claim: The 2025 International AI Safety Report noted wide expert disagreement on loss-of-control likelihood and no scientific consensus on manipulation risk.