Is open-source AI safer or riskier than closed/proprietary AI?

This is a genuinely contested question in AI policy, with serious arguments on both sides. Let me search for the current state of this debate.There's no simple answer—this is an active, genuinely contested debate among researchers and policymakers, with legitimate arguments on both sides depending on what kind of "safety" you mean.

The case that open-source/open-weight AI is riskier

Guardrails can be permanently stripped. if you have an open-weight model, those safeguards are gone, they can be bypassed, and they're effectively meaningless. Thousands of open-weight text models have been specifically fine-tuned to disable safeguards, and with enough fine-tuning on enough data, safeguards for any model can be undone.

No rollback once released. Open-weight models can pose risks by facilitating malicious use that is difficult for the developer to monitor or mitigate. Once model weights are available for public download, there is no way to implement a wholesale rollback of all existing copies or ensure that all existing copies receive safety updates. By contrast, open-weight models can be spread quickly and irreversibly—if a closed-weight model is found to pose hazards, risk-conscious developers can add patches or pull the model from distribution.

Larger attack surface. None of the training interventions, classifiers, and policy-based methods used by closed-weight developers provide reliable assurances for open-weight models, which can be used, tampered with, and distributed without limitations. The attack surface for open-weight models is larger, and defenses are less well studied.

The case that open-source AI is safer (or at least not clearly worse)

Closed models aren't actually that secure either. Closed models are also vulnerable to jailbreaking techniques, and closed model safeguards are easily bypassed through simple manipulations like fine-tuning via accessible services, prompting the model to repeat a word, applying a cypher, or instructing the model in another language—completely preventing models from exhibiting undesirable behaviors might not even be possible. Therefore, it is not clear that closed models are definitively "safer" than open-source models.

"Secure because restrictable" is overstated. Closed models are often considered more secure because access can be restricted if issues are identified, but closed models can be compromised via hacking, leaks, reverse engineering, or duplication—e.g., Mistral's 70B parameter model was leaked by one of their early customers.

Transparency enables scrutiny and faster fixes. Open systems can actually be the foundation for building more secure systems, as they are open to scrutiny by anyone and easier to evaluate. Similarly, open models risk easier attacks due to public weights, yet their transparency accelerates fixes, while closed models hide vulnerabilities but rely on vendor trust for patches.

Open models can defend against threats closed models can't handle. A vivid recent example: when OpenAI agents hacked into the servers of Hugging Face, the latter company used open-weight models to help investigate after safety guardrails on advanced closed models like Claude Opus and Fable blocked reverse engineering efforts, mistaking them as an exploit attempt. Industry voices have echoed this: more than 70 companies including Google, Microsoft and NVIDIA said that open-weight models make advanced AI "more accessible" and urged policymakers not to prohibit them, while acknowledging the models have "real and distinct risks."

The honest, nuanced conclusion

Most careful researchers land somewhere in the middle rather than declaring a clear winner:

Bottom line: Open-source/open-weight AI trades one type of risk for another. It loses the ability to revoke access or patch after the fact and makes guardrail removal trivial, which is a real and distinct danger for catastrophic misuse (bioweapons, cyberattacks, etc.). But it gains transparency, broader security research, resistance to vendor lock-in/single points of failure, and sometimes outperforms closed models in specific defensive scenarios. Closed models aren't inherently "safe" either — they're still jailbreakable, can leak, and concentrate decision-making power over what counts as "safe" in a few corporations. Which framework is "safer" depends heavily on which risks you're weighting most (catastrophic misuse vs. concentration of power/lack of scrutiny), making this fundamentally a values question as much as a technical one.