Is open-source AI safer or riskier than closed/proprietary AI?

Open vs closed AI safety is hotly debated — I'll map the arguments for each side.

It is not binary - experts disagree because each model has different safety advantages and risks.

1. Why Open-Source is Argued to be Safer

Transparency and auditability: Open-source AI provides unprecedented transparency into how AI systems function, enabling rigorous auditing, identification of biases, verification of safety properties, and detection of security vulnerabilities.

Unlike closed proprietary systems that operate as black boxes, open-source agentic AI can be examined for biases, security vulnerabilities, and alignment issues by independent experts worldwide. This allows anyone to inspect the architecture, trace decision-making processes, and understand system limitations.

Other benefits cited include:

•Increased transparency: The datasets and codes of open-source models can be audited and verified by third parties, which helps to ensure their quality and reliability.

•Collective intelligence: Open-source development leverages collective intelligence through distributed community scrutiny, a model proven successful by projects like Linux. When applied to AI safety, this collaborative approach accelerates the identification and resolution of security flaws.

•Preventing concentration of power: Concentration of AI development within a few large companies raises significant concerns about regulatory capture, where major industry players shape regulations to protect their interests. Without open sourcing, there is less transparency and accountability, which could lead to a concentration of power and knowledge in the hands of a few select organizations.

Advocates summarize this as: the clearest, biggest argument [for open models] is around preventing mass concentration of power—it is about making sure that this foundational technology is a public good. With traditional proprietary AI systems, organizations must trust vendor claims about model behavior, bias testing, safety measures, and data handling practices without independent verification capability.

Open models also challenge closed architectures that create a dangerous concentration risk, where an organization’s strategic roadmap, compliance posture, and even its basic ability to operate are tethered to a third party’s commercial stability and opaque architectures. Concentration can limit independent oversight and restrict transparency.

Proponents also argue security through obscurity is a fallacy, and that open-source AI is actually safer because it eliminates the single point of failure and prevents a small group of engineers from having unilateral control over the world's most powerful cognitive tools.

Stanford Institute for Human-Centered AI (HAI) researchers noted in this context that an open-foundation LLM tailored to help build chemical weapons is dangerous, but the danger is marginal because it would largely derive its knowledge from websites people can already access with web search.

2. Why Open-Source is Argued to be Riskier

Survey respondents consider open source AI tools to be riskier than proprietary AI for most types of AI risks, with the most relevant risks cited as cybersecurity (62%), regulatory compliance (54%), and intellectual property.

Anthropic's position: Anthropic PBC's top sales executive outside the US argued that open alternatives carry “more risk” of cybersecurity incidents or other dangers: “We need to be really careful with open-source on safety... By definition, there’s more risk with them”.

Specific technical reasons:

•No controllable filters or revocation: Before Anthropic passes a user's query to one of its closed models, it uses automated systems to detect whether that query relates to hacking, biological weapons, or other risks. When someone downloads a model and runs it on their own hardware, there's no filter checking whether their prompts are safe, and there's no way to revoke their access.

•Easy removal of safeguards: Researchers have shown that it’s surprisingly easy to retrain a "safe" open model to agree to engage in the behaviors it was previously trained to avoid. Research demonstrates that safety guardrails can be removed from open models through fine-tuning with relatively minimal computational resources.

•High-impact misuse categories: Stanford HAI researchers recognize certain areas where open-foundation AIs are indeed riskier than closed-foundation models, such as disinformation, bioweapons, hacking, and various forms of illicit pornography.

•Biosecurity / AIxBio: Scientists are moving quickly to provide widely distributed open-source AIxBio capabilities, however there is limited shared understanding of potential biosecurity risks and no guidelines for how to guard against potential misuse. Bill Gates has warned that open-source AI software, while valuable for innovation, could lower barriers for malicious actors to build bioweapons.

The paradox is summarized by MIT Technology Review: So closed models are risky enough—and open models would likely be much easier to exploit, and many experts still perceive open models as riskier than closed models because the time-tested open-source security logic doesn't necessarily hold up when dealing with tools as powerful as today's models.

Closed models also retain the ability to pull the plug - e.g., if someone used an open-foundation LLM to build a chatbot with comprehensive knowledge of how to build chemical weapons, the original developer could not shut it down once downloaded, unlike an API-based closed model.

3. Why Closed/Proprietary AI is Also Risky

Defenses that AI companies build around their closed models are far from perfect, and people misuse those models all the time. Closed models are inherently more opaque, and that is, in fact, its own form of risk - the less transparent developers are about how foundation models are built, the harder it will be to root out problems like racial and gender bias.

Conclusion: Trade-off Depends on Threat Model

> The debate is not "safe vs. dangerous" but what kind of risk you prioritize.

•If your priority is defense against misuse by bad actors, decentralization, and catastrophic one-off events: closed models are considered safer in the short term because access can be monitored, filtered, and revoked, and retraining to remove safety is not possible by the end user.

•If your priority is systemic, long-term safety, accountability, equity, and resilience: open models are considered safer because they enable independent auditing, rapid community patching, prevention of power concentration, and democratic oversight - addressing the argument that keeping AI closed doesn't guarantee safety—it guarantees opacity, and that's far riskier long term.

Most current policy research therefore seeks a middle ground, rather than an all-or-nothing framing, developing techniques like tamper-resistant training, pre-training data filtering, and Responsible AI Licenses (RAIL) that allow open distribution while embedding use-based restrictions.

Citations (3)

1planetcrust.comNot yet classified
How Can Open-Source Make Agentic AI Safer?
Backs: “Transparency and auditability: Open-source AI provides unprecedented transparency into how AI systems function, enabling rigorous auditing, identification of biases, verification of safety properties, and detection of security vulnerabilities. Unlike closed proprietary systems that operate as black boxes, open-source…”
2technologyreview.comNews and mediaquality 0.91
Open models have real benefits. Do they outweigh the risks?
Backs: “Without open sourcing, there is less transparency and accountability, which could lead to a concentration of power and knowledge in the hands of a few select organizations. Advocates summarize this as: the clearest, biggest argument [for open models] is around preventing mass concentration of power—it is about making…”
3thirdway.orgNonprofits and think tanks
What is “Open” AI?
Backs: “Proponents also argue security through obscurity is a fallacy, and that open-source AI is actually safer because it eliminates the single point of failure and prevents a small group of engineers from having unilateral control over the world's most powerful cognitive tools. Stanford Institute for Human-Centered AI…”