Is open-source AI safer or riskier than closed/proprietary AI?

This debate splits the AI world — I'll lay out the safety and risk cases for open versus closed systems.

Open-source AI is neither universally safer nor universally riskier than closed / proprietary AI - both have distinct safety advantages and risks, and recent research emphasizes marginal risk vs absolute risk.

1. What the terms mean

2. Arguments that open-source is SAFER

a. Transparency and scrutiny:
Open-weight systems increase transparency, allow for widespread red teaming, and decrease market concentration. The transparency of open-source frameworks often results in stronger security, as vulnerabilities are quickly identified and resolved by a global community of experts.

This openness:

Proponents also argue proprietary, company-controlled services are not inherently safer: their operators can keep flaws from public scrutiny, monitor how people use them, impose restrictions, raise prices, or withdraw access.

b. Avoiding concentration of power:
Restricting access can stifle innovation by restricting engineers' ability to improve and adapt the models; increase secrecy of the models' operation; and concentrate power in the hands of a few individuals and corporations who control access to the technology.

One panelist argued: "In some ways, the closed AI is a bigger risk simply because of the massive distribution".

3. Arguments that open-source is RISKIER

a. Irreversible proliferation and removal of safeguards:
Open-weight models also pose unique risks. They can allow harmful AI capabilities to proliferate rapidly and irreversibly. Just as they can be modified for beneficial uses, they can also be modified for harmful ones. Safeguards designed to prevent harmful behaviours can also be quickly and cheaply removed from open-weight models.

If a model is deployed with closed weights, its deployer can draw from a rich toolkit for making it safer, such as content filters and control of points of access to enforce acceptable usage policies. Open-weight models, on the other hand, are harder to safeguard, because they can be shared and modified arbitrarily without oversight.

On open models, jailbreaks and guardrail bypasses are easier.

b. Lower barrier to misuse - CBRN and cyber:
The U.S. National Telecommunications and Information Administration (NTIA) identified two discrete public safety risks:

Open model weights could possibly increase this risk because they are more accessible to a wider range of actors, including actors who otherwise could not develop advanced AI models and easy to distribute, which means that the original model and augmented, offshoot models can be proliferated and used for harm without developer knowledge.

Cyber attacks that rely on dual-use foundation models with widely available model weights could take various forms, such as social engineering and spear-phishing, malware attack generation, and exploitation of other models' vulnerabilities.

> As with all digital data in the Internet age, the release of model weights also cannot feasibly be reversed.

4. What does the evidence say about comparative risk?

A 2024 Science article by researchers including Princeton that reviewed these threat vectors concluded:

> The researchers found that available evidence does not show that open models are riskier than closed models or information already available through standard research techniques such as online searching.

They also concluded that evidence shows that restricting access to models does not necessarily limit the risk of misuse. This is partly because even closed models can be subverted and partly because information for malicious actors might already be available on the internet through search engines.

The key analytical question, they argue, is marginal risk:

> "Correctly characterizing the distinct risk of open foundation models requires centering marginal risk: To what extent do open foundation models increase risk relative to closed foundation models, or to preexisting technologies such as search engines".

The NTIA report similarly notes that no current models, proprietary or widely available, offer uplift on CBRN tasks relative to open source information resources on the Internet, and that the marginal cybersecurity risk posed by widely available weights remains unclear, and likely varies by attack vector.

The UK AI Security Institute (AISI) notes that while open-weight systems are the lifeblood of open research and innovation, managing risks is challenging and requires a multi-pronged toolkit including training data curation, tamper-resistant fine-tuning, and full-access audits.

5. Consensus view

Most experts and policy reports now frame it as a trade-off, not a binary choice:

Current best-practice recommendations often include staged deployment - ranging from fully closed to fully open rather than an immediate full release - plus training data filtering, which can be over 10x more effective at resisting adversarial fine-tuning than defences added after training, and requiring full-access audits and transparency documentation before any open-weight release.

> In short: For current-generation models, research suggests closed control does not automatically make AI safer, and openness does not automatically make it more dangerous. For future highly capable models (especially with potential WMD or advanced cyber capabilities), open release is considered irreversibly riskier, which is why proposals like "not deploying high-risk systems with open weights" are part of the risk-management toolkit.