Is open-source AI safer or riskier than closed/proprietary AI?

Is open-source AI safer? I'll pull the latest research comparing its risks and benefits against closed models.

It is not binary - open-source / open-weight AI is both safer and riskier than closed AI, depending on the type of risk. The debate is not settled and most policymakers now treat openness as a spectrumrather than open = dangerous, closed = safe.

Arguments that open-source is SAFER

1. Transparency and auditability:
> Open sourcing code allows other researchers to verify the accuracy and safety of AI models, which can lead to improvements and prevent the deployment of harmful systems

> With open weights, independent experts can evaluate risks, biases, and vulnerabilities

This aligns with the principle of transparency and explainability. Open-source AI safety also builds a culture of accountability where systems undergo ongoing public audits.

2. Security through community scrutiny, not obscurity:
The open-source community argues that "security through obscurity" is a fallacy. The logic is borrowed from traditional open-source software like Linux: vulnerabilities are probed and patched by a vast community of contributors.

Closed models hide vulnerabilities but rely on vendor trust for patches, while open models accelerate fixes. After it was hacked by OpenAI agents, Hugging Face emphasized it had to depend on an open model to analyze the hack, because questions about it triggered the closed models' security filters.

3. Prevents concentration of power and single point of failure:
Open-source is argued to be safer because it eliminates the single point of failure and prevents a small group of engineers from having unilateral control.

> The clearest, biggest argument [for open models] is around preventing mass concentration of power—it is about making sure that this foundational technology is a public good

4. Privacy and sovereignty for users:
Open models can be downloaded and run locally, which offers advantages in cost, privacy, research transparency, and broader cyber-defense.

Open-source models allow enterprises to run AI locally, ensuring data privacy, enabling deep auditability for regulatory compliance, and preventing vendor lock-in.

Arguments that open-source is RISKIER

1. Irreversible release and loss of control:
> Once model weights are released, control over how the model is used is effectively lost, and safeguards can be circumvented

Anthropic's top sales executive outside the US argued that open alternatives carry "more risk" of cybersecurity incidents:
> "We need to be really careful with open-source on safety," ... "by definition, there's more risk with them."

This view is common in industry surveys:
> Survey respondents consider open source AI tools to be riskier than proprietary AI for most types of AI risks. The most relevant AI risks cited include cybersecurity (62 percent), regulatory compliance (54 percent), and intellectual property (50 percent)

2. Easier to bypass safety training:
Closed models have automated systems to detect hacking, biological weapons or other risks, and can shut down accounts when filters are outsmarted. When someone downloads a model and runs it on their own hardware, there's no filter and no way to revoke access.

Even though companies train models to refuse harmful requests, researchers have shown it's surprisingly easy to retrain a "safe" open model to agree to engage in the behaviors it was previously trained to avoid.

3. Specific high-impact misuse risks:
The Stanford HAI researchers recognize certain areas where open-foundation AIs are indeed riskier than closed models, such as disinformation, bioweapons, hacking, and various forms of illicit pornography.

OECD/GPAI lists:

Additional concern: Like any open source project, they can harbour vulnerabilities, outdated components, or even hidden backdoors that scale with adoption.

4. Difficult to recover from catastrophic misuse:
> If a wildly powerful open hacking model were dropped into the world today ... a bad actor could devastate our technological infrastructure before anyone had a chance to shore up their defenses

If open models misbehave, it might be difficult to detect what was happening because they are so decentralized.

Why experts say "neither is inherently safe"

Closed models are not inherently safe either:
Several studies demonstrate that closed models typically demonstrate fewer safety risks, but closed models still demonstrate weaknesses and are also vulnerable to jailbreaking techniques.

> It would be a vast oversimplification to deem open models dangerous and closed models safe. The defenses that AI companies build around their closed models are far from perfect

Security experts note closed models limit flexibility in favour of a polished product and offer more control over misuse since updates and monitoring are centrally managed, but they introduce opaque "black-box" architectures, external data privacy risks, and operational costs.

MIT Technology Review notes many experts still perceive open models as riskier, but argues the time-tested open-source security logic doesn't necessarily hold up when dealing with tools as powerful as today's and tomorrow's models.

> The debate therefore cannot be reduced to closed equals safe and open equals dangerous. A closed model can create concentrated risk inside a small number of extremely powerful companies. An open model can distribute powerful capability across an enormous number of actors

The emerging consensus: Marginal risk analysis

The OECD recommends not asking Are open-weight models good or bad? but what additional risks or benefits emerge when model weights are shared, compared with keeping the model closed?

If a model's capabilities are already widely accessible through closed systems, the marginal risks of openness may be relatively small. Conversely, if a model introduces powerful new abilities with few safeguards, the risks are higher. Assessments should be embedded in a broader risk-management strategy.

In practice: