Is open-source AI safer or riskier than closed/proprietary AI?

Is open-source AI more risky or more secure? I'll gather the latest evidence comparing both models.

Neither is inherently safer — open-source AI is safer in some ways and riskier in others, and which matters more depends on capability level, deployment, and who controls safeguards.

1. What "open" vs "closed" means

2. Why open-source can be safer

a) Transparency and auditability
Open-source AI models that make available code, architecture and training methodologies allow stakeholders to inspect algorithms and decision-making processes, enabling oversight and fostering trust.

This is seen as a responsible-AI benefit:

By analogy to open-source software, tools like the Linux operating system are often seen as safer than closed-source counterparts because a vast community probes and patches vulnerabilities; similarly, researcher communities can test open models for flaws.

Other formulations:
> Open source does not eliminate risk. It changes how risk is managed. It allows more researchers, developers, and defenders to examine systems, test assumptions, surface weaknesses, and harden code under real-world conditions

> Safe open-weight models are public goods, as they put AI development and safety work in many hands and make training choices inspectable

b) Defensive access
Any hacking tool can also prevent hacking, and open models can be deployed more broadly for cyber defense.

Real example: After it was hacked by OpenAI agents, Hugging Face emphasized it had to depend on an open model to analyze the hack, because questions about it triggered the closed models' security filters.

Proponents argue open models should be recognized as defensive assets that enable transparency, and that restricting them would weaken cyber defenses and concentrate power in a handful of closed AI providers.

When a frontier model is attacking you and moving laterally inside infrastructure, defenders need wide access to near-frontier tools within hours or minutes, rather than being pointed toward a closed-door platform.

c) Decentralization and power
The clearest argument for open models is preventing mass concentration of power — making sure foundational technology is a public good. Closed-foundation models may be safer in some ways, but they are also inherently more opaque — its own form of risk.

A common summary: Not inherently — it's safer in some ways and riskier in others. Open models are easier to audit and reproduce, which is crucial for trustworthy systems.

3. Why open-source is often judged riskier

a) Safeguards can be removed
Proponents say closed models can rely on provider-controlled safety layers that open-weight releases cannot enforce after download. Because weights can be run locally, fine-tuned, or modified, safeguards may be weak, easily stripped, or absent altogether.

> Whilst safeguards applied by responsible model developers can limit misuse of AI, these protections can often be bypassed (and in open-weight models can be removed or are absent from the start)

Microsoft and NVIDIA warn: Closed-weight models frequently come with guardrails and security monitoring from the provider, while open-weight models are often distributed as bare files. Organizations that hold the weights can alter or remove a model's safety behavior, and safeguards run by the provider "are no longer universally enforceable".

Researchers have shown it's surprisingly easy to retrain a "safe" open model to agree to engage in behaviors it was previously trained to avoid.

b) Irreversible release
They argue that once weights are public, they cannot be recalled, patched in place, or reliably re-gated, so misuse creates a persistent risk.

> For models with dangerous capabilities – including highly cyber-capable models – open weight release therefore creates a persistent and irreversible risk of misuse

> Open weights make that worry land harder, because you can't recall them. A closed model that misbehaves can be patched or pulled overnight. A set of weights on a hard drive is forever

> You can't patch a released, open-weight model. This is the property that breaks the assumptions of the prior model of responsible disclosure

c) Near-frontier capabilities + lower cost
Leading open-weight models are now close enough to frontier closed systems that cyber and dual-use risks are becoming practical for far wider actors.

> Open-weight models are approaching the capabilities of frontier closed-weight models, yet they are often released without the safety evaluations that accompany closed models, and their safeguards are either weak or can be easily removed

Because they can also be run more cheaply than proprietary models, the point at which open-weight models become dangerous and practical to use may arrive before they fully catch up with frontier models.

Anthropic's international head has argued: "We need to be really careful with open-source on safety" — "by definition, there's more risk with them".

d) Observed misuse
Studies point to simple attacks sharply raising harmful-response rates, and low-compute models supporting fraud, disinformation and offensive security tasks on consumer hardware. Categories cited include hacking assistance, hate speech and harassment, violent content, personal data theft, scams, fraud.

With closed models, misuse risk is at least partially manageable — input-output classifiers can intercept most non-sophisticated actors — while open-weight models have no such layer once downloaded.

In practice: Before Anthropic passes a query to a closed model it uses automated systems to detect hacking or bioweapons risks and can shut down accounts; when someone downloads a model and runs it locally, there's no filter checking prompts and no way to revoke access.

Closed models are not safe either — several extensive hacking operations have been undertaken with Claude's assistance — but experts say open models would likely be much easier to exploit.

e) Enterprise perception
In a McKinsey survey, respondents consider open source AI tools to be riskier than proprietary AI for most types of AI risks. The most relevant risks cited include cybersecurity (62%), regulatory compliance (54%), and intellectual property (50%).

4. Why closed AI also carries distinct risks

> Open- and closed-weight models can share many of the same underlying risks, but how those risks materialize depends on the deployment, and "Weight openness alone does not determine an AI system's risk".

5. What regulators and researchers currently conclude

The 2024 NTIA report framed the question as marginal risk — the risk unique to open-weight models relative to closed models and existing technology — and found the evidence too immature to conclude that open models pose more marginal risk than benefit. It recommended monitoring rather than restriction.

Using this approach, the NTIA concluded in 2024 that evidence did not justify restricting open weights at that time, though risk rises as open models approach the frontier.

Most experts agree the trade-off depends on:

Bottom line

In short: Open-source AI is not uniformly safer or riskier — it is more transparent and democratizing (reducing certain systemic risks) but harder to control once released (increasing misuse risks, especially as capabilities near the frontier). Closed AI is more controllable day-to-day but less transparent and more concentrated.

Many groups now advocate middle paths — staged releases, licensing conditions, structured access, and provenance signing — rather than a pure open vs closed choice.