Is open-source AI safer or riskier than closed/proprietary AI?

Is open-source AI safer than closed AI? I'll weigh the evidence on risks versus benefits for each.

Open-source and closed AI have different, not strictly greater, risk profiles - current evidence does not show a clear winner.

> "Available evidence does not show that open models are riskier than closed models" - a conclusion echoed by a Stanford Institute for Human-Centered AI study.

> "There is currently limited evidence of the marginal risk of OFMs compared to closed models or existing technologies"

The key question researchers say to ask is marginal risk: To what extent do open foundation models increase risk relative to closed models or to preexisting technologies such as search engines. On that measure:

> "current evidence of this marginal risk remains quite limited"

> Evidence shows that restricting access to models does not necessarily limit the risk of misuse. This is partly because even closed models can be subverted and partly because information for malicious actors might already be available on the internet through search engines

Why open-source is argued to be safer

> Open source software may offer potential security advantages over the traditional proprietary development model. Specifically, access by developers to source code allows for a thorough examination that decreases the potential for embedded trap doors and/or Trojan horses. In addition, the open source model increases the number of programmers searching for software bugs and subsequently developing fixes

Restricting them can also have downsides:
> stifling innovation by restricting engineers' ability to improve and adapt the models; increasing secrecy of the models' operation; and concentrating power in the hands of a few individuals and corporations who control access to the technology

> "In some ways, the closed AI is a bigger risk simply because of the massive distribution"
Closed/proprietary systems also introduce opaque "black-box" architectures, external data privacy risks, and lack of democratic accountability.

Researchers promoting openness argue transparency through open source is needed to protect against bad actors, bias, data exploitation and national security threats and that there is "absolutely no place for black-box AI" in sensitive deployments.

Why open-source is argued to be riskier

> While mainstream generative AI models have built-in safety barriers, open-source alternatives have no such restrictions
> these models belong to entirely closed-source ecosystems, where the developers behind them have the power to dictate what they can and cannot be used for
> Once AI tools are open-sourced, the organizations they originate from lose control over the development and use of the technology

> While the closed US models generally come with guardrails to prevent chatbots from spewing out harmful material, open-source models have fewer such limits and the ones that exist are easier to remove
> you won’t run into any such restrictions with open-source AI systems developed by organizations like Stability AI, EleutherAI or Hugging Face

This has led to rogue models:
> Rogue AIs like FraudGPT and WormGPT are widely available on dark web markets. Both are based on the open-source large language model (LLM) GPT-J
> Malicious actors are also using open-source image synthesizers like Stable Diffusion to build specialized models capable of generating abusive content

> The growing openness of AI models fosters transparency, collaboration, and faster iteration...But those benefits come with familiar risks. AI models are still software – often bundled with extensive codebases, dependencies, and data pipelines. Like any open source project, they can harbour vulnerabilities, outdated components, or even hidden backdoors that scale with adoption

> But easier access to AI also means bad actors will be able to fine-tune systems for nefarious purposes, such as generating disinformation

For biorisks, the dual-use nature of wholly open-sourced models coupled with declining cost to synthesize DNA significantly increases likelihood of misuse has been flagged to policymakers.

Why closed/proprietary is argued to be safer and riskier

Safer: Closed allows centralized enforcement of safety filters, usage policies, monitoring, rapid patching, and legal accountability. Developers can refuse prompts for phishing, weapons instructions or abusive images in a way open weights cannot guarantee after release.

Riskier: Closed obscures one or more of code, data, weights and training methods, making independent safety auditing, bias detection, and alignment research harder - researchers' access to proprietary models was limited, reducing transparency and safety. It also concentrates power and creates single points of failure at massive distribution scale.

Consensus emerging

Most recent academic assessments do not frame it as binary:

> "The ‘open versus closed’ binary obscures much"

Policy researchers recommend:
> Some interventions are better targeted at choke points downstream of the foundation model layer
and that regulators should explicitly consider unintended consequences on innovation.

The Linux Foundation / Economist synthesis captures the trade-off:

> On the plus side, it makes monopolistic control of AI by a handful of companies far less likely. It will make access to AI much cheaper, accelerate innovation across the field and make it easier for researchers to analyse the behaviour of AI systems...boosting transparency and safety. But easier access to AI also means bad actors will be able to fine-tune systems for nefarious purposes

Bottom line:

Most studies to date conclude open does not add substantial marginal risk beyond what closed models and the open web already enable, but that both paradigms need layered defenses: transparency, audits, downstream controls (e.g., email/browser protections for phishing, gene synthesis screening), and shared threat intelligence rather than secrecy alone.