Is open-source AI safer or riskier than closed/proprietary AI?

Both sidesSays it is contested

Open vs closed AI safety is hotly debated — I'll pull current research and expert views to compare the risks.

Neither open-source nor closed/proprietary AI is inherently safer - the expert, industry and government consensus is that each has different safety advantages and risks, and the debate is moving beyond a simple open-vs-closed binary.

1. Why open-source is argued to be safer

•Transparency and inspection: Access to source code allows for thorough examination that decreases the potential for embedded trap doors and/or Trojan horses. The open source development model is described as a viable strategy for producing high quality software.

•Community bug-finding: Diversity in an OSS community stimulates improved software quality, faster innovation and errors in OSS are detected and corrected with everyday use, and much faster than in closed-source software. Similarly, increasing the number of programmers searching for bugs reduces potential areas for exploitation.

•Faster iteration and accountability: The growing openness of AI models fosters transparency, collaboration, and faster iteration across the AI community. It will make access to AI much cheaper, accelerate innovation and make it easier for researchers to analyse the behaviour of AI systems boosting transparency and safety.

•Avoiding concentration of power: Limiting public access to AI could stifle innovation by restricting engineers' ability to improve models, increase secrecy, and concentrate power in the hands of a few individuals and corporations who control access. Transparency through open source is also framed as the only way to protect against bad actors, bias, data exploitation when black-box AI is used, especially in government.

•Possible secure-by-design: Even open models can have protections built in - the developer could take a secure by design approach and build the model in a responsible manner, resilient to common vulnerabilities.

2. Why open-source is argued to be riskier

•Loss of control and easy removal of guardrails: Once AI tools are open-sourced, the organizations they originate from lose control over the development and use of the technology. While closed US models generally come with guardrails to prevent harmful material, open-source models have fewer such limits and the ones that exist are easier to remove.

•Misuse at scale: Easier access means bad actors will be able to fine-tune systems for nefarious purposes, such as generating disinformation. It also means attempts to prevent hostile regimes from gaining access will fail and it makes AI harder to regulate because the genie is out of the bottle.

•Software supply-chain risks: Like any open source project, they can harbour vulnerabilities, outdated components, or even hidden backdoors that scale with adoption. AI models are black boxes built from massive, opaque datasets and hard-to-trace training processes, and there is no definitive way to prove a model is safe.

•Broad harms: Foundation models with widely-available model weights could engender substantial harms, such as risks to security, equity, civil rights, or other harms due to affirmative misuse, failures of oversight, or lack of accountability. Availability of weights could also allow countries of concern to enhance military and intelligence capabilities.

3. Why closed/proprietary is argued to be safer

•Built-in controls: If an enterprise prefers to offload environmental overhead and utilize robust, built-in safety filters with minimal upfront workforce investment, closed-source proprietary models are highly advantageous.

•Controllable deployment: Closed models keep parts secret and allow developers to block outside access and be legally responsible for misuse, which regulators can use to combat misuse.

•Narrative of responsible AI: The most common narrative equates closed AI with responsible AI, and open models with AI risk - proponents argue closed allows centralized oversight and rapid patching.

4. Why closed is argued to be riskier

•Opacity and black-box risk: Closed models introduce opaque black-box architectures, external data privacy risks, and operational token costs. AI's hidden capabilities make open-source essential because consumers cannot easily detect AI manipulation in closed-source code.

•Security through obscurity fails: Available evidence does not show that open models are riskier than closed models or information already available through standard search engines. This is partly because even closed models can be subverted and partly because information for malicious actors might already be available on the internet.

•Less scrutiny: Secrecy often doesn't improve security, and restricting access requires more rigorous analysis to substantiate policy interventions because marginal risk is limitedand.

5. What government and research reviews conclude

The U.S. NTIA, directed by the President's Executive Order on AI to review risks and benefits of large AI models with widely available weights, issued policy recommendations embracing openness while calling for active monitoring of risks.

Its report recommends the U.S. government develop new capabilities to monitor for potential risks, but refrain from immediately restricting the wide availability of open model weights in the largest AI systems, while positioning the government to quickly respond to risks that may arise from future models.

Specifically it calls for an ongoing program to collect evidence of risks and benefits, evaluate that evidence, and act on those evaluations, including possible restrictions if warranted.

Princeton and Stanford researchers in Science argue correctly characterizing risk requires centering marginal risk: to what extent do open foundation models increase risk relative to closed models or to preexisting technologies such as search engines.

> In short: Open-source tends to be safer for transparency, auditability, decentralized innovation, privacy (local hosting) and resisting power concentration; riskier for misuse, removal of safeguards, and proliferation. Closed-source tends to be safer for centralized safety filtering and control; riskier for opacity, lack of independent verification, privacy of data sent to vendor, and concentration of power. Most policymakers now advocate a nuanced, risk-based approach rather than banning either model.

Citations (4)

1linuxfoundation.orgNonprofits and think tanks
Are Open AI Models Safe?
Backs: “Neither open-source nor closed/proprietary AI is inherently safer - the expert, industry and government consensus is that each has different safety advantages and risks, and the debate is moving beyond a simple open-vs-closed binary. 1. Why open-source is argued to be safer. Transparency and inspection: Access to…”
evidence, article, points to yes. Claim: Open-source code access allows thorough examination, reducing trap doors and Trojan horses; open development is a viable strategy for high quality software.
2computerweekly.comNews and media
The hidden security risks of open source AI | Computer Weekly
Backs: “Community bug-finding: Diversity in an OSS community stimulates improved software quality, faster innovation and errors in OSS are detected and corrected with everyday use, and much faster than in closed-source software. Similarly, increasing the number of programmers searching for bugs reduces potential areas for…”
evidence, article, points to yes. Claim: Open-source community bug-finding and faster iteration improve quality, transparency and safety, though open models carry hidden security risks.
3princeton.eduAcademic
For AI, secrecy often doesn't improve security - Princeton Engineering
Backs: “It will make access to AI much cheaper, accelerate innovation and make it easier for researchers to analyse the behaviour of AI systems boosting transparency and safety. Avoiding concentration of power: Limiting public access to AI could stifle innovation by restricting engineers' ability to improve models, increase…”
evidence, article, points to yes. Claim: Limiting public access could stifle innovation and concentrate power; secrecy often doesn't improve security; marginal risk of open models is limited.
4ntia.govGovernment
NTIA Supports Open Models to Promote AI Innovation | National Telecommunications and Information Administration
Backs: “NTIA, directed by the President's Executive Order on AI to review risks and benefits of large AI models with widely available weights, issued policy recommendations embracing openness while calling for active monitoring of risks.”
evidence, official document, neutral. Claim: NTIA review recommends monitoring risks but not immediately restricting open model weights, while preserving ability to act on future risks.