Is open-source AI safer or riskier than closed/proprietary AI?
Neither is inherently safer or riskier overall—the answer is nuanced and depends on the specific risks considered, model capabilities, modality (e.g., language vs. image), and time horizon. Open-source/open-weight AI and closed/proprietary AI each have distinct advantages and drawbacks for safety. Expert opinion and evidence remain divided, with strong arguments on both sides, and the balance shifts as capabilities advance.[1]
Key Definitions
Arguments That Open AI Is Riskier
Open models lose centralized control once released:
Closed models are not invulnerable—jailbreaks, prompt injections, and fine-tuning via provider APIs still occur, and misuse (including hacking assisted by models like Claude) has been documented—but the provider retains more levers.[1]
Arguments That Open AI Is Safer (or Not Riskier)
Openness brings transparency, scrutiny, and distribution of power that can enhance safety:
Closed models can over-refuse useful queries, concentrate power, and still leak or get jailbroken, while their safety claims can be hard to independently verify.
Important Nuances and Current Context
Bottom Line
Open AI is not clearly safer or riskier in a blanket sense. It introduces unique, hard-to-reverse proliferation risks and easier customization for harm, particularly as models grow more powerful, making it riskier for certain catastrophic or hard-to-monitor misuse scenarios. At the same time, it offers unique safety benefits through transparency, broader scrutiny, defensive democratization, and reduced power concentration—and empirical evidence of greater real-world risk for many current threats (especially LLMs) has often been limited compared to the hype.[9][7]
A healthy ecosystem with both open and closed models, strong norms/practices for responsible release (staged evaluation, capability thresholds), focus on high-risk uses rather than banning openness outright, investment in safety research usable by everyone, and evidence-based policy is likely the most robust path. Panic or blanket restrictions on open models are generally viewed as unwarranted by many researchers who have studied the evidence, while complacency about advancing capabilities would also be unwise. The debate continues as the technology evolves.
Citations (12)
- https://www.technologyreview.com/2026/10/13/1146366/open-models-have-real-benefits-do-they-outweigh-the-risks/
Backs: “Expert opinion and evidence remain divided, with strong arguments on both sides, and the balance shifts as capabilities advance.”
- https://benchlm.ai/blog/posts/open-weight-models-explained
Backs: “This is often loosely called “open-source,” though true open-source AI (per OSI definitions) would also include full training code, data details, etc.—most “open” releases are primarily open-weight.”
- https://www.cbsnews.com/news/open-weight-ai-models-safety-risks/
Backs: “Safeguards (refusal training via RLHF or similar) can often be removed cheaply via fine-tuning with a few hundred examples.”
- https://spectrum.ieee.org/open-source-ai-2666932122
Backs: “Safeguards (refusal training via RLHF or similar) can often be removed cheaply via fine-tuning with a few hundred examples.”
- https://www.ibm.com/think/insights/unregulated-generative-ai-dangers-open-source
Backs: “Easier for sophisticated or even moderately resourced bad actors to customize for cybercrime (e.g., variants like FraudGPT/WormGPT based on earlier open models), disinformation, or other harms. “Abliterated” (safety-stripped) versions proliferate.”
- https://www.rand.org/pubs/research_reports/RRA5112-1.html
Backs: “An open model advanced enough for detailed bioweapon guidance, novel pathogen design, or sophisticated autonomous hacking could be catastrophic if misused, with no “kill switch.” Some recent analyses (e.g., RAND work) note that removing refusals from capable open-weight models is feasible and low-cost, potentially…”
- https://engineering.princeton.edu/news/2024/05/10/ai-too-dangerous-release-openly
Backs: “Open image/voice models (e.g., derivatives of Stable Diffusion) have clearly enabled widespread non-consensual intimate imagery (NCII) and related harms that are harder to fix post-release, unlike closed models where loopholes can be patched centrally.”
- https://techpolicy.press/is-open-source-ai-really-the-dangerous-path
Backs: “Closed providers can (and do) detect and respond to misuse, even if imperfectly.”
- https://the-decoder.com/stanford-study-open-source-ai-models-pose-no-greater-risks-than-closed-models/
Backs: “Policy proposals heavily restricting open models risk harming the ecosystem disproportionately. Yann LeCun (Meta) has strongly endorsed this view.”
- https://law.stanford.edu/publications/on-the-societal-impact-of-open-foundation-models/
Backs: “Policy proposals heavily restricting open models risk harming the ecosystem disproportionately. Yann LeCun (Meta) has strongly endorsed this view.”
- https://www.rand.org/pubs/research_reports/RRA2977-2.html
Backs: “Early RAND red-teaming found LLMs did not measurably increase viability of bio attack plans vs. internet-only access. Cyber capabilities often help defenders more.”
- https://www.scmp.com/news/us/article/3362974/us-ai-leaders-turn-chinese-open-weight-models-challenging-closed-source-safety-claims
Backs: “Experts like Andrew Ng have argued that open-weight models can appear safer in practice for certain legitimate security tasks.”