Is open-source AI safer or riskier than closed/proprietary AI?

Is open-source AI safer or riskier? I'll weigh both sides with the latest research.

There is no consensus that either is universally safer - they have different risk profiles, and experts increasingly talk about marginal risk rather than absolute risk.

1. Why open-source is argued to be SAFER

Transparency and auditability:

•The growing openness of AI models fosters transparency, collaboration, and faster iteration across the AI community.

•Many researchers have suggested that since a large community contributes to and scrutinizes open-source code, security holes are less likely to occur than in proprietary software, and can be caught and fixed more quickly.

•Security experts argue transparency through open source is the only way to protect against bad actors, bias, data exploitation and national security threats, and there is no place for black-box AI in sensitive uses.

•Proponents argue open design enables collective oversight, while closed systems' opacity enables biased perspectives to steer development unchecked and deficiencies hide until harm emerges.

Decentralization and resilience:

•Limiting public access to the underlying structures of AI systems could stifle innovation by restricting engineers' ability to improve and adapt the models; increasing secrecy of the models' operation; and concentrating power in the hands of a few individuals and corporations who control access to the technology.

•A report notes promoting key attributes of AI openness can lead to benefits in security, innovation and competition, public transparency and democratic accountability, and education and research.

Privacy and control for deployers:

•Open-source alternatives are often cheaper and more customizable, enabling companies to reduce reliance on vendors and build specialized internal tools, and to keep proprietary data local rather than sending it to a vendor API.

2. Why open-source is argued to be RISKIER

Irreversible loss of control:

•Once AI tools are open-sourced, the organizations they originate from lose control over the development and use of the technology.

•You won't run into restrictions with open-source AI systems developed by organizations like Stability AI, EleutherAI or Hugging Face — or with a proprietary system you're building in-house, unlike closed systems that refuse harmful requests.

Easy removal of safety guardrails:

•A joint Financial Times investigation demonstrated that a free tool called Heretic hosted on GitHub can strip all safety protections from open-weight AI models, including those from Meta, Google, OpenAI, and others, in under ten minutes, using only a standard laptop.

•The tool's creator reports it has been used to produce over 3,500 modified model variants with 13 million cumulative downloads.

•Once modified, these models responded to prompts involving biological weapons, malware generation, and child sexual abuse material that the original systems were designed to refuse.

•Current safety alignment methods create identifiable, isolated neural pathways dedicated to refusal behavior rather than integrating safety throughout the model's processing. Abliteration identifies these refusal pathways and surgically removes them by modifying the model's weights.

This vulnerability applies to open-weight models where parameters are publicly downloadable, including Meta's Llama family, Google's Gemma family, Mistral's open models, and thousands of derivatives hosted on Hugging Face.

It does not apply in the same way to proprietary, API-based models such as Anthropic's Claude or OpenAI's ChatGPT where underlying weights are not accessible and safety protections operate server-side and cannot be modified by end users.

Traditional software supply-chain risks amplified:

•AI models are still software – often bundled with extensive codebases, dependencies, and data pipelines. Like any open source project, they can harbour vulnerabilities, outdated components, or even hidden backdoors that scale with adoption.

•Even when datasets or tuning parameters are available, they're often too large to audit. Malicious behaviours can be trained in, and the non-deterministic nature of AI makes exhaustive testing impossible.

•Open-source models can be vulnerable to traditional software supply-chain exploits, unvetted package dependencies, and local server misconfigurations.

Survey data reflects this perception: survey respondents consider open source AI tools to be riskier than proprietary AI for most types of AI risks, with cybersecurity (62 percent), regulatory compliance (54 percent), and intellectual property (50 percent) cited most.

The paradox is long-standing: the fact that open-source code is audited by many eyes is an advantage in finding security holes, but this same practice also makes it easier for individuals to find weaknesses to exploit than in proprietary software.

3. Why closed/proprietary is argued to be SAFER

•Closed-source insulates enterprises from infrastructure-level vulnerabilities.

•Abliteration does not apply to closed systems where weights are not accessible.

•A closed-source model ensures that the technology is released to the public in a manner that is tested, mature, and safe - vendors can enforce server-side filters, monitoring, and rapid patching.

•Leaders who need strict data privacy, IP protection, and complete explainability with built-in safety filters and minimal workforce investment often find closed-source models advantageous.

4. Why closed/proprietary is argued to be RISKIER

Opacity and concentration of power:

•Until recently AI was developed in the open; use of systems built solely behind closed corporate doors would bring unwelcome centralization of control, and could result in unequal distribution of AI's benefits.

•The most maximally open AI systems can provide some level of transparency, reusability, and extensibility – but do not themselves 'democratize' access to AI by themselves, yet closed systems are seen as even less democratic.

Security through obscurity is weak:

•Many researchers have shown that we can get around the guardrails for the closed systems quite easily, and many risks for both open and closed are overlapping.

•In some ways, the closed AI is a bigger risk simply because of the massive distribution - a closed system with a nice user interface that's cheap is the biggest risk, because hostile actors already have capabilities and open models don't add much marginal risk.

•Closed-source exposes workflows to vendor-side outages, API key leaks, and even prompt jailbreaks, and black-box AI models can be manipulated to perform unauthorized actions, making transparent code crucial.

5. Expert synthesis: Marginal risk and governance

Princeton and Stanford researchers in Science (Oct 2024) concluded available evidence does not show that open models are riskier than closed models or information already available through standard search engines. This is partly because even closed models can be subverted and partly because information for malicious actors might already be available on the internet through search engines.

They argue correctly characterizing distinct risk requires centering marginal risk: To what extent do open foundation models increase risk relative to closed foundation models, or to preexisting technologies such as search engines.

For many threat vectors, existing evidence of marginal risk is limited, which does not mean open models pose no risk but that more rigorous analysis is required to substantiate policy interventions.

The Georgetown panel in June 2025 similarly concluded many of the risks associated with open-source AI also exist in closed systems, challenging the perception that openness inherently makes AI more dangerous.

Researchers who reviewed open-source ML model risks concluded they firmly believe that the benefits outweigh the drawbacks and encourage developers to continue embracing open-source approaches, thereby promoting transparency and fostering innovation, while acknowledging trade-offs between transparency and privacy/security.

Bottom line:

•Choose open-source/open-weight if: you need auditability, data sovereignty, customizability, avoidance of vendor lock-in, and ability for community scrutiny - but you must treat models like any other critical software component, scanning for vulnerabilities, validating training data sources, managing updates, and investing in continuous monitoring for prompt injection and adversarial exploits, because governance, not built-in alignment, becomes the primary safety mechanism.

•Choose closed if: you need provider-managed safety filters, faster time to value (48% of respondents see faster time to value from proprietary tools), and protection against local weight tampering - but you accept opacity, external data privacy risks, and concentration of control.

Most enterprises now use a mixture: more than 70 percent say they are open to either open source or proprietary technologies across areas of the tech stack, and 50%+ already leverage open source in data, models, and tools, suggesting the practical answer is hybrid governance rather than absolutist choice.

5.[### April 2025](https://www.mckinsey.com.br/~/media/mckinsey/business functions/quantumblack/our insights/open source technology in the age of ai/open-source-technology-in-the-age-of-ai_final.pdf)

Citations (5)

1computerweekly.comNot yet classified
The hidden security risks of open source AI | Computer Weekly
Backs: “There is no consensus that either is universally safer - they have different risk profiles, and experts increasingly talk about marginal risk rather than absolute risk. 1. Why open-source is argued to be SAFER. Transparency and auditability: The growing openness of AI models fosters transparency, collaboration, and…”
2princeton.eduAcademic
For AI, secrecy often doesn't improve security - Princeton Engineering
Backs: “Proponents argue open design enables collective oversight, while closed systems' opacity enables biased perspectives to steer development unchecked and deficiencies hide until harm emerges. Decentralization and resilience: Limiting public access to the underlying structures of AI systems could stifle innovation by…”
3jdsupra.comNews and media
Open-Weight AI Models: Safety Guardrails Can Be Removed in Minutes Using Free, Publicly Available Tools | Akerman LLP - JDSupra
Backs: “You won't run into restrictions with open-source AI systems developed by organizations like Stability AI, EleutherAI or Hugging Face — or with a proprietary system you're building in-house, unlike closed systems that refuse harmful requests. Easy removal of safety guardrails: A joint Financial Times investigation…”
4broadbandbreakfast.comNews and media
Risks of Open Source AI Also Exist in Closed Systems: Panelists
Backs: “The most maximally open AI systems can provide some level of transparency, reusability, and extensibility – but do not themselves 'democratize' access to AI by themselves, yet closed systems are seen as even less democratic. Security through obscurity is weak: Many researchers have shown that we can get around the…”
5mckinsey.com.brCompanies
### April 2025
Backs: “exploits, because governance, not built-in alignment, becomes the primary safety mechanism. Choose closed if: you need provider-managed safety filters, faster time to value (48% of respondents see faster time to value from proprietary tools), and protection against local weight tampering - but you accept opacity,…”