Is open-source AI safer or riskier than closed/proprietary AI?

Is open-source AI a safety net or a risk amplifier? I'll weigh how transparency and access compare to closed safeguards.

Open-source AI is not strictly safer or riskier than closed/proprietary AI - each model has distinct safety trade-offs, and most experts describe it as a balance of risks rather than a binary.

1. Arguments that Open-Source is Safer

Transparency and auditability: Open models foster transparency, collaboration, and faster iteration across the AI community. Because many eyes can inspect code, security holes are less likely to occur and can be caught and fixed more quickly.

Some advocates argue this makes it safer, claiming open-source models are faster, more customizable, more private, and pound-for-pound more capable.

Decentralization and accountability: Proponents argue that implemented responsibly, open source can facilitate wide review to reinforce defenses, citing that Linux and OpenSSL are open source yet power critical infrastructure, demonstrating security is achievable. On accountability, closed models concentrate control in few hands, while open ecosystems cultivate collective responsibility through transparency and peer review.

Benefits outweigh risks view: Some research concludes that nevertheless, the benefits are likely to outweigh the downsides of open-source AI, especially if a responsible open-source approach with clear barriers against misuse is pursued. A large survey paper on open-source generative AI argued that, overall, the benefits of open source Gen AI outweigh its risks.

Advantages often cited for enterprises include:

2. Arguments that Open-Source is Riskier

Loss of control and easy misuse: Once AI tools are open-sourced, the organizations they originate from lose control over the development and use of the technology. Unlike closed systems, you won't run into restrictions with open-source AI systems when asking them to misbehave.

Anyone can modify bypassing review: Open source software can be modified by anyone and it can be difficult to establish the dependencies present within open source code... anyone can deploy changes without review or approval – including adversaries.

Specific risks include:

Weak guardrails in practice: Open-weight AI models' safety guardrails can be removed in minutes using free, publicly available tools, as demonstrated in a joint investigation that showed a tool called Heretic hosted on GitHub can strip all safety protections from open-weight models including those from Meta, Google, OpenAI in under ten minutes using only a standard laptop.

Example: Cisco warned DeepSeek R1 has "critical safety flaws" and failed to block a single harmful prompt when tested against 50 random prompts taken from the HarmBench dataset, and researchers found it was significantly easier to bypass its weak safeguards to generate harmful content with little to no specialized knowledge. Researchers concluded open models like DeepSeek lack the guardrails found in more established models and have not undergone extensive red teaming.

Inherent limits to auditing: AI models are black boxes built from massive, opaque datasets and hard-to-trace training processes. Even when datasets are available, they're often too large to audit. Malicious behaviours can be trained in and the non-deterministic nature makes exhaustive testing impossible. Bias is subtle and dangerous as skewed training data bakes in systemic flaws that are hard to detect.

Survey data reflects this perception: respondents consider open source AI tools to be riskier than proprietary AI for most types of AI risks, with the most relevant risks cited as cybersecurity (62%), regulatory compliance (54%), and intellectual property (50%).

Other enterprise risks include:

3. Arguments that Closed / Proprietary is Safer

Built-in controls and maturity: A closed-source model ensures that the technology is released to the public in a manner that is tested, mature, and safe.

Closed providers often provide robust, built-in safety filters with minimal upfront workforce investment, including guardrails and controls to ensure outputs align with ethical guidelines and mitigate hate, sexual, violence, self-harm, and prompt injection attacks.

Advantages cited:

4. Arguments that Closed / Proprietary is Riskier

Opacity and concentration of power: Closed-source proprietary models introduce opaque "black-box" architectures, external data privacy risks, and concentrate control in few hands.

Critics argue: "In some ways, the closed AI is a bigger risk simply because of the massive distribution". Because a single flaw or bias in a widely used proprietary model affects millions, and external audits are impossible.

There is also concern that transparency through open source is the only way to protect against bad actors, bias, data exploitation and national security threats, with no place for black-box AI in sensitive contexts.

Closed systems also have data privacy risks from sending proprietary competitive information to an external provider and escalating token costs at scale.

5. Synthesis: Context Matters

Most governance experts now argue that government and policy stakeholders should consider more nuanced approaches towards AI safety, moving beyond the binary of open vs. closed model access.

In practice, many organizations adopt a hybrid approach: using open models locally with added governance - scanning for vulnerabilities, validating training data sources, managing updates, implementing model approval processes, dependency tracking, and continuous monitoring for prompt injection and manipulation, plus sandboxing and training data audits.