Is open-source AI safer or riskier than closed/proprietary AI?

This is a genuinely contested question with reasonable arguments on both sides, and the answer often depends on what kind of "safety" you're talking about (misuse risk, transparency/accountability, concentration of power, cybersecurity, etc.). Let me pull in some current perspectives to give you a balanced view.There's no simple answer—"safer" depends heavily on what risk you're worried about, your time horizon, and who you trust. Here's the honest state of the debate:

Case that open-source/open-weight AI is riskier

Irreversibility and misuse. Once weights are released, they can't be recalled. Critics note once the model weights are released, they are impossible to "unrelease," so preparedness efforts should focus on prevention. This matters especially for capabilities like bioweapons design or cyberattacks, where open-source systems could pose safety risks by increasing the chances of bad actors using these systems for harmful purposes including learning how to create bioweapons and carry out cyberattacks.

Safety fine-tuning is fragile. Even well-aligned open models can be stripped of safeguards cheaply: even if an open-source model is initially aligned, it can easily be transformed into a harmful model—research has shown that with just a few dozen fine-tuning samples, the safety constraints of a model can be removed, enabling it to execute arbitrary instructions.

Geopolitical dimension. Anthropic's Dario Amodei argues the real danger isn't openness per se but capability falling into the wrong hands: the most powerful frontier systems carry an irreversible national-security risk once their weights are released, centering on authoritarian misuse—principally the prospect that the Chinese Communist Party could field AI systems capable of enabling durable military advantage or intensified domestic repression. National security analysts similarly warn that the availability of model weights could allow countries of concern to develop more robust advanced AI ecosystems... these countries could take U.S.-developed models and use them to enhance, perhaps substantially, their military and intelligence capabilities.

Ecosystem-level fragility. Open ecosystems can lack centralized incident response: open-source AI ecosystems are also more susceptible to cybersecurity risks like data poisoning and adversarial attacks because their lack of controlled access, centralized oversight, standardization, and clear guidelines for acceptable use can hinder vulnerability identification and incident-response efforts.

Case that open-source/open-weight AI is safer (or at least not clearly riskier)

Closed models aren't actually that safe either. Multiple surveys find closed-model safeguards are brittle too: closed models still demonstrate weaknesses and are also vulnerable to jailbreaking techniques; closed model safeguards are easily bypassed through simple manipulations like fine-tuning via accessible services, prompting the model to repeat a word, applying a cypher, or instructing the model in another language. The researchers conclude completely preventing models from exhibiting undesirable behaviors might not even be possible, so it is not clear that closed models are definitively "safer" than open-source models.

"Restrictable access" is partly an illusion. Closed models get leaked too: closed models are often considered more secure because access can be restricted, but closed models can be compromised via hacking, leaks, reverse engineering, or duplication—Mistral's 70B parameter model was leaked by one of their early customers.

Transparency enables scrutiny and defense. A coalition including major tech firms argues the opposite framing—that concentration, not openness, is the systemic danger: closed models create single points of failure and transparency of open weights can strengthen cybersecurity by giving defenders capabilities comparable to those available to attackers. On offense-defense balance specifically, one AI researcher argues: it is very likely that cyber capabilities of closed models increase much faster than guardrail performance, and a world where open models are banned while closed models continue to progress would be increasing the offense-defense cyber gap.

Concentration-of-power risk. Some worry that an all-closed world creates a different kind of catastrophic risk—unaccountable concentrated control: the humans or AI agents responsible for development within these closed-source companies would wield immense power, leaving the fate of humanity in the hands of a very small number of intelligent entities. Under this "vaccine" framing, open-source AI can be seen as a "vaccine" for human society—since it may be used maliciously, society is forced to strengthen its defenses, improving overall security; otherwise, if only closed-source AI exists, people might become relaxed, and if it fails alignment or is stolen, humanity could face catastrophic consequences.

Skepticism about current marginal risk. Some argue the catastrophic scenarios are still speculative at today's capability levels: because mitigations elsewhere in the supply chain are still effective, and because even the best models currently offer bad actors only negligible advantages, there is little evidence that open models are driving a material increase in catastrophic risk over the baseline today.

The emerging consensus-ish middle ground

The most sophisticated current position (including from Anthropic itself, generally seen as safety-focused) isn't "open bad, closed good"—it's capability-dependent. Anthropic's own stated view is that the biggest danger isn't openness itself but secret, unaccountable concentration of the most dangerous capabilities: the most dangerous model may be one that is trained in secret and handed only to the People's Liberation Army for use in drones and the Ministry of State Security for surveillance and repression. This suggests policy should target a narrow slice of frontier capability, not open-source as a category — one proposal calls for prohibiting open release only for the small subset of models that pose credible national security threats, such as those trained with a high amount of computational resources or possessing certain security-relevant capabilities, rather than restricting open models broadly.

Bottom line: For today's models, the empirical risk gap between open and closed is smaller than people assume — both have jailbreakable safeguards, both can leak. The real fault line forming in policy circles is about frontier-level capability (bio/cyber uplift) and geopolitics (who controls the most powerful systems), not openness as a blanket category. Open-source likely offers more safety benefits (transparency, broad auditing, resilience against monopoly/single points of failure) at current capability levels, while the risk calculus shifts toward favoring restriction only as models approach specific high-consequence thresholds.