Is open-source AI safer or riskier than closed/proprietary AI?

The debate over whether open-source AI is safer or riskier than closed (proprietary) AI is one of the most consequential issues in technology, national security, and policy.

To understand the debate, it is helpful to first clarify the terminology. While "fully open-source" AI would include the open publication of training data, code, and architecture, the debate today primarily centers on open-weight models (where the core mathematical "weights" of the model are made freely downloadable, such as Meta’s Llama or Mistral’s models) versus closed-source models (where the model is kept behind a corporate server and accessed only via an API, such as OpenAI's GPT models or Google’s Gemini).

Both paradigms present distinct types of safety advantages and hazards. The consensus among researchers and policymakers is not that one is objectively "safer," but rather that they shift different kinds of risk.

Why Open-Source AI is Considered Safer (and Closed is Riskier)

Proponents of open-source AI—including many academic researchers, civil liberties groups, and software developers—argue that openness is essential for long-term safety and security. Their arguments include:

Closed-source models are "black boxes." External researchers cannot fully inspect their training data, alignment processes, or hidden biases. Open-source models allow the global scientific community to audit the code, identify security vulnerabilities, detect systemic biases, and study model alignment in ways that proprietary companies cannot or will not do publicly.

Relying entirely on a small handful of massive tech conglomerates to control the world’s leading cognitive infrastructure poses a systemic risk. If a closed-source provider experiences a massive outage, is hacked, or decides to abruptly alter its terms of service, it can disrupt entire industries. Open-source democratizes access, ensuring that no single company holds a monopoly on intelligence.

In cybersecurity, there is a famous principle known as Linus’s Law: "Given enough eyeballs, all bugs are shallow." By open-sourcing models, the global developer community can quickly build defensive tools, patch security vulnerabilities, and create robust guardrails. Proprietary software security is dependent on a single company’s internal security team, which is a single point of failure.

Proprietary models are heavily steered by the corporate, legal, and ideological incentives of their parent companies. Open-source allows users to customize models to their specific values, legal jurisdictions, and cultural contexts rather than forcing them to adopt a worldview dictated by Silicon Valley.

Why Closed/Proprietary AI is Considered Safer (and Open-Source is Riskier)

Proponents of closed-source AI—including many safety-oriented AI labs, national security officials, and regulators—argue that highly capable models are too dangerous to be distributed freely without centralized oversight. Their arguments include:

Once an open-source model is published to the internet, it cannot be recalled. If a critical safety exploit, dual-use biological hazard, or catastrophic capability is discovered in an open-source model after its release, it is out in the wild forever. By contrast, if a vulnerability is found in a closed model, the provider can patch it globally or take the API offline within seconds.

While open-source developers try to implement safety guardrails (via reinforcement learning from human feedback, or RLHF), these guardrails are incredibly fragile. Researchers have repeatedly shown that with a very small amount of compute (often costing under $100), a malicious actor can fine-tune an open-weight model to completely strip away its safety filters. Once stripped, the model can be used to generate malicious code, optimize cyberattacks, or assist in chemical/biological weapon design.

Closed-source API providers can monitor query telemetry in real-time. If a user attempts to generate propaganda, coordinate a cyberattack, or search for nuclear weapon instructions, the API provider can flag the activity, block the prompt, and ban the user. With open-source AI, the model is run locally and offline. There is zero surveillance, giving bad actors complete privacy to abuse the system.

Open-source AI lowers the technical and financial barriers to entry for rogue states, terrorist groups, and cybercriminals. They do not need to invest hundreds of millions of dollars to train a model; they can simply download a highly capable open-weights model and customize it for harm.

The Regulatory Perspective: "Marginal Risk"

Government agencies and policy groups have had to grapple with this trade-off. A pivotal study on this topic was released by the U.S. National Telecommunications and Information Administration (NTIA).

The NTIA concluded that the U.S. government should not restrict the availability of open-weight models. Instead of assuming open-source is inherently dangerous, the NTIA advocated for "marginal risk analysis":

Summary

Ultimately, closed-source AI reduces tactical risk (preventing immediate, malicious use by bad actors), while open-source AI reduces systemic risk (preventing monopolies, black-box vulnerabilities, and central points of failure). Finding the right balance between the two remains one of the most critical challenges of AI governance.