Is open-source AI safer or riskier than closed/proprietary AI?

Is open-source AI the safer bet? I'll compare its risks against closed systems.

Open-source AI is neither categorically safer nor categorically riskier than closed/proprietary AI - experts are divided and the answer depends on what type of safety risk you mean. Both models have distinct safety advantages and disadvantages:

Arguments that Open-Source is Safer

1. Transparency and auditability:
Open models can be scrutinized, reused and built on top of to varying degrees. The transparency of open source allows for independent scrutiny and auditing of AI systems' behaviors and ethics.

Proponents argue this many-eyes principle improves security:

> Open source software may offer potential security advantages over the traditional proprietary development model. Specifically, access by developers to source code allows for a thorough examination that decreases the potential for embedded trap doors and/or Trojan horses

> In addition, the open source model increases the number of programmers searching for software bugs and subsequently developing fixes, thus reducing potential areas for exploitation by malicious programmers

Making AI open source means it is also open to external parties to audit the system without needing special access, permissions or NDAs, like they might with closed source AI.

2. Democratization and decentralization:
> It will make access to AI much cheaper, accelerate innovation across the field and make it easier for researchers to analyse the behaviour of AI systems (their access to proprietary models was limited), boosting transparency and safety

Open-source approaches also lower barriers to innovation, accelerate science, and improve transparency and promote competition while supporting democratization instead of leaving it solely in the hands of big corporations.

Researchers using this framing argue that overall, the benefits of open source GenAI outweigh its risks and that the significant benefits enumerated allow them to strongly support the open sourcing of models in the near to midterm stage.

3. Privacy and control:
Open models can be run locally, which strongly favours local execution for strict data privacy, IP protection, and complete explainability, avoiding the need to upload proprietary company data into a third-party black box.

Arguments that Open-Source is Riskier

1. Cannot be recalled and guardrails can be removed:

> Once AI tools are open-sourced, the organizations they originate from lose control over the development and use of the technology

While open-source models offer distinct advantages, the argument that open models cannot be rolled back or forced to update is a net negative of these models.

Fine-tuning large language models on even a small number of harmful data points can easily disrupt safety mechanisms, and while mainstream models have guardrails, you won't run into any such restrictions with open-source AI systems developed by organizations like Stability AI, EleutherAI or Hugging Face.

As The Economist summarized the trade-off:

> But easier access to AI also means bad actors will be able to fine-tune systems for nefarious purposes, such as generating disinformation. It means Western attempts to prevent hostile regimes from gaining access to powerful AI technology will fail. And it makes AI harder to regulate, because the genie is out of the bottle

2. Greater misuse potential:

> However, without safeguards, generative AI is the next frontier in cyber crime. Rogue AIs like FraudGPT and WormGPT are widely available on dark web markets. Both are based on the open-source large language model (LLM) GPT-J

> Malicious actors are also using open-source image synthesizers like Stable Diffusion to build specialized models capable of generating abusive content

The open-source nature of dual-use AI models coupled with both the declining cost and skills required to synthesize DNA significantly increase the likelihood of the misuse of such models. Researchers have shown open-source AI protein-design tools could be used to paraphrase the amino acid sequences of toxic proteins.

Other risks cited include data poisoning, vulnerabilities in codebases and dependencies, and hidden backdoors that scale with adoption, and prompt injection attacks which are arguably greater in open-source environments lacking sufficient oversight.

Arguments that Closed/Proprietary is Riskier

1. Black-box opacity:
Closed models are proprietary systems controlled by companies like OpenAI, Google, or Anthropic.

Their internal workings are unknown to users (and sometimes, even developers).

> Further, the opacity of black box systems heightens security risks, leaving room for exploitation by malicious actors, susceptibility to human biases in training, and potential violations of regulations governing the use of sensitive personal data

Closed-source proprietary models introduce opaque "black-box" architectures, external data privacy risks, and make it impossible to inspect how models operate, preventing research into how they work.

2. Concentration of power and single-point-of-failure:

Some argue closed AI is a bigger risk simply because of the massive distribution:

> In some ways, the closed AI is a bigger risk simply because of the massive distribution

There is also concern that Western attempts to control access fail, monopolistic control becomes likely, and users must trust a few powerful companies to act as gatekeepers for safety, bias, and censorship decisions — a concern cited by Meta's chief AI scientist Yann LeCun who argued secrecy is a huge mistake.

Proponents of closing models note that a closed-source model ensures that the technology is released to the public in a manner that is tested, mature, and safe.

Bottom Line

Most recent comprehensive reviews conclude the distinction is not binary. As one policy panel noted, risks of open source AI also exist in closed systems, and the academic framing in 2024 was that benefits outweigh risks provided responsible development principles and liability for misuse (not for open sourcing itself) are enforced.

In practice, many deployed models are "semi-open" - offering weights but not training data or code under non-open restrictions - which blends both sets of risks.