Both, depending on the capability level, release details, and risk type—there is no universal answer. Open-source (or open-weights) AI is often safer for transparency, auditing, and distributed scrutiny on current systems, but riskier for irreversible proliferation of advanced dual-use capabilities. Closed/proprietary AI concentrates control and enables staged mitigations but introduces opacity, single points of failure, and misaligned incentives. The net effect hinges on whether the dominant risks are misuse/accidents from widely available models or failures/capture by a few developers.
Transparency and Scrutiny Favor Open Approaches
•More eyes find flaws: Open weights, code, and (ideally) data let researchers, governments, and outsiders probe for backdoors, biases, jailbreaks, memorization of secrets, or deceptive alignment. Closed models rely on the developer's internal red-teaming and external API access, which is incomplete. Historical software analogy: widely scrutinized open projects frequently surface vulnerabilities faster than proprietary ones (though disclosure quality varies).
•Safety research accelerates: Techniques in interpretability, adversarial robustness, unlearning, and alignment evaluations have benefited heavily from open models and datasets. Closed labs publish selectively; outsiders cannot easily replicate or extend their safety work.
•Reduces concentrated power risks: A small number of companies or states controlling frontier models creates risks of misuse by those entities, regulatory capture, insider threats, or sudden capability jumps without broader preparation. Open releases distribute know-how and enable defensive applications (e.g., better detection tools).
•Empirical pattern so far: Releases like Llama, Mistral, and earlier models have driven rapid iteration without clear evidence of uniquely catastrophic misuse beyond what closed APIs already enable via jailbreaks, fine-tuning proxies, or leaks. Near-term harms (deepfakes, scams, low-level cyber aids) scale with accessibility but are not exclusive to open models.
Proliferation and Irreversibility Favor Caution with Closed or Controlled Releases
•Anyone can use or adapt them: Once weights are public, actors with modest resources (criminals, terrorists, rogue states, or reckless amateurs) can fine-tune, quantize, agentize, or combine them for bioweapon design assistance, sophisticated cyber offense, mass persuasion/disinformation, or autonomous scams. Closed APIs allow rate limits, monitoring, refusal training, and revocation. Open models cannot be "recalled."
•Capability overhang and dual-use: At frontier levels, models may cross thresholds for high-stakes misuse before defenses mature. Open-sourcing compresses the timeline for bad actors while safety techniques (scalable oversight, robust alignment, hardware-level controls) lag. This is the core concern of many researchers focused on catastrophic risk: diffusion makes coordination and containment far harder.
•Evaluation and governance gaps: Dangerous capability evals (CBRN, cyber, self-replication, scheming) are imperfect. Closed developers can gate releases on internal tests + external audits; open releases shift the burden to post-hoc community response. Regulation of open models is also harder (export controls, liability, and compute monitoring apply imperfectly).
•Real-world constraints on "closed": Leaks, insider exfiltration, side-channel extraction, and fine-tuning of smaller models already erode moats. Closed systems can still be insecure or deceptively aligned. Proprietary incentives sometimes prioritize deployment speed or PR over rigorous safety.
Key Variables That Decide the Balance
•Capability threshold: Open-sourcing GPT-2- or early Llama-class models is low-stakes and net-positive for science. Hypothetical systems with reliable autonomous R&D, novel bioweapon ideation, or strategic deception are different—proliferation risk dominates. Most experts distinguishing "current LLMs" from "AGI-level" treat them separately.
•What is released: Fully open (training code + data + weights + process) enables the best science and auditing. Weights-only is common and still proliferates capability. Partial opens (e.g., with safety layers or delayed release) are compromises.
•Defensive vs. offensive asymmetry: If open models disproportionately help defenders (detection, verification, hardening), openness wins. If offense scales easier, caution wins. This is domain-specific (cyber often favors offense short-term; some biosecurity tooling may not).
•Ecosystem maturity: Better evals, standardized dangerous-capability benchmarks, unlearning methods, watermarking, and hardware-enforced controls tilt toward safer openness. Their absence tilts against.
•Actor landscape: Nation-states and well-resourced groups can already train capable models; openness mainly affects the long tail and speed. Concentrated closed development can create race dynamics among labs.
Expert and Institutional Landscape
Views are split and often track risk models. Advocates of broad openness (parts of Meta, academic groups, Hugging Face-style ecosystems) emphasize democratization, innovation speed, and distrust of gatekeepers. Labs and researchers prioritizing extreme risks (Anthropic, OpenAI historically on frontiers, many in the alignment community, some government assessments) argue for controlled release of the most capable systems until safety catches up—sometimes paired with open research on narrower models. Governments increasingly discuss compute thresholds, reporting, and export rules that implicitly treat highly capable open weights as higher diffusion risk.
Evidence remains limited: we lack clear causal studies showing open models uniquely driving large-scale harm relative to closed ones plus open-source software/tools already available. Absence of catastrophe to date is weak reassurance as capabilities grow.
Practical Synthesis
•For today's models: Openness is frequently net safer or neutral due to scrutiny and research benefits; the biggest near-term issues (bias, reliability, petty misuse) are not solved by secrecy.
•For future frontier systems with credible catastrophic potential: Default to closed or highly staged/controlled release, heavy external auditing, and differential investment in safety/defensive tech—while still publishing research and enabling verified access. Pure secrecy is brittle; pure immediate open-weights on the absolute frontier is reckless if evals show high-risk capabilities.
•Hybrid paths exist: open research and smaller models, delayed weight releases, licensing with use restrictions (imperfect), structured access, and focusing open efforts on safety tools themselves.
•Regardless of open vs. closed: Prioritize hard safety work (mechanistic interpretability, robust alignment, scalable oversight, evals that detect sandbagging/scheming), secure development practices, and international norms on irresponsible scaling. Neither paradigm substitutes for that.
The question is empirical and will be clarified by better measurements of capabilities, misuse incidence, and defense effectiveness—not slogans. Treat each release by its demonstrated properties rather than ideology.