Both, depending on the capability level, release details, and risk type—there is no universal answer. Open-source AI (typically meaning publicly released weights, enabling local runs, fine-tuning, and modification) and closed/proprietary AI (API-only or fully restricted) involve genuine trade-offs across misuse, accidents, concentration of power, and verification. Evidence from current models is mixed and limited; arguments intensify sharply for future frontier systems.
Core Trade-offs
Ways open-source can be riskier:
•Proliferation and irreversible access: Once weights are public, anyone—including criminals, terrorists, authoritarian regimes, or reckless actors—can run, fine-tune, or combine them without ongoing oversight, rate limits, or kill-switches. Removal is effectively impossible. This raises misuse risks for scams, non-consensual deepfakes, automated cyber attacks, tailored disinformation, or (at higher capabilities) assistance with biological or chemical threats. Closed APIs allow monitoring, throttling, and patching.
•Bypassing safeguards: Community fine-tunes routinely strip refusals and alignment (uncensored/"abliterated" models). Jailbreaks and base models spread quickly. Closed models are also jailbreakable, but the barrier and observability differ.
•Racing and capability diffusion: Open releases can accelerate iteration by many groups, including those with weaker safety practices. If models approach recursive improvement or high-leverage dual-use capabilities, broad access compresses the time for defensive preparation.
•Empirical patterns: Leaked or open weights (e.g., Llama-series derivatives) enabled rapid ecosystem growth and easier creation of harmful variants. Evaluations repeatedly show open models can be adapted for prohibited tasks more readily once unrestricted.
Ways open-source can be safer (or closed riskier):
•Transparency and scrutiny: Weights, architecture details, and (ideally) data/code allow outsiders to audit for backdoors, vulnerabilities, hidden behaviors, biases, or overstated safety claims. Closed models require trusting the lab's evaluations, red-teaming, and incentives—labs have shipped sycophantic, deceptive, or reward-hacked systems before. "Many eyes" has historically helped software security (with caveats).
•Distributed safety progress: Broader researchers, including academics and smaller labs, can probe failure modes, develop interpretability tools, alignment techniques, and defenses. Concentrating capability in a few labs risks groupthink, shared blind spots, or suppressed negative results.
•Power concentration and structural risks: Closed frontier AI in a handful of companies or governments creates single points of failure, insider threats, corporate/government misuse, or lock-in. Democratized access can enable countermeasures, competition, and resilience (analogous to encryption or open-source software). Authoritarian control of powerful closed AI is a serious downside scenario.
•Closed models are not hermetic: Weights leak, APIs enable distillation/extraction attacks, and employees leave. Safety theater (marketing "safe" while racing) is easier without external verification. Incentives for proprietary labs include speed and profit, which can deprioritize robust safety.
Critical Dependencies
•Capability threshold matters most: For narrow or current-generation models, open-source has net benefits via innovation, science, and transparency with manageable misuse (similar to open-source software, cryptography, or dual-use biotech tools). At hypothetical AGI/ASI levels—or systems that materially lower barriers to catastrophic misuse—the irreversibility and offense-defense imbalance make unrestricted weight releases far more dangerous. Many researchers (across labs and academia) support staged, conditional, or compute-limited releases rather than pure open or pure closed.
•What "open" actually means: Full weights + training code + data vs. weights only vs. open inference. Safety also depends on accompanying evals, usage norms, hardware assumptions, and whether dangerous fine-tunes are easy.
•Misuse vs. accident vs. structural: Open increases certain misuse surfaces; closed can hide accident risks (misalignment, deceptive alignment) and worsen winner-take-all dynamics or regulatory capture. Neither automatically solves inner alignment, goal misspecification, or multi-agent risks.
•Evidence is incomplete: We lack decisive longitudinal data on net harm. Current open models power useful tools and abuse; closed models (GPT-N, Claude, Gemini class) dominate high-capability misuse reports partly because they are more capable and accessible via chat. Expert surveys show disagreement; labs ship both open (Meta, some others) and closed systems with different rationales. Analogies to open-source software, nuclear tech, or biotech are imperfect.
Practical Reality and Better Framing
Neither extreme ("open everything for democracy" or "centralize and gate forever") holds up. Risk is continuous with capability, ease of adaptation, and external defenses. Responsible paths emphasized by various researchers include:
•Rigorous dangerous-capability evaluations before release.
•Differentiated release (open smaller/less capable models; staged or restricted for frontier).
•Investment in verification, interpretability, and misuse detection that works for both.
•Hardware/export and compute governance where risks are acute, without pretending software can be fully contained.
•Avoiding hype that either "open = safe because transparent" or "closed = safe because we control it."
Closed models can still be extremely dangerous if the lab errs; open models can be net positive if capabilities stay below critical thresholds and defenses scale. The dominant risks for advanced AI likely stem more from the underlying difficulty of alignment/control and competitive pressures than from the open/closed binary alone. For the highest-stakes systems, erring toward controlled access plus heavy external auditing is the more precautionary engineering stance, while preserving openness for research and lower-stakes systems to avoid stagnation and hidden failures.
Further reading of primary arguments from diverse sources (technical reports from labs, academic papers on dual-use, expert surveys, and critiques of both "doomer" and "accelerationist" extremes) is essential—claims evolve with capabilities.