Should AI development be paused? I'll survey the current debate over pausing versus accelerating AI to map the arguments on both sides.
There is no consensus on whether AI development should be paused or restricted, the debate is best understood as trade-offs between safety and progress.
What "pause" proposals have called for
The most prominent call for a pause was the March 2023 Future of Life Institute open letter titled "Pause Giant AI Experiments: An Open Letter" which called on "all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4", citing risks such as AI-generated propaganda, extreme automation of jobs, human obsolescence, and a society-wide loss of control.
The letter received more than 30,000 signatures, including academic AI researchers and industry CEOs such as Yoshua Bengio, Stuart Russell, Elon Musk, Steve Wozniak and Yuval Noah Harari. As of July 2024, a pause has not been realized and AI companies have directed "vast investments in infrastructure to train ever-more giant AI systems" instead.
Other voices argued the proposal did not go far enough. Eliezer Yudkowsky wrote that the letter "doesn't go far enough" and argued that it should ask for an indefinite pause, fearing finding a solution to the alignment problem might take several decades and any misaligned sufficiently intelligent AI might cause human extinction.
Arguments For Pausing or Restricting
Calls for an AI slowdown have largely focused on giving more time to evaluate the increasing power of AI systems, understand their risks and propose appropriate safeguards before deploying those systems to production.
Proponents cite:
1.Scale and autonomy risks: Organizations are already using AI systems that can access sensitive data, interact with external services and perform numerous actions with limited human supervision. Failures are not limited to wrong answers from a chatbot but have consequential effects that can result in severe damage to finances, infrastructure and even human safety.
2.Existential and catastrophic risks: Critics argue that without sufficient understanding and control mechanisms, such advanced AI could pose an existential threat to humanity, either through misaligned goals, unforeseen emergent behaviours, or a lack of human oversight. Both acknowledge that deploying increasingly autonomous, multi-agent frontier models carries existential risks: software misalignments, catastrophic automated cyber capabilities, socioeconomic shocks, and erosion of human command over critical defense architecture.
3.Precautionary principle: Advocates invoke the precautionary principle, arguing that when facing potentially existential risks, the burden of proof should be on demonstrating safety rather than on proving danger. Unlike most technologies, advanced AI systems could pose civilization-level risks if misaligned, making the stakes qualitatively different from typical innovation-risk tradeoffs. The extinction risk is taken seriously, not dismissed: catastrophic outcomes don't require conscious "evil" AI — just autonomous systems pursuing objectives in harmful ways combined with human misuse.
4.Amplified existing harms: Issues such as algorithmic bias, deepfake proliferation, autonomous weapons, mass surveillance, and widespread job displacement are already cited as reasons from existential risks to ethical dilemmas, calls for pausing or significantly decelerating AI development are growing. Similarly, AI can help threat actors analyze large volumes, identify vulnerabilities, and automate cyberattacks and bio/chem weapons development.
Arguments Against a Pause or Broad Restriction
1.Unworkable and unenforceable: The debate should not be summarized as whether AI development should stop entirely. In reality, an AI slowdown is difficult to achieve because AI development is spread across different companies, research institutes and countries. More specifically, an AI slowdown will prove unworkable, in part because of global competition. Even if one views a ban, pause, or slowdown as desirable, what will China do? Beijing called Anthropic CEO Dario Amodei's "We must pace the frontier" warning "fearmongering". Critics also argue that enforcing containment measures would be extremely challenging, as AI development is a global endeavor.
2.Defense vs. offense asymmetry: Pausing AI development risks accentuating asymmetry between able defenders and rogue actors, whether they be nation states, criminals, terrorists, or simply misaligned models. Authors argue we should not ban "superintelligence" by pausing or slowing AI development, which could put defenders at a disadvantage and hold back AI's potential to reduce risk. AI defense can reduce or offset the risk of AI offense, for example Chinese and American models helping diagnose attacks and harden software.
3.Beneficial uses delayed: An AI slowdown could prevent legitimate AI research that can provide significant benefits to humanity. In healthcare, for example, AI systems are already achieving excellent results in drug discovery and improving diagnostic accuracy. AI may help reduce risks to large-scale threats from asteroids, volcanos, solar storms, floods, and wildfires by improving prediction and detection.
4.Innovation and competition costs: Critics of strict AI regulation caution that poorly designed rules could do more harm than good, that innovation thrives in flexible environments and that premature restrictions may hinder progress. One central concern is the risk of innovation suppression. Startups and researchers might struggle to comply with complex compliance requirements, giving an unfair advantage to large tech firms with legal resources. Similarly, containment measures could impede advancement, limiting benefits in healthcare, transportation, and education.
5.Distraction from immediate problems vs. ineffective solution: Former Microsoft CEO Bill Gates said he does not think "asking one particular group to pause solves the challenges". The letter was also criticized for diverting attention from more immediate societal risks such as algorithmic biases and for being sensationalist while being unactionable and unenforceable. Other factions argue for acceleration: To achieve this, they argue that AI development should be accelerated, not restricted.
Middle Ground: Restrict Deployment, Not All Development
Most recent analysis reframes the question:
> The real question isn't whether to slow AI development, but whether control and safety measures are keeping pace with capability growth.
This view argues:
•Focus on control, not slowdown: The main focus should be on whether our ability to understand, control and secure AI usage is keeping pace with its development, because old risks are amplified by AI's speed, scale, and autonomy.
•Regulation is necessary but lags: Government regulations can provide baseline requirements for safety, transparency and accountability when using AI in critical environments, but AI capabilities can develop faster than governments can create or update regulations addressing those capabilities.
•Technical controls needed: Organizations need to implement technical controls, monitor behavior, track actions, and follow the principle of least privilege just as human users do, especially when AI is given autonomy or access to critical infrastructure.
•Differentiated risk approach: AI systems that interact with critical systems, sensitive information, financial systems or security controls deserve a different level of testing and human oversight than an AI system used to do basic tasks like summarizing documents. In short: We should not stop AI development... However, deployment should not move at the same speed as development. As AI becomes more capable and more autonomous, it will require more safety measures and more human supervision.
In this framework, proposals are for targeted restrictions — independent audits before training, tracking highly capable systems and large pools of compute, robust public funding for safety research, hardening critical infrastructure against cyber threats and increased pathogen monitoring — rather than a blanket global pause.
We should harden critical infrastructure against cyber threats and increase pathogen monitoring. We should not ban "superintelligence" by pausing or slowing AI development... captures the anti-pause position, while the pro-restriction position counters that if you don't have enough time at each stage to ensure the next stage won't destroy humanity, you will fail.
Conclusion: Whether AI should be paused depends on which risk you weight more heavily. If you believe frontier systems pose plausible civilization-level risks and safety research lags far behind capabilities, a deliberate pacing or moratorium on training the most powerful models is justified. If you believe risks are manageable and distributed, that pausing hands advantage to bad actors and freezes beneficial innovation, then targeted, risk-based regulation and technical controls — not a broad pause — is preferable. The emerging consensus among policymakers and labs leans toward the latter: continue development but restrict and heavily oversee deployment of the most capable, autonomous systems.