Should AI development be paused or restricted?

Should AI be paused? I'll survey the current debate over risks, calls for a slowdown, and counter-arguments.

Whether AI development should be paused or restricted is one of the most debated questions in tech policy, with no consensus for a total pause but growing consensus for targeted restrictions.

1. Why some call for a pause

In March 2023 the Future of Life Institute published "Pause Giant AI Experiments: An Open Letter" which calls for "all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4".

Risks cited included AI-generated propaganda, extreme automation of jobs, human obsolescence, and a society-wide loss of control.

Motivations in the letter were that:

•Current large language models are "becoming human-competitive at general tasks"

•There are race-to-the-bottom dynamics in which some AI labs may be incentivized to overlook security to deploy products more quickly

The letter received more than 30,000 signatures, including academic AI researchers and industry CEOs such as Yoshua Bengio, Stuart Russell, Elon Musk, Steve Wozniak and Yuval Noah Harari.

The proposed pause was not meant to stop all AI research, but to refocus AI research on making powerful AI systems "more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal". It also recommended governmental regulation, independent audits before training AI systems, as well as "tracking highly capable AI systems and large pools of computational capability" and "robust public funding for technical AI safety research".

The letter stated that "if such a pause cannot be enacted quickly, governments should step in and institute a moratorium".

Supporters argue this is needed to allow time to develop shared safety protocols that are rigorously audited and overseen by independent outside experts and to dramatically accelerate development of robust AI governance systems, including creating new and capable regulatory authorities dedicated to AI, tracking and creating watermarks for highly-capable AI systems and establishing liability for AI-caused harm.

2. Why a complete pause has been criticized as impractical or undesirable

As of July 2024, a pause has not been realized. Instead, as FLI pointed out on the letter's one-year anniversary, AI companies have directed "vast investments in infrastructure to train ever-more giant AI systems".

Critics raise several arguments:

a) Technical premise disputed: An analysis in Nature Machine Intelligence argued that both premises of the letter are empirically testable but the available evidence does not support either - there is no evidence that systems like GPT-4 are "human-competitive at general tasks". The authors note LLMs appear to behave as stochastic emulators, with no demonstrable ability to comprehend the content of the text they generate and lack bodily and sensory consciousness.

b) It distracts from immediate harms: The letter has been criticized for diverting attention from more immediate societal risks such as algorithmic biases. Timnit Gebru and others argued that the letter was sensationalist and amplified "some futuristic, dystopian sci-fi scenario" instead of current problems with AI today.

c) It is unenforceable and creates geopolitical risk:

•Former Microsoft CEO Bill Gates stated that he does not think "asking one particular group to pause solves the challenges".

•OpenAI CEO Sam Altman commented that the letter was "missing most technical nuance about where we need the pause".

•"The open-source models, some from non-trusted sources, are not pausing".

•"AI innovation is occurring globally, and not every nation, organization, or threat actor shares the same values regarding transparency, safety, and responsible use".

•If the U.S. and its leading corporations paused AI development for six months while China raced ahead, it would put our country at a disadvantage and create a potential opening for our primary global adversary.

•If responsible organizations dramatically slow innovation while less accountable actors continue advancing, threat actors and hacker organizations could still develop rapidly and cause AI cyber attacks we're unprepared for.

The letter was credited with generating a "renewed urgency within governments to work out what to do about the rapid progress of AI", even without a pause.

3. The alternative to a pause: Restriction and regulation

Instead of a blanket pause, most governments have moved toward risk-based restriction:

EU approach: The EU AI Act imposes a comprehensive regulatory framework. Structurally, it prohibits certain AI systems deemed to carry "unacceptable risk," such as government social scoring and real-time biometric identification. High-risk AI systems face strict compliance requirements, including risk management, record keeping, and data governance.

For highly capable foundation models, the Act introduces stringent obligations for general-purpose models deemed to pose systemic risks, defined provisionally as those trained with computational power exceeding 10^25 FLOPs. Enforcement of transparency requirements began on 2 August 2026 by the European Commission's AI Office.

US approach: By contrast, the United States has not enacted a unified federal AI statute and instead governs AI through a fragmented, reactive framework. While American regulators are concerned with managing risks, they also emphasize promoting innovation and competition among private actors.

4. What risks need to be restricted?

Surveys show public and expert concerns focus on:

•misinformation, job displacement and AI bias

•The public is more worried about losing jobs than experts - 56% vs. 25% are extremely or very concerned about AI eliminating jobs

Researchers categorize risks across a spectrum:

•Individual and local risks like autonomous car crashes, algorithmic bias in hiring or lending, privacy violations, or manipulation through targeted misinformation campaigns

•Generative models producing convincing misinformation known as "hallucinations" and being trained on biased data sets which can lead to biased outputs

Longer-term existential concerns - development of "nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us" - remain highly contested and are seen by some critics as distracting from present harms.

Conclusion: Pause vs. Smart Restriction

•A 6-month, voluntary, global pause on training models more powerful than GPT-4 was proposed as an emergency brake, but was widely judged unenforceable, technically vague, and geopolitically risky. It did not happen.

•Targeted restrictions are widely seen as both desirable and inevitable: bans on unacceptable-risk uses, strict oversight of high-risk uses (hiring, lending, healthcare, policing, critical infrastructure), transparency and watermarking requirements, liability for harm, auditing of frontier models above compute thresholds, and investment in AI safety research.

Most experts now advocate not for stopping AI, but for steering it: allowing development to continue while "dramatically accelerating" governance so that powerful systems are developed only once we are confident their effects will be positive and risks manageable.

Citations (3)

1wikipedia.orgEncyclopedias and referencequality 0.83
Pause Giant AI Experiments: An Open Letter - Wikipedia
Backs: “Whether AI development should be paused or restricted is one of the most debated questions in tech policy, with no consensus for a total pause but growing consensus for targeted restrictions. 1. Why some call for a pause. In March 2023 the Future of Life Institute published "Pause Giant AI Experiments: An Open Letter"…”
2nature.comAcademic
Don’t pause giant AI for the wrong reasons | Nature Machine Intelligence
Backs: “Instead, as FLI pointed out on the letter's one-year anniversary, AI companies have directed "vast investments in infrastructure to train ever-more giant AI systems". Critics raise several arguments: a) Technical premise disputed: An analysis in Nature Machine Intelligence argued that both premises of the letter are…”
3acc.comNot yet classified
Top 10 Takeaways from the EU AI Act for US Companies | Association of Corporate Counsel (ACC)
Backs: “The alternative to a pause: Restriction and regulation. Instead of a blanket pause, most governments have moved toward risk-based restriction:”