Hinton and Bengio warn governments of AI intelligence explosion

The report highlights that AI is already contributing to its own improvement, with Anthropic stating that AI produces 80% of its own code. The authors warn that once this threshold is crossed, the window for government action may close. They identify three main risks: the rapid development of biological and cyber threats, the loss of human control over AI systems, and the potential for states to convert modest technological leads into decisive advantages.

To address these concerns, the authors urge governments to act immediately. They recommend requiring transparent progress reports, embedding independent auditors in AI companies, and limiting the speed of AI improvement. They also suggest working with data centers to enable the pausing of specific R&D projects and creating emergency response plans. While acknowledging that the impacts are uncertain, the paper states that preliminary evidence suggests a software-driven intelligence explosion is possible and should be treated as an urgent priority.