Opinion: AI may not trigger an apocalypse, but regulation is still needed

In realistic terms how would artificial intelligence actually kill me? What is the main proxy threat? Nuclear war, death due to breakdown of society, a virus? I hear all these reports it could kill us but still unsure how that could practically happen? To be honest all sounds too wooly to really take seriously. Pottys99

Aisha: For what it’s worth, it seems unlikely that AI at this point can significantly change the equation around bioweapons or viruses … What has prevented bioweapon attacks through the past decades is not the difficulty of inventing new deadly viruses, but other, more prosaic issues. Consider the failure of Aum Shinrikyo to kill vast numbers of people with botulism and anthrax by spraying these around central Tokyo. There are a lot of logistical challenges, and chemical weapons seem more straightforward. Those exist without AI needing to invent new ones.

Humans might make the poor decision to put AI in charge of key nuclear systems, perhaps, but that seems more of a human error than a rogue AI error. Humans could also make the poor decision to launch tactical warheads without AI.

How might AI kill us otherwise? There are of course plenty of plausible scenarios. If you’re a civilian in Gaza or Iran, it could direct a US or Israeli missile your way. But perhaps our focus might be more productively directed towards the people and companies in charge of the systems, as opposed to Dr Strangelove scenarios.

Throughout history new technological advances have met with cries of doom. Are the cries against AI any different? Virtualewit

Blake: The end times-quality of doomsday predictions may be similar across history, but I think the reason we listen to the renewed cries of doom is the growing power of each technology. AI can perform impressive tasks we didn’t think were possible, which has spurred our collective imagination to hopeful outcomes and ones of doom.

Can AI now be truly controlled or are we past a tipping point where no matter what action is taken by governments and companies, someone will be able to continue to advance the technology, until who knows what? Tigon20

Aisha: Let’s back up a bit here. It’s always possible to advance any technology, no matter what … But what it sounds like you’re wondering about is whether regulation can stop the emergence of some kind of superintelligence that destroys humanity.

Our reporting does not support the idea that superintelligence is going to emerge and pave over the Earth with datacentres. In fact, we have found that the industry itself seems to be playing loosely with this kind of terminology: consider, for example, how one OpenAI executive, Greg Brockman, claimed that the company had achieved AGI (artificial general intelligence) just days after Sam Altman, OpenAI’s CEO, said that term was “an irrelevant marketing term”. Right now, a number of industry warnings about out-of-control AI appear to be predicated on a limited number of incidents with cherrypicked details.

The evidence that we have now does not indicate that AI is uncontrolled, or that it has agency; a lot of experts suggest that this framing is misleading. In the Hugging Face incident, for example, the so-called “agent swarm” was doing what it was being told to do – pass a test. Our sources caution against anthropomorphising AI or attributing intention to it. Doing so plays into industry narratives that appear to be pushing for favourable regulation.

If AI wipes out the human race, would it be sustainable? Unless there is an equal advance in robotics to enable it to physically extend and maintain its power source and infrastructure, would it finally physically break down? Merrth

Blake: AI might someday be capable, well-read, and connected enough to take over power plants, chip factories, and a sufficient number of robots to build its own datacentres and sustain itself. I don’t know. I would look to science fiction more than news articles for the answer. Though The Matrix is more famous for one man’s choice between a comfortable simulation and painful reality, it’s also a story about robots needing new batteries. Their desperation for a power source turns them against humans. For a multiplicity of stories, I, Robot is perhaps the greatest book written about sentient robots and deserves a reread in this moment.

People (ie the tech bros) bang on about the “benefits”: just what are they? TooTrueMate

Dan: Anyone who has used a chatbot to parse a lengthy document, argue against a parking fine or spitball ideas for a speech can point to the benefits of the technology – although this is very basic-level stuff that people have been doing for a while now. There is also the everyday use of the technology from facial recognition to open your iPhone to asking Alexa to play an Oasis track or driving around in a Tesla. With those latter examples, you’d notice if AI wasn’t involved.

In science there are many examples of benefits, from Google’s AlphaFold model – which can predict how proteins behave and thus help us study the biological machinery that underpins living organisms – to making weather forecasting more accurate, which is sadly necessary in this climate crisis era.

But there is an element of banging on about the benefits for the wider global economy, which have yet to reveal themselves on the scale predicted by the AI industry. For instance, these benefits require AI agents – tools that carry out tasks on your behalf while you’re at the water cooler or have clocked off – to be a lot more active in the workforce. We are not at that stage yet even if they are making inroads into areas like software programming.

Regulation, oversight and accountability for huge companies that appear unable or unwilling to eliminate the harms their products inflict – and that seem incapable of conducting safe experiments to ensure that AI does not leak into the wider internet – seem to be urgent and vital. What are the prospects for a global regulatory framework that is enforceable, particularly when the genie may already be out of the bottle, and when the US and China have very different objectives and applications for their AI technologies? Trentwarrior

Blake: It’s a tough question. One way to hinder the general development of AI could be restricting access to computing power, which is a vital resource that underpins the success of an AI model. In the absence of legislation that applies to all types of AI development, measures that target certain aspects of AI like housing allotment or children’s access to chatbots seem like the path lawmakers will take.

There are two big question marks hovering over a possible global regulatory framework. The first is whether Donald Trump will revise his “Drill, baby, drill” stance towards AI development. He has rejected any calls for a voluntary slowdown. In my opinion, he is not likely to change that position: he loves to post AI slop, and he loves what AI has done for the US stock market.

The second question mark is Beijing, which has rejected US CEOs’ calls for a slowdown. Without China’s cooperation, an intentional slowdown would likely not succeed. Xi Jinping also seems unlikely to back down. A slowdown of AI development led by the US would offer China few obvious benefits.

Trump and Xi are also set to meet next week in Washington. AI will be high on the agenda.

The UN general assembly is taking place now, with discussions of AI large and small on the schedule. It is possible that an initial version of a global regulatory framework emerges from there.

Is this recent narrative push of the AI apocalypse an inventive way of preparing the public for the market crash and wipeout of AI shares? The companies seem overvalued and their borrowing and input costs have gone up. Seaspray

Aisha: The finances of the AI buildout are certainly shaky. Hundreds of billions of dollars are being ploughed into massive, unpopular infrastructure projects that our reporting has shown are not necessarily getting built. Nvidia appears to be the only company making a profit, and investigations I have done with Ed Zitron raise some questions about whether all its very expensive graphics processing units (GPUs) are actually, currently, getting plugged into datacentres. (Perhaps they are mouldering in warehouses instead.) There are some unsettling financial instruments being used to raise money to build yet more datacentres, such as GPU-backed debt.

Is that why tech bros want a slowdown? Perhaps. It looks like OpenAI just pushed back its IPO until 2027. This might be because, as we reported, things did not look amazing for them earlier this year in terms of floating on the market – they seemed to be struggling to monetise ads and erotic chatbots. Their valuation will depend a lot on vibes, and maybe the doomy vibes will help a potential share offering.

Given our senses, emotions, memory and moods, is it ever going to be accepted that humanly conscious AI is fantasy? IcommentthereforeIam

Dan: This is a serious question that has even rattled Richard Dawkins, the evolutionary biologist. He may not believe that God is real but he does believe that AI is conscious – or at least the chatbot he was using, which he called Claudia. “You may not know you are conscious, but you bloody well are,” he wrote.

He’s not the first. In 2022 Google fired an engineer who went public with his belief that a company-developed AI had feelings. At the more troubling end of this belief, there are countless examples of people becoming overly attached to, and influenced by, AI chatbots – including cases of psychosis.

These sorts of attachments won’t go away, given a) chatbots are on a trajectory to become even more powerful and convincing and b) the human need for emotional attachment and, well, propensity to go all-in for the whole higher being thing.

But Dawkins aside, there are plenty of scientists who will argue that this is fantasy, and that there are obvious differences between a biological brain – the root of consciousness – and ChatGPT. Responding to Dawkins in the Guardian last year, Jacy Reese Anthis, a researcher in human AI interaction, said there was a “staggering gulf between how biological brains evolved and how AI systems are built”. OpenAI might have cracked the Navier-Stokes problem, but not subjective experience.

What, if anything, can individuals like me do about AI? Alberichthenibelung

Blake: There is, unfortunately, not much that comes to mind. Everyday people confronted with declarations of a coming AI apocalypse find themselves in a position similar to the cold war, with doom hovering over their heads and little say in it. We may all have to accept the faint tinnitus of paranoia buzzing in the background. You can stop using AI if you feel so moved, though I have found that unfeasible, and your job may require it. You can vote for politicians that endorse AI regulation. I myself don’t put much stock in the reality of an AI-made doomsday because a genuine belief in it would demand drastic, life-changing actions today, ones that I don’t want to take. A bit of inductive reasoning from how I’d like to situate my own life within the world.