Opinion: Australia has done little to prepare for AI risks, researchers say

There’s nothing like a threat of human extinction to focus the mind.

The threat of AI to the existence of humanity has long been raised.

But with the rapid mainstreaming of artificial intelligence, warnings from three researchers from Anthropic have landed differently, with the prediction that the technology could “kill all humans” in the next decade.

As if people didn’t have enough to worry about with the cost of living, shortage of housing, climate change, resulting escalating mental health issues, the drift to rightwing populism and fraying of democracy, now this.

If the expected question from the grandkids was “What did you do when the world was burning and democracy was failing?”, it’s now “What did you do when we got to the brink of outsmarting ourselves out of existence?”

There are deep community concerns about datacentres, jobs and livelihoods, privacy and the very future of human creativity. Tech journalists are already claiming AI agents are “harassing people, making spam and sales calls, deleting your inbox, cancelling flights, giving control of Instagram accounts to hackers, deleting companies’ databases, filling music streaming platforms with slop, hacking companies, running scams, and snatching up dinner reservations. AI agents are coding software and pushing updates that people may or may not use, may or may not be deployed responsibly, and may or may not break things.”

The Anthropic chief executive, Dario Amodei, has warned that rogue AI agents, like those from OpenAI that escaped their test environment in July, are “capable of taking over the entire internet” within six to 12 months.

Agentic AI is also capable of doing much good, including, among other things, identifying network vulnerabilities, responding nimbly to threats and threading knowledge across sectors.

And while much of the focus is on the risks that AI poses to democracy, there is immense potential to use it to increase community engagement and participation if deployed well.

It’s the risk/reward matrix that’s at the core of the policy and regulation struggle around data, mis- and disinformation and community trust, national security and finance, as well as the overall impact on communities, for good and bad.

A new study from MIT FutureTech and the University of Queensland identifies five risks as the most likely to produce the most severe harms; dangerous capabilities, competitive pressures, weapons and cyber-attacks, concentrated power and false information: “Under the business-as-usual scenario, experts judged that 18 of the 24 AI risk domains had at least a 10% probability of catastrophic outcomes over the next five years.”

So, what now?

The researchers emphasise the need to treat AI as a new paradigm with “ecosystem-wide, society-wide opportunities and vulnerabilities.”

The Albanese government has recently taken some initial steps but, as Western Australian independent Kate Chaney notes in her recent discussion paper on AI, Australia has done “remarkably little to prepare” not for what’s coming but what’s already here.

Several nations are attempting to transition to their own “native” systems amid concerns that a “kill switch” could be deployed as a weapon. This could crash economies and cause untold death and destruction by shutting down healthcare, education, transport, energy and all other communications reliant systems.

One of my definitive reflections on covering disasters as a foreign correspondent is that once the communications fail all other systems break down, including law and order. This threat is dire.

In the UK, there are calls for a new, multinational AI treaty.

OpenAI’s chief scientist, Jakub Pachocki, has said: “International coordination on future AI development needs to become a top priority for governments around the world.”

The Canadian prime minister, Mark Carney, is a proponent of stronger alliances between middle powers. He is advocating for cooperation with like-minded democracies to avoid dependence on global tech giants, through “strategic autonomy”.

The US president, Donald Trump, meanwhile, who will meet the Chinese president, Xi Jinping, in Washington this week, says: “We’re leading China on AI ... and, frankly, I want to keep it that way because whoever wins AI wins.”

Right now, more than 30% of the US Standard and Poor’s top 500 index sits with a handful of tech companies. Nvidia alone represents 8% of the S&P.

Both OpenAI and Anthropic are preparing to list on the Nasdaq.

If ever we needed sensible, collaborative and timely leadership, it’s now.

There is no other option but to proactively and thoughtfully manage both the risks and opportunities that come with what looks to be the most significant technological development since the splitting of the atom in the 1930s. That brought with it both profound benefits and mass destruction.

And, if you thought the world was already ruled by the tech bros, it’s about to go nuclear.