The world will face consequences that "look like a bad Hollywood movie" if it does not act to safeguard control of nuclear weapons, an Australian member of a key global body warns.
In a report to the UN, the Global Commission on the Responsible Use of AI in the Military called for legally binding agreements to ensure decisions to use nuclear weapons remain under human control, not AI systems.
Foreign Minister Penny Wong backed the report in a speech to the United Nations Security Council on Friday (AEST), saying AI had extraordinary potential, but also presented significant dangers if not kept in check.
"AI's potential use in nuclear weapons and unmanned systems challenges the future of humanity," she told the council.
UNSW Professor of AI Toby Walsh, who is a member of the commission, said he was pleased Senator Wong had "called out our work .. and endorsed our recommendation to ... not hand over the decision to set off something as catastrophic as a nuclear weapon to an algorithm".
"We have to be realistic and accept the fact that there are significant operational advantages to using AI in a military context ... but AI is very much a double-edged sword," he said.
"One incredibly important risk is that it's going to change the speed, accuracy and deadliness of war, and not in a good way."
The world could potentially face a situation where "as soon as the fighting starts, all the humans will be killed because they won't have the speed of response, they won't have the accuracy the machines have", he said.
Prof Walsh was "pretty confident" the world does not want AI outcomes to resemble apocalyptic sci-fi films.
"What worries me is that we'll have to see that bad Hollywood movie before we take appropriate action," he said.
Senator Wong said nuclear warfare had so far been constrained by human judgment, by leaders who bear responsibility and by human conscience.
"AI has no such concern, nor can it be held accountable," she said.
"These weapons threaten to change war itself, and they risk escalation without warning."
Decisions of life and death "must never be delegated to machines, and together, we must set the rules and establish the norms", Senator Wong said.
Senator Wong also used her speech to call for tougher standards to ensure the use of AI was safe and ethical.
"While once we grappled to discern fact from propaganda, we are now witness to a collapse of truth altogether," she said.
The foreign minister urged the UN to ensure AI threats do not undermine the work of the global body and the security council.
Her speech coincides with Australia's stated ambition to secure a spot on the body charged with ensuring international peace from 2029.
Australia wants a two-year term as one of the 10 countries given a rotating spot, alongside permanent members from the UK, US, France, China and Russia.
Australia last held a seat on the Security Council in 2013-14.
