6 hours ago
13
Drones will need to be programmed with moral guidelines as AI-driven decision making reduces human involvement in autonomous warfare, according to a former UK spy chief.
David Omand told the Guardian that he had changed his mind on unmanned weapons systems, more than a decade after concluding that autonomous drones could not comply with international humanitarian law.
The former head of GCHQ – the UK’s listening agency – said he now believed AI could create a “moral” framework for unmanned weapons that could differentiate between combatants and civilians.
Omand said he been swayed by the “speeding-up” of modern warfare, as shown by the deployment of weapons such as drones and hypersonic missiles, and the emergence of generative AI which had given states a “potentially ethical way” of allowing drones to operate under morally compliant systems.
“My call is to really get some work done on this, so that we’re not left in a situation where there isn’t a moral component built into future AI-powered weapon systems,” he said.
His intervention came as the armed forces minister Al Carns said there might be circumstances where machines made their own targeting decisions, saying “you must have the ability to take the human out of the loop when required”.
Meanwhile, the US is leaning heavily into AI-powered warfare. Its 2027 budget, released in April, sets aside $54bn (£40bn) to fund “autonomous and remotely operated systems across air, land, and above and below the sea”, including a “drone dominance” programme. The former CIA director David Petraeus, however, has argued that the US has no military doctrine for autonomous formations.
Speaking to the Guardian before an appearance at the Cheltenham science festival on Wednesday, Omand said AI technology was now capable of weighing the factors that go into a human drone operator’s targeting decisions, such as whether a target was legitimate, whether there would be civilian casualties and whether the target has been correctly identified. This was not inventing new ethics, added Omand, but putting the current one used by the military into a form that can be deployed by a machine.
AI is already featuring heavily in recent US military missions, with technology developed by Palantir and Anthropic being deployed to help shorten the “kill chain” – the process from selecting targets to launching a strike – in the Iran conflict.
Omand, who was director of the UK spy agency GCHQ between 1996 and 1997, chaired a 2014 University of Birmingham commission on armed drones that expressed “doubts” over whether autonomous weapons systems could safely distinguish between civilians and combatants and whether they could ever “exercise the proportionality necessary for compliance with international humanitarian law”.
Omand, a visiting professor in the department of war studies at King’s College London, believes that a machine can now attack legitimate military targets including humans in a way that reflects “sound moral reasoning”, while still being under some form of human oversight.
Omand said that as warfare becomes ever more technologically sophisticated, humans will inevitably need to be “on the loop” rather than “in the loop” with AI systems. The term “in the loop” is commonly used in debates about reining in powerful AI systems and refers to a human being intimately involved in the decision-making process.
Omand said “on the loop” meant a human would supervise the system but not authorise every single combat action, with generative AI breakthroughs making it possible for machines to operate within a moral framework designed by the human. These moral frameworks must be adjustable from operation to operation, he added.
Moving to an “on the loop” set-up was inevitable in a world where warfare is conducted at speed, said Omand.
“It’s a physical and operational inevitability,” Omand said. “The term ‘on the loop’ means you still have human supervision and it’s humans setting the parameters of a mission. In that sense humans still have moral control. But individual decisions in the heat of combat, or where time is very short, you just won’t have time for a human to make them.”
The question now, said Omand, was how humans ensured that autonomous weapons systems could adhere to international law. UK military policy on the use of AI in defence states that “there must be context-appropriate human involvement in weapons which identify, select and attack targets”, adding that this could include “control exercised through the setting of a system’s operational parameters”.
Omand argues that a drone’s actions can be split into to six variables which can be assigned levels of importance by a human operator – the one “on the loop” – which are then used to guide the drone’s decision on a mission.
The ultimate result could be a moral decision-making system that is ethically superior to human decision making, said Omand, who has also held board positions at defence companies and is an adviser to investment firm Paladin Capital, which invests in cybersecurity companies.
“It could actually work, whereas relying on humans in a very fast-moving seconds matter for warfare is probably going to lead to far worse results in terms of collateral damage,” he said.
Morality would be programmed via what Omand calls an “adaptive moral control layer” where a human sets the parameters of the “moral” system before the mission, such as expected proportion of civilians around the target. Omand describes this as a “formalisation of moral authority”.
One anti-drone campaigner said Omand’s stance was “as nonsensical as it is dangerous”.
Chris Cole, the director of Drone Wars UK, said: “AI is simply not capable of making a judgment. It merely processes data, completely lacking the ability, for example, to distinguish civilians from combatants or to judge whether loss of life is proportionate to military advantage.”
