TORONTO — Researchers at the University of Toronto and the cybersecurity firm CleverHans developed a proof-of-concept AI-powered computer worm. The study outlining their experiments was uploaded to the arXiv preprint server on June 2, 2024; the research has not been peer-reviewed.

The proof-of-concept malware combines a locally running large language model with an autonomous software agent. This software agent can scan networks, evaluate potential attack paths, and make decisions on how to compromise new targets without requiring human intervention. Michael Agee, an adjunct professor of information technology at Trinity Washington University who was not involved in the research, said: "The main finding is that this type of system can do more than run a fixed exploit; it can examine the target environment, reason about possible vulnerabilities, use tools to attempt attacks, and then replicate itself after a successful compromise." Agee added: "The AI-driven part of the attack is mainly the reasoning and decision-making." He said: "The LLM is not magically hacking the system; it is being used to reason about what the information means, suggest possible attack strategies, decide which tool or action should be tried next, and help adjust the approach when something fails."

The worm was tested on a simulated corporate network that included 33 hosts, such as Linux servers, Windows workstation computers, and internet-connected devices. Over approximately one week, the system identified vulnerabilities, compromised new machines, and replicated itself across about 62% of the network.

Bob Hutchins, an instructor of AI strategy courses at Lipscomb University, distinguished the AI worm from traditional worms. He said: "Traditional worms follow a scripted sequence: Once a vulnerability is identified, the worm replicates." Hutchins stated: "In contrast, the researchers demonstrated that an easily downloaded AI model could be used as the decision-making component of the worm." He added: "The worm would analyze each device it encountered to determine its most effective strategy to breach that particular system." Hutchins also noted: "Intelligence does not exist in discovering new vulnerabilities; rather, intelligence exists in determining how quickly an attacker can choose and sequence attacks against previously identified vulnerabilities." He said: "The researchers employed lightweight open-weight models during their demonstration, which are relatively easy to download, remove guardrail components from, and utilize."

The researchers designed the worm to function across devices with varying computing power. They used an open-weight large language model running on local hardware connected to a software framework, rather than a frontier model from a major AI company. Tom Vazdar, a professor of AI and cybersecurity at the Open Institute of Technology, said: "What made it particularly dangerous was a clever tiered design." Vazdar explained: "GPU-equipped compromised machines provided reasoning capacity for lightweight agents running on low-power IoT devices that couldn't run an AI model locally." He added: "A camera becomes a thinking node in the attack network, not just another door."