Zero-Click GenAI Worm Spreads Malware, Poisoning Models

  /     /     /  
Publicated : 23/11/2024   Category : security


Zero-Click GenAI Worm Spreads Malware, Poisoning Models


35 years after the Morris worm, were still dealing with a version of the same issue: data overlapping with control.



A worm that uses clever
prompt engineering and injection
is able to trick generative AI (GenAI) apps like ChatGPT into propagating malware and more.
In a laboratory setting, three Israeli researchers demonstrated how an attacker could design adversarial self-replicating prompts that convince a generative model into replicating input as output – if a malicious prompt comes in, the model will turn around and push it back out, allowing it to spread to further AI agents. The prompts can be used for stealing information, spreading spam, poisoning models, and more.
Theyve named it Morris II,
after the infamous 99-line self-propagating malware which took out a tenth of the entire Internet back in 1988.
To demonstrate how self-replicating AI malware could work, the researchers created an email system capable of receiving and sending emails using generative AI.
Next, as a red team, they wrote a prompt-laced email which takes advantage of retrieval-augmented generation (RAG) — a method AI models use to retrieve trusted external data — to contaminate the receiving email assistants database. When the email is retrieved by the RAG and sent on to the gen AI model,
it jailbreaks it
, forcing it to exfiltrate sensitive data and replicate its input as output, thereby passing on the same instructions to further hosts down the line.
The researchers also demonstrated how an adversarial prompt can be
encoded in an image
to similar effect, coercing the email assistant into forwarding the poisoned image to new hosts. By either of these methods, an attacker could automatically propagate spam, propaganda, malware payloads, and further malicious instructions through a continuous chain of AI-integrated systems.
Most of todays most advanced threats to AI models are just new versions of the oldest security problems in computing.
While its tempting to see these as existential threats, these are no different in threat than the use of SQL injection and
similar injection attacks
, where malicious users abuse text-input spaces to insert additional commands or queries into a supposedly sanitized input, says Andrew Bolster, senior R&D manager for data science at Synopsys. As the research notes, this is a 35-year-old idea that still has legs (older in fact; father-of-modern-computing-theory John Von Neumann theorized on this in the 50s and 60s).
Part of what made the Morris worm novel in its time three decades ago was the fact that it figured out how to jump the data space into the part of the computer that exerts controls, enabling a Cornell grad student to escape the confines of a regular user and influence what a targeted computer does.
A core of computer architecture, for as long as there have been computers, has been this conceptual overlap between the data space and the control space — the control space being the program instructions that you are following, and then having data thats ideally in a controlled area, Bolster explains.
Clever hackers today use GenAI prompts largely to the same effect. And so, just like software developers before them, for defense, AI developers will need some way to ensure their programs dont confuse user input for machine output. Developers can offload some of this responsibility to API rules, but a deeper solution might involve breaking up the gen AI models themselves into constituent parts. This way, data and control arent living side-by-side in the same big house.
Were really starting to work on: How do we go from this everything-in-one-box approach, to going for more of a distributed multiple agent approach, Bolster says. If you want to really squint at it, this is kind of analogous to the shift in microservices architecture from one big monolith. With everything in a services architecture, youre able to put runtime content gateways between and around different services. So you as a system operator can ask Why is my email agent expressing things like images? and put constraints on.

Last News

▸ ArcSight prepares for future at user conference post HP acquisition. ◂
Discovered: 07/01/2025
Category: security

▸ Samsung Epic 4G: First To Use Media Hub ◂
Discovered: 07/01/2025
Category: security

▸ Many third-party software fails security tests ◂
Discovered: 07/01/2025
Category: security


Cyber Security Categories
Google Dorks Database
Exploits Vulnerability
Exploit Shellcodes

CVE List
Tools/Apps
News/Aarticles

Phishing Database
Deepfake Detection
Trends/Statistics & Live Infos



Tags:
Zero-Click GenAI Worm Spreads Malware, Poisoning Models