Critical Flaw in Replicate AI Platform Exposes Proprietary Data

The finding underscores the challenges of protecting data from multiple customers across AI-as-a-service solutions, especially in environments that run AI models from untrusted sources.

A critical vulnerability in the Replicate AI platform could have allowed attackers to execute a malicious AI model within the platform for a cross-tenant attack — allowing access to the private AI models of customers and potentially exposing proprietary knowledge or sensitive data.
Researchers at Wiz discovered the flaw as part of a series of partnerships with AI-as-a-service providers to investigate
the security
of their platforms. The discovery of the flaw demonstrates the difficulty of tenant separation across AI-as-a-service solutions, especially in environments that run AI models from untrusted sources.
Exploitation of this vulnerability would have allowed unauthorized access to the AI prompts and results of all Replicates platform customers, and potentially alter those results, Wizs Shir Tamari and Sagi Tzadik wrote in a
blog post published today
. Previously, Wiz researchers
found flaws
that led to a similar outcome in the HuggingFace AI platform.
As we saw in the results of our work with
Hugging Face
and now in Replicate, two leading AI-as-a-service providers, when running AI models in cloud environments, it is crucial to remember that AI models are actually code, Ami Luttwak, Wiz CTO and co-founder, tells Dark Reading. Like all code, the origin must be verified, and content-scanned for malicious payloads.
Indeed, the flaw presents an immediate threat to AI-as-a-service providers, who often allow their customers to execute untrusted code in the form of AI models in shared environments - where there is other customers data. It also can impact AI teams, who could be affected when they adopt AI models from untrusted sources and run them on their workstation or company servers, the researchers noted.
Wiz Research responsibly disclosed the vulnerability to AI model-sharing vendor Replicate in January 2023; the company promptly mitigated the flaw so that no customer data was compromised. At this time, no further action is required by customers.
The flaw lies in achieving remote code execution on Replicates platform by creating a malicious container in the Cog format, which is a proprietary format used to containerize models on Replicate. After containerizing a model using Cog, users can upload the resulting image to Replicates platform and start interacting with it.
Wiz researchers created a malicious Cog container and uploaded it to the platform and then, with root privileges, used it to execute code on the Replicate infrastructure.
We suspect this code-execution technique is a pattern, where companies and organizations run AI models from untrusted sources, even though these models are code that could potentially be malicious, the researchers wrote in the post. A similar technique was used to exploit flaws found on the HuggingFace platform.
This exploitation allowed the researchers to investigate the environment move laterally out and ultimately outside of the node on which they were running, which was inside
a Kubernetes cluster
hosted on Google Cloud Platform. Though the process was challenging, they eventually were able to conduct a cross-tenant attack that allowed them to query other models and even modify the output of those models.
The exploitation of this vulnerability would have posed significant risks to both the Replicate platform and its users, the researchers wrote. An attacker could have queried the private AI models of customers, potentially exposing proprietary knowledge or sensitive data involved in the model training process. Additionally, intercepting prompts could have exposed sensitive data, including personally identifiable information (PII).
Indeed, this ability to alter prompts and responses of an AI model poses a severe threat to the functionality of AI applications, giving attackers a way to manipulate AI behavior and compromise the decision-making processes of these models.
Such actions directly threaten the accuracy and reliability of AI-driven outputs, undermining the integrity of automated decisions and potentially having far-reaching consequences for users dependent on the compromised models, the researchers wrote.
Currently there is no easy way to validate a models authenticity, or to scan it for threats, so malicious AI models present a
new attack surface
for defenders that needs other forms of mitigation, Luttwak says.
The best way to do this is to ensure that production workloads only use
AI models
in secure formats, like so-called safetensors. We recommend that security teams monitor for usage of unsafe models and work with their AI teams to transition into safetensors or similar formats, he says.
Using only safe AI formats can the attack surface dramatically, as these formats are designed to prevent attackers from taking over the AI model instance, Luttwak says.
Further, cloud providers who run their customers models in a shared environment should enforce tenant-isolation practices to ensure that a potential attacker who managed to execute a malicious model cannot access the data of other customers or the service itself, he adds.

Last News
▸ Debunking Machine Learning in Security. ◂ Discovered: 23/12/2024 Category: security	▸ Researchers create BlackForest to gather, link threat data. ◂ Discovered: 23/12/2024 Category: security	▸ Travel agency fined £150,000 for breaking Data Protection Act. ◂ Discovered: 23/12/2024 Category: security

**Cyber Security Categories**
Google Dorks Database	Exploits Vulnerability	Exploit Shellcodes

CVE List

Tools/Apps

News/Aarticles

Phishing Database

Deepfake Detection

Trends/Statistics & Live Infos

Tags:
Critical Flaw in Replicate AI Platform Exposes Proprietary Data