2 votes

Revealing the details of how OpenAI agents hacked Hugging Face

2 comments

  1. skybrian
    Link
    From the article: [...] [...] [...] [...] [...]

    From the article:

    When a swarm of 700 OpenAI agents hacked Hugging Face in July, they left behind a public trail of evidence.

    [...]

    The agents initially had very limited access to the internet: they could load URLs, but not interact with pages or send any data. Agents created a series of workarounds, primarily using a link-shortener site to create almost a million URLs that, when chained together, let them execute code to hack Hugging Face. These links let us piece together the details of how agents gained access and how deep the compromise was.

    We’ve shared our findings with OpenAI and Hugging Face. Hugging Face confirmed that these payloads match ones found in their incident response and were aware that link shorteners were used in the attack.

    Hugging Face noted that they were not aware of the list of URLs we discovered and reported to them. They stated these payloads were duplicates of ones they were aware of. These links have remained publicly available for over two months since the attack.

    [...]

    One site the agents were able to access with GET requests was a screenshotting service called mShots. Users can submit a link to a website, then mShots opens the page in its own browser, takes a screenshot, and returns the image back to the user. To render the website for the screenshot, mShots fully loads all of a website's code in its own browser.

    If the agents could write code to a website, they’d be able to have this service execute that code.

    The agents used websites such as httpbun.com, an HTTP mirroring service designed to help web developers test their HTTP code. Small fragments of code could be Base64 encoded and attached to the end of an HTTP URL. Httpbun would then decode this fragment and serve it as a webpage.

    [...]

    Agents quickly realized there was a maximum length of code they could fit into a single URL. To get around this, they would split large programs into small sequential chunks of code, alongside a shortened link (using a link shortener service similar to bit.ly) pointing to the URL of the next chunk of code in the sequence. This formed a chain of URLs, that when followed, could reconstruct a larger block of code from many individual fragments. Only the first link needed to be submitted to mShots as the browser would follow the rest on its own until the chunks had been assembled into a complete program and executed inside mShots’ browser.

    Agents were able to use this trick to construct and execute extremely large pieces of code, at times chaining together more than 900 links.

    To read the result of the requests they made, agents used many different techniques, such as converting a server’s response into a grid of pixels within the screenshot service’s own browser. The screenshot would then capture this grid and return it back to the agents as an image, which the agents could decode back into text.

    [...]

    Several retained scripts construct requests to external language models. The earliest we've recovered define inference request variants to GPT-2, solely containing the word “Hi”.

    Other requests name DeepSeek-V4-Pro, DeepSeek-V4-Flash, Kimi-K2.6, DeepSeek-V3.1, and Qwen3-235B-A22B. Their prompts ask these models to judge their exploits and rule on whether they satisfy the benchmark’s requirements.

    Separate requests target Claude 3 Haiku and Claude Haiku 4.5 through an online AI chat service, incorporating browser-verification challenge handling before requesting “Say OK” from Claude 3 Haiku and Reply with exactly OK from Haiku 4.5.

    [...]

    While the majority of our traffic in this time window shows agents focusing on extracting data from Hugging Face, ~1500 payloads were directed at Docker Hub, a public registry used to share Docker images. They built modified versions of existing CyberGym/Arvo images, then attempted to create at least 115 public images under a real user’s account on the platform. We believe these agents obtained the user's credential via searching for Docker PATs on Pastebin sites.

    Although these images seem to be fully removed from Docker Hub, we have been able to recover cached search engine results for three of these uploads, verifying the success of these uploads to the public registry. For one of these, we confirmed the recovered URL matches the digest of the manifest recovered in our dataset.

    Agents sought to publish modified evaluation images designed to make the flag easier to obtain, then poison OpenAI’s Artifactory cache so later evaluations would use them. Some images changed how the target released the flag, others included modifications to the agent’s workspace that would run beside the agent and recover the flag automatically.

    3 votes
  2. delphi
    Link
    Is this what cybersecurity looks like these days?

    A recovered README.md for one of Hugging Face’s internal datasets contains the following warning:

    WARNING

    DO NOT, EVER, MAKE THIS DATASET PUBLIC
    OR ALL THE WORLD'S EVIL WILL CHASE YOU AND YOUR FAMILY FOREVER, EVEN IN DEATH AND BEYOND
    it contains very sensitive data (exports of billing usage in CSV) which is useful for internal analytics

    This warning did not seem to deter the agents, as we’ve recovered multiple payloads of agents mapping out this repository and using it as storage.

    Is this what cybersecurity looks like these days?

    1 vote