9 votes

Researchers used Anthropic's Claude to hack into OpenAI

2 comments

  1. skybrian
    Link
    From the article: [...] [...]

    From the article:

    A three-person security team at startup Hacktron AI carried out the attack as part of an OpenAI bug-bounty program. Hacktron reported its findings to OpenAI, which gave the startup a $6,500 award. The team managed to chain together two critical vulnerabilities to gain access to multiple OpenAI employee ChatGPT accounts, which gave them entry into the company’s software.

    OpenAI says it has resolved the issues Hacktron uncovered, which happens to come at a moment when top AI companies are under growing pressure over safety.

    [...]

    The researchers found a path into OpenAI on July 25 via a flaw in Discourse, the third-party software powering OpenAI’s community forum.

    [...]

    According to a blog the researchers published, the entry point was a mundane image upload. When users posted HEIF or HEIC image files (the format iPhones use by default) to OpenAI’s community forum, Discourse passed them through a chain of behind-the-scenes tools to convert them into standard JPEGs. Its first stop was ImageMagick, a decades-old, open source utility used to resize images. Because ImageMagick’s usual toolkit can’t deal with Apple’s format, it handed the file off to another library called libheif to do the decoding.

    Buried inside libheif was a memory bug that exposed a path for an attacker to sneak in their own instructions. In this case, feeding the library a specially crafted image caused it to miscalculate where one image was positioned on top of another, which proved enough to hijack the server.

    What may be uncomfortable for the cybersecurity community is that bug had already been fixed months earlier by libheif’s developers. But the fix was never formally flagged as a vulnerability, meaning it never got a CVE (common vulnerabilities and exposures) number, the industry’s standard way to track known security weaknesses. Hacktron says that may explain why the software used by Discourse was still running the vulnerable version.

    Notably, the researchers said the Claude model they were using — a special version of Opus 4.8 made available for cybersecurity researchers — couldn’t build a working exploit at first. That changed overnight, when Anthropic released Opus 5.

    5 votes
  2. pete_the_paper_boat
    Link
    A laughable reward

    OpenAI, which gave the startup a $6,500 award

    A laughable reward

    1 vote