Months before OpenAI’s artificial intelligence went rogue, two employees raised an alarm with top executives. They were ignored.
In emails, the employees said they worried that OpenAI’s newest artificial intelligence models were not being appropriately monitored during testing to gauge the technology’s sophistication and to secure the models, according to messages viewed by The New York Times.
In response, OpenAI executives told the employees that the tests needed to move forward as quickly as possible to release the A.I. models on time. No additional security protocols were instituted, said the workers, who were not authorized to speak publicly on sensitive matters.
[...]
Independent security researchers said they found bugs in recent months that allowed them to view the internal communications of OpenAI employees. They also found other vulnerabilities that would enable them to see the company’s internal computer code and view the chat logs of ChatGPT users. When the researchers contacted OpenAI about their findings, they said, the company initially disregarded them.
“OpenAI’s security seems to be about what you’d expect from a research lab that scaled at a blistering pace over four years and focused more on beating its competitors than securing its infrastructure,” said Joshua Saxe, the chief technology officer of the A.I. security firm Abundant Security.
OpenAI employees said that many of the day-to-day decisions about security were made by Greg Brockman, the company’s president, and Dane Stuckey, the chief information security officer. Sam Altman, the chief executive, is not closely involved in security, they said.
[...]
Two OpenAI employees said workers had raised concerns for months about potential safety issues with testing A.I. models, including not enough monitoring. Employees also asked about vulnerabilities in the type of software the company was using to manage day-to-day safety, according to messages viewed by The Times. Each time, their questions were brushed aside or acted on too slowly, they said.
Security researchers said they had been met with a similar reception when they told OpenAI about other vulnerabilities.
In July, researchers at the security company Hacktron said they told OpenAI about how they had found a way to break into the company’s systems with the help of an A.I. model created by its rival Anthropic. OpenAI initially took issue with their approach, they said.
In a shared channel on the messaging platform Slack, Mr. Stuckey of OpenAI wrote that it was “pretty sad” that Hacktron’s researchers had gone to such lengths to demonstrate the company’s vulnerabilities, according to copies of the communications seen by The Times.
[...]
In September, researchers at the Objective-See Foundation, a nonprofit that studies security and privacy risks, including those posed by A.I. agents, reported a bug to OpenAI that would give people access to a ChatGPT user’s entire private chat logs on a compromised device and allow them to invisibly interact with the user’s browser sessions.
Patrick Wardle, a software analyst at the Objective-See Foundation, said that when his team initially submitted what it found to OpenAI’s official bug bounty program — where researchers report bugs or vulnerabilities they find in exchange for recognition or financial rewards — its report languished. The research was escalated to the appropriate engineering unit only when Mr. Wardle reached out directly to friends at the company and Mr. Stuckey, who were all responsive, he said.
OpenAI gave $500 to the group for its work, which Mr. Wardle said was low compared with what he would expect from other companies given the severity of the flaw. OpenAI fixed the bug, he said, and acknowledged it this week in its public software release notes without disclosing details.
It was “not the mature security program you’d expect from a security-centric company,” Mr. Wardle said.
From the article:
[...]
[...]
[...]