OpenAI’s report is the first of its kind from the company, which has swiftly become one of the leading players in AI. ChatGPT has gained more than 100 million users since its public launch in November 2022.
But even though AI tools have helped the people behind influence operations produce more content, make fewer errors, and create the appearance of engagement with their posts, OpenAI says the operations it found didn’t gain significant traction with real people or reach large audiences. In some cases, the little authentic engagement their posts got was from users calling them out as fake.
…
That echoes Facebook owner Meta’s quarterly threat report published on Wednesday. Meta's report said several of the covert operations it recently took down used AI to generate images, video, and text, but that the use of the cutting-edge technology hasn’t affected the company’s ability to disrupt efforts to manipulate people.
…
In the past three months, OpenAI banned accounts linked to five covert influence operations, which it defines as “attempt[s] to manipulate public opinion or influence political outcomes without revealing the true identity or intentions of the actors behind them.”
That includes two operations well known to social media companies and researchers: Russia’s Doppelganger and a sprawling Chinese network dubbed Spamouflage.
…
Another previously unreported Russian network banned by OpenAI focused its efforts on spamming the messaging app Telegram. It used OpenAI tools to debug code for a program that automatically posted on Telegram, and used AI to generate the comments its accounts posted on the app. Like Doppelganger, the operation's efforts were broadly aimed at undermining support for Ukraine, via posts that weighed in on politics in the U.S. and Moldova.
…
Another campaign that both OpenAI and Meta said they disrupted in recent months traced back to a political marketing firm in Tel Aviv called Stoic. Fake accounts posed as Jewish students, African-Americans, and concerned citizens. They posted about the war in Gaza, praised Israel’s military, and criticized college antisemitism and the U.N. relief agency for Palestinian refugees in the Gaza Strip, according to Meta. The posts were aimed at audiences in the U.S., Canada, and Israel. Meta banned Stoic from its platforms and sent the company a cease and desist letter.
…
OpenAI said the Israeli operation used AI to generate and edit articles and comments posted across Instagram, Facebook, and X, as well as to create fictitious personas and bios for fake accounts. It also found some activity from the network targeting elections in India.
None of the operations OpenAI disrupted only used AI-generated content. “This wasn't a case of giving up on human generation and shifting to AI, but of mixing the two,” Nimmo said.
He said that while AI does offer threat actors some benefits, including boosting the volume of what they can produce and improving translations across languages, it doesn’t help them overcome the main challenge of distribution.
From the article:
…
…
…
…
…