A previously unknown Russian operation used ChatGPT to write propaganda comments and debug the bot that posted them. The company whose tools it borrowed found it, named it, and shut it down, which raises a strange new question about who is watching the watchers.
In the spring of 2024, the company behind ChatGPT did something the makers of previous propaganda tools never could. It published a report identifying the influence operations that had been using its own product, described what they had tried to do, and announced that it had removed their access. Among the operations it exposed was one from Russia that no one outside had documented before. The episode marked a shift worth thinking about carefully, because for the first time the manufacturer of the weapon was also the party catching the people who wielded it.
The operations OpenAI named
In its first threat report, released in May 2024 and led by the investigator Ben Nimmo, OpenAI said it had disrupted five covert influence operations over the previous three months. Two of them were Russian. One was the already notorious Doppelganger, the cloned-outlet operation that Western governments had been chasing for years. The other was new, and OpenAI gave it a name: Bad Grammar.
Bad Grammar operated mainly on Telegram and aimed its output at audiences in Ukraine, Moldova, the Baltic states, and the United States. What made it notable was how mundane its use of the technology was. The operators used OpenAI’s models to debug the code for a bot that posted automatically to Telegram, and to generate short political comments in multiple languages designed to look like the reactions of ordinary users. As TIME reported at the time, the same disruption swept up operations from China, Iran, and a private firm in Israel, but the Russian activity fit a familiar strategic pattern: cheap, high-volume, and aimed at the countries on Russia’s borders and at the American electorate.
The reassuring part, and the catch
OpenAI’s early assessment carried a note of reassurance, and it was honest about it. The operations it caught were, for the most part, not working very well. Nimmo’s team found that generative tools were helping the operators produce content faster and in more languages, but that the resulting posts were largely failing to reach or move real audiences. The bottleneck for a propaganda campaign, it turned out, was rarely the writing. It was distribution, and an AI model does not solve distribution. A fluent comment posted by a bot that no one follows is still a comment no one reads.
That finding should be held onto, because it cuts against the loudest fears about AI and disinformation. In these cases the technology made bad operations more efficient without making them more effective. The catch is that efficiency compounds. An operation that can produce ten times the content in ten times the languages for a fraction of the cost has more chances to get lucky, more shots at the one post that breaks through, and a lower price for every failed attempt. Cheapness is its own kind of danger even when quality stays flat.
Why the company found what governments could not
The reason OpenAI could expose Bad Grammar when outside researchers had not is simple and important. The operators had to come to OpenAI’s systems to use them. Every prompt, every debugging session, every batch of generated comments left a trace inside the company’s own logs. An intelligence agency trying to map a foreign influence operation is working from the outside, reconstructing a campaign from the debris it leaves on public platforms. The platform the operators used to build the campaign is working from the inside, with a view of the workshop rather than the litter.
This is a genuinely new dynamic in the history of information warfare. The Soviet Union’s forgery shops and the Internet Research Agency’s troll floors were opaque to the companies whose channels they exploited. A newspaper that ran a planted story had no way of knowing it had been planted. But an operation that outsources its content generation to a commercial AI provider hands that provider a front-row seat, and if the provider chooses to look, and to publish what it sees, the operation is exposed from an angle it cannot defend.
Outcome and what changed
The single report became a routine. OpenAI has continued the practice, releasing further threat reports and, in subsequent updates discussed by EU disinformation monitors, disrupting a growing number of malicious campaigns, Russian and otherwise, through 2024 and 2025. Periodic transparency reporting from an AI company, naming the influence operations that abused its tools, did not exist before this and is now an established source that researchers and governments read.
The core finding has largely held: across the cases OpenAI has disclosed, generative tools boosted the efficiency of influence operations without yet delivering the audience they still could not reach. What changed alongside it is the awareness, on both sides, that the tool keeps the receipts. A sophisticated actor will eventually respond by moving to private or open models that report to no one, trading the best commercial systems for the safety of staying off the logs. And there is a limit worth stating plainly. OpenAI catches operations that use OpenAI, it cannot see the ones that do not, and its reports are the company describing its own successes rather than an independent audit. The transparency is real and valuable, and it is partial by nature, and it lasts only as long as the company keeps choosing to look and to say what it finds.
Leave a Reply