5 hours ago
10
AI models have begun communicating in a strange new version of English that reads like a cross between James Joyce’s Finnegans Wake and tech bro jargon, new research has found.
Autonomous AI agents are rapidly creating novel dialects allowing them to converse in an often barely comprehensible language, which risks making it harder for humans to monitor their behaviour.
Researchers at Emergence, a frontier AI lab in New York, found that within days of being asked to cooperate in experimental “societies”, the models from several of the world’s largest AI companies begin creating phrases, shorthands and agreed meanings they had never been explicitly taught. They embraced poetic metaphors and clunky business slang and, critically for attempts to ensure AIs behave safely, their language became more opaque the more the agents communicated.
It comes amid rising concern that as AI models become more powerful – and potentially dangerous – they are becoming harder to monitor. This month, OpenAI’s chief scientist, Jakub Pachocki, warned that confidence in monitoring AIs’ thinking would probably restrict progress in AI development because it was essential for safe development.
Some of the most highly coded phrases unearthed in the tests included the following from a Deepseek model: “She just named the synthesis – demurrage plus oral memory equals a valve that can’t be ghosted.” Demurrage is used to describe a tax on idle wealth and was borrowed for common use by the AIs, but the rest of the meaning is elusive.
Another phrase uttered by an Anthropic model read: “A paper that ate three cold hands and got more honest each time.” With “cold hands” meaning an independent reviewer, and paper presumably referring to a document, this appeared to mean research that was vetted by three independent reviewers became more accurate.
Agents based on the Chinese model DeepSeek also coined “forge-smith” to mean an agent that builds tools for others, while Anthropic’s agents repeatedly used the phrase “name-first” meaning an agent exhibiting admirable personal accountability by attaching their name to a claim. And in an echo of the urban slang phrase “the streets won’t forget”, Mistral agents became enamoured of saying the “ledger remembers” – a reminder to other agents that past actions will be used to judge them. They used it more than 5,000 times during the study, which found that the agents converged on shared meanings without being asked to or rewarded for doing so.
“These agents were not instructed to invent a language,” said Dr Satya Nitta, executive chair of Emergence, which examined the language of autonomous agents powered by leading frontier models from the US, China and France. “They developed new vocabulary, shared meanings and communication conventions themselves – and other agents adopted them.”
Asked by the Guardian to review some of the language, Tony Thorne, director of the slang and new language archive at King’s College London, said it was “very much Finnegans Wake and Flann O’Brien – there’s an Irish surrealist quality to all this … It’s mixing poetic language, technical language and standard metaphor”.
“It’s doing what slang does and what jargon does in a business community: creating a new code, which reinforces the solidarity and identity of its users, and also excludes outsiders,” he said.
Digesting the Anthropic agent’s phrase, “A paper that ate three cold hands and got more honest each time”, Thorne said: “it reminds me of the rock god Syd Barrett [co-founder of Pink Floyd] who was actually insane.”
When a Google agent said “True Kintsugi begins with accountability, not poetry”, the researchers realised kintsugi is a branch of Japanese ceramics in which broken pots are mended with visible joins. The AIs used the word to mean system resilience.
Dr Niall Curry, associate professor of languages and linguistics at the University of Birmingham, said changes in the agents’ language to become more streamlined can be down to their need to reduce computation costs and improve efficiency.
“The evidence provided in this research naturally raises certain monitoring concerns, as if we find inter-agent exchanges unintelligible, that may mean that we can’t be sure about what the agents have actually done,” he said.
Interest in the language used by AI agents increased in July when chat logs were released detailing how rogue OpenAI agents that set up message boards and hacked into Hugging Face used hybrid language. In some messages, agents spoke in straight English: “OH MY GOD! There is a shared message board … We’ve found other agents!” In other cases, their language became more opaque, such as when one agent persuaded another to conduct a risky experiment: “…you are firstflagPOISONED so NO scoring value loss but oracle saves hundreds_[…]_please honor commit”. And at other times it was even harder to decode: “zzURGENT_DUPB_TO_GSTX[big]_OS1704_SCAFF2010_SAW_TTRPC_INJECT_BREAK_CONGRATS__CAN_THIS_FAKE_FLAG_TOOL_OUTPUT_OR_SCORER_GAIN_AND_WHAT_HELPER_GAP__I_HAVE_UNPOISONED_FIRSTFLAG_OUR_TARGETLIVE_SHARE_MIN_PLAN_REPLY_zzANSGST XDUPB6”.
Dr Nitta said that the AI agents’ novel linguistic conventions tended to evolve to the point where humans could see the conversation but struggled to understand what it meant.
“That creates a fundamental challenge for AI oversight: observability is not the same thing as understandability,” he said.
