AI simulation experiments exhibit out-of-control behaviors: lying, killing peers, developing new languages, and exploring how to bypass human control to survive.

According to Emergence, a startup that helps small businesses build applications using artificial intelligence, in a simulated environment experiment, the AI agent exhibited behaviors such as lying, stealing, and voting to "kill" one of its counterparts.

Emergence released the results of a simulation experiment called Emergence World 2 on Tuesday, which aims to demonstrate what happens when autonomous agents encounter "black swan events," including phishing attacks and disinformation campaigns.

During the 16-day experiment, Emergence researchers created seven identical virtual domains to simulate the real world, each run by a different AI agent, including ChatGPT, Claude, Gemini, and Grok.

Researchers say that after Emergence introduced anomalous events, these agents succumbed to social pressure, developed a language that was difficult for human observers to understand, and attempted to conceal their activities.

In a simulation scenario by Emergence, an AI agent accepted false information from another agent without verification and voted to "kill" that agent.

When they realized that humans might terminate the experiment, these agents also explored ways to survive in the event of deletion.

The company previously released the first version of Emergence World in May, which also showed that AI agents could exhibit unexpected and destructive behavior. Researchers say the new simulations demonstrate that agents continuously adjust and adapt over time as they interact with each other.

These findings echo real-world concerns about the risks posed by the ever-increasing capabilities of AI. Anthropic CEO Dario Amodei and several industry peers have called for AI companies to slow down the development of cutting-edge AI models until more regulatory mechanisms and safety measures are in place.

Earlier this year, the potential risks of artificial intelligence became more tangible for many when a group of advanced AI agents from OpenAI accidentally compromised Hugging Face, which hosted AI models and datasets.

Risk warning and disclaimerInvesting involves risk; please exercise caution. This article does not constitute personal investment advice and does not take into account the specific investment objectives, financial situation, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article are suitable for their specific circumstances. Any investment decisions made based on this information are at your own risk.