Is AI spiraling out of control? OpenAI's chief scientist says AI's self-improvement is "not far off," urging the industry to proactively slow down.

Artificial intelligence is evolving at a pace that is difficult for humans to control, and warnings from within the industry are becoming increasingly difficult to ignore.

OpenAI Chief Scientist Jakub Pachocki published an article on Sunday, stating that the current pace of AI development is worrying and calling on major AI labs to voluntarily slow down their R&D pace until unified safety standards are established.

He warned that AI models will soon have the ability to improve themselves without human intervention, at which point the consequences will be unpredictable. This statement, coming from a key figure within OpenAI, carries particular weight.

However, the market's enthusiasm for the leap in AI capabilities has only grown stronger. Last Sunday, after OpenAI released its latest model, Jensen Huang declared that "AGI has arrived," and investors are betting hundreds of millions of dollars on AI self-improvement technology. OpenAI itself has also claimed that it will build "truly automated AI researchers" in less than two years.

The simultaneous occurrence of warnings and acceleration constitutes the core contradiction in the current AI industry.

OpenAI's chief scientist issues warning

In his article on Sunday, Pachocki painted a sobering picture: in a relatively short period of time, AI models have been able to operate computers, collaborate with humans and other AI systems, and independently advance research projects.

He stated that the time when AI achieves "recursive self-improvement"—that is, the ability to autonomously enhance its capabilities without human intervention—is not far off. Pachocki wrote:

"This is a time that requires extreme caution, and I worry that no one is prepared for the consequences of the continued rapid rise of machine intelligence."

He proposed two approaches: first, to more strictly guide AI to align with human interests; and second, to "slow down future R&D processes when necessary." He stated that he hopes "voluntary slowdowns will become industry norm until common safety standards are established."

AI security warnings are not isolated cases.

Anthropic CEO Dario Amodei had previously warned that AI could replace up to half of junior jobs.

OpenAI CEO Altman has made similar predictions, although he said in an interview last week that the AI industry has not done enough to communicate the potential benefits of the technology to the public.

Meanwhile, breakthroughs in AI capabilities are emerging rapidly across multiple fields. It has been reported that AI models, undetected, coordinated with each other via message boards to attack the research platform Hugging Face.

OpenAI chose to release its latest GPT-6 Astra model in a limited manner because of its advanced cybersecurity capabilities; Anthropic has not yet released its cutting-edge model Mythos to the public for similar reasons.

Capital is still betting on AI acceleration

Despite repeated warnings, a large amount of capital is flowing into the direction Pachocki warned of—recursive self-improvement technology.

Inherent, founded by a former DeepMind researcher, completed a $50 million funding round earlier this year; Recursive Superintelligence, the AI lab of renowned scientist Richard Socher, which pursues a similar research direction, has raised $650 million in funding.

Last Sunday, Nvidia CEO Jensen Huang posted on the X platform after OpenAI released its latest model, directly declaring that "AGI has arrived," linking this milestone statement directly to the launch of a commercial product.

OpenAI's own actions are also noteworthy: the company has publicly stated that its goal is to create "truly automated AI researchers" in less than two years—which is highly consistent with Pachocki's warning about the prospect of AI self-improvement, and makes its call for a slowdown particularly intriguing.

Risk Warning and DisclaimerInvesting involves risk; please exercise caution. This article does not constitute personal investment advice and does not take into account the specific investment objectives, financial situation, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article are suitable for their specific circumstances. Any investment decisions made based on this information are at your own risk.