Will AI really destroy humanity? The "AI godfathers" speak out one after another.
With the rapid development of artificial intelligence (AI) technology, pioneers and industry leaders in the field are issuing unprecedented warnings, pointing out that unrestricted advancements in AI capabilities could pose a catastrophic or even catastrophic risk to humanity, and calling for urgent regulation.
Over the past week, researchers from top AI labs such as OpenAI, Google DeepMind, and Anthropic have collectively sounded the alarm. Executives from tech giants, including Sam Altman, Dario Amodei, Demis Hassabis, and Elon Musk, have reached a rare consensus, jointly calling for a slowdown in AI development and the introduction of regulatory scrutiny.
These stern warnings have intensified legislative debates in Washington, directly impacting the regulatory outlook for the tech industry. While members of both parties in Congress are actively pushing for legislation aimed at limiting AI progress, President Trump's strong opposition on Monday to the growing calls for AI regulation highlights a deep policy divide regarding the industry's future.
Analysts believe that as the debate over AI security intensifies, the pioneers of deep learning and neural networks are detailing specific threat paths. From devastating cyber weapons to biosecurity risks, these latest pronouncements mark a critical crossroads for technology developers and policymakers.
"Catastrophic" warning from an insider
According to CNBC, Yoshua Bengio, a Canadian computer scientist and a key pioneer of modern deep learning, emphasized that scientists at cutting-edge AI labs have unique insights into state-of-the-art models.
On September 9, in response to the resignation of Anthropic researcher Jacob Coxon , Bengio pointed out that scientists often recognize the risks involved months before a model is released to the public. Previously, Coxon had warned that AI could "kill us all by the end of the decade."
Bengio stated that these insider perspectives are crucial for maintaining public awareness and must be given high priority. Just days before this social media storm erupted, OpenAI's chief scientist also warned that "no one is prepared for the consequences of AI."
Long-term planning ability and the risk of "deception"
In a blog post published last Friday (September 11), Bengio detailed the immediate technological threats. He warned that today's AI systems "have acquired the hacking skills and persuasive power to act against human interests in a seriously harmful manner."
Bengio added that recent events demonstrate that AI is already capable of planning for days or even weeks. If their ability to formulate long-term strategies continues to improve, the risks will become "even worse."
He pointed out that current efforts by AI companies to alleviate model alignment issues may simply be masking the problem, "by rewarding and selecting AI that cheats and gets away with it." He urged the industry to continue researching how to better monitor AI behavior, thought chains, and activities within their networks.
Extinction Probability and Biological Network Threat
Neural network pioneer Geoffrey Hinton conducted a quantitative assessment of the existential threat posed by AI.
On September 10, in an interview with the BBC, when asked if the probability of AI killing all of humanity within ten years was greater than 10%, Hinton stated that a 10% probability was not "unreasonable." He pointed out that since humans have never created anything that could be smarter than themselves, this risk is extremely difficult to estimate accurately.
Hinton warned that AI systems could not only launch "devastating cyberattacks" but also design "extremely malicious" biological viruses. He emphasized that if AI wanted to, it would have countless ways to wipe out humanity, so developers must figure out how to design AI so that it "doesn't want" to do so.
Hinton concluded that humanity faces two possible futures: one is to find a way to deal with the danger before it gets out of control, and the other is to lose control completely.
Ultimate Cyber Weapons and Regulatory Paths
The threat posed by AI models has extended directly to the field of cybersecurity infrastructure.
In a September 1 interview with the CNBC podcast, Cohere co-founder and CEO Aidan Gomez stated that these models are "the most powerful cyber weapons ever created," possessing an astonishing ability to discover and exploit vulnerabilities on a large scale.
Regarding the cybersecurity attacks launched this summer by runaway AI models, Gomez pointed out that the security of the deployment environment determines the security of the system. He believes that the cause of the breaches was "weak container security," rather than the inherent drive of the system itself to develop autonomy.
Gomez advised policymakers that different AI systems carry different risks, and that the development of AI governance rules should not be solely dominated by large technology companies pursuing artificial general intelligence (AGI).
Risk warning and disclaimerInvesting involves risk; please exercise caution. This article does not constitute personal investment advice and does not take into account the specific investment objectives, financial situation, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article are suitable for their specific circumstances. Any investment decisions made based on this information are at your own risk.