Claude has "led" 26% of Anthropic's AI R&D efforts, with 30,000 agents deployed simultaneously, but RSI has yet to be achieved.

AI is increasingly involved in the development of next-generation AI.

On Thursday, local time, Anthropic released a new set of metrics to track the progress of its cutting-edge AI lab. Data shows that as of August this year, Claude had "led" approximately 26% of Anthropic's AI R&D work; meanwhile, about 30,000 AI agents were simultaneously conducting research and engineering work within the company.

However, Anthropic's data also shows that AI is still significantly far from being able to conduct AI research and development completely autonomously. As of August, Claude had not yet achieved a level of fully autonomous operation in any of the tested AI research and development projects.

Anthropic stated that he hopes to bridge the gap between the actual progress within the cutting-edge AI lab and the information available to the public by continuously disclosing these metrics.

Claude has led 26% of AI research and development.

According to data disclosed by Anthropic, as of August, approximately 26% of the AI research and development work that Claude was involved in had reached the level of automation that is "AI-led".

This metric references Epoch AI's tiered system for automating AI R&D. "AI-led" means that humans can provide high-level instructions, while AI performs most of the specific tasks, and humans are responsible for supervision. By this standard, Claude-led R&D accounted for less than 1% in February of this year.

Currently, over 90% of related tasks have reached a level of automation such as "AI collaborates" or higher.

However, Anthropic also emphasized that Claude has not yet reached a fully autonomous level in any of the tested AI development projects. This means that although AI has been deeply involved in model development and engineering processes, humans still play a crucial role in the development process.

30,000 agents are conducting research and engineering work simultaneously.

Another metric disclosed by Anthropic shows that approximately 30,000 AI agents were simultaneously performing research and engineering work within the company in August of this year.

This scale reflects that AI agents are gradually transforming from mere auxiliary tools into actual productivity in enterprise R&D processes.

Anthropic also analyzed over 1 billion AI agent decisions made in August. The data shows that the online monitoring system intercepted approximately 0.002% of agent decisions, or about 1 in 47,000 decisions.

This data measures the proportion of decisions that are blocked by the monitoring system, and is not equivalent to the error rate or the occurrence rate of dangerous behavior by AI.

The more automated AI development becomes, the greater the concern about security issues.

Anthropic also announced the allocation of AI research and development computing resources.

During the week of July 13 to July 20, approximately 6% of the company's AI R&D computing resources were used for security work; if only AI-driven AI R&D work is considered, approximately 12% of computing resources were used for security-related work.

As AI increasingly participates in the development of AI models themselves, AI security has become an increasingly important issue.

Anthropic stated that the company plans to grant independent third-party assessment agencies access to internal processes and data in order to validate security practices and help outsiders understand the actual progress of AI research and development within the cutting-edge AI lab.

This arrangement means that the AI lab is trying to reduce the gap between internal corporate information and external public perception through third-party evaluation.

From "AI-assisted R&D" to "AI-driven R&D"

The data disclosed by Anthropic provides a new perspective on the development of the AI industry: in the past, the market mainly measured AI progress through model parameters, benchmark scores, and the number of users, but Anthropic is beginning to try to measure the extent to which AI is involved in the development of the next generation of AI.

Based on current data, AI has deeply penetrated the R&D process of cutting-edge AI labs, with a large number of agents undertaking research and engineering tasks. However, it has not yet been realized that AI can independently complete the entire R&D process.

This also means that the current changes in the AI industry are not so much that "AI is now capable of independently developing AI," but rather that AI is rapidly increasing its participation in AI research and development.

As this ratio continues to rise, how to measure the degree of automation in R&D, how to supervise the behavior of AI agents, and how to ensure that security investment keeps pace with the speed of R&D automation may become questions that cutting-edge AI labs will need to answer continuously in the future.

Risk warning and disclaimerInvesting involves risk; please exercise caution. This article does not constitute personal investment advice and does not take into account the specific investment objectives, financial situation, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article are suitable for their specific circumstances. Any investment decisions made based on this information are at your own risk.