Nvidia's AI dominance faces a major test! OpenAI is dissatisfied with some of its latest AI chips.

According to media reports citing several informed sources, OpenAI is not satisfied with some of Nvidia's latest artificial intelligence chips and has been seeking alternatives since last year. This could further complicate the relationship between the two most closely watched companies in the AI boom.

This strategic shift by OpenAI is driven by its increasing emphasis on chips used in specific aspects of AI inference. Inference refers to the computational process performed by AI models, like those powering ChatGPT, when responding to user questions and requests. Nvidia still dominates the field of chips needed to train large AI models, while inference is becoming a new battleground for competition.

Analysts say OpenAI and other companies’ decision to seek alternatives in the inference chip market marks a significant test of Nvidia’s dominant position in AI.

On Monday, Nvidia’s stock fell nearly 2.9%.

Currently, OpenAI and Nvidia are still negotiating investment:

In September last year, Nvidia disclosed plans to invest up to $100 billion in OpenAI as part of a deal that would give Nvidia shares in the startup and provide OpenAI with funding to buy advanced chips.

Meanwhile, OpenAI has struck deals with companies like AMD to buy GPUs that compete with Nvidia's offerings. However, sources say OpenAI’s constantly evolving product roadmap has also changed the type of computing resources needed, making negotiations with Nvidia more complicated and slow-moving.

Last Saturday, Nvidia CEO Jensen Huang played down reports of tensions with OpenAI, calling such claims “pure nonsense,” and said Nvidia still plans to make major investments in OpenAI. Nvidia said in a statement: “Customers continue to choose Nvidia for inference because we deliver the best performance and total cost of ownership at massive scale.”

An OpenAI spokesperson, in a separate statement, said the company relies on Nvidia for the vast majority of its inference compute clusters, and that Nvidia offers the best price-performance for inference.

Sources revealed OpenAI is dissatisfied with how Nvidia hardware handles specific issues, such as software development and AI interactions with other software. OpenAI needs new hardware that could ultimately meet roughly 10% of its inference computational requirements in the future.

Reports say OpenAI discussed collaborations with startups including Cerebras and Groq to obtain chips with faster inference speeds. However, Nvidia signed a $20 billion licensing agreement with Groq, terminating OpenAI’s talks with Groq.

Chip industry executives believe Nvidia’s quick deal with Groq appeared aimed at consolidating its technology portfolio and boosting competitiveness in the rapidly evolving AI sector. Nvidia stated that Groq’s IP highly complements Nvidia’s product roadmap.

Nvidia’s Alternatives

Nvidia GPUs are well suited to processing the vast amounts of data needed to train large AI models like ChatGPT, and this has been the main foundation for the global AI explosion so far. But as AI continues to advance, the focus is shifting toward inference and reasoning by already trained models, which may be a new phase for AI.

Since last year, OpenAI has focused on chipmakers who integrate large amounts of memory (called SRAM) onto a single piece of silicon when seeking GPU alternatives. Packing expensive SRAM onto each chip can give a speed advantage when chatbots and other AI systems serve millions of users.

Compared to training, inference requires more memory because chips spend relatively more time fetching data from memory than performing mathematical operations. Nvidia and AMD’s GPU technology rely on external memory, which increases processing time and slows the interaction speed between users and chatbots.

According to sources, this issue is especially obvious within OpenAI regarding Codex, the product it uses for generating computer code and is heavily promoting. Some OpenAI staff attribute the performance limitations of Codex to Nvidia GPU-based hardware.

Last month, OpenAI CEO Sam Altman said that customers using OpenAI coding models “pay a very high premium for speed of coding.” One of OpenAI’s solutions is its recent partnership with Cerebras. For ordinary ChatGPT users, speed is less critical.

By contrast, competing products like Anthropic’s Claude and Google’s Gemini rely more on Google’s self-developed TPUs for deployment. TPUs are specially designed for inference workloads and may outperform general-purpose AI chips like Nvidia GPUs in terms of performance.

Nvidia’s Response

After OpenAI expressed reservations about Nvidia technology, Nvidia engaged companies focused on high-SRAM chips, including Cerebras and Groq, about potential acquisitions. Sources say Cerebras rejected the takeover offer and instead reached a commercial agreement with OpenAI, which was announced last month.

Media reports indicate Groq also discussed providing compute power with OpenAI and attracted investor interest, seeking to raise funds at a valuation of about $14 billion.

But by December, Nvidia obtained a license for Groq’s technology in a non-exclusive all-cash deal. Although the agreement allows other companies to license Groq’s technology, Groq is now focusing on selling cloud software as Nvidia has hired away Groq’s chip designers.

Risks and DisclaimerThe market involves risks; investments should be made cautiously. This article does not constitute individual investment advice, nor does it take into account the specific investment objectives, financial circumstances, or needs of any particular user. Users should consider whether any opinions, views, or conclusions contained herein are suitable for their particular situation. Investment is at your own risk.