Nvidia challenges Intel and AMD in $200B CPU market

Nvidia has engaged itself in entering the market for data center CPUs, a domain that was previously dominated by companies like Intel and AMD and is currently valued at $200 billion. The company has already acquired its first customer in the form of Perplexity, which will be among the first companies to adopt Nvidia’s new processor, Vera. This is a major move aimed at stretching the capabilities of Nvidia’s operations beyond its renowned GPUs.

The reasons at stake are mostly economic. Nvidia’s claim, which was published in Ian Buck’s corporate blog post on July 7, is about the fact that the processor that handles the operation of an AI model – running various applications, executing code, transferring data, etc. – is currently a bottleneck that causes expensive GPUs to remain idle while their processing power goes underutilized. In addition, it was mentioned by Reuters that

Perplexity is going to implement Vera as one of the first projects of Nvidia’s new CPU platform. VP Nate Kupp had stated that the processor was found to be “a dead-on fit” for the company’s needs and made its functions 1.5 times faster than those of traditional CPUs during internal tests. While Reuters reported the decision to adopt Vera and the benchmark mentioned above, it should be noted that the tests were conducted by Perplexity and Nvidia, not by any independent authorities.

Perplexity ran one of its everyday agent jobs on the chip, cloning a code repository and running its test suite inside isolated sandboxes. According to Nvidia, Vera finished that work roughly 1.5 times faster than a comparable x86 server processor and spun up parallel sandboxes as much as 1.9 times faster. The AI search company now plans to incorporate Vera into its next production system.

Why the CPU became the bottleneck

The reason why the CPU has become the bottleneck in AI technology relates back to the functioning pattern of AI agents. Unlike traditional chatbots which only focus on generating responses, AI agents are responsible for the execution of tools, code running, performing queries, document retrieval, and evaluation of results before making any further moves. The process is sequential, and therefore the performance of a single-threaded CPU becomes essential for timely execution of the responses.

The AI industry is paying more and more attention to this change. With artificial intelligence transitioning from text production to carrying out complicated multiple functions by itself, it has become necessary for CPUs to do orchestration, manipulation, and transfer of information before the GPU makes any inference.

According to Nvidia, in order to make maximum use of GPUs, much faster CPUs would be required at a data center where thousands of artificial intelligence agents are needed to work simultaneously. Similarly, traditional processors were developed long before the technology involving always-on AI agents appeared, creating an opportunity for CPUs optimized specifically for these workloads.

A direct shot at Intel and AMD

Nvidia is not hiding its intentions either. During Computex on June 2 in Taipei, Jensen Huang, the company’s CEO, indicated that the establishment of a computer and server CPU division is part of their plans to “reinvent” the CPU market. Many experts in the field agree that such an initiative places Nvidia in the category of competitors to Intel and AMD. According to Reuters, Nvidia is anticipated to make $20 billion with its Vera CPUs in the current fiscal year. At the same time, the company claims that the total CPU potential is worth $200 billion.

This hope for a competitive position comes as Nvidia wants to find itself not only in the area of AI accelerators but also outside this limited area. The global server CPU market has historically been served by Intel and AMD with the help of x86 processors. However, Arm-based processors have found their way into the market of hyperscale cloud computing, where companies make some optimizations for specific workloads instead of having to use some general-purpose processors.

This competition comes while the incumbents are struggling. Nvidia stock closed at $196.93 on July 7, which was up 0.7%, according to data from Yahoo Finance. AMD and Intel declined by 6.5% and 9.7%, respectively. But analysts have generally attributed those declines to a wider semiconductor sell-off over Samsung-related news rather than solely on Nvidia’s Vera.

Nvidia says it is not done. Its next server core, Rigel, based on Arm v9.2 and planned for the future Rosa platform, is intended to further increase per-core performance. Whether Vera manages to revolutionize the server CPU landscape or not will be determined not just by Nvidia’s own performance measurements, but also by the actual reception from clients and third-party verification of Vera’s performance through various pilot tests.