1 hour ago
Nvidia CEO Jensen Huang revealed a new AI inference system during the company's annual GTC conference on Monday, marking a significant step to maintain Nvidia's leadership as inference becomes a critical focus in AI development.
Huang highlighted the growing demand for inference technology, projecting at least $1 trillion in demand for Nvidia's Blackwell and Rubin AI systems through 2027, up from about $500 billion projected through 2026.
The newly announced Nvidia Groq 3 LPX chip, which integrates technology from AI chip startup Groq with Nvidia's Vera Rubin architecture, can accelerate inference workloads by up to 35 times. Samsung manufactures this chip, and Nvidia expects the system to ship in the second half of this year.
"The inflection point of inference has arrived," Huang stated during his keynote address.
This new system builds on the approximately $20 billion licensing agreement Nvidia made with Groq in December, which included licensing Groq's technology and hiring its top engineers.
Huang had previously hinted at this collaboration during Nvidia's latest earnings call. The Wall Street Journal also reported that Nvidia was preparing a new inference system incorporating Groq technology.
While Nvidia's GPUs continue to dominate AI workloads for both training and inference, an increasing number of competitors—from hyperscalers to chip startups—are developing specialized, cost-effective systems optimized for the repetitive and cost-sensitive nature of inference tasks.
The rise of AI agents, which perform tasks autonomously on behalf of users, is expected to significantly increase inference demand.
In response, some AI companies like OpenAI have explored alternatives to Nvidia hardware. Reuters reported that OpenAI was dissatisfied with Nvidia's inference chips and in January signed a reported $10 billion compute deal with inference chip startup Cerebras.
Have a tip? Contact reporter Geoff Weiss via email at [email protected] or Signal at @geoffweiss.25. Use a personal email, nonwork WiFi, and a nonwork device. For more, see our guide to sharing information securely.
