NVIDIA's $20 billion acquisition of Groq marks its first strategic discussion: reasoning tokens should be priced based on quality, with low latency and high unit price as the new battleground.

ME News message, April 16 (UTC+8), according to Beating Monitoring, Jensen Huang, in an interview, for the first time provided a detailed explanation of the strategic rationale behind NVIDIA’s acquisition of Groq. In December last year, NVIDIA acquired Groq’s inference chip business for $20 billion. Groq founder Jonathan Ross and the core team joined NVIDIA, and Groq continued to operate as an independent company. At this March’s GTC conference, NVIDIA released its first chip after the merger, Groq 3 LPU, manufactured using Samsung’s 4nm process. NVIDIA said that for trillion-parameter models, its inference throughput per megawatt is 35 times that of Blackwell NVL72.

Jensen Huang said the driving force behind acquiring Groq is the stratification of the inference market. Previously, inference optimization only pointed in one direction: increasing throughput. But the commercial value of tokens has risen significantly, and different users are willing to pay different prices for different response speeds. “If I can provide software engineers with faster-response tokens, making them more efficient than they are today, I’m willing to pay for that. But this market only emerged recently.” He described this as an expansion of the Pareto frontier in the inference market: beyond existing high-throughput solutions, adding a new market segment characterized by low latency and a high unit price. For the same model, with differentiated pricing based on response time, “although throughput is lower, the higher unit price can make up for it.” Groq’s LPU architecture is known for deterministic low latency, complementing NVIDIA’s high-throughput GPU roadmap, and the acquisition fills a missing piece in NVIDIA’s inference product lineup. (Source: BlockBeats)