Nvidia Launches Groq 3 LPX Inference Chip into Full Production

Nvidia announced at the Hot Chips 2026 conference that its Groq 3 LPX inference accelerator has entered full production. The chip, built on the Vera Rubin platform, is marketed as an interactive AI inference accelerator designed to deliver ultrafast token generation for agentic AI systems. Nvidia plans to roll the technology out later this year.

The move follows Nvidia’s late‑December acquisition of Groq for roughly $20 billion, bringing founder Jonathan Ross and president Sunny Madra into the Nvidia fold. Groq has long focused on processors tuned for inference rather than training, and the 3 LPX is the latest iteration of that line.

In an independent benchmark conducted by Artificial Analysis, the Groq 3 LPX achieved 3,400 tokens per second on the open‑source Gemma 4 31B model with a 100,000‑token context window. The test ran 50 back‑to‑back requests and maintained steady performance across input lengths from 10,000 to 100,000 tokens. Nvidia claims this figure is the highest ever recorded for the model and that it is four times faster than the next best option, Cerebras, which achieved 882 tokens per second.

However, the raw speed numbers hide a key scaling factor. Each Groq 3 LPX processing unit (LPU) contains only 500 MB of SRAM, about 576 times less than a Rubin GPU’s 288 GB. To run Gemma 4 31B, the system requires at least 64 LPUs, whereas Cerebras needs only one or two. For larger mixture‑of‑experts models such as DeepSeek V3, the Groq architecture would need 1,342 LPUs, or a little over five racks, compared to Cerebras’ single or dual‑chip requirement.

The architecture relies on a SRAM‑heavy dataflow design. Models are split across multiple LPUs over Ethernet, with GPUs handling the compute‑heavy prefill phase and the LPUs handling the bandwidth‑heavy decode phase. This hybrid setup can deliver high throughput, but it also introduces complexity in coordination and potential latency.

The comparison also omits Cerebras’ newer CS‑4 generation, which could narrow the performance gap. Moreover, the benchmark used a dense model that fits entirely in one rack, leaving open how the Groq 3 LPX scales with larger, more complex MoE models. The cost of deploying dozens or hundreds of LPUs, as well as the operational overhead of managing a multi‑rack system, remains unclear.

Nebius plans to be the first cloud provider to offer the Groq 3 LPX through its Token Factory, and Groq itself is among the early adopters. The production launch positions Nvidia to compete directly with Cerebras in the high‑speed inference market, but the real‑world advantage will depend on how quickly the technology can be integrated into agentic AI workflows and how the cost‑performance balance plays out.

What to watch next includes the rollout timeline, pricing models, and performance on a broader set of models, especially those with large MoE layers. Investors and developers will also be interested in how the chip’s architecture affects power consumption, latency, and ease of integration with existing AI pipelines.