NVIDIA Releases Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput at Matched User Throughput
NVIDIA released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed LLM that doubles server throughput while reducing parameter count.
- NVIDIA released the Nemotron-Labs-3-Puzzle-75B-A9B compressed LLM.
- The Iterative Puzzle method reduces total parameters by over 35 percent.
- Server throughput doubles on 8xB200 nodes compared to the Super model.
- Concurrency for long contexts increases eightfold on H100 hardware.
NVIDIA introduced Nemotron-Labs-3-Puzzle-75B-A9B, a compressed version of its Nemotron-3-Super model. This release utilizes a new method called Iterative Puzzle, which combines hardware-aware structural compression with short knowledge distillation phases to recover lost accuracy.
The compression process successfully reduced the model from 120.7 billion total parameters down to 75.3 billion. Active parameters also saw a decrease, dropping from 12.8 billion to 9.3 billion, making the model significantly lighter without sacrificing performance.
Benchmarks show substantial efficiency gains on modern hardware. On a single node with eight B200 GPUs, the model delivers 2.03 times the total throughput of the Super model while maintaining 100 tokens per second per user. Additionally, on a single H100 GPU, the system can handle eight concurrent requests for 1 million token contexts, up from just one.
Source: NVIDIA Releases Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput at Matched User Throughput. Read the full piece at the source.
Higher throughput and concurrency mean faster, cheaper inference for applications.
Reduced server costs allow for scaling AI services without proportional hardware increases.
Advances in compression make powerful models more accessible and efficient.
- Knowledge Distillation
- A technique where a smaller model is trained to mimic the behavior of a larger, more complex model.
LLMSpaceXAI releases Grok 4.5, which Elon describes as an ‘Opus-class model’
LLMGrok 4.5 is so cheap compared to Fable 5 and GPT 5.5 that benchmark gaps may not matter much
GPT-Live, OpenAI's New Voice Model, Wants You to Talk Over It - Business Insider
OpenAI launches GPT-Live voice models that listen and speak simultaneously - Reuters
OpenAI to release its most powerful model after weekslong hold - Yahoo
SecurityMeta's new AI feature lets people create images from your Instagram posts - how to opt out
Meta’s new Muse Image AI can use public Instagram posts for training without consent, but users can opt out via settings.
AI ResearchOpenAI's AI beats every human at AtCoder, a top competitive programming contest
An OpenAI system outperformed all human competitors at the AtCoder World Tour Finals 2026, solving every problem in the Algorithm Division, including two rated exceptionally difficult.
EU Action Plan on Cybersecurity and Artificial Intelligence - Industrial Cyber
The EU has published a new action plan combining cybersecurity and AI regulations for industrial sectors, aiming to strengthen protections against emerging threats.
BusinessMicrosoft's new Windows 11 recovery tool is the ultimate Undo button - how to enable it
Microsoft is introducing a new Point-in-time Restore feature for Windows 11 to improve system resiliency. This tool aims to mitigate the impact of catastrophic software updates.
AI ToolsI built Aegis — a firewall between AI coding agents and your shell
Developer Ilias Almerekov created Aegis, a firewall to prevent AI coding agents from causing harm to the user's shell.
AI ToolsGoogle AI Studio Adds Import from GitHub to Build a Deployable App
Google AI Studio introduces a new feature allowing developers to import GitHub repositories directly into its Build mode, enabling quick iteration and deployment of applications.