nvidia/Nemotron-Labs-Audex-30B-A3B · Hugging Face
NVIDIA’s Nemotron-Labs has launched Audex-30B-A3B, a 30-billion-parameter MoE model combining text and audio capabilities. It builds on Nemotron-Cascade-2-30B-A3B and introduces discrete audio tokens for speech and general audio.
- Audex-30B-A3B is a 30B MoE model combining text and audio capabilities, built on Nemotron-Cascade-2-30B-A3B.
- The model introduces discrete audio tokens for speech and general audio outputs, along with an audio encoder for inputs.
- It features 3B activated parameters out of 30B total, emphasizing efficiency in multimodal AI processing.
- The release is available on Hugging Face, indicating accessibility for developers and researchers.
NVIDIA’s Nemotron-Labs has released Audex-30B-A3B, a 30-billion-parameter mixture-of-experts (MoE) model designed for unified audio-text processing. The model extends the Nemotron-Cascade-2-30B-A3B architecture by adding discrete audio tokens for speech and general audio outputs, alongside an audio encoder for inputs. This integration allows the model to handle both text and audio modalities within a single framework, marking a step toward more versatile multimodal AI systems.
The release highlights NVIDIA’s push into multimodal AI, where models can process and generate speech and audio alongside text. By leveraging discrete audio tokens, Audex-30B-A3B aims to bridge the gap between traditional text-based LLMs and audio-specific applications, such as speech synthesis, transcription, and general audio understanding. The model’s architecture, with 3 billion activated parameters out of 30 billion total, suggests a focus on efficiency without sacrificing performance.
While the announcement comes via Hugging Face and a Reddit post, the technical details indicate a significant advancement in NVIDIA’s AI toolkit. The model’s availability on Hugging Face suggests it is designed for broader accessibility, enabling developers to experiment with multimodal AI applications.
Source: nvidia/Nemotron-Labs-Audex-30B-A3B · Hugging Face. Read the full piece at the source.
Provides a new tool for building multimodal AI systems that handle both text and audio.
Enables companies to integrate speech and audio processing into their AI applications.
Advances the field of multimodal AI with a unified approach to text and audio.
- MoE (Mixture of Experts)
- A machine learning architecture where only a subset of parameters (experts) are activated for each input, improving efficiency.
- Discrete audio tokens
- Encoded representations of audio data used for processing and generation in AI models.