Tencent releases Hy3 open-source model that allegedly matches models up to five times its active size
Tencent released Hy3, an open-source 295-billion-parameter mixture-of-experts model where only 21 billion parameters are active at once. The company claims Hy3 matches models two to five times its size while halving hallucination rates to 5.4%.
- Hy3 uses a mixture-of-experts architecture with 295B total parameters but only 21B active at inference, reducing computational costs.
- Tencent claims Hy3 matches models two to five times its active size in performance while halving hallucination rates to 5.4%.
- The model is open-source, enabling broader access and customization for researchers and developers.
- This release highlights Tencent's expanding role in the open-source AI ecosystem.
Tencent has launched Hy3, an open-source language model built on a mixture-of-experts (MoE) architecture. The model boasts 295 billion parameters, but only 21 billion are active during inference, significantly reducing computational costs. According to Tencent, Hy3 achieves performance comparable to models two to five times its active size, a claim backed by internal benchmarks.
The model also reportedly cuts hallucination rates by half, reaching 5.4%, which could address a critical challenge in large language models. Hy3 is positioned as a cost-effective alternative for developers seeking high performance without the overhead of massive active parameter sets. The release underscores Tencent's growing influence in the open-source AI space, following its earlier contributions like Hunyuan-DiT.
The open-source nature of Hy3 allows researchers and developers to fine-tune and deploy the model freely, potentially accelerating innovation in efficiency-focused AI systems.
Source: Tencent releases Hy3 open-source model that allegedly matches models up to five times its active size. Read the full piece at the source.
Provides a cost-effective, high-performance alternative to larger models with reduced inference costs.
Offers a scalable solution for enterprises needing efficient AI deployment without sacrificing performance.
Demonstrates Tencent's strategic push into open-source AI, potentially influencing market dynamics.
Showcases advancements in model efficiency, addressing both performance and cost barriers in AI.
- Mixture-of-Experts (MoE)
- A model architecture where only a subset of parameters (experts) are activated for each input, improving efficiency.
- Hallucination rate
- The frequency at which an AI model generates incorrect or fabricated information.
NVIDIA and Hugging Face Bring New Models and Frameworks to LeRobot for the Open Robotics Community
Tencent Releases Hy3: An Open 295B Mixture-of-Experts (MoE) Model with 21B Active Parameters and 256K Context
LeRobot v0.6.0: Imagine, Evaluate, Improve
