tencent/Hy3
Evolving story · 2 updatesTencent’s Hy3 MoE Model LaunchTimeline →Tencent released Hy3, an Apache 2.0 licensed Mixture-of-Experts model with 295B total parameters but only 21B active at inference. It outperforms similar-sized models and rivals larger open-source models.
- Hy3 is a 295B-parameter MoE model with only 21B active parameters during inference, reducing computational costs.
- It outperforms similar-sized models and rivals larger open-source models by 2-5x in parameter count.
- The model is released under Apache 2.0, enabling broad research and commercial adoption.
- Tencent gathered feedback from 50+ internal products to refine Hy3’s post-training process.
Tencent’s Hy Team has launched Hy3, a groundbreaking Mixture-of-Experts (MoE) model that challenges conventional scaling laws. With a total of 295 billion parameters but only 21 billion active during inference, Hy3 achieves performance comparable to flagship open-source models that are 2 to 5 times larger. The model is released under the Apache 2.0 license, making it freely available for research and commercial use.
The development of Hy3 follows the earlier Hy3 Preview release in late April, during which Tencent gathered feedback from over 50 internal products. This feedback informed post-training improvements, including the use of higher-quality data to refine the model’s capabilities. The result is a model that not only matches but exceeds the performance of similarly sized models while remaining computationally efficient.
Hy3’s architecture leverages the MoE paradigm, where only a subset of parameters is activated for each input, reducing inference costs without sacrificing performance. This approach aligns with the growing trend of optimizing large language models for practical deployment, particularly in resource-constrained environments.
Source: tencent/Hy3. Read the full piece at the source.
Hy3 offers a high-performance, efficient alternative to larger models, enabling cost-effective deployment.
Companies can leverage Hy3 for advanced AI applications without the prohibitive costs of larger models.
Tencent’s release of Hy3 underscores the growing importance of efficient MoE architectures in AI.
Hy3 serves as a practical case study in optimizing large language models for real-world use.
- Mixture-of-Experts (MoE)
- A machine learning architecture where only a subset of model parameters is activated for each input, improving efficiency.
- Active parameters
- The subset of a model’s total parameters that are engaged during inference for a given input.
Anthropic says Claude has carved out its own space to ponder - Axios
![[AINews] The Field Guide to Fable](https://images.weserv.nl/?url=substackcdn.com%2Fimage%2Fyoutube%2Fw_728%2Cc_limit%2F9fubhllmsBU&w=520&fit=cover&q=70&output=webp&dpr=2&we=1&il=1)
LLM[AINews] The Field Guide to Fable
LLMCompose your agent prompts once, compile them to every harness
New model: GigaChat3.5-432B-A28B (with day-0 GGUF support!)
LLMThe LLM API Failure Policy I Wish I Had Before My First Production Incident
The New Playbook for Enterprise AI Contracts - Emerj Artificial Intelligence Research
Emerj Research outlines a new framework for structuring enterprise AI contracts, focusing on risk management, compliance, and vendor accountability.
Police use of artificial intelligence grows as rules lag behind - Macomb Daily
Law enforcement agencies are increasingly deploying AI tools despite a lack of comprehensive regulations to govern their use.
RoboticsBritish Space Startup Launches Longevity Lab Into Orbit
A British space startup launched a lab into orbit to collect data on protein behavior linked to age-related diseases, aiming to train AI models for better predictions.
Business - Samsung Electronics profits surge 1,800% annually amid artificial intelligence spending boom - France 24
Samsung Electronics reported a staggering 1,800% annual profit surge, driven by soaring AI-related chip sales and infrastructure investments.
Stymied datacentre projects threaten global AI revolution - The Guardian
Major data center projects critical for AI growth face delays, threatening the global AI revolution.
Sam Altman Offers a Trojan Horse to American Taxpayers - Bloomberg.com
Sam Altman suggests the U.S. government invest in AI infrastructure to boost national competitiveness, framing it as a strategic public good.