Soofi Consortium Releases Soofi S 30B-A3B: An Open Hybrid Mamba-Transformer MoE Foundation Model For German And English
The Soofi Consortium has released Soofi S 30B-A3B, an open-source foundation model combining Mamba and Transformer architectures. It supports German and English, activating 3.2 billion of its 31.6 billion parameters.
- Soofi S 30B-A3B is an open-source hybrid Mamba-Transformer MoE foundation model.
- The model supports German and English languages.
- It features a sparse activation mechanism, using 3.2B of 31.6B parameters.
The Soofi Consortium has introduced Soofi S 30B-A3B, a novel open-source foundation model that merges the strengths of Mamba and Transformer architectures. This hybrid approach aims to leverage the efficiency of Mamba with the established capabilities of Transformers.
The model is designed to support both German and English languages. Notably, it employs a Mixture of Experts (MoE) design, activating only a fraction of its total parameters for any given task. Specifically, it activates 3.2 billion out of its 31.6 billion total parameters, making it a sparse model.
This release contributes to the growing landscape of open-source large language models, offering a new option for researchers and developers working with multilingual capabilities, particularly for German and English.
Provides a new open-source multilingual model option.
Offers potential for cost-effective German and English language AI applications.
Indicates continued innovation in open-source LLM architectures.
A new resource for studying hybrid and sparse model designs.
Expands the availability of specialized open-source AI models.
- Mamba
- A deep learning architecture known for its efficiency in processing long sequences, often compared to Transformers.
- Transformer
- A dominant deep learning architecture in NLP, known for its attention mechanism.
- MoE
- Mixture of Experts, a type of neural network architecture that uses multiple 'expert' sub-networks, activating only a subset for each input.
LLMMeet GPT-Red: an LLM super-hacker OpenAI built to make its models safer
LLMBonsai 27B is a full open reasoning model that fits on an iPhone
Elon Musk promises ‘no exceptions’ after Sam Altman’s one word response on xAI’s Grok ‘uploading’ custome - The Times of India
LLMApple opens its new Siri AI to everyone with the iOS 27 public beta
LLMBonsai 27B: 1-bit dense LLM running locally in your browser using custom WebGPU kernels
Green approves bills that cracks down on artificial intelligence - Hawaii Public Radio
Hawaii State Senator Green has approved bills to regulate artificial intelligence, aiming to curb its potential misuse.
Meta’s AI-based layoffs allegedly targeted workers who had taken protected leave - HR Dive
Meta's AI-based layoffs allegedly targeted workers who had taken protected leave, sparking concerns over the use of AI in hiring decisions.
Claude for Teachers: Anthropic exec. explains why educators get a year for free - Yahoo Finance
Anthropic is providing a free one-year subscription to its Claude AI assistant for educators. This initiative aims to integrate AI into classrooms responsibly.
The American Council of the Blind to Distribute Ray-Ban Meta AI Glasses to Blind Veterans in St. Louis on July 27 - PR Newswire
The American Council of the Blind will distribute Ray-Ban Meta AI Glasses to blind veterans in St. Louis on July 27.
3 Questions: Neural transparency and the future of AI design - MIT News
MIT researchers discuss the importance of neural transparency in AI design, and its potential to improve the future of AI. They explore how transparency can lead to more trustworthy AI systems.
SecurityWindows 0-day drops the same day Microsoft releases record number of patches
A Windows 0-day exploit has been discovered, coinciding with Microsoft's release of a record number of patches. The exploit, known as HiveLegacy, is considered a powerful primitive that could be used for various malicious activities.