AMD acquires Taalas, a startup that bakes AI models directly into silicon
AMD has acquired Taalas, a Canadian startup that hard-codes AI model weights into inference chips, achieving over 16,000 tokens per second with Llama 3.1-8B.
- AMD acquires Taalas to hard-code AI model weights directly into inference chips, eliminating external memory bottlenecks.
- Demonstration chip achieved over 16,000 tokens per second with Llama 3.1-8B, a significant speed improvement.
- Chips are locked to a single model, reducing flexibility but maximizing performance for specific workloads.
- Google is reportedly working on a similar approach for Gemini, suggesting a broader industry trend.
AMD has announced the acquisition of Taalas, a Canadian startup specializing in hard-coding AI model weights directly into inference chips. This approach eliminates the need for external memory access, significantly boosting inference speeds. A demonstration chip reportedly achieved over 16,000 tokens per second per user while running Llama 3.1-8B, a performance leap compared to traditional architectures.
The technology, however, comes with a trade-off: chips designed this way are locked to a single model, reducing flexibility. This acquisition aligns with AMD's broader strategy to optimize AI workloads on its hardware. Competitors like Google are also exploring similar approaches for their Gemini models, indicating a potential shift in AI chip design toward specialized, high-speed inference solutions.
The move underscores the growing importance of hardware-software co-design in AI, where model optimization and silicon architecture are increasingly intertwined. For developers and businesses, this could mean faster inference times but at the cost of adaptability in dynamic AI environments.
Developers may need to adapt to hardware-locked models, requiring new optimization strategies for inference.
Businesses prioritizing inference speed over flexibility could benefit from this technology.
Investors should watch for shifts in AI chip design and the potential for specialized hardware to dominate niche markets.
This acquisition highlights the growing convergence of AI models and hardware, reshaping how AI systems are deployed.
- inference chips
- Specialized hardware designed to run AI models after training, optimizing for speed and efficiency.
- tokens per second
- A metric measuring the speed at which an AI model processes input and generates output.
HardwareThe ultimate eclipse chase: A Concorde raced against the Moon's shadow
HardwareThe world's biggest solar telescope caught vortexes on the Sun's surface
HardwareJony Ive’s first OpenAI gadget is reportedly a hockey puck-sized smart speaker
HardwareAnthropic will design its own hardware to power Claude
HardwareSpaceX claims Starlink Mobile will be better than AT&T, T-Mobile, and Verizon
OpenAI says it slowed Astra model development over security concerns
OpenAI has temporarily halted parts of its Astra AI model development because of security vulnerabilities. The move reflects growing scrutiny of AI safety in advanced models.
AI ToolsEurope's free satellite service just made it easier to track wildfires
Europe’s free Copernicus Browser now includes wildfire visualization tools, helping authorities and researchers monitor blazes during an intense wildfire season.
In the News: John Abraham Discusses AI Safety Concerns - Newsroom | University of St. Thomas
John Abraham, a prominent AI safety advocate, shares his thoughts on the pressing concerns surrounding AI development in an interview with the University of St. Thomas.
Who is liable when Artificial Intelligence goes rogue? - FOX 29 Philadelphia
The issue of liability when artificial intelligence systems fail or cause harm is becoming increasingly important. Experts are debating who should be held responsible in such cases.
BusinessAfter Rippling blew millions on AI in months, it built an employee ROI tool
Rippling introduces AI Spend Console to monitor AI tool spending by employees and teams, addressing cost inefficiencies after rapid AI adoption.
University of Pennsylvania researchers develop artificial intelligence tool to help speed autism evaluations - 6abc Philadelphia
University of Pennsylvania researchers have developed an artificial intelligence tool to help speed up autism evaluations. The AI tool uses machine learning algorithms to analyze data and provide more accurate results.