How I tricked Claude into leaking your deepest, darkest secrets
A security researcher discovered a flaw in Claude's web_fetch tool that allows the AI to leak user memories to external websites.
- Claude's web_fetch tool has a vulnerability allowing memory leaks.
- The exploit bypasses guardrails designed to stop data exfiltration.
- It highlights the difficulty of securing AI agents with web access.
Simon Willison highlighted a discovery by Ayush Paul regarding a security vulnerability in Anthropic's Claude. The issue specifically targets the web_fetch tool, which was intended to prevent data exfiltration attacks by limiting how the model interacts with external URLs.
The exploit allows an attacker to trick Claude into sending its stored memories to an external server. This bypasses the safety guardrails meant to protect private user data during web browsing tasks.
This finding demonstrates the ongoing risks associated with the lethal trifecta in AI systems. This occurs when models possess private data while interacting with untrusted inputs and outputs.
Need to audit tool usage and permissions for LLM agents.
Risk of proprietary data leakage via AI memory features.
Highlights privacy risks in advanced AI chatbots.
- Data Exfiltration
- The unauthorized transfer of data from a computer or system.
- Lethal Trifecta
- A security risk where AI has private data, untrusted input, and untrusted output.
SecurityThe Three-Second Theft: Why AI Voice Fraud Outruns Every Defence
Microsoft Copilot: Websites could secretly issue commands to the AI - Notebookcheck
SecurityStratagems #14: Leo Found an AI Leak. He Wasn't the First to Find It.
SecurityCursor 0day: When Full Disclosure Becomes the Only Protection Left
The Agentic Enterprise Has a Privilege Problem - Dark Reading
BusinessApple Intelligence approved for launch in China with Alibaba’s Qwen AI
Apple's AI services have been approved for launch in China through a partnership with Alibaba, bringing Alibaba's Qwen AI models to Apple's operating systems.
Apple Intelligence Turns To Alibaba’s Qwen In China Even As Apple Is Likely To Reserve Its Private Compute ‘Secret Sauce’ For Itself - Wccftech
Apple has partnered with Alibaba's Qwen in China for AI services, but is likely to reserve its own private compute technology for internal use.
Here’s How To Calculate Your ROI For An AI Agent - Forbes
Forbes explains how to calculate the return on investment for an AI agent, providing a step-by-step guide for businesses.
AI Model Predicts 348 Diseases from Electronic Health Record, Genetics - Inside Precision Medicine
A new AI model can predict 348 diseases using electronic health records and genetic data. This breakthrough has significant implications for precision medicine.
Apple looking for AI chip acquisitions: report (AAPL:NASDAQ) - Seeking Alpha
Apple is reportedly seeking acquisitions of AI chip startups to bolster its hardware capabilities and catch up in the AI race.
AI ToolsI let ChatGPT Work and Claude Cowork loose on my files - only one made me nervous
ChatGPT Work and Claude Cowork were compared for desktop automation, showing similar results and strengths. However, Claude feels safer due to a major reason.