Perplexity AI Releases WANDR: An Open Benchmark Evaluating Research Agents That Must Search Wide And Deep
Perplexity AI has launched WANDR, an open-source benchmark designed to evaluate the depth and accuracy of AI research agents using 500 complex tasks.
- WANDR uses 500 tasks to test research agent depth and breadth.
- The benchmark emphasizes the necessity of cited, re-verifiable evidence.
- Current performance metrics indicate a high level of difficulty for existing research agents.
Perplexity AI has introduced WANDR, a specialized evaluation harness aimed at measuring the performance of autonomous research agents. Unlike standard LLM benchmarks, WANDR focuses on the ability of an agent to perform wide and deep searches to find specific entities and support them with verifiable evidence.
The benchmark consists of 500 evidence-heavy tasks that require agents to navigate complex information landscapes. This approach addresses a critical gap in current AI evaluation, which often fails to account for the iterative search and citation requirements of true research workflows.
Initial results show that Perplexity's own Search as Code implementation achieved a soft F1 score of 0.363 and a hard F1 score of 0.133. These metrics highlight the significant technical challenges involved in ensuring agents can find and validate information accurately.
Provides a standardized way to test and improve autonomous research agent capabilities.
Helps companies validate the reliability of AI agents used for market research or data gathering.
Ensures the AI tools used for academic research are capable of deep, cited investigation.
- F1 score
- A statistical measure that combines precision and recall to evaluate the accuracy of a model.
- Research Agent
- An AI system capable of autonomously performing multi-step searches to fulfill a complex information request.
New AI blood test predicts heart disease 15 years early - ScienceDaily
AI Doesn’t Absolve You of Getting Facts Right the First Time - mindmatters.ai
AI ResearchNonprofit Current AI is racing to build the World Wide Web of AI, free for all
Alibaba previews Qwen3.8, claiming its strength trails only Anthropic’s Fable 5 - South China Morning Post
AI ResearchAlibaba's Qwen takes on Kimi K3 with open-weight Qwen 3.8, says model is "second only to Fable 5"
Tellurian Research Launches AI-Driven Intelligence Platform for Complex and Emerging Markets - EIN News
Tellurian Research has launched an AI-driven intelligence platform for complex and emerging markets. The platform aims to provide insights and analysis for these markets.
Current AI wants to build a free World Wide Web for artificial intelligence, and it has $400 million to start - Crypto Briefing
Current AI, a startup, has secured $400 million to develop a free, decentralized web for artificial intelligence. This initiative aims to provide a platform for AI models to interact and learn from each other.
AI may make you 'boring' at work, Columbia Business School professor says: How it can hurt your career - CNBC
A Columbia Business School professor warns that AI may make workers appear 'boring' at their jobs, potentially hurting their careers. This is due to AI's ability to automate routine tasks, making human workers seem less valuable.
Banks ramp up digital assistants in productivity race - crossroadstoday.com
Banks are investing in digital assistants to improve employee productivity, a trend that could reshape the industry.
Meta sued: AI helped choose whom to lay off, but missed an important detail - The Jerusalem Post
Meta is being sued due to its AI system missing a crucial detail when selecting employees for layoff. The lawsuit claims the AI's decision-making process was flawed.
AI ToolsBuilding AI Agents for Social Media with TypeScript and Hono.js
A tutorial on creating AI agents for social media using TypeScript and Hono.js, taking a step beyond basic LLM integration.