MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agents
Researchers released MM-ToolSandBox, a benchmark framework for evaluating agents that use tools based on visual inputs across 16 domains.
- MM-ToolSandBox offers a benchmark for visual tool-calling agents.
- It includes 500+ tools across 16 domains in a stateful environment.
- The framework handles multi-image, multi-turn tasks with state mutations.
- An automated pipeline creates diverse evaluation scenarios.
A new benchmark called MM-ToolSandBox has been introduced to evaluate visual tool-calling agents. It provides a stateful environment with over 500 tools spanning 16 different application domains.
The system supports complex multi-image and multi-turn interactions where agents must ground visual inputs into executable tool calls. It handles realistic conversational elements like goal revisions, error corrections, and state mutations.
An automated pipeline generates diverse scenarios using information-flow-guided planning. This allows for rigorous testing of how well agents can interpret visual data to perform actions over time.
Provides a standard to test agentic systems that use vision.
Helps assess the reliability of AI agents in visual workflows.
- Visual Tool-Calling
- The ability of an AI agent to interpret visual data and trigger specific software functions or tools based on that input.
RouteRec: Strict Evaluation of Recommender-Agent Selection and Aggregation
Alan Turing's biggest AI assumption may have been wrong - ScienceDaily
UPDATE: Fresno State creates minor in artificial intelligence - EdSource
09 Radiologists’ and Ultrasound Artificial Intelligence Decision-Support Assessment of Benign and Malignant Cystic Breast Lesions - CancerNetwork
Editorial: AI needs chemistry for better or worse - Chemical & Engineering News
Mistral AI Releases Robostral Navigate: An 8B Model Enabling Robots to Navigate Complex Environments Using a Single RGB Camera
Mistral AI introduced Robostral Navigate, an 8B embodied navigation model. It moves robots from a plain-language instruction using only a single RGB camera, with no LiDAR or depth sensors. The model reaches 76.6% success on R2R-CE validation unseen through a pointing method, prefix-caching training, and CISPO online reinforcement learning. The post Mistral AI Releases Robostral Navigate: An 8B Model Enabling Robots to Navigate Complex Environments Using a Single RGB Camera appeared first on MarkTechPost.
Google Delays Gemini 3.5 Pro Over Early Performance Flaws - Geeky Gadgets
Google has reportedly delayed the release of its Gemini 3.5 Pro model, citing early performance issues. This postponement impacts the anticipated update to its advanced AI capabilities.
To AI-Proof Lawyers, Some Law Schools Restrict Technology - Inside Higher Ed
Some law schools are implementing restrictions on technology use during exams and assignments to ensure students develop fundamental legal skills, anticipating the transformative impact of AI on the legal profession.
I tried Google Maps' new 3D Immersive View for Android Auto, and it fixed my biggest navigation problems
The update is the biggest visual change in a decade, and it makes navigating much easier.
CRS finds Trump's frontier AI order relies on voluntary industry participation, leaves definitions and funding unresolved - Industrial Cyber
CRS finds Trump's frontier AI order relies on voluntary industry participation, leaves definitions and funding unresolved Industrial Cyber
OpenAI CEO Sam Altman Throws Shade At Anthropic's New Ad Amid Tussle With Elon Musk, Apple: 'Thought This Was Satire' - Yahoo Finance
OpenAI CEO Sam Altman Throws Shade At Anthropic's New Ad Amid Tussle With Elon Musk, Apple: 'Thought This Was Satire' Yahoo Finance