Global Benchmark Ecosystem

Foundational Benchmarks

AgentV provides first-class support for the world’s most rigorous academic and industrial benchmarks. By using native URI schemes, researchers can execute standardized evaluations without manual data munging or environment setup.

Supported Benchmarks

GAIA

General AI Assistants
Benchmark for multi-hop reasoning and real-world tool use. View Guide

AssistantBench

Enterprise Intelligence
Specialized for business reasoning and industrial accuracy. View Guide

OpenCompass

Coming Soon
Native support for OpenCompass integration is currently in the v1.5.x roadmap.

---

The URI Schema Advantage

AgentV abstracts benchmark loading behind the Unified Benchmark URI Schema. This allows researchers to target specific datasets, versions, and difficultly tiers directly from the CLI.

How it works:

# General syntax
agentv evaluate --path <benchmark_scheme>://<version> --agent <agent_url>

# Examples
agentv evaluate --path gaia://2025_validation --agent http://localhost:5001/execute_task
agentv evaluate --path assistantbench://v1_standard --agent http://localhost:5001/execute_task

Benefits for Researchers:

1. Zero-Munge Extraction: AgentV automatically maps benchmark-specific fields (e.g., GAIA’s Correct Answer) to high-fidelity AES Success Criteria.
2. Version Consistency: By pinning the URI (e.g., gaia://2023), researchers ensure they are always evaluating against a specific, immutable dataset snapshot.
3. Cross-Bench Comparisons: Because all benchmarks are normalized to the AgentV Forensic Ledger, you can compare results from different benchmarks in a single Comparative Leaderboard.