Benchmarks Overview - AgentCompass Documentation

AgentCompass lists benchmark reference pages alphabetically. Capability labels are metadata tags rather than sidebar categories because many benchmarks touch more than one evaluation dimension.

Registered Benchmarks

The public reference currently covers 22 benchmark entries.

Benchmark	Tags	Typical execution
`browsecomp`	`Deep Research`, `Web Browsing`, `Judge-Scored`	local with judge
`browsecomp_zh`	`Deep Research`, `Web Browsing`, `Chinese`, `Judge-Scored`	local with judge
`deepsearchqa`	`Deep Research`, `Search QA`, `Judge-Scored`	local with judge
`frontierscience`	`Deep Research`, `Science`, `Judge-Scored`	local with judge
`gaia`	`Deep Research`, `Tool Use`, `File Reasoning`, `General Assistant`	local with judge
`gdpval_ac`	`Productivity`, `Long-Horizon`, `Judge-Scored`	remote environment
`hle`	`Deep Research`, `Reasoning`, `Judge-Scored`	local with judge
`hle_verified`	`Deep Research`, `Reasoning`, `Verified`, `Judge-Scored`	local with judge
`pinchbench`	`Productivity`, `OpenClaw`, `Agentic Coding`	remote environment
`researchclawbench`	`Deep Research`, `Scientific Research`, `Workspace`, `Agentic Coding`	local or remote workspace
`scicode`	`Agentic Coding`, `Scientific Coding`, `Tool Use`	local
`screenspot`	`GUI Grounding`, `Vision`	local
`sgi_deep_research`	`Deep Research`, `Scientific Research`, `Judge-Scored`	local with judge
`skillsbench`	`Terminal`, `Skills`, `Productivity`, `Agentic Coding`	remote environment
`swebench_multilingual`	`Agentic Coding`, `Repository Repair`, `Multilingual`	remote environment
`swebench_pro`	`Agentic Coding`, `Repository Repair`, `Long-Horizon`	remote environment
`swebench_verified`	`Agentic Coding`, `Repository Repair`, `Verified`	remote environment
`taubench`	`Tool Use`, `Conversational`, `User Simulation`, `Judge-Scored`	local (in-process)
`terminal_bench_2`	`Terminal`, `Tool Use`	remote environment
`terminal_bench_2_1`	`Terminal`, `Tool Use`	remote environment
`terminal_bench_2_verified`	`Terminal`, `Tool Use`, `Verified`	remote environment
`wildclawbench`	`Productivity`, `OpenClaw`, `Long-Horizon`, `Tool Use`	container workspace

Runtime Controls

Shared benchmark params include sample_ids, category, k, avgk, and resume where supported. Use recipes for heavyweight benchmarks so task images and workspace roots come from task metadata instead of CLI overrides.

​Registered Benchmarks

​Runtime Controls

Registered Benchmarks

Runtime Controls