Skip to main content
Results are the durable record of an AgentCompass run. They are designed for both human inspection and later automation: summary regeneration, post-analysis, badcase mining, and result reuse all start from the run directory.

​
Directory Layout

details/*.json files are the source of truth. Summary files are derived views.

​
Artifact Roles

​
Local Result Browser

tools/result-browser provides a local web UI for inspecting completed or in-progress run directories. It is served from a source checkout and requires Node.js and npm on the machine running the UI service. From the repository root:
Open the Vite URL printed by npm run dev, usually http://localhost:5173, then enter the absolute run directory path that contains summary.md and details/, for example /path/to/AgentCompass/results/swebench_verified/your-model/20260703_120000. The entered path is resolved on the machine running npm run dev. If you access the UI through SSH port forwarding or a remote forwarded URL, still enter the server-side absolute path to the run directory. To check the production build locally:

​
Summary vs Analysis

summary is lightweight. It reads existing details and recomputes aggregate benchmark metrics.
analysis is heavier. It re-runs analyzers over task attempts, writes analysis_result.<AnalyzerId> into details, and regenerates analysis summaries.
By default, analysis copies the input run into a new timestamped sibling. Use --override only when you intentionally want to mutate the original run directory.

​
Reading Details

Per-task detail files should contain enough context to debug without rerunning:
  • task id, category, and benchmark metadata;
  • attempt status and correctness;
  • benchmark metric fields;
  • harness prediction and trajectory;
  • environment or evaluation errors;
  • analyzer output under analysis_result.

​
Reuse Behavior

Runtime can materialize and load existing details before executing pending tasks. This is useful when a run was interrupted or when only a subset needs to be recomputed. Use stable output paths, stable task_id values, and avoid deleting details/ if you want reuse and summary regeneration to work.

​
Data And Output Roots

runtime.data_dir and runtime.results_dir are YAML keys in config/defaults.yaml with matching CLI flags. --run-name and --run-id are per-run CLI flags (fields of the RunRequest output spec, not configuration file keys).