Setup

Configure Model Access

Start a local GLM-5.2 endpoint with an OpenAI-compatible chat API, then export its connection settings. Use valid shell variable names with underscores:

export MODEL_BASE_URL="http://localhost:8000/v1"
export MODEL_API_KEY="EMPTY"

Use the real key instead of EMPTY if your local server enforces authentication. The examples below pass these values into each run and use the openai-chat API protocol.

agentcompass run \
  swebench_verified \
  mini_swe_agent \
  glm-5.2 \
  --env docker \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat \
  --task-concurrency 1

Check SWE-bench Dependencies

SWE-bench Verified and mini-SWE-agent use optional dependencies that are not part of the minimal install. Install them in the active environment before running the quick start:

uv pip install -r requirements/swe.txt
uv pip install -r requirements/mini-swe-agent.txt

The Docker quick start also needs a working Docker daemon and access to pull the SWE-bench task image inferred by the recipe:

docker version

Recommended Config Setup

Keep secrets in environment variables, and use YAML config files for defaults you want to reuse. agentcompass config show prints YAML by default. Without component selectors it prints the global runtime and execution sections, which is useful for user-level or project-level defaults:

mkdir -p ~/.config/agentcompass

agentcompass config show > ~/.config/agentcompass/config.yaml

Project-level defaults can live in <repo-root>/config.yaml:

agentcompass config show > config.yaml

For a run-specific editable template, pass the components you plan to use. The selected benchmark, harness, and environment defaults are included in the YAML:

mkdir -p configs

agentcompass config show --benchmark swebench_verified --harness mini_swe_agent --env docker > configs/swebench-glm52-docker.yaml

Apply that file when starting the run:

agentcompass run \
  swebench_verified \
  mini_swe_agent \
  glm-5.2 \
  --config configs/swebench-glm52-docker.yaml \
  --env docker \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat \
  --task-concurrency 1

Use agentcompass config docs <kind> <component-id> to inspect fields before editing a component block:

agentcompass config docs benchmark swebench_verified
agentcompass config docs harness mini_swe_agent
agentcompass config docs env docker

Quick Start

Start with one SWE-bench Verified instance before increasing concurrency or removing sample_ids.

Docker Smoke Test

Run one SWE-bench Verified task with mini-SWE-agent in Docker against your local GLM-5.2 endpoint:

agentcompass run \
  swebench_verified \
  mini_swe_agent \
  glm-5.2 \
  --env docker \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat \
  --model-params '{"temperature":0}' \
  --task-concurrency 1

Remote Sandbox

Remote sandboxes require provider credentials. They are the preferred path for heavier terminal and coding benchmarks because recipes can select task images and workspace roots from benchmark metadata.

Daytona: Single SWE-bench Verified Instance

Use Daytona to run the same SWE-bench smoke test without consuming local Docker compute:

export DAYTONA_API_KEY="..."

agentcompass run \
  swebench_verified \
  mini_swe_agent \
  glm-5.2 \
  --env daytona \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY"

This is a good provider smoke test: sample_ids limits the run to one task, and the Daytona recipe can infer the SWE-bench task image and workspace from metadata. Use Modal to run the complete SWE-bench Verified benchmark with concurrent tasks:

export MODAL_TOKEN_ID="..."
export MODAL_TOKEN_SECRET="..."

agentcompass run \
  swebench_verified \
  mini_swe_agent \
  glm-5.2 \
  --env modal \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --task-concurrency 32

Because this command does not set sample_ids, it runs the full benchmark split. Lower --task-concurrency if your model endpoint or sandbox provider has tighter rate limits.

For benchmark/provider combinations with recipes, image and workspace root are usually inferred from task metadata. You do not need to pass image in --env-params unless you intentionally want to override the recipe.

Model API setup

Learn protocol and judge-model configuration.

Environment selection

Choose local, Docker, Daytona, Modal, or cluster-backed execution.

Get Started

Key Modules

Environments

Cookbooks

Configure Model Access

Check SWE-bench Dependencies

Recommended Config Setup

Quick Start

Docker Smoke Test

Remote Sandbox

Daytona: Single SWE-bench Verified Instance

Model API setup

Environment selection

​Configure Model Access

​Check SWE-bench Dependencies

​Recommended Config Setup

​Quick Start

​Docker Smoke Test

​Remote Sandbox

​Daytona: Single SWE-bench Verified Instance

​Modal: Full SWE-bench Verified With Concurrency

Model API setup

Environment selection

Configure Model Access

Check SWE-bench Dependencies

Recommended Config Setup

Quick Start

Docker Smoke Test

Remote Sandbox

Daytona: Single SWE-bench Verified Instance

Modal: Full SWE-bench Verified With Concurrency