Description
agent-eval is a benchmark skill for CodeGraph, the code intelligence tool. It measures how much CodeGraph helps an agent compared with plain grep and file reading, for a chosen CodeGraph version on a chosen real-world repository. It runs the harness in the repository's scripts/agent-eval folder, so the comparison is made the same way each time. The skill is for maintainers who need evidence before they claim that a CodeGraph change improves retrieval, and for developers who want to decide whether CodeGraph is worth using on their own codebase. It can be started with the agent-eval command or by asking for a quality audit of CodeGraph. The prerequisites are tmux 3 or newer, a logged-in claude command-line client, and node and git on macOS or Linux, and it must be run from the CodeGraph repository root. Because the benchmark starts agent sessions and reads a real repository, it takes time and uses model usage, so pick the repository and version on purpose. The skill's security audit shows a warning from Gen Agent Trust Hub and passes Socket and Snyk on skills.sh. It has about 1,700 installs. Install it with npx skills add https://github.com/colbymchenry/codegraph --skill agent-eval.…
Highlights
- Compares agent behavior with and without CodeGraph
- Runs against a chosen CodeGraph version and repository
- Uses the scripts/agent-eval harness in the repo
- Requires tmux 3 or newer and a logged-in claude CLI
Ratings & Reviews
No ratings yet
Be the first to rate agent-eval.
People also visit
agent-eval FAQ
What does the audit measure?
How much CodeGraph helps an agent compared with plain grep and read, on a chosen version and a real repository.
What do I need?
tmux 3 or newer, a logged-in claude CLI, node and git on macOS or Linux, run from the CodeGraph repository root.
Does it cost model usage?
Yes. The benchmark starts agent sessions on a real repository, so it takes time and uses model usage.
How do I install it?
Run npx skills add https://github.com/colbymchenry/codegraph --skill agent-eval.
Which security audits does it have?
On skills.sh, Gen Agent Trust Hub shows a warning, and Socket and Snyk pass.
What are the best agent-eval alternatives?
Popular alternatives to agent-eval include agent-eval, dynamic-workflow-mode and eval-harness. See all agent-eval alternatives