eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session
by alirezarezvaniFree
Description
Eval is an agent skill that ranks the results of agents working in an AgentHub session, so you can pick a winner. It supports three modes: a metric-based evaluation that runs a command you configure in each agent's worktree, an LLM judge that compares the diffs the agents produced, and a hybrid of the two. You can evaluate the latest session, a specific session by its ID, or force the LLM judge mode even if a metric is configured. The metric mode runs the evaluation command in each agent's worktree and scores the results against the criteria, while the judge mode reads the changes and compares them. The skill is triggered by the hub eval command, or when you ask to score, compare or pick a winner among agent results. It is useful when several agents have attempted the same task and you want an objective ranking before you merge one of them. It is part of the claude-skills collection by alirezarezvani and is listed on skills.sh, where its audit shows a warning from Gen Agent Trust Hub.…
Highlights
- Metric mode runs your evaluation command in each worktree
- LLM judge mode compares the diffs the agents produced
- Hybrid mode combines the metric and the judge
- Evaluates the latest session or a chosen session ID
Ratings & Reviews
No ratings yet
Be the first to rate eval.
People also visit
benchmark-optimization-loop
Convert 'make it faster' requests into a bounded measured optimization loop
eval FAQ
What does eval do?
It ranks the results of agents in an AgentHub session by a metric, an LLM judge, or both.
How does metric mode work?
It runs the evaluation command you configured in each agent's worktree and ranks the results.
When would I use the judge?
When you want the diffs compared directly, or when you force judge mode with the judge option.
Who is it for?
Developers who run several agents on the same task and need to choose the best result.
How do I install it?
Run npx skills add https://github.com/alirezarezvani/claude-skills --skill eval in your terminal.
Is it free?
The listing shows the skill as free.
What security audits has it passed?
skills.sh lists three audits: Gen Agent Trust Hub Warn, Socket Pass and Snyk Pass. A Warn or Fail is worth reading before you install.
What are the best eval alternatives?
Popular alternatives to eval include merge, spawn and github. See all eval alternatives