google-agents-cli-eval
Evaluate ADK agents with metrics, evalsets, and the iterative eval-fix loop
by googleFree
Description
google-agents-cli-eval is an agent skill from Google's agents-cli project for evaluating Agent Development Kit (ADK) agents. It covers metrics, evaluation sets and an iterative eval-and-fix loop, in which the agent runs an evaluation, reads the results, changes the agent and runs it again. Evaluations run with the agents-cli eval run command, which chains generation and grading. The criteria are configurable and include tool trajectory, response matching, rubric-based scoring, hallucination detection and safety checks. Match types are EXACT, IN_ORDER and ANY_ORDER. The skill has references for the canonical evaluation dataset schema, with examples for single-turn, multi-turn and multi-agent cases and a list of common mistakes; a complete metrics guide that includes custom metrics and judge model configuration; user simulation for dynamic conversation testing; how built-in tools such as Google Search behave in evaluations; multimodal inputs; and live and voice agents. If the project was scaffolded with the scaffold skill, a dataset and a custom metric already exist in the tests/eval folder for Python projects.…
Highlights
- Metrics for tools, responses, rubrics and safety
- Eval run chains generation and grading
- Dataset schema and user simulation
- Multimodal and live agent evaluation
Ratings & Reviews
No ratings yet
Be the first to rate google-agents-cli-eval.
People also visit
skill-comply
Visualize whether skills, rules, and agent definitions are actually followed
google-agents-cli-adk-code
Quick reference for ADK Python patterns: agents, tools, callbacks
chief-of-staff
Pursue a long-running goal in a single session by co-ordinating subagents
google-agents-cli-eval FAQ
What does the google-agents-cli-eval skill do?
It helps evaluate an ADK agent with metrics and evaluation sets, and runs an iterative loop of evaluating and fixing the agent.
How do I install it?
Run: npx skills add https://github.com/google/agents-cli --skill google-agents-cli-eval
How are evaluations run?
With agents-cli eval run, which chains generation and grading using the configured criteria.
Which match types does it use?
EXACT, IN_ORDER and ANY_ORDER.
What security checks has it passed?
skills.sh lists three audits: Gen Agent Trust Hub Pass, Socket Pass and Snyk Pass.
What are the best google-agents-cli-eval alternatives?
Popular alternatives to google-agents-cli-eval include agent-eval, eval-harness and skill-comply. See all google-agents-cli-eval alternatives