Agents CLI: Agent Evaluation
Part of Google's agents-cli skills suite. Documents the Agent Platform's evaluation methodology: preparing an EvaluationDataset (single-turn, multi-turn, and multi-agent JSON schemas), running `agents-cli eval run` to execute the agent over the dataset and grade traces, analyzing failed metrics, and iterating via prompt/tool/instruction fixes. Also covers opt-in commands for user-simulated multi-turn datasets, LLM-based failure clustering, and ADK GEPA prompt optimization. Applies to any agents-cli project regardless of framework. Does not cover ADK API code patterns (google-agents-cli-adk-code), deployment (google-agents-cli-deploy), or scaffolding (google-agents-cli-scaffold).
This skill hasn't been reviewed yet.
What it does
Evaluation methodology for agents built with Google's agents-cli — dataset schema, built-in metrics, LLM-as-judge scoring, and the failure-analysis loop (the Quality Flywheel).
How to use this
Once it is installed, this skill picks itself up. You do not need to name it.
- Created by
- Google LLC
- Category
- Engineering
- Source
- View source
Run this on Skills and Agents
You can run the skill on Skills and Agents by creating an account. You don't need to install anything. We provide the model and tokens. Your connected tools and company brain are ready to use.
Skills give you best practices and templates to get the most out of your AI
Add this skill
Pick where you want it. Nothing to configure after.
These install links go live once the signup flow ships. Until then, ask Claude, ChatGPT, or Gemini to fetch this skill's file directly from GitHub.
Not sure which? See the install guide.
Share this skill
Agents CLI: Agent Evaluation
Part of Google's agents-cli skills suite. Documents the Agent Platform's evaluation methodology: preparing an EvaluationDataset (single-turn, multi-turn, and multi-agent JSON schemas), running `agents-cli eval run` to execute the agent over the dataset and grade traces, analyzing failed metrics, and iterating via prompt/tool/instruction fixes. Also covers opt-in commands for user-simulated multi-turn datasets, LLM-based failure clustering, and ADK GEPA prompt optimization. Applies to any agents-cli project regardless of framework. Does not cover ADK API code patterns (google-agents-cli-adk-code), deployment (google-agents-cli-deploy), or scaffolding (google-agents-cli-scaffold).
Third-party, by Google LLC
September 13, 2026
Install from google/agents-cli
This skill is one of 7 bundled in Google's agents-cli repo (skills/google-agents-cli-eval). Installing pulls all 7 skills in the repo — there's no way to grab just this one.
- 1
Install all 7 agents-cli skills
Installs this skill along with the other 6 (workflow, adk-code, scaffold, deploy, publish, observability) from google/agents-cli's skills/ folder.
npx skills add google/agents-cli - 2
Install the CLI itself (required to run eval commands)
The eval commands (eval run, eval generate, eval grade) come from the CLI, not the skill text alone.
uv tool install google-agents-cli - 3
Activate in Claude
Ask to run an evaluation, write an eval dataset, or analyze why an eval is failing.
Your skill is downloading.
Create a free account and the platform does the rest:
- Get an email the moment this skill updates
- See every skill and agent you've installed, in one place
- Get your next recommendation based on what you actually use
No credit card. Takes about a minute.
What this skill does
This is an overview of the skill from the site, not the skill's real instructions. The full SKILL.md Claude reads when this skill is active lives on GitHub at the source below.
A skill from Google’s
google/agents-cli. The canonical source — including the CLI, the other 6 skills, and release notes — lives in that repo.
What this skill does
Documents the “Quality Flywheel” for improving agent quality: prepare data, run the eval, analyze failures, optimize and fix. Each stage has a default path (do the work directly) and an opt-in CLI command that delegates to the Agent Platform Eval Service for more scale.
The loop
- Prepare data — start from the scaffolded
tests/eval/datasets/basic-dataset.json, or opt in toagents-cli eval dataset synthesizeto user-simulate multi-turn datasets when you lack real data. - Run the eval —
agents-cli eval runruns the agent over the dataset and grades the traces, writingresults_<ts>.{json,html}toartifacts/grade_results/. Can be decoupled intoeval generate+eval gradefor custom trace locations or re-grading without re-running the agent. - Analyze failures — open the results HTML/JSON and check failed metrics against a failure-to-fix mapping, or opt in to
agents-cli eval analyzefor LLM-based failure clustering once you have 10+ failing cases. - Optimize and fix — edit prompts, tool descriptions, or instructions based on the failure analysis.
agents-cli eval optimizeruns ADK GEPA prompt optimization for prompt-only failures, but the skill is explicit that this is long-running and expensive: exhaust manual fixes first, then run one final optimization pass rather than looping on it.
Reference files it points to
Dataset schema (single-turn/multi-turn/multi-agent JSON shapes), a complete metrics guide (built-in metrics, match types, judge model config), user-simulation flags, built-in-tools eval behavior, advanced opt-in commands, and multimodal eval handling.
Scope
Framework-agnostic — applies to any agents-cli project. Does not cover ADK API code patterns (google-agents-cli-adk-code), deployment (google-agents-cli-deploy), or project scaffolding (google-agents-cli-scaffold).