bmad-eval
Runs the evals of a BMad skill through the agent harness the user works in and
by bmad-code-orgFree
Description
BMad Eval runs the evals of a BMad skill and reports what they show. It works through the agent harness you already use, and covers four modes. Baseline compares the skill with the bare model on the same input. Variant checks whether a section of the skill earns its place, or whether a stripped version does as well. Quality grades an output against a named rubric. Trigger checks that the skill's description fires on the right queries and stays quiet on the rest. Cases are stored in the skill's evals folder as an input, a rubric and optional fixture files, and trigger queries as labelled examples. The skill is built to report honestly: it cites specific findings, flags evals that pass for trivial reasons, and does not widen a tolerance to make a run look successful. It can also improve a skill's description across rounds. A skill-creator evals file can be converted into the BMad case format. It is part of the BMad Method project on GitHub, which has about 54,000 stars. The listing shows it as free. Its security audits show a pass from Gen Agent Trust Hub, and warnings from Socket and Snyk.…
Highlights
- Baseline runs against the bare model
- Variant tests whether a section earns its place
- Quality grades output against a rubric
- Trigger tests the skill description
- Converts skill-creator eval files
Ratings & Reviews
No ratings yet
Be the first to rate bmad-eval.
People also visit
bmad-correct-course
Assess the impact of a significant change during sprint execution across the
bmad-eval FAQ
What is BMad Eval for?
Testing a BMad skill. It runs the skill's evals and reports whether the skill beats the bare model, whether each section is needed, whether outputs meet a rubric, and whether the description triggers correctly.
What are the four modes?
Baseline, variant, quality and trigger. Baseline compares against the bare model, variant tests sections, quality grades against a rubric, and trigger tests when the description fires.
Where are the cases stored?
In the skill's evals folder. Cases are input, rubric and optional fixture files, and trigger queries are stored as labelled examples.
Can I convert an existing eval file?
Yes. A skill-creator evals.json can be converted into the BMad case format, and converted back if needed.
How do I install it?
Run npx skills add https://github.com/bmad-code-org/bmad-method --skill bmad-eval.
Is it free and checked for security?
The listing shows it as free. Its security audits show a pass from Gen Agent Trust Hub, and warnings from Socket and Snyk.
What are the best bmad-eval alternatives?
Popular alternatives to bmad-eval include eval-harness, bmad-build and bmad-code-review. See all bmad-eval alternatives