Alternatives to eval-harness-first
50 AI Skills apps to use instead, best rated first.
Top picks
- W
webapp-testing
LaunchNative Python Playwright scripts for testing local web applications with server lifecycle management. Includes with_server.py helper to automatically start and manage single or multiple servers before running automation scripts. Supports re
- C
content-research-writer
LaunchAssists in writing high-quality content by conducting research, adding citations, improving hooks, iterating on outlines, and providing real-time feedback on
- O
openhue
Launchopenhue is an agent skill from the OpenClaw project, listed on skills.sh. It lets an agent control Philips Hue lights and scenes through a Hue Bridge with the OpenHue CLI: turn lights on and off, dim rooms, set scenes such as movie mode, an
- B
batch-grill-me
Launchbatch-grill-me is an agent skill from Matt Pocock's skills collection, listed on skills.sh. It interviews you relentlessly about a plan until you share an understanding. It maps decisions as a tree and asks every question it can answer now
- I
implement
Launchimplement is an agent skill from Matt Pocock's skills collection, listed on skills.sh. It implements work from a spec or tickets: it fetches the ticket, works test-first at agreed seams, runs type checks and tests as it goes, reviews the re
- F
frontend-design
LaunchDistinctive, production-grade frontend interfaces that reject generic AI aesthetics through intentional design choices. Guides aesthetic direction selection (brutalist, maximalist, retro-futuristic, luxury, organic, etc.) before implementat
More AI Skills apps
brainstorming
Transform rough ideas into fully-formed designs through structured questioning
finishing-a-development-branch
Guides completion of development work by presenting clear options
test-driven-development
Use when implementing any feature or bugfix, before writing implementation code
About eval-harness-first
eval-harness-first
Build the evaluation harness that gates every fine-tuning run
eval-harness-first is an agent skill from the wshobson/agents collection for building the evaluation harness that gates a fine-tuning run. It is the first phase of a fine-tuning workflow: the other skills in that sequence assume the harness exists before any training configuration is written. The harness is treated as more than a report at the end of a run. It is the engine for data curation, because the same labeled traces that build the golden test sets can feed the training data, with an explicit holdout kept back for testing. The skill covers golden sets of expected behavior, graders that check each failure mode separately, calibration of an LLM judge against human labels, and baselines measured on the base model before any fine-tuning, so a later result can be compared with where the model started. Its inputs are production or agent traces when they exist, or a task specification when they do not, plus labelers willing to grade at least 100 examples. Its output is an eval directory with goldens, graders, a drift suite and the base-model baseline that later phases check against. It is installed with the skills CLI from the wshobson/agents repository. Its skills.sh page showed about 2.4K installs and passing results from three security audits at the time of the scan.
- Price
- Free
- Made by
- wshobson
- Category
- AI Skills
- Website
- github.com/wshobson/agents/eval-harness-first