Open source alternatives to eval-harness-first
24 open source AI Skills apps to use instead, best rated first.
Top picks
- G
github-ops
Launchgithub-ops is an agent skill from the ECC collection by affaan-m, listed on skills.sh. It manages a GitHub repository with a focus on community health, CI reliability and the experience of contributors. It covers triaging issues by classify
Open source
- O
opensource-pipeline
Launchopensource-pipeline is an agent skill from the ECC collection by affaan-m, listed on skills.sh. It prepares a private project for safe public release through a three-stage pipeline. The fork stage strips secrets from a copy. The sanitise st
Open source
- A
andrej-karpathy-skills
Launchandrej-karpathy-skills is a single CLAUDE.md file from the multica-ai project that is meant to improve how Claude Code behaves on coding tasks. It is derived from Andrej Karpathy's public observations about LLM coding pitfalls. The file dis
Open source
- M
microsoft/markitdown
Launchmarkitdown-mcp is the MCP server package for MarkItDown, Microsoft's tool for converting many file formats to Markdown for language models. It is a lightweight server that runs over stdio, Streamable HTTP or SSE, and exposes one tool, conve
Open source
- O
od-contribute
Launchod-contribute is an agent skill in the OpenDesign project that provides a guided, one-click contribution flow to the OpenDesign repository, written so that people without a coding background can send a real pull request. It is locked to the
Open source
- G
github-deep-research
Launchgithub-deep-research is an agent skill from the DeerFlow project that researches a single GitHub repository in several rounds and writes a report in Markdown. It is for requests that need more than a README summary, such as a comprehensive
Open source
More AI Skills apps
modelcontextprotocol/server-memory
Knowledge graph-based persistent memory for maintaining context
netdata/netdata#Netdata
Discovery, exploration, reporting and root cause analysis over Netdata metrics
Learn Claude Code
by shareAI-lab - A really interesting analysis of how coding agents like Claude
About eval-harness-first
eval-harness-first
Build the evaluation harness that gates every fine-tuning run
eval-harness-first is an agent skill from the wshobson/agents collection for building the evaluation harness that gates a fine-tuning run. It is the first phase of a fine-tuning workflow: the other skills in that sequence assume the harness exists before any training configuration is written. The harness is treated as more than a report at the end of a run. It is the engine for data curation, because the same labeled traces that build the golden test sets can feed the training data, with an explicit holdout kept back for testing. The skill covers golden sets of expected behavior, graders that check each failure mode separately, calibration of an LLM judge against human labels, and baselines measured on the base model before any fine-tuning, so a later result can be compared with where the model started. Its inputs are production or agent traces when they exist, or a task specification when they do not, plus labelers willing to grade at least 100 examples. Its output is an eval directory with goldens, graders, a drift suite and the base-model baseline that later phases check against. It is installed with the skills CLI from the wshobson/agents repository. Its skills.sh page showed about 2.4K installs and passing results from three security audits at the time of the scan.
- Price
- Free
- Made by
- wshobson
- Category
- AI Skills
- Website
- github.com/wshobson/agents/eval-harness-first