
Domain Rating
by Ahrefs
Domain Rating by Ahrefs27
Description
FrontierHarness Eval is a public benchmark that compares AI coding agent harnesses while keeping the model fixed. It tested 9 harnesses in 12 configurations, among them Codex, Claude Code, Pi, Oh My Pi, Kimi Code, OpenCode, Hermes, Exo Harness and several DSH variants, on the same software engineering tasks. Every run used the Kimi K3 model on Runta, starting from an identical fresh checkpoint with the same CPU, memory and disk, for 360 trials in all. Results are shown for pass rate, median cost per task and per successful task, cache hit rate and time. The headline finding is that pass rates were close, roughly 50 to 67 percent, while cost per success differed by about 17 times. The evaluation code and a skill for testing your own harness are on GitHub.…
Highlights
- Compares 9 harnesses in 12 configurations
- Same model, tasks and hardware for every run
- Pass rate, cost, cache hit rate and time metrics
- 360 trials from identical cold starts
- Open evaluation code to test your own harness
Ratings & Reviews
No ratings yet
Be the first to rate FrontierHarness Eval.
People also visit

Learn Claude Code
by shareAI-lab - A really interesting analysis of how coding agents like Claude
FrontierHarness Eval FAQ
What does FrontierHarness Eval measure?
How different coding agent harnesses perform on identical software engineering tasks using the same model, comparing pass rate, cost, cache hit rate and time.
Which harnesses were tested?
Codex, DSH Creator, Claude Code, Pi, DSH PTC, DSH Standard, Oh My Pi, Kimi Code, DSH Minimal, Exo Harness, OpenCode and Hermes.
How was fairness ensured?
Every run used Kimi K3 on Runta with a fresh checkpoint restore and identical vCPU, memory, disk and memory state, and formal tasks were never run early to avoid warm-cache bias.
Can I evaluate my own harness?
Yes. The evaluation repository and a skill for running it are on GitHub at github.com/frontier-harness-eval/eval, and you can also run it by hand.
What are the best FrontierHarness Eval alternatives?
Popular alternatives to FrontierHarness Eval include Frugal Tokens. See all FrontierHarness Eval alternatives







