Description
autoresearch is an agent skill from the github/awesome-copilot collection for running an autonomous experimentation loop on any programming task that has a measurable outcome. The skill is inspired by Karpathy's autoresearch and generalizes it from machine learning training to any task. The user first defines the goal, the metric that measures it, and the scope constraints that limit what may change. The agent then iterates on its own: it modifies code, runs an experiment, measures the result, and keeps the change if it improved the metric or discards it if it did not, repeating until the user interrupts. The skill sets out agent behavior rules that keep the loop disciplined, such as changing only what the scope allows and recording each experiment so the history of what was tried is clear. It is useful for tuning performance, improving a model's score on a test, or reducing the size of a build, wherever the result can be measured automatically. The loop can run for a long time, so the metric must be trustworthy, and the scope should keep the agent away from code that must not change. Because the agent is changing code without a person reviewing each step, the kept changes should be reviewed before they are merged. The skill is installed with the skills CLI from the github/awesome-copilot repository. Its skills.sh page showed about 3.4K installs and passing results from three security audits at the time of the scan.…
Highlights
- Autonomous modify, run, measure loop
- Keeps changes that improve the metric
- Goal, metric and scope defined by the user
- Behavior rules for disciplined experiments
- Works for any measurable programming task
- Installs with the skills CLI
Ratings & Reviews
No ratings yet
Be the first to rate autoresearch.
People also visit
autoresearch FAQ
What does the loop do?
It modifies code, runs an experiment, measures the result, and keeps or discards the change, repeating until interrupted.
What must I define first?
The goal, how to measure it, and the scope constraints that limit what the agent may change.
What kinds of tasks fit?
Any programming task with an outcome that can be measured automatically.
Should I review the kept changes?
Yes. The agent changes code without reviewing each step, so kept changes should be reviewed before they are merged.
How do I install it?
Run npx skills add https://github.com/github/awesome-copilot --skill autoresearch.
What security audits does it have?
skills.sh lists three audits: Gen Agent Trust Hub Pass, Socket Pass and Snyk Pass.
What are the best autoresearch alternatives?
Popular alternatives to autoresearch include python-performance-optimization, automate-this and github-actions-efficiency. See all autoresearch alternatives