Skip to content
store
preference-optimization

Self-hosted alternatives to preference-optimization

5 self-hosted AI Skills apps to use instead, best rated first.

Top picks

  1. C

    claude-to-deerflow

    Launch

    claude-to-deerflow is an agent skill that lets an assistant send messages and questions to a running DeerFlow instance through its HTTP API. DeerFlow is an AI agent platform built on LangGraph that orchestrates sub-agents for research, code

    Self-hosted

  2. O

    open-notebook

    Launch

    open-notebook is an agent skill for organising research with Open Notebook, a self-hosted alternative to NotebookLM. It arranges sources, notes and AI conversations into research notebooks. Sources can be PDFs, web pages, audio, video and O

    Self-hosted

  3. R

    Reactive Resume

    Launch

    Reactive Resume is a free and open-source resume builder for writing resumes and cover letters, checking the work, and tracking job applications. You create a free account, import an existing resume or start from scratch, and see the finish

    Open sourceSelf-hosted

  4. S

    self-hosted-funnel-launch

    Launch

    Self-hosted funnel launch is an agent skill that takes a funnel from an empty install to published on infrastructure you control, using the Autonnel funnel builder, which is licensed Apache-2.0. The result covers a landing page, checkout, a

    Self-hosted

  5. G

    get-convex/convex-backend

    Launch

    Convex is an open-source reactive database for web app developers. It combines a database, a place to write server functions in pure TypeScript, and client libraries that keep the app's data live, so screens update as the data changes. Beca

    Self-hosted

About preference-optimization

P

preference-optimization

Align a fine-tuned model with preference data using DPO, ORPO, KTO, or SimPO

preference-optimization is an agent skill from the wshobson/agents collection for aligning a fine-tuned language model with preference data, using methods from the DPO family: DPO, ORPO, KTO and SimPO. Preference data comes either as pairs, where a reviewer picks which of two responses is better, or as unpaired thumbs-up and thumbs-down feedback. The skill applies when that kind of feedback exists and the model should learn from it rather than from demonstrations or a verifiable reward. Within the wshobson fine-tuning sequence, the routing step is assumed to have already sent the work here, and the other skills cover demonstration-based training and reward-based reinforcement learning. The skill covers the choice of method within the DPO family, the evidence on how much that choice actually changes results, the production training pattern, and how to build the preference pairs in the first place. Preference data is only as good as the judgments behind it, so the skill's guidance on building pairs matters as much as the training configuration. The skill is installed with the skills CLI from the wshobson/agents repository. Its skills.sh page showed about 2.4K installs and passing results from three security audits at the time of the scan.

View preference-optimizationAll AI Skills apps
Price
Free
Made by
wshobson
Category
AI Skills
Website
github.com/wshobson/agents/preference-optimization