Self-hosted alternatives to automate-this

A

automate-this

Analyze a screen recording of a manual process and produce targeted

automate-this is an agent skill from the github/awesome-copilot collection for turning a screen recording of a repetitive manual task into working automation. The user records themselves doing the task and hands over the video, and the agent works out what is being done, why, and how to script it away. The skill begins with a prerequisites check. It needs ffmpeg to extract frames from the recording, and it uses Whisper, an optional speech-to-text tool, when the recording has spoken narration; without Whisper the agent can still analyze the visuals. Phase 1 extracts the content from the recording, pulling out frames at key moments and the audio narration where it exists. Later phases analyze the steps, identify the parts that can be scripted, and build the automation and test it. The approach works best for tasks that are repetitive, rule-based and done the same way each time, such as moving data between screens, renaming files or filling in forms. Automating a task can change how it is done, so the skill's output should be tested on copies before it is run on real data. The recording may show sensitive information on screen, so it should be reviewed before it is shared with the agent. The skill is installed with the skills CLI from the github/awesome-copilot repository. Its skills.sh page showed about 2.3K installs and passing results from three security audits at the time of the scan.