Self-hosted alternatives to robots-txt

No self-hosted apps like robots-txt yet.

About robots-txt

robots-txt

Configure, audit, or optimize robots.txt

When the user wants to configure, audit, or optimize robots.txt. Also use when the user mentions "robots.txt," "crawler rules," "block crawlers," "AI