robots.txt Generator

Generate a robots.txt file

A robots.txt file tells search engine crawlers which pages or sections of your site they can or cannot access. Place the generated file at the root of your domain (e.g. https://example.com/robots.txt).

Everything runs in your browser — nothing is uploaded to a server.

FAQ

What does Allow: / mean? It tells all bots they may crawl the entire site.

Should I list every page to disallow? No — just list the path prefixes you want to block (e.g. /admin blocks /admin/*).