Robots.txt Generator

Build a correct robots.txt in seconds — choose what crawlers may index, block private folders and point to your sitemap.

Default policy Sitemap URL Disallowed paths (one per line) Block AI training bots (GPTBot, CCBot, etc.) Crawl-delay (optional)

Where to put it

Upload the file to your website root so it is reachable at https://yoursite.com/robots.txt. Robots.txt controls crawling, not indexing — to remove a page from Google entirely, use a noindex meta tag as well.

Complete your SEO setup

Preview titles and descriptions with the Meta Tag Checker and create browser icons with the Favicon Generator.