Robots.txt Generator

Create a customizable robots.txt file to guide search engine crawlers on which parts of your site should be indexed. Essential for SEO management.

Crawler Rules

User Agent

Access

Path

Sitemap URL (Optional)

File Content

User-agent: *
Allow: /

Sitemap: https://example.com/sitemap.xml

This rule applies to all web crawlers. You can also specify individual crawlers like "Googlebot" or "Bingbot" for specific instructions.

Always disallow your administration or private user-data folders to prevent them from showing up in public search results.