Robots.txt Generator
v1.0.0Generate a robots.txt file to control crawler access to your website.
Generates and validates robots.txt files with a visual rule builder for allowing, disallowing, and setting crawl-delay.
How to use- →Add rules using the + buttons for each user-agent/path combination.
- →Enable sitemap reference to include your sitemap URL in the output.
- →Validate an existing robots.txt by pasting it into the Validate tab.
- →Try a preset from the example picker to see sample input.
User-agent:
disallow:
allow: