QuickFreeTools

Online Robots.txt Generator

Generate robots.txt content for your domain. Set User-agent rules and Allow/Disallow paths for search engine crawlers.

User-agent: *
Allow: /
Disallow: /admin/
Disallow: /private/

Frequently Asked Questions

Where do I upload robots.txt?
Upload robots.txt to the root of your domain (e.g. example.com/robots.txt).
What does User-agent: * mean?
The asterisk applies rules to all crawlers. You can target specific bots like Googlebot with User-agent: Googlebot.
Should I disallow admin or private paths?
Yes. Use Disallow for paths you don't want indexed (e.g. /admin/, /api/).

Related SEO Tools