Online Robots.txt Generator
Generate robots.txt content for your domain. Set User-agent rules and Allow/Disallow paths for search engine crawlers.
User-agent: * Allow: / Disallow: /admin/ Disallow: /private/
Frequently Asked Questions
- Where do I upload robots.txt?
- Upload robots.txt to the root of your domain (e.g. example.com/robots.txt).
- What does User-agent: * mean?
- The asterisk applies rules to all crawlers. You can target specific bots like Googlebot with User-agent: Googlebot.
- Should I disallow admin or private paths?
- Yes. Use Disallow for paths you don't want indexed (e.g. /admin/, /api/).
Related SEO Tools
Meta Description Generator
Generate SEO meta descriptions for your pages. Optimize length and keywords for search results.
UTM Builder
Build UTM-tagged URLs for campaign tracking. Add source, medium, campaign, term, and content parameters.
Google SERP Preview Optimization Tool
Preview how your title and meta description will appear in Google search results (~60 / ~160 chars).
Keyword Density Checker
Check keyword density: enter text and a keyword to see count, occurrences, and density (count/total words × 100).
Open Graph Generator
Generate Open Graph meta tags (og:title, og:description, og:image, etc.) as HTML or JSON.
Twitter Card Generator
Generate Twitter Card meta tags from title, description, and image URL. Copy HTML for rich previews on Twitter/X.