Générateur gratuit de Robots.txt en ligne
Build a robots.txt file for your website quickly.
Quick Start Presets
How to Use
- Choose a preset or build rules manually.
- Add user-agent groups with Allow/Disallow paths.
- Add your Sitemap URL(s).
- Copy or download the file and upload it to your site's root directory.
Frequently Asked Questions
Where should robots.txt be placed?
It must be at the root of your domain: https://yourdomain.com/robots.txt. It won't work in subdirectories.
Does robots.txt block pages from Google?
Robots.txt prevents crawlers from accessing pages, but it doesn't remove them from search results. For that, use a noindex meta tag.
What are AI crawlers?
AI crawlers like GPTBot, CCBot, and Google-Extended scrape content for training AI models. You can block them specifically in your robots.txt.