Wix Robots.txt Generator

Build a custom robots.txt file for your Wix site with individual controls for search engine crawlers and AI bots. Toggle GPTBot, ClaudeBot, PerplexityBot and others on or off, set crawl-delay and add your sitemap URL. No sign-up required, no data stored, everything runs in your browser.

How this tool works

Choose a base crawl policy: allow all crawlers, block all crawlers, or start blank. Then fine-tune which specific paths to allow or disallow using the path editor. The tool has dedicated toggle sections for AI crawlers (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Bytespider, Meta-ExternalAgent) and for unwanted bots (AhrefsBot, SemrushBot, DotBot, MJ12bot, PetalBot). Set an optional crawl-delay and add your sitemap URL. The generated output follows the Robots Exclusion Protocol and is ready to paste into your Wix SEO settings.

Where to add robots.txt on Wix

In your Wix dashboard, go to Settings, then SEO Tools, then robots.txt Editor. Paste the generated directives there. Changes go live within minutes. Note that Wix adds some default directives automatically. You are adding to them, not replacing the whole file.

Crawling vs indexing

robots.txt controls crawling, not indexing. A blocked page can still appear in search results if other sites link to it. To prevent indexing, use a noindex meta tag or X-Robots-Tag HTTP header instead. Use the meta tag generator to create the correct tags.

AI crawlers explained

GPTBot and ChatGPT-User are operated by OpenAI. GPTBot collects training data while ChatGPT-User powers live ChatGPT browsing. ClaudeBot is Anthropic's crawler for Claude. PerplexityBot powers Perplexity AI search answers. Bytespider is ByteDance's crawler. Meta-ExternalAgent is Meta's AI training crawler. Each can be blocked independently in your robots.txt without affecting traditional search engines like Google or Bing.

By Michael Andrews - UK's #1 Wix SEO Expert | Get a Free SEO Audit