Free tools
Robots.txt Examples
Robots.txt file content for deepl.com.
Robot.txt file for: deepl.com
User-agent: * User-agent: AdsBot-Google User-agent: Adsbot-Google-Mobile User-agent: Mediapartners-Google User-agent: AdIdxBot Allow: */translator/c/ Disallow: /*docs-api-legacy$ Disallow: /*docs-api-legacy/ # Cloudflare Disallow: /cdn-cgi/bm/cv/ Disallow: /cdn-cgi/challenge-platform/ Disallow: /cdn-cgi/images/trace/ Disallow: /cdn-cgi/rum Disallow: /cdn-cgi/scripts/ Disallow: /cdn-cgi/styles/ Disallow: /cdn-cgi/zaraz/ User-agent: ChatGPT-User Disallow: / User-agent: Googlebot User-agent: Bingbot Disallow: */translator/c/ Disallow: /*docs-api-legacy$ Disallow: /*docs-api-legacy/ # Cloudflare Disallow: /cdn-cgi/bm/cv/ Disallow: /cdn-cgi/challenge-platform/ Disallow: /cdn-cgi/images/trace/ Disallow: /cdn-cgi/rum Disallow: /cdn-cgi/scripts/ Disallow: /cdn-cgi/styles/ Disallow: /cdn-cgi/zaraz/