Free tools

Robots.txt Examples

Robots.txt file content for cylex.de.

Robot.txt file for: cylex.de

      User-agent: *
Disallow: /google/
Disallow: /reviews/
Disallow: /webmastertools/
Disallow: /fir_news/
Disallow: /webservices/
Disallow: /Homepage/internet-shops.asp
Disallow: /webshop/
Disallow: /correct/
Disallow: /webservices/
Disallow: /ContactForm.aspx
Disallow: /ContactForm2.aspx
Disallow: /ContactForm.ashx
Disallow: /uploadHandler.ashx
Disallow: /userContent.ashx
Disallow: /korrektur_firmendaten.asp
Disallow: /s?
Disallow: /ScriptResource.axd 
Disallow: /combinescriptshandler.axd 
Disallow: /WebResource.axd
Disallow: /info/terms-and-conditions-customer-reviews.html
Disallow: /info/terms-and-conditions-quote-requests.html
Disallow: /cdn-cgi/

User-agent: Mediapartners-Google
Allow: /s?
User-agent: GPTBot
Disallow: /