Free tools

Robots.txt Examples

Robots.txt file content for aaa.com.

Robot.txt file for: aaa.com

      User-agent: * 

Allow: /autorepair/* 

Allow: /carbatteryquote/* 

Allow: /tripcanvas/* 
 
Disallow: /aaa/* 
Disallow: /travel2/tourbook/* 
Disallow: /travel2/destination/* 
Disallow: /travel2/sycs/* 
Disallow: /scripts/WebObjects.dll/* 

User-agent: GPTBot 
Disallow: / 
  
Sitemap: https://www.aaa.com/sitemapindex.xml