Free tools

Robots.txt Examples

Robots.txt file content for aaa.com.

Robot.txt file for: aaa.com

      # For domain: http://www.aaa.com
 
User-agent: *

Allow: /aaa/common/travel/car/*
Allow: /autorepair/*
Allow: /carbatteryquote/*
Disallow: /aaa/*
Disallow: /travel2/tourbook/*
Disallow: /travel2/destination/*
Disallow: /travel2/sycs/*
Disallow: /scripts/WebObjects.dll/*
 
Sitemap: http://www.aaa.com/sitemapindex.xml