Free tools

Robots.txt Examples

Robots.txt file content for maryland.gov.

Robot.txt file for: maryland.gov

      User-agent: *
Disallow: /_layouts/
Disallow: /_catalogs/
Disallow: /_vti_bin/
Disallow: /shutdown