Free tools

Robots.txt Examples

Robots.txt file content for illinois.edu.

Robot.txt file for: illinois.edu

      User-agent: *
Disallow: /temp/ 
Disallow: /test/ 
Disallow: /mobile/