Free tools
Robots.txt Examples
Robots.txt file content for osu.edu.
Robot.txt file for: osu.edu
User-agent: * Disallow: /map/building/ Sitemap: https://www.osu.edu/sitemap.xml User-agent: * Disallow: /static/
Robots.txt file content for osu.edu.
User-agent: * Disallow: /map/building/ Sitemap: https://www.osu.edu/sitemap.xml User-agent: * Disallow: /static/