Free tools

Robots.txt Examples

Robots.txt file content for collider.com.

Robot.txt file for: collider.com

      User-agent: *
Disallow: /pixel.png*
Disallow: /search/
Disallow: /profile/
User-agent: GPTBot
Disallow: /
Sitemap: https://collider.com/sitemap.xml