Free tools

Robots.txt Examples

Robots.txt file content for go.com.

Robot.txt file for: go.com

      User-Agent: *
Disallow: /7046/
Disallow: /*?
Disallow: /products/
Disallow: /_xd/
Disallow: /_did/

User-agent: GPTBot
Disallow: /

Sitemap: https://www.go.com/sitemap.xml

User-agent: Google-Extended
Disallow: /