Free tools

Robots.txt Examples

Robots.txt file content for cbssports.com.

Robot.txt file for: cbssports.com

      User-agent: *
Disallow: /info/search
Disallow: /data/*
Disallow: /partners
Disallow: /component/*
Disallow: /user*
Disallow: /watch/navigation-api

User-agent: GPTBot
Disallow: /

Sitemap: https://www.cbssports.com/index-sitemap.xml
Sitemap: https://www.cbssports.com/news-fantasy-sitemap.xml
Sitemap: https://www.cbssports.com/news-gametracker-sitemap.xml
Sitemap: https://www.cbssports.com/news-general-sitemap.xml
Sitemap: https://www.cbssports.com/news-mlb-sitemap.xml
Sitemap: https://www.cbssports.com/news-mma-sitemap.xml
Sitemap: https://www.cbssports.com/news-nba-sitemap.xml
Sitemap: https://www.cbssports.com/news-nfl-sitemap.xml
Sitemap: https://www.cbssports.com/news-nhl-sitemap.xml
Sitemap: https://www.cbssports.com/news-sitemap.xml
Sitemap: https://www.cbssports.com/news-tennis-sitemap.xml
Sitemap: https://www.cbssports.com/watch/sitemap.xml
Sitemap: https://www.cbssports.com/watch/index-sitemap.xml