Free tools

Robots.txt Examples

Robots.txt file content for chron.com.

Robot.txt file for: chron.com

      Sitemap: https://www.chron.com/sitemap.xml
Sitemap: https://www.chron.com/sitemap_news.xml

User-agent: *
Disallow: /style/beauty/hearstmagazines/
Disallow: /style/fashion/hearstmagazines/
Disallow: /living/relationships/hearstmagazines/
Disallow: /homeandgarden/home/hearstmagazines/
Disallow: /living/wellness/hearstmagazines/
Disallow: /adtest
Disallow: /sponsored
Disallow: /events/
Disallow: /search

User-agent: Googlebot-News
Disallow: /business/press-releases

User-agent: CCBot
Disallow: /

User-agent: *
Disallow: /413gkwMT/

User-agent: Applebot-Extended
Disallow: /private/