Free tools

Robots.txt Examples

Robots.txt file content for newsbreak.com.

Robot.txt file for: newsbreak.com

      Sitemap: https://www.newsbreak.com/sitemap.xml
Sitemap: https://www.newsbreak.com/sitemap-publisher.xml
Sitemap: https://www.newsbreak.com/sitemap-local-index.xml
Sitemap: https://www.newsbreak.com/sitemap-news-index.xml

User-agent: *
Disallow: /_api/
Disallow: /api/
Disallow: /channels/
Disallow: /privacy
Disallow: /terms
Disallow: /t-*
Disallow: /redirect-external
Disallow: /me/
Disallow: /_next/data/*.json
Disallow: /following
Disallow: /following/