Free tools

Robots.txt Examples

Robots.txt file content for dexerto.com.

Robot.txt file for: dexerto.com

      User-Agent: *
Disallow: /search/
Disallow: /cdn-cgi/
Allow: /cdn-cgi/image/

Sitemap: https://www.dexerto.com/sitemap_index.xml
Sitemap: https://www.dexerto.com/news-sitemap.xml