Free tools
Robots.txt Examples
Robots.txt file content for clarin.com.
Robot.txt file for: clarin.com
User-agent: * Disallow: /api/ Disallow: /redaccionfiler/* Disallow: /viafoura/* Disallow: /autor/*? Disallow: /videos/*? Disallow: /videoloop/*_3_ Sitemap: https://www.clarin.com/sitemaps/index.xml Sitemap: https://www.clarin.com/sitemaps/index_services.xml Sitemap: https://www.clarin.com/sitemaps/sitemap_google_news.xml