Free tools
Robots.txt Examples
Robots.txt file content for indiatvnews.com.
Robot.txt file for: indiatvnews.com
User-agent: * Allow: / Disallow: /testfiles/ Disallow: /8323530/ Disallow: /weatherapis/* Disallow: /cricket/getmatchlist* Disallow: /pincode/ajaxpostofficeload/* Disallow: /sports/cricket/seriesembed* Disallow: /sports/cricketfixtures/* Disallow: /print/* Disallow: /sports/cricket/getcricketseriesdetaildata/* Disallow: /newsdata/* Disallow: /sports/getlivematchdatamobile/* Disallow: /iframe/* User-agent: Twitterbot Allow: / Disallow: /speedtest User-agent: Adsbot-Google Disallow: /testfiles/ User-agent: Googlebot-Video Disallow: /search-video/ Disallow: /testfiles/ User-agent: GPTBot Disallow: / User-agent: CCBots Disallow: / User-agent: AhrefsSiteAudit Allow: / User-agent: AhrefsBot Allow: / Sitemap: https://www.indiatvnews.com/news-sitemap.xml Sitemap: https://www.indiatvnews.com/sitemap.xml