Free tools
Robots.txt Examples
Robots.txt file content for indiatimes.com.
Robot.txt file for: indiatimes.com
#robots.txt User-agent: * Allow: / Disallow: *.php* Disallow: */7176/* Disallow: */whatshot/* Disallow: */wattpad/* Disallow: */uc_feed/* Disallow: */brief/* Disallow: */search/* Disallow: */user/* Disallow: */idtoin/* Disallow: */video_player/* Disallow: */blog/* Disallow: */content/* Disallow: */seoarchive/* Disallow: */trends/audios/* Disallow: */trends/videos/* Disallow: *sw.js* Disallow: *{* Disallow: *&* Disallow: *?utm_source* Disallow: *?ref=* Disallow: *?v=* Disallow: *?google_editors_picks* Disallow: *?no_fullscreen_* Disallow: *?symbol=* Disallow: */?date=* Disallow: *?fb_comment_id=* Disallow: *?_twitter* Disallow: */2011/* Disallow: */2012/* Disallow: */2013/* Disallow: */2014/* Disallow: */2015/* Disallow: */2016/* Disallow: */2017/* Disallow: */2018/* Disallow: */2019/* Disallow: */2020/* Disallow: */2021/* Disallow: */2022/* Disallow: */2023/* Disallow: */2024/* #Sitemaps Sitemap:https://www.indiatimes.com/sitemap.xml Sitemap:https://www.indiatimes.com/sitemap-news.xml Sitemap:https://www.indiatimes.com/sitemap-posts-index-2024.xml Sitemap:https://www.indiatimes.com/hindi/sitemap-ampstories-2024.xml User-agent: GPTBot Disallow: /